2024-09-12T12:05:21.304 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T12:05:21.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T12:05:21.429 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/sessionmap} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901713' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - client session with non-allowable root sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAff60/SFZ9MgWP4VvK3P9pI+z79kgSgUPfuxUNpSMfaabZXNMv4GQRgY4GVTnvRukzP66vNtY4fN3abh9kgyac= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKvqt2M+a9vqwj94NVuvAgxnY7dJhwPVxuZOEmRdLN0vf7C2+QbjpcQVX9VpHhcc7jE2NUUWMNYG3lCBm1PT3wM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_sessionmap teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T12:05:21.429 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T12:05:21.430 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T12:05:21.430 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T12:05:21.432 INFO:teuthology.task.internal:Checking packages... 2024-09-12T12:05:21.453 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T12:05:21.453 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T12:05:21.453 INFO:teuthology.packaging:ref: None 2024-09-12T12:05:21.453 INFO:teuthology.packaging:tag: None 2024-09-12T12:05:21.453 INFO:teuthology.packaging:branch: squid 2024-09-12T12:05:21.453 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:05:21.453 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-12T12:05:21.663 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314-g086c737f-1jammy 2024-09-12T12:05:21.664 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T12:05:21.671 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T12:05:21.672 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T12:05:21.741 INFO:teuthology.task.internal:Saving configuration 2024-09-12T12:05:21.757 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T12:05:21.817 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T12:05:21.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 11:59:55.334799', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK1ZsNgLk4WqbpH0lUT2CQq5fj/E+1IAcfmpF7kZ8utL'} 2024-09-12T12:05:21.872 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 11:59:55.336860', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICF9rMrThbuDZbi3E+/mB88jY/k/2aU7wfaQ/SCs0re2'} 2024-09-12T12:05:21.872 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T12:05:21.879 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-12T12:05:21.879 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-12T12:05:21.879 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T12:05:21.995 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb59baf4310>, signals=[15]) 2024-09-12T12:05:21.995 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T12:05:22.001 INFO:teuthology.task.internal:Opening connections... 2024-09-12T12:05:22.001 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:05:22.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:05:22.096 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:05:22.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:05:22.187 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T12:05:22.194 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-09-12T12:05:22.199 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-09-12T12:05:22.199 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-09-12T12:05:22.244 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-12T12:05:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Ubuntu" 2024-09-12T12:05:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="22.04" 2024-09-12T12:05:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-12T12:05:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_CODENAME=jammy 2024-09-12T12:05:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:ID=ubuntu 2024-09-12T12:05:22.245 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE=debian 2024-09-12T12:05:22.246 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-12T12:05:22.246 INFO:teuthology.orchestra.run.smithi003.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-12T12:05:22.246 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-12T12:05:22.246 INFO:teuthology.orchestra.run.smithi003.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-12T12:05:22.246 INFO:teuthology.orchestra.run.smithi003.stdout:UBUNTU_CODENAME=jammy 2024-09-12T12:05:22.246 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-09-12T12:05:22.276 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-09-12T12:05:22.281 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-09-12T12:05:22.282 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-09-12T12:05:22.328 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-12T12:05:22.328 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Ubuntu" 2024-09-12T12:05:22.328 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="22.04" 2024-09-12T12:05:22.328 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_CODENAME=jammy 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:ID=ubuntu 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE=debian 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-12T12:05:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:UBUNTU_CODENAME=jammy 2024-09-12T12:05:22.329 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-09-12T12:05:22.350 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T12:05:22.359 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T12:05:22.381 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T12:05:22.382 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-09-12T12:05:22.384 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-09-12T12:05:22.388 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T12:05:22.490 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T12:05:22.490 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-09-12T12:05:22.493 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-09-12T12:05:22.499 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T12:05:22.576 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T12:05:22.594 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-12T12:05:22.594 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-12T12:05:22.595 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T12:05:22.595 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T12:05:22.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:05:22.599 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-12T12:05:22.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:05:22.600 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-09-12T12:05:22.645 INFO:teuthology.orchestra.run.smithi003.stdout:5.15.0-56-generic 2024-09-12T12:05:22.645 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.15.0-56-generic 2024-09-12T12:05:22.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get clean 2024-09-12T12:05:22.648 INFO:teuthology.orchestra.run.smithi142.stdout:5.15.0-56-generic 2024-09-12T12:05:22.648 INFO:teuthology.task.kernel:Running kernel on smithi142: 5.15.0-56-generic 2024-09-12T12:05:22.649 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get clean 2024-09-12T12:05:22.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2024-09-12T12:05:22.796 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get update 2024-09-12T12:05:22.921 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T12:05:22.921 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-12T12:05:22.921 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-12T12:05:22.953 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T12:05:22.963 INFO:teuthology.orchestra.run.smithi142.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-12T12:05:22.963 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-12T12:05:22.993 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-12T12:05:23.026 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-12T12:05:23.172 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-12T12:05:23.196 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-12T12:05:23.239 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-12T12:05:23.248 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-12T12:05:23.269 INFO:teuthology.orchestra.run.smithi142.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-12T12:05:23.277 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-12T12:05:23.279 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-12T12:05:23.280 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-12T12:05:23.297 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-12T12:05:23.301 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-12T12:05:23.303 INFO:teuthology.orchestra.run.smithi142.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-12T12:05:23.304 INFO:teuthology.orchestra.run.smithi142.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-12T12:05:23.304 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-12T12:05:23.304 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-12T12:05:23.304 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-12T12:05:23.305 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-12T12:05:23.310 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-12T12:05:23.316 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-12T12:05:23.317 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-12T12:05:23.318 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-12T12:05:23.318 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-12T12:05:23.323 INFO:teuthology.orchestra.run.smithi142.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-12T12:05:23.324 INFO:teuthology.orchestra.run.smithi142.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-12T12:05:23.326 INFO:teuthology.orchestra.run.smithi142.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-12T12:05:23.326 INFO:teuthology.orchestra.run.smithi142.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-12T12:05:23.330 INFO:teuthology.orchestra.run.smithi142.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-12T12:05:23.334 INFO:teuthology.orchestra.run.smithi142.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-12T12:05:23.336 INFO:teuthology.orchestra.run.smithi142.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-12T12:05:23.337 INFO:teuthology.orchestra.run.smithi142.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-12T12:05:23.339 INFO:teuthology.orchestra.run.smithi142.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-12T12:05:23.339 INFO:teuthology.orchestra.run.smithi142.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-12T12:05:23.351 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-12T12:05:23.351 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-12T12:05:23.352 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-12T12:05:23.357 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-12T12:05:23.360 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-12T12:05:23.361 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-12T12:05:23.361 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-12T12:05:23.376 INFO:teuthology.orchestra.run.smithi142.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-12T12:05:23.377 INFO:teuthology.orchestra.run.smithi142.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-12T12:05:23.377 INFO:teuthology.orchestra.run.smithi142.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-12T12:05:23.380 INFO:teuthology.orchestra.run.smithi142.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-12T12:05:23.380 INFO:teuthology.orchestra.run.smithi142.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-12T12:05:23.382 INFO:teuthology.orchestra.run.smithi142.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-12T12:05:23.400 INFO:teuthology.orchestra.run.smithi142.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-12T12:05:23.404 INFO:teuthology.orchestra.run.smithi142.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-12T12:05:23.404 INFO:teuthology.orchestra.run.smithi142.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-12T12:05:23.414 INFO:teuthology.orchestra.run.smithi142.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-12T12:05:23.419 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-12T12:05:23.420 INFO:teuthology.orchestra.run.smithi142.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-12T12:05:23.422 INFO:teuthology.orchestra.run.smithi142.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-12T12:05:23.422 INFO:teuthology.orchestra.run.smithi142.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-12T12:05:23.423 INFO:teuthology.orchestra.run.smithi142.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-12T12:05:23.423 INFO:teuthology.orchestra.run.smithi142.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-12T12:05:23.424 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-12T12:05:23.424 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-12T12:05:23.430 INFO:teuthology.orchestra.run.smithi142.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-12T12:05:23.430 INFO:teuthology.orchestra.run.smithi142.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-12T12:05:23.431 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,123 kB] 2024-09-12T12:05:23.432 INFO:teuthology.orchestra.run.smithi142.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-12T12:05:23.439 INFO:teuthology.orchestra.run.smithi142.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-12T12:05:23.440 INFO:teuthology.orchestra.run.smithi142.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-12T12:05:23.440 INFO:teuthology.orchestra.run.smithi142.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-12T12:05:23.441 INFO:teuthology.orchestra.run.smithi142.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-12T12:05:23.441 INFO:teuthology.orchestra.run.smithi142.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-12T12:05:23.447 INFO:teuthology.orchestra.run.smithi142.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-12T12:05:23.470 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-12T12:05:23.473 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-12T12:05:23.474 INFO:teuthology.orchestra.run.smithi003.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-12T12:05:23.474 INFO:teuthology.orchestra.run.smithi003.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-12T12:05:23.475 INFO:teuthology.orchestra.run.smithi003.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-12T12:05:23.475 INFO:teuthology.orchestra.run.smithi003.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-12T12:05:23.475 INFO:teuthology.orchestra.run.smithi003.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-12T12:05:23.476 INFO:teuthology.orchestra.run.smithi003.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-12T12:05:23.477 INFO:teuthology.orchestra.run.smithi003.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-12T12:05:23.490 INFO:teuthology.orchestra.run.smithi003.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-12T12:05:23.507 INFO:teuthology.orchestra.run.smithi003.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-12T12:05:23.508 INFO:teuthology.orchestra.run.smithi003.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-12T12:05:23.508 INFO:teuthology.orchestra.run.smithi003.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-12T12:05:23.509 INFO:teuthology.orchestra.run.smithi003.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-12T12:05:29.819 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 15.8 MB in 2s (7,480 kB/s) 2024-09-12T12:05:29.819 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 15.8 MB in 2s (7,135 kB/s) 2024-09-12T12:05:31.134 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-09-12T12:05:31.147 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-12T12:05:31.152 DEBUG:teuthology.orchestra.run.smithi142:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T12:05:31.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T12:05:31.212 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-09-12T12:05:31.221 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-12T12:05:31.410 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2024-09-12T12:05:31.410 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2024-09-12T12:05:31.421 INFO:teuthology.orchestra.run.smithi142.stdout:Building dependency tree... 2024-09-12T12:05:31.421 INFO:teuthology.orchestra.run.smithi142.stdout:Reading state information... 2024-09-12T12:05:31.580 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:05:31.580 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:05:31.580 INFO:teuthology.orchestra.run.smithi003.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:05:31.581 INFO:teuthology.orchestra.run.smithi003.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:05:31.581 INFO:teuthology.orchestra.run.smithi003.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-12T12:05:31.581 INFO:teuthology.orchestra.run.smithi003.stdout: python2.7-minimal 2024-09-12T12:05:31.581 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:05:31.581 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2024-09-12T12:05:31.582 INFO:teuthology.orchestra.run.smithi003.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T12:05:31.582 INFO:teuthology.orchestra.run.smithi003.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-12T12:05:31.582 INFO:teuthology.orchestra.run.smithi003.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-12T12:05:31.583 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2024-09-12T12:05:31.583 INFO:teuthology.orchestra.run.smithi003.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-12T12:05:31.583 INFO:teuthology.orchestra.run.smithi003.stdout:Recommended packages: 2024-09-12T12:05:31.583 INFO:teuthology.orchestra.run.smithi003.stdout: thermald 2024-09-12T12:05:31.594 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:05:31.594 INFO:teuthology.orchestra.run.smithi142.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:05:31.594 INFO:teuthology.orchestra.run.smithi142.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:05:31.595 INFO:teuthology.orchestra.run.smithi142.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:05:31.595 INFO:teuthology.orchestra.run.smithi142.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-12T12:05:31.595 INFO:teuthology.orchestra.run.smithi142.stdout: python2.7-minimal 2024-09-12T12:05:31.595 INFO:teuthology.orchestra.run.smithi142.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:05:31.596 INFO:teuthology.orchestra.run.smithi142.stdout:The following additional packages will be installed: 2024-09-12T12:05:31.596 INFO:teuthology.orchestra.run.smithi142.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T12:05:31.596 INFO:teuthology.orchestra.run.smithi142.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-12T12:05:31.597 INFO:teuthology.orchestra.run.smithi142.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-12T12:05:31.598 INFO:teuthology.orchestra.run.smithi142.stdout:Suggested packages: 2024-09-12T12:05:31.598 INFO:teuthology.orchestra.run.smithi142.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-12T12:05:31.598 INFO:teuthology.orchestra.run.smithi142.stdout:Recommended packages: 2024-09-12T12:05:31.598 INFO:teuthology.orchestra.run.smithi142.stdout: thermald 2024-09-12T12:05:31.629 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2024-09-12T12:05:31.629 INFO:teuthology.orchestra.run.smithi003.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T12:05:31.629 INFO:teuthology.orchestra.run.smithi003.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-12T12:05:31.629 INFO:teuthology.orchestra.run.smithi003.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-12T12:05:31.631 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2024-09-12T12:05:31.631 INFO:teuthology.orchestra.run.smithi003.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-12T12:05:31.650 INFO:teuthology.orchestra.run.smithi142.stdout:The following NEW packages will be installed: 2024-09-12T12:05:31.650 INFO:teuthology.orchestra.run.smithi142.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-12T12:05:31.650 INFO:teuthology.orchestra.run.smithi142.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-12T12:05:31.650 INFO:teuthology.orchestra.run.smithi142.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-12T12:05:31.652 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages will be upgraded: 2024-09-12T12:05:31.652 INFO:teuthology.orchestra.run.smithi142.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-12T12:05:31.813 INFO:teuthology.orchestra.run.smithi003.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T12:05:31.813 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 113 MB of archives. 2024-09-12T12:05:31.813 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-12T12:05:31.813 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-12T12:05:31.832 INFO:teuthology.orchestra.run.smithi142.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T12:05:31.832 INFO:teuthology.orchestra.run.smithi142.stdout:Need to get 113 MB of archives. 2024-09-12T12:05:31.832 INFO:teuthology.orchestra.run.smithi142.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-12T12:05:31.832 INFO:teuthology.orchestra.run.smithi142.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-12T12:05:33.218 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-12T12:05:33.339 INFO:teuthology.orchestra.run.smithi142.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-12T12:05:33.572 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-12T12:05:33.692 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-12T12:05:35.800 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-12T12:05:35.800 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-12T12:05:35.800 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-12T12:05:35.930 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-12T12:05:35.930 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-12T12:05:35.930 INFO:teuthology.orchestra.run.smithi142.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-12T12:05:36.412 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-12T12:05:36.505 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-12T12:05:36.893 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 113 MB in 5s (23.4 MB/s) 2024-09-12T12:05:37.039 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-12T12:05:37.082 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-12T12:05:37.197 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-12T12:05:37.647 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 113 MB in 6s (20.4 MB/s) 2024-09-12T12:05:37.830 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-12T12:05:39.006 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-12T12:05:39.010 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:39.095 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:39.796 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-12T12:05:39.801 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:39.887 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:41.178 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-12T12:05:41.199 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:41.265 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:41.723 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-12T12:05:41.743 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:41.781 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:42.238 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-12T12:05:42.260 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:42.353 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:42.757 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-12T12:05:42.778 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:42.807 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:48.410 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T12:05:48.537 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T12:05:48.869 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T12:05:49.006 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T12:05:49.174 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T12:05:49.298 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T12:05:49.326 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-12T12:05:49.348 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-12T12:05:49.384 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T12:05:49.614 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T12:05:49.719 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T12:05:50.030 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-12T12:05:50.052 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-12T12:05:50.096 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T12:05:55.254 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-12T12:05:55.293 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:55.337 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:55.623 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-12T12:05:55.660 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-12T12:05:55.698 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:57.956 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T12:05:58.079 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T12:05:58.421 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T12:05:58.444 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-12T12:05:59.367 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:59.400 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-12T12:05:59.485 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-12T12:05:59.597 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:59.742 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-12T12:05:59.855 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:05:59.956 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-12T12:06:00.068 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:06:00.935 INFO:teuthology.orchestra.run.smithi142.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-12T12:06:00.936 INFO:teuthology.orchestra.run.smithi142.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-12T12:06:00.936 INFO:teuthology.orchestra.run.smithi142.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-12T12:06:00.936 INFO:teuthology.orchestra.run.smithi142.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:01.081 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:06:01.409 INFO:teuthology.orchestra.run.smithi003.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-12T12:06:01.409 INFO:teuthology.orchestra.run.smithi003.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-12T12:06:01.409 INFO:teuthology.orchestra.run.smithi003.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-12T12:06:01.409 INFO:teuthology.orchestra.run.smithi003.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:01.569 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:06:02.474 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:06:03.020 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:06:03.884 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-12T12:06:04.010 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-12T12:06:04.136 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:06:04.199 INFO:teuthology.orchestra.run.smithi142.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-12T12:06:04.200 INFO:teuthology.orchestra.run.smithi142.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:04.446 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-12T12:06:05.423 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-12T12:06:05.524 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-12T12:06:05.579 INFO:teuthology.orchestra.run.smithi003.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-12T12:06:05.579 INFO:teuthology.orchestra.run.smithi003.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:18.193 INFO:teuthology.orchestra.run.smithi142.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-12T12:06:18.194 INFO:teuthology.orchestra.run.smithi142.stdout:Sourcing file `/etc/default/grub' 2024-09-12T12:06:18.210 INFO:teuthology.orchestra.run.smithi142.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T12:06:18.234 INFO:teuthology.orchestra.run.smithi142.stdout:Generating grub configuration file ... 2024-09-12T12:06:18.513 INFO:teuthology.orchestra.run.smithi142.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T12:06:18.529 INFO:teuthology.orchestra.run.smithi142.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:18.822 INFO:teuthology.orchestra.run.smithi142.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T12:06:18.828 INFO:teuthology.orchestra.run.smithi142.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T12:06:18.900 INFO:teuthology.orchestra.run.smithi142.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T12:06:18.906 INFO:teuthology.orchestra.run.smithi142.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T12:06:19.086 INFO:teuthology.orchestra.run.smithi142.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T12:06:19.086 INFO:teuthology.orchestra.run.smithi142.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T12:06:19.086 INFO:teuthology.orchestra.run.smithi142.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T12:06:19.125 INFO:teuthology.orchestra.run.smithi142.stdout:done 2024-09-12T12:06:19.539 DEBUG:teuthology.orchestra.run.smithi142:> dpkg -s linux-image-generic 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Package: linux-image-generic 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Status: install ok installed 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Priority: optional 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Section: kernel 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Installed-Size: 21 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: amd64 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Source: linux-meta 2024-09-12T12:06:19.575 INFO:teuthology.orchestra.run.smithi142.stdout:Version: 5.15.0.119.119 2024-09-12T12:06:19.576 INFO:teuthology.orchestra.run.smithi142.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T12:06:19.576 INFO:teuthology.orchestra.run.smithi142.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T12:06:19.576 INFO:teuthology.orchestra.run.smithi142.stdout:Recommends: thermald 2024-09-12T12:06:19.576 INFO:teuthology.orchestra.run.smithi142.stdout:Description: Generic Linux kernel image 2024-09-12T12:06:19.576 INFO:teuthology.orchestra.run.smithi142.stdout: This package will always depend on the latest generic kernel image 2024-09-12T12:06:19.576 INFO:teuthology.orchestra.run.smithi142.stdout: available. 2024-09-12T12:06:19.576 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-12T12:06:19.576 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T12:06:19.576 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T12:06:19.577 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-12T12:06:19.577 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T12:06:19.577 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get clean 2024-09-12T12:06:19.774 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get update 2024-09-12T12:06:20.024 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T12:06:20.024 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-12T12:06:20.050 INFO:teuthology.orchestra.run.smithi003.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-12T12:06:20.051 INFO:teuthology.orchestra.run.smithi003.stdout:Sourcing file `/etc/default/grub' 2024-09-12T12:06:20.055 INFO:teuthology.orchestra.run.smithi003.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T12:06:20.081 INFO:teuthology.orchestra.run.smithi003.stdout:Generating grub configuration file ... 2024-09-12T12:06:20.096 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T12:06:20.178 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T12:06:20.337 INFO:teuthology.orchestra.run.smithi003.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T12:06:20.357 INFO:teuthology.orchestra.run.smithi003.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:20.653 INFO:teuthology.orchestra.run.smithi003.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T12:06:20.659 INFO:teuthology.orchestra.run.smithi003.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T12:06:20.724 INFO:teuthology.orchestra.run.smithi003.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T12:06:20.731 INFO:teuthology.orchestra.run.smithi003.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T12:06:20.916 INFO:teuthology.orchestra.run.smithi003.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T12:06:20.916 INFO:teuthology.orchestra.run.smithi003.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T12:06:20.916 INFO:teuthology.orchestra.run.smithi003.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T12:06:20.958 INFO:teuthology.orchestra.run.smithi003.stdout:done 2024-09-12T12:06:21.456 DEBUG:teuthology.orchestra.run.smithi003:> dpkg -s linux-image-generic 2024-09-12T12:06:21.477 INFO:teuthology.orchestra.run.smithi003.stdout:Package: linux-image-generic 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Status: install ok installed 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Priority: optional 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Section: kernel 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Installed-Size: 21 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: amd64 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Source: linux-meta 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Version: 5.15.0.119.119 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T12:06:21.478 INFO:teuthology.orchestra.run.smithi003.stdout:Recommends: thermald 2024-09-12T12:06:21.479 INFO:teuthology.orchestra.run.smithi003.stdout:Description: Generic Linux kernel image 2024-09-12T12:06:21.479 INFO:teuthology.orchestra.run.smithi003.stdout: This package will always depend on the latest generic kernel image 2024-09-12T12:06:21.479 INFO:teuthology.orchestra.run.smithi003.stdout: available. 2024-09-12T12:06:21.479 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-12T12:06:21.479 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T12:06:21.479 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T12:06:21.479 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-12T12:06:21.479 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T12:06:21.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get clean 2024-09-12T12:06:21.485 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-09-12T12:06:21.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T12:06:21.568 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-09-12T12:06:21.690 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2024-09-12T12:06:21.777 INFO:teuthology.orchestra.run.smithi142.stdout:Building dependency tree... 2024-09-12T12:06:21.778 INFO:teuthology.orchestra.run.smithi142.stdout:Reading state information... 2024-09-12T12:06:21.819 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-12T12:06:21.820 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T12:06:21.830 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T12:06:21.848 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T12:06:21.958 INFO:teuthology.orchestra.run.smithi142.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-12T12:06:21.959 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:06:21.959 INFO:teuthology.orchestra.run.smithi142.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:06:21.959 INFO:teuthology.orchestra.run.smithi142.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:06:21.959 INFO:teuthology.orchestra.run.smithi142.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:06:21.959 INFO:teuthology.orchestra.run.smithi142.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T12:06:21.960 INFO:teuthology.orchestra.run.smithi142.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T12:06:21.960 INFO:teuthology.orchestra.run.smithi142.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:06:21.997 INFO:teuthology.orchestra.run.smithi142.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T12:06:21.999 DEBUG:teuthology.orchestra.run.smithi142:> dpkg -s linux-image-generic 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Package: linux-image-generic 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Status: install ok installed 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Priority: optional 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Section: kernel 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Installed-Size: 21 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: amd64 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Source: linux-meta 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Version: 5.15.0.119.119 2024-09-12T12:06:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T12:06:22.013 INFO:teuthology.orchestra.run.smithi142.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T12:06:22.013 INFO:teuthology.orchestra.run.smithi142.stdout:Recommends: thermald 2024-09-12T12:06:22.013 INFO:teuthology.orchestra.run.smithi142.stdout:Description: Generic Linux kernel image 2024-09-12T12:06:22.013 INFO:teuthology.orchestra.run.smithi142.stdout: This package will always depend on the latest generic kernel image 2024-09-12T12:06:22.014 INFO:teuthology.orchestra.run.smithi142.stdout: available. 2024-09-12T12:06:22.014 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-09-12T12:06:22.059 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.RNQSiaRjOb 2024-09-12T12:06:22.059 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub/grub.cfg /tmp/tmp.RNQSiaRjOb 2024-09-12T12:06:22.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.RNQSiaRjOb 2024-09-12T12:06:22.230 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.RNQSiaRjOb is 10KB 2024-09-12T12:06:22.242 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.RNQSiaRjOb 2024-09-12T12:06:22.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-12T12:06:22.304 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:06:22.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-12T12:06:22.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-12T12:06:22.373 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-12T12:06:22.373 DEBUG:teuthology.orchestra.run.smithi142:> sudo update-grub 2024-09-12T12:06:23.280 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-12T12:06:23.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-12T12:06:23.354 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-12T12:06:23.544 INFO:teuthology.orchestra.run.smithi142.stderr:Sourcing file `/etc/default/grub' 2024-09-12T12:06:23.545 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2024-09-12T12:06:23.545 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2024-09-12T12:06:23.546 INFO:teuthology.orchestra.run.smithi142.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T12:06:23.549 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-09-12T12:06:23.715 INFO:teuthology.orchestra.run.smithi003.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-12T12:06:23.715 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:06:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:06:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:06:23.715 INFO:teuthology.orchestra.run.smithi003.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:06:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T12:06:23.716 INFO:teuthology.orchestra.run.smithi003.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T12:06:23.716 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:06:23.739 INFO:teuthology.orchestra.run.smithi142.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T12:06:23.745 INFO:teuthology.orchestra.run.smithi142.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:23.758 INFO:teuthology.orchestra.run.smithi003.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-12T12:06:23.760 DEBUG:teuthology.orchestra.run.smithi003:> dpkg -s linux-image-generic 2024-09-12T12:06:23.821 INFO:teuthology.orchestra.run.smithi003.stdout:Package: linux-image-generic 2024-09-12T12:06:23.821 INFO:teuthology.orchestra.run.smithi003.stdout:Status: install ok installed 2024-09-12T12:06:23.821 INFO:teuthology.orchestra.run.smithi003.stdout:Priority: optional 2024-09-12T12:06:23.821 INFO:teuthology.orchestra.run.smithi003.stdout:Section: kernel 2024-09-12T12:06:23.821 INFO:teuthology.orchestra.run.smithi003.stdout:Installed-Size: 21 2024-09-12T12:06:23.821 INFO:teuthology.orchestra.run.smithi003.stdout:Maintainer: Ubuntu Kernel Team 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: amd64 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Source: linux-meta 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Version: 5.15.0.119.119 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Recommends: thermald 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout:Description: Generic Linux kernel image 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: This package will always depend on the latest generic kernel image 2024-09-12T12:06:23.822 INFO:teuthology.orchestra.run.smithi003.stdout: available. 2024-09-12T12:06:23.823 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-12T12:06:23.869 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.i1mUpRj2QN 2024-09-12T12:06:23.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub/grub.cfg /tmp/tmp.i1mUpRj2QN 2024-09-12T12:06:23.942 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.i1mUpRj2QN 2024-09-12T12:06:23.979 INFO:teuthology.orchestra.run.smithi142.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T12:06:23.985 INFO:teuthology.orchestra.run.smithi142.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T12:06:24.028 INFO:teuthology.orchestra.run.smithi142.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T12:06:24.033 INFO:teuthology.orchestra.run.smithi142.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T12:06:24.054 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.i1mUpRj2QN is 10KB 2024-09-12T12:06:24.066 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.i1mUpRj2QN 2024-09-12T12:06:24.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-12T12:06:24.128 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:06:24.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-12T12:06:24.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-12T12:06:24.174 INFO:teuthology.orchestra.run.smithi142.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T12:06:24.174 INFO:teuthology.orchestra.run.smithi142.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T12:06:24.174 INFO:teuthology.orchestra.run.smithi142.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T12:06:24.186 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-09-12T12:06:24.202 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-09-12T12:06:24.203 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-12T12:06:24.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo update-grub 2024-09-12T12:06:25.408 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub' 2024-09-12T12:06:25.409 INFO:teuthology.orchestra.run.smithi003.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-12T12:06:25.411 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-12T12:06:25.593 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-12T12:06:25.599 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-12T12:06:25.856 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-12T12:06:25.861 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-12T12:06:25.903 INFO:teuthology.orchestra.run.smithi003.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-12T12:06:25.909 INFO:teuthology.orchestra.run.smithi003.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-12T12:06:26.034 INFO:teuthology.orchestra.run.smithi003.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-12T12:06:26.034 INFO:teuthology.orchestra.run.smithi003.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-12T12:06:26.034 INFO:teuthology.orchestra.run.smithi003.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-12T12:06:26.057 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-12T12:06:26.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-09-12T12:06:54.236 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-12T12:06:54.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T12:06:54.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:06:56.069 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-12T12:06:56.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T12:06:56.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:07:12.701 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-09-12T12:07:21.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T12:07:21.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:07:24.797 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-09-12T12:07:36.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T12:07:36.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:07:55.197 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-09-12T12:07:56.081 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T12:08:05.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T12:08:05.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:08:06.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-09-12T12:08:10.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T12:08:10.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:08:10.599 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-09-12T12:08:11.730 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T12:08:11.730 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-119-generic"... 2024-09-12T12:08:11.730 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-09-12T12:08:11.775 INFO:teuthology.orchestra.run.smithi142.stdout:5.15.0-119-generic 2024-09-12T12:08:11.776 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-12T12:08:11.776 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T12:08:11.776 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-12T12:08:12.777 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-12T12:08:12.777 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T12:08:13.027 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-09-12T12:08:13.057 DEBUG:teuthology.parallel:result is None 2024-09-12T12:08:18.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T12:08:18.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:08:18.304 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-12T12:08:18.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T12:08:18.891 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-119-generic"... 2024-09-12T12:08:18.891 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-12T12:08:18.938 INFO:teuthology.orchestra.run.smithi003.stdout:5.15.0-119-generic 2024-09-12T12:08:18.939 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-12T12:08:18.939 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T12:08:18.939 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-12T12:08:19.940 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-12T12:08:19.940 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T12:08:20.019 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2024-09-12T12:08:20.052 DEBUG:teuthology.parallel:result is None 2024-09-12T12:08:20.052 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T12:08:20.062 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T12:08:20.062 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T12:08:20.065 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T12:08:20.071 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T12:08:20.090 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T12:08:20.195 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T12:08:20.196 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T12:08:20.198 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T12:08:20.220 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T12:08:20.239 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T12:08:20.240 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T12:08:20.255 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T12:08:20.271 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:08:20.279 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:08:20.282 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:08:20.292 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:08:20.293 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T12:08:20.352 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T12:08:20.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T12:08:20.354 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T12:08:20.368 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T12:08:20.401 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T12:08:20.401 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T12:08:20.412 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T12:08:20.417 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T12:08:20.463 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T12:08:20.511 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:08:20.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T12:08:20.576 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T12:08:20.581 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T12:08:20.627 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:08:20.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T12:08:20.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-09-12T12:08:20.693 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-09-12T12:08:20.765 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T12:08:20.773 INFO:teuthology.task.internal:Starting timer... 2024-09-12T12:08:20.773 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T12:08:20.819 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T12:08:20.867 DEBUG:teuthology.task.selinux:Excluding smithi003: OS 'ubuntu' does not support SELinux 2024-09-12T12:08:20.867 DEBUG:teuthology.task.selinux:Excluding smithi142: OS 'ubuntu' does not support SELinux 2024-09-12T12:08:20.867 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T12:08:20.867 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-12T12:08:20.867 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T12:08:20.867 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T12:08:20.917 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T12:08:20.920 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T12:08:21.065 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T12:08:21.076 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T12:08:21.077 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T12:16:00.672 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-09-12T12:16:00.674 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T12:16:00.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:16:00.768 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-12T12:16:00.845 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T12:16:00.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T12:16:00.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:16:00.936 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-09-12T12:16:01.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-09-12T12:16:01.013 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T12:16:01.024 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T12:16:01.024 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T12:16:01.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:16:01.028 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T12:16:01.028 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:16:01.056 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-12T12:16:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Command line: ntpd -gq 2024-09-12T12:16:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: ---------------------------------------------------- 2024-09-12T12:16:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: ntp-4 is maintained by Network Time Foundation, 2024-09-12T12:16:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-12T12:16:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: corporation. Support and training for ntp-4 are 2024-09-12T12:16:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: available at https://www.nwtime.org/support 2024-09-12T12:16:01.057 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: ---------------------------------------------------- 2024-09-12T12:16:01.058 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: proto: precision = 0.065 usec (-24) 2024-09-12T12:16:01.058 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: basedate set to 2022-02-04 2024-09-12T12:16:01.058 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: gps base set to 2022-02-06 (week 2196) 2024-09-12T12:16:01.058 INFO:teuthology.orchestra.run.smithi003.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T12:16:01.059 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T12:16:01.059 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: restrict ::: KOD does nothing without LIMITED. 2024-09-12T12:16:01.059 INFO:teuthology.orchestra.run.smithi003.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-12T12:16:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Listen and drop on 0 v6wildcard [::]:123 2024-09-12T12:16:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-12T12:16:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Listen normally on 2 lo 127.0.0.1:123 2024-09-12T12:16:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Listen normally on 3 ens1f0 172.21.15.3:123 2024-09-12T12:16:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Listen normally on 4 lo [::1]:123 2024-09-12T12:16:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15dc%4]:123 2024-09-12T12:16:01.060 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:01 ntpd[18367]: Listening on routing socket on fd #22 for interface updates 2024-09-12T12:16:01.088 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-12T12:16:01.088 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Command line: ntpd -gq 2024-09-12T12:16:01.088 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: ---------------------------------------------------- 2024-09-12T12:16:01.088 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: ntp-4 is maintained by Network Time Foundation, 2024-09-12T12:16:01.088 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-12T12:16:01.088 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: corporation. Support and training for ntp-4 are 2024-09-12T12:16:01.088 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: available at https://www.nwtime.org/support 2024-09-12T12:16:01.089 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: ---------------------------------------------------- 2024-09-12T12:16:01.089 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: proto: precision = 0.045 usec (-24) 2024-09-12T12:16:01.089 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: basedate set to 2022-02-04 2024-09-12T12:16:01.089 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: gps base set to 2022-02-06 (week 2196) 2024-09-12T12:16:01.089 INFO:teuthology.orchestra.run.smithi142.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T12:16:01.090 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-12T12:16:01.090 INFO:teuthology.orchestra.run.smithi142.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: restrict ::: KOD does nothing without LIMITED. 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Listen and drop on 0 v6wildcard [::]:123 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Listen normally on 2 lo 127.0.0.1:123 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Listen normally on 3 enp3s0f1 172.21.15.142:123 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Listen normally on 4 lo [::1]:123 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e0d%5]:123 2024-09-12T12:16:01.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:01 ntpd[18482]: Listening on routing socket on fd #22 for interface updates 2024-09-12T12:16:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:02 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:02 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:02 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:02 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:02 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:02 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:03.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:03 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:03.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:03 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:03.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:03 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:03.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:03 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:03 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:03.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:03 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:04.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:04.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:04 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:04.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:04 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:05.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:05 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:05.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:05 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:05.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:05 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:05.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:05 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:05.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:05 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:05.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:05 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:06.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:06 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:06.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:06.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:06 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:07.057 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:07 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:07 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:07 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:07 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:07 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:07.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:07 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:08.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.058 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:08.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:08.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:08.059 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:08.059 INFO:teuthology.orchestra.run.smithi003.stdout:12 Sep 12:16:08 ntpd[18367]: ntpd: time slew +0.001582 s 2024-09-12T12:16:08.059 INFO:teuthology.orchestra.run.smithi003.stdout:ntpd: time slew +0.001582s 2024-09-12T12:16:08.060 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-12T12:16:08.060 INFO:teuthology.orchestra.run.smithi003.stderr:12 Sep 12:16:08 ntpd[18367]: can't open /var/log/ntpstats/loopstats.20240912: Permission denied 2024-09-12T12:16:08.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:08.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:08.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:08.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:08 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:08.106 INFO:teuthology.orchestra.run.smithi003.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T12:16:08.106 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================== 2024-09-12T12:16:08.106 INFO:teuthology.orchestra.run.smithi003.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:08.107 INFO:teuthology.orchestra.run.smithi003.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:08.107 INFO:teuthology.orchestra.run.smithi003.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:08.107 INFO:teuthology.orchestra.run.smithi003.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:10.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:10.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:10.089 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240912: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-12T12:16:10.090 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240912: Permission denied 2024-09-12T12:16:10.091 INFO:teuthology.orchestra.run.smithi142.stdout:12 Sep 12:16:10 ntpd[18482]: ntpd: time slew -0.003236 s 2024-09-12T12:16:10.091 INFO:teuthology.orchestra.run.smithi142.stdout:ntpd: time slew -0.003236s 2024-09-12T12:16:10.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-12T12:16:10.091 INFO:teuthology.orchestra.run.smithi142.stderr:12 Sep 12:16:10 ntpd[18482]: can't open /var/log/ntpstats/loopstats.20240912: Permission denied 2024-09-12T12:16:10.257 INFO:teuthology.orchestra.run.smithi142.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T12:16:10.258 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================== 2024-09-12T12:16:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:10.258 INFO:teuthology.orchestra.run.smithi142.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-12T12:16:10.259 INFO:teuthology.run_tasks:Running task install... 2024-09-12T12:16:10.266 DEBUG:teuthology.task.install:project ceph 2024-09-12T12:16:10.266 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T12:16:10.266 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T12:16:10.267 INFO:teuthology.task.install:Using flavor: default 2024-09-12T12:16:10.272 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T12:16:10.273 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T12:16:10.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-key list | grep Ceph 2024-09-12T12:16:10.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-key list | grep Ceph 2024-09-12T12:16:10.341 INFO:teuthology.orchestra.run.smithi003.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-12T12:16:10.364 INFO:teuthology.orchestra.run.smithi142.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-12T12:16:10.373 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-12T12:16:10.373 INFO:teuthology.orchestra.run.smithi003.stdout:uid [ unknown] Ceph.com (release key) 2024-09-12T12:16:10.374 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-12T12:16:10.374 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-12T12:16:10.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:16:10.392 INFO:teuthology.orchestra.run.smithi142.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-12T12:16:10.393 INFO:teuthology.orchestra.run.smithi142.stdout:uid [ unknown] Ceph.com (release key) 2024-09-12T12:16:10.393 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-12T12:16:10.393 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-12T12:16:10.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:16:10.524 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default/ 2024-09-12T12:16:10.524 INFO:teuthology.task.install.deb:Package version is 19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:16:10.554 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default/ 2024-09-12T12:16:10.554 INFO:teuthology.task.install.deb:Package version is 19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:16:10.658 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:16:10.659 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-12T12:16:10.673 DEBUG:teuthology.orchestra.run.smithi003:> sudo apt-get update 2024-09-12T12:16:10.676 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:16:10.676 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-12T12:16:10.690 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get update 2024-09-12T12:16:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-12T12:16:10.847 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T12:16:10.860 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-12T12:16:10.863 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T12:16:10.881 INFO:teuthology.orchestra.run.smithi003.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T12:16:10.900 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy InRelease 2024-09-12T12:16:10.913 INFO:teuthology.orchestra.run.smithi142.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy InRelease 2024-09-12T12:16:10.921 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-12T12:16:10.934 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-12T12:16:10.946 INFO:teuthology.orchestra.run.smithi003.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-12T12:16:10.956 INFO:teuthology.orchestra.run.smithi142.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-12T12:16:10.967 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,404 B] 2024-09-12T12:16:10.977 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,404 B] 2024-09-12T12:16:10.983 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-12T12:16:10.994 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-12T12:16:11.004 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-12T12:16:11.060 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-12T12:16:11.142 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-12T12:16:11.236 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 29.0 kB in 0s (78.3 kB/s) 2024-09-12T12:16:11.284 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 29.0 kB in 0s (70.0 kB/s) 2024-09-12T12:16:12.295 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-12T12:16:12.302 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-09-12T12:16:12.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-298-gb53b6d3e-1jammy cephadm=19.1.1-298-gb53b6d3e-1jammy ceph-mds=19.1.1-298-gb53b6d3e-1jammy ceph-mgr=19.1.1-298-gb53b6d3e-1jammy ceph-common=19.1.1-298-gb53b6d3e-1jammy ceph-fuse=19.1.1-298-gb53b6d3e-1jammy ceph-test=19.1.1-298-gb53b6d3e-1jammy ceph-volume=19.1.1-298-gb53b6d3e-1jammy radosgw=19.1.1-298-gb53b6d3e-1jammy python3-rados=19.1.1-298-gb53b6d3e-1jammy python3-rgw=19.1.1-298-gb53b6d3e-1jammy python3-cephfs=19.1.1-298-gb53b6d3e-1jammy python3-rbd=19.1.1-298-gb53b6d3e-1jammy libcephfs2=19.1.1-298-gb53b6d3e-1jammy libcephfs-dev=19.1.1-298-gb53b6d3e-1jammy librados2=19.1.1-298-gb53b6d3e-1jammy librbd1=19.1.1-298-gb53b6d3e-1jammy rbd-fuse=19.1.1-298-gb53b6d3e-1jammy python3-cephfs=19.1.1-298-gb53b6d3e-1jammy cephfs-shell=19.1.1-298-gb53b6d3e-1jammy cephfs-top=19.1.1-298-gb53b6d3e-1jammy cephfs-mirror=19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:16:12.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-298-gb53b6d3e-1jammy cephadm=19.1.1-298-gb53b6d3e-1jammy ceph-mds=19.1.1-298-gb53b6d3e-1jammy ceph-mgr=19.1.1-298-gb53b6d3e-1jammy ceph-common=19.1.1-298-gb53b6d3e-1jammy ceph-fuse=19.1.1-298-gb53b6d3e-1jammy ceph-test=19.1.1-298-gb53b6d3e-1jammy ceph-volume=19.1.1-298-gb53b6d3e-1jammy radosgw=19.1.1-298-gb53b6d3e-1jammy python3-rados=19.1.1-298-gb53b6d3e-1jammy python3-rgw=19.1.1-298-gb53b6d3e-1jammy python3-cephfs=19.1.1-298-gb53b6d3e-1jammy python3-rbd=19.1.1-298-gb53b6d3e-1jammy libcephfs2=19.1.1-298-gb53b6d3e-1jammy libcephfs-dev=19.1.1-298-gb53b6d3e-1jammy librados2=19.1.1-298-gb53b6d3e-1jammy librbd1=19.1.1-298-gb53b6d3e-1jammy rbd-fuse=19.1.1-298-gb53b6d3e-1jammy python3-cephfs=19.1.1-298-gb53b6d3e-1jammy cephfs-shell=19.1.1-298-gb53b6d3e-1jammy cephfs-top=19.1.1-298-gb53b6d3e-1jammy cephfs-mirror=19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:16:12.377 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-12T12:16:12.382 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-09-12T12:16:12.590 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2024-09-12T12:16:12.590 INFO:teuthology.orchestra.run.smithi142.stdout:Building dependency tree... 2024-09-12T12:16:12.591 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2024-09-12T12:16:12.591 INFO:teuthology.orchestra.run.smithi142.stdout:Reading state information... 2024-09-12T12:16:12.782 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:16:12.782 INFO:teuthology.orchestra.run.smithi142.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:16:12.783 INFO:teuthology.orchestra.run.smithi142.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:16:12.783 INFO:teuthology.orchestra.run.smithi142.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:16:12.783 INFO:teuthology.orchestra.run.smithi142.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T12:16:12.783 INFO:teuthology.orchestra.run.smithi142.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T12:16:12.783 INFO:teuthology.orchestra.run.smithi142.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:16:12.783 INFO:teuthology.orchestra.run.smithi142.stdout:The following additional packages will be installed: 2024-09-12T12:16:12.783 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-12T12:16:12.785 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-12T12:16:12.786 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout:Suggested packages: 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-12T12:16:12.787 INFO:teuthology.orchestra.run.smithi142.stdout:Recommended packages: 2024-09-12T12:16:12.788 INFO:teuthology.orchestra.run.smithi142.stdout: btrfs-tools 2024-09-12T12:16:12.809 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:16:12.809 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:16:12.809 INFO:teuthology.orchestra.run.smithi003.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:16:12.809 INFO:teuthology.orchestra.run.smithi003.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:16:12.810 INFO:teuthology.orchestra.run.smithi003.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T12:16:12.810 INFO:teuthology.orchestra.run.smithi003.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T12:16:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:16:12.810 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2024-09-12T12:16:12.810 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-12T12:16:12.811 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-12T12:16:12.811 INFO:teuthology.orchestra.run.smithi003.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-12T12:16:12.811 INFO:teuthology.orchestra.run.smithi003.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-12T12:16:12.812 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-12T12:16:12.812 INFO:teuthology.orchestra.run.smithi003.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-12T12:16:12.813 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-12T12:16:12.814 INFO:teuthology.orchestra.run.smithi003.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-12T12:16:12.815 INFO:teuthology.orchestra.run.smithi003.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-12T12:16:12.815 INFO:teuthology.orchestra.run.smithi003.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-12T12:16:12.815 INFO:teuthology.orchestra.run.smithi003.stdout:Recommended packages: 2024-09-12T12:16:12.815 INFO:teuthology.orchestra.run.smithi003.stdout: btrfs-tools 2024-09-12T12:16:12.900 INFO:teuthology.orchestra.run.smithi142.stdout:The following NEW packages will be installed: 2024-09-12T12:16:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-12T12:16:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-12T12:16:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-12T12:16:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-12T12:16:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-12T12:16:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-12T12:16:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-12T12:16:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-12T12:16:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-12T12:16:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-12T12:16:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-12T12:16:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-12T12:16:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-12T12:16:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: socat unzip xmlstarlet zip 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-12T12:16:12.930 INFO:teuthology.orchestra.run.smithi003.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-12T12:16:12.931 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-12T12:16:12.932 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-12T12:16:12.933 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-12T12:16:12.933 INFO:teuthology.orchestra.run.smithi003.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-12T12:16:12.933 INFO:teuthology.orchestra.run.smithi003.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-12T12:16:12.933 INFO:teuthology.orchestra.run.smithi003.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-12T12:16:12.933 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-12T12:16:12.933 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-12T12:16:12.933 INFO:teuthology.orchestra.run.smithi003.stdout: socat unzip xmlstarlet zip 2024-09-12T12:16:12.992 INFO:teuthology.orchestra.run.smithi003.stdout:0 upgraded, 124 newly installed, 0 to remove and 331 not upgraded. 2024-09-12T12:16:12.992 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 181 MB of archives. 2024-09-12T12:16:12.992 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-12T12:16:12.992 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-12T12:16:13.010 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-12T12:16:13.019 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-12T12:16:13.048 INFO:teuthology.orchestra.run.smithi142.stdout:0 upgraded, 124 newly installed, 0 to remove and 331 not upgraded. 2024-09-12T12:16:13.048 INFO:teuthology.orchestra.run.smithi142.stdout:Need to get 181 MB of archives. 2024-09-12T12:16:13.049 INFO:teuthology.orchestra.run.smithi142.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-12T12:16:13.049 INFO:teuthology.orchestra.run.smithi142.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-298-gb53b6d3e-1jammy [3,588 kB] 2024-09-12T12:16:13.060 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-12T12:16:13.075 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-298-gb53b6d3e-1jammy [3,588 kB] 2024-09-12T12:16:13.081 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-12T12:16:13.084 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-12T12:16:13.093 INFO:teuthology.orchestra.run.smithi142.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-12T12:16:13.095 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-12T12:16:13.150 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-12T12:16:13.155 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-12T12:16:13.167 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-12T12:16:13.171 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-12T12:16:13.171 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-12T12:16:13.175 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-12T12:16:13.180 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-12T12:16:13.182 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-12T12:16:13.184 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-12T12:16:13.185 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-12T12:16:13.186 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-12T12:16:13.189 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-12T12:16:13.191 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-12T12:16:13.206 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-12T12:16:13.206 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-12T12:16:13.207 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-12T12:16:13.208 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-12T12:16:13.208 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-12T12:16:13.209 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-12T12:16:13.209 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-12T12:16:13.209 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-12T12:16:13.221 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-12T12:16:13.223 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-12T12:16:13.224 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-12T12:16:13.239 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-298-gb53b6d3e-1jammy [3,243 kB] 2024-09-12T12:16:13.240 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-12T12:16:13.241 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-12T12:16:13.241 INFO:teuthology.orchestra.run.smithi003.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-12T12:16:13.242 INFO:teuthology.orchestra.run.smithi003.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-12T12:16:13.243 INFO:teuthology.orchestra.run.smithi003.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-12T12:16:13.243 INFO:teuthology.orchestra.run.smithi003.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-12T12:16:13.244 INFO:teuthology.orchestra.run.smithi003.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-12T12:16:13.245 INFO:teuthology.orchestra.run.smithi003.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-12T12:16:13.257 INFO:teuthology.orchestra.run.smithi003.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-12T12:16:13.258 INFO:teuthology.orchestra.run.smithi003.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-12T12:16:13.274 INFO:teuthology.orchestra.run.smithi003.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-12T12:16:13.275 INFO:teuthology.orchestra.run.smithi003.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-12T12:16:13.275 INFO:teuthology.orchestra.run.smithi003.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-12T12:16:13.276 INFO:teuthology.orchestra.run.smithi003.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-12T12:16:13.277 INFO:teuthology.orchestra.run.smithi003.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-12T12:16:13.279 INFO:teuthology.orchestra.run.smithi003.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-12T12:16:13.280 INFO:teuthology.orchestra.run.smithi003.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-12T12:16:13.281 INFO:teuthology.orchestra.run.smithi003.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-12T12:16:13.291 INFO:teuthology.orchestra.run.smithi003.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-298-gb53b6d3e-1jammy [3,243 kB] 2024-09-12T12:16:13.291 INFO:teuthology.orchestra.run.smithi003.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-12T12:16:13.294 INFO:teuthology.orchestra.run.smithi003.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-12T12:16:13.308 INFO:teuthology.orchestra.run.smithi003.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-12T12:16:13.309 INFO:teuthology.orchestra.run.smithi003.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-12T12:16:13.313 INFO:teuthology.orchestra.run.smithi003.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-12T12:16:13.317 INFO:teuthology.orchestra.run.smithi142.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-298-gb53b6d3e-1jammy [719 kB] 2024-09-12T12:16:13.331 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-298-gb53b6d3e-1jammy [356 kB] 2024-09-12T12:16:13.347 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-298-gb53b6d3e-1jammy [32.8 kB] 2024-09-12T12:16:13.347 INFO:teuthology.orchestra.run.smithi142.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-298-gb53b6d3e-1jammy [180 kB] 2024-09-12T12:16:13.356 INFO:teuthology.orchestra.run.smithi142.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-298-gb53b6d3e-1jammy [67.6 kB] 2024-09-12T12:16:13.358 INFO:teuthology.orchestra.run.smithi142.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-298-gb53b6d3e-1jammy [333 kB] 2024-09-12T12:16:13.364 INFO:teuthology.orchestra.run.smithi142.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-298-gb53b6d3e-1jammy [6,875 kB] 2024-09-12T12:16:13.369 INFO:teuthology.orchestra.run.smithi003.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-12T12:16:13.370 INFO:teuthology.orchestra.run.smithi003.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-12T12:16:13.371 INFO:teuthology.orchestra.run.smithi003.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-12T12:16:13.371 INFO:teuthology.orchestra.run.smithi003.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-12T12:16:13.372 INFO:teuthology.orchestra.run.smithi003.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-12T12:16:13.390 INFO:teuthology.orchestra.run.smithi003.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-12T12:16:13.392 INFO:teuthology.orchestra.run.smithi003.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-12T12:16:13.392 INFO:teuthology.orchestra.run.smithi003.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-12T12:16:13.396 INFO:teuthology.orchestra.run.smithi003.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-298-gb53b6d3e-1jammy [719 kB] 2024-09-12T12:16:13.408 INFO:teuthology.orchestra.run.smithi003.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-12T12:16:13.408 INFO:teuthology.orchestra.run.smithi003.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-12T12:16:13.411 INFO:teuthology.orchestra.run.smithi003.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-12T12:16:13.411 INFO:teuthology.orchestra.run.smithi003.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-12T12:16:13.411 INFO:teuthology.orchestra.run.smithi003.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-12T12:16:13.412 INFO:teuthology.orchestra.run.smithi003.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-12T12:16:13.412 INFO:teuthology.orchestra.run.smithi003.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-12T12:16:13.417 INFO:teuthology.orchestra.run.smithi142.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-12T12:16:13.427 INFO:teuthology.orchestra.run.smithi003.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-12T12:16:13.431 INFO:teuthology.orchestra.run.smithi003.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-12T12:16:13.431 INFO:teuthology.orchestra.run.smithi003.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-298-gb53b6d3e-1jammy [356 kB] 2024-09-12T12:16:13.432 INFO:teuthology.orchestra.run.smithi003.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-12T12:16:13.433 INFO:teuthology.orchestra.run.smithi003.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-12T12:16:13.434 INFO:teuthology.orchestra.run.smithi003.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-12T12:16:13.441 INFO:teuthology.orchestra.run.smithi003.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-298-gb53b6d3e-1jammy [32.8 kB] 2024-09-12T12:16:13.441 INFO:teuthology.orchestra.run.smithi003.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-298-gb53b6d3e-1jammy [180 kB] 2024-09-12T12:16:13.443 INFO:teuthology.orchestra.run.smithi003.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-298-gb53b6d3e-1jammy [67.6 kB] 2024-09-12T12:16:13.444 INFO:teuthology.orchestra.run.smithi003.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-12T12:16:13.445 INFO:teuthology.orchestra.run.smithi003.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-298-gb53b6d3e-1jammy [333 kB] 2024-09-12T12:16:13.447 INFO:teuthology.orchestra.run.smithi003.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-12T12:16:13.447 INFO:teuthology.orchestra.run.smithi003.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-12T12:16:13.448 INFO:teuthology.orchestra.run.smithi003.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-12T12:16:13.448 INFO:teuthology.orchestra.run.smithi003.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-12T12:16:13.463 INFO:teuthology.orchestra.run.smithi003.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-12T12:16:13.463 INFO:teuthology.orchestra.run.smithi003.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-12T12:16:13.465 INFO:teuthology.orchestra.run.smithi003.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-12T12:16:13.465 INFO:teuthology.orchestra.run.smithi003.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-12T12:16:13.466 INFO:teuthology.orchestra.run.smithi003.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-12T12:16:13.474 INFO:teuthology.orchestra.run.smithi003.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-298-gb53b6d3e-1jammy [6,875 kB] 2024-09-12T12:16:13.479 INFO:teuthology.orchestra.run.smithi003.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-12T12:16:13.480 INFO:teuthology.orchestra.run.smithi003.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-12T12:16:13.481 INFO:teuthology.orchestra.run.smithi003.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-12T12:16:13.487 INFO:teuthology.orchestra.run.smithi003.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-12T12:16:13.487 INFO:teuthology.orchestra.run.smithi003.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-12T12:16:13.496 INFO:teuthology.orchestra.run.smithi003.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-12T12:16:13.498 INFO:teuthology.orchestra.run.smithi003.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-12T12:16:13.499 INFO:teuthology.orchestra.run.smithi003.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-12T12:16:13.509 INFO:teuthology.orchestra.run.smithi142.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-12T12:16:13.524 INFO:teuthology.orchestra.run.smithi142.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-12T12:16:13.565 INFO:teuthology.orchestra.run.smithi142.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-298-gb53b6d3e-1jammy [112 kB] 2024-09-12T12:16:13.567 INFO:teuthology.orchestra.run.smithi142.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-298-gb53b6d3e-1jammy [472 kB] 2024-09-12T12:16:13.571 INFO:teuthology.orchestra.run.smithi142.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-12T12:16:13.575 INFO:teuthology.orchestra.run.smithi142.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-298-gb53b6d3e-1jammy [25.9 MB] 2024-09-12T12:16:13.715 INFO:teuthology.orchestra.run.smithi003.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-298-gb53b6d3e-1jammy [112 kB] 2024-09-12T12:16:13.717 INFO:teuthology.orchestra.run.smithi003.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-298-gb53b6d3e-1jammy [472 kB] 2024-09-12T12:16:13.746 INFO:teuthology.orchestra.run.smithi142.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-12T12:16:13.753 INFO:teuthology.orchestra.run.smithi003.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-298-gb53b6d3e-1jammy [25.9 MB] 2024-09-12T12:16:13.754 INFO:teuthology.orchestra.run.smithi142.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-12T12:16:13.782 INFO:teuthology.orchestra.run.smithi142.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-12T12:16:13.791 INFO:teuthology.orchestra.run.smithi142.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-12T12:16:13.792 INFO:teuthology.orchestra.run.smithi142.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-12T12:16:13.809 INFO:teuthology.orchestra.run.smithi142.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-12T12:16:13.813 INFO:teuthology.orchestra.run.smithi142.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-12T12:16:13.818 INFO:teuthology.orchestra.run.smithi142.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-12T12:16:13.820 INFO:teuthology.orchestra.run.smithi142.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-12T12:16:13.829 INFO:teuthology.orchestra.run.smithi142.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-12T12:16:13.903 INFO:teuthology.orchestra.run.smithi142.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-12T12:16:13.908 INFO:teuthology.orchestra.run.smithi142.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-12T12:16:13.912 INFO:teuthology.orchestra.run.smithi142.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-12T12:16:13.915 INFO:teuthology.orchestra.run.smithi142.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-12T12:16:13.917 INFO:teuthology.orchestra.run.smithi142.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-12T12:16:13.919 INFO:teuthology.orchestra.run.smithi142.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-12T12:16:13.922 INFO:teuthology.orchestra.run.smithi142.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-12T12:16:13.927 INFO:teuthology.orchestra.run.smithi142.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-12T12:16:13.929 INFO:teuthology.orchestra.run.smithi142.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-12T12:16:13.992 INFO:teuthology.orchestra.run.smithi142.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-12T12:16:14.014 INFO:teuthology.orchestra.run.smithi142.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-12T12:16:14.014 INFO:teuthology.orchestra.run.smithi142.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-12T12:16:14.020 INFO:teuthology.orchestra.run.smithi142.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-12T12:16:14.021 INFO:teuthology.orchestra.run.smithi142.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-12T12:16:14.022 INFO:teuthology.orchestra.run.smithi142.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-12T12:16:14.023 INFO:teuthology.orchestra.run.smithi142.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-12T12:16:14.025 INFO:teuthology.orchestra.run.smithi142.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-12T12:16:14.025 INFO:teuthology.orchestra.run.smithi142.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-12T12:16:14.073 INFO:teuthology.orchestra.run.smithi142.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-12T12:16:14.108 INFO:teuthology.orchestra.run.smithi142.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-12T12:16:14.110 INFO:teuthology.orchestra.run.smithi142.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-12T12:16:14.111 INFO:teuthology.orchestra.run.smithi142.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-12T12:16:14.122 INFO:teuthology.orchestra.run.smithi142.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-12T12:16:14.122 INFO:teuthology.orchestra.run.smithi142.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-12T12:16:14.155 INFO:teuthology.orchestra.run.smithi142.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-12T12:16:14.156 INFO:teuthology.orchestra.run.smithi142.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-12T12:16:14.159 INFO:teuthology.orchestra.run.smithi142.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-12T12:16:14.163 INFO:teuthology.orchestra.run.smithi142.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-12T12:16:14.236 INFO:teuthology.orchestra.run.smithi142.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-12T12:16:14.239 INFO:teuthology.orchestra.run.smithi142.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-12T12:16:14.241 INFO:teuthology.orchestra.run.smithi142.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-12T12:16:14.245 INFO:teuthology.orchestra.run.smithi142.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-12T12:16:14.246 INFO:teuthology.orchestra.run.smithi142.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-12T12:16:14.318 INFO:teuthology.orchestra.run.smithi142.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-12T12:16:14.322 INFO:teuthology.orchestra.run.smithi142.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-12T12:16:14.435 INFO:teuthology.orchestra.run.smithi142.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-298-gb53b6d3e-1jammy [5,145 kB] 2024-09-12T12:16:14.489 INFO:teuthology.orchestra.run.smithi142.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-12T12:16:14.491 INFO:teuthology.orchestra.run.smithi142.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-12T12:16:14.494 INFO:teuthology.orchestra.run.smithi142.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-12T12:16:14.494 INFO:teuthology.orchestra.run.smithi142.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-12T12:16:14.497 INFO:teuthology.orchestra.run.smithi142.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-12T12:16:14.532 INFO:teuthology.orchestra.run.smithi142.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-12T12:16:14.534 INFO:teuthology.orchestra.run.smithi142.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-12T12:16:14.535 INFO:teuthology.orchestra.run.smithi142.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-12T12:16:14.553 INFO:teuthology.orchestra.run.smithi142.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-298-gb53b6d3e-1jammy [245 kB] 2024-09-12T12:16:14.558 INFO:teuthology.orchestra.run.smithi142.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-298-gb53b6d3e-1jammy [124 kB] 2024-09-12T12:16:14.561 INFO:teuthology.orchestra.run.smithi142.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-298-gb53b6d3e-1jammy [1,555 kB] 2024-09-12T12:16:14.594 INFO:teuthology.orchestra.run.smithi142.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-12T12:16:14.595 INFO:teuthology.orchestra.run.smithi142.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-12T12:16:14.610 INFO:teuthology.orchestra.run.smithi142.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-298-gb53b6d3e-1jammy [6,143 kB] 2024-09-12T12:16:14.656 INFO:teuthology.orchestra.run.smithi142.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-12T12:16:14.657 INFO:teuthology.orchestra.run.smithi142.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-12T12:16:14.658 INFO:teuthology.orchestra.run.smithi142.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-12T12:16:14.659 INFO:teuthology.orchestra.run.smithi142.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-12T12:16:14.660 INFO:teuthology.orchestra.run.smithi142.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-12T12:16:14.661 INFO:teuthology.orchestra.run.smithi142.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-12T12:16:14.668 INFO:teuthology.orchestra.run.smithi142.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-12T12:16:14.738 INFO:teuthology.orchestra.run.smithi142.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-12T12:16:14.740 INFO:teuthology.orchestra.run.smithi003.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-298-gb53b6d3e-1jammy [5,145 kB] 2024-09-12T12:16:14.741 INFO:teuthology.orchestra.run.smithi142.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-12T12:16:14.744 INFO:teuthology.orchestra.run.smithi142.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-12T12:16:14.787 INFO:teuthology.orchestra.run.smithi142.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-298-gb53b6d3e-1jammy [22.8 MB] 2024-09-12T12:16:14.820 INFO:teuthology.orchestra.run.smithi142.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-12T12:16:14.827 INFO:teuthology.orchestra.run.smithi142.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-12T12:16:14.828 INFO:teuthology.orchestra.run.smithi142.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-12T12:16:14.830 INFO:teuthology.orchestra.run.smithi142.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-12T12:16:14.832 INFO:teuthology.orchestra.run.smithi142.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-12T12:16:14.843 INFO:teuthology.orchestra.run.smithi142.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-12T12:16:14.844 INFO:teuthology.orchestra.run.smithi003.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-298-gb53b6d3e-1jammy [245 kB] 2024-09-12T12:16:14.844 INFO:teuthology.orchestra.run.smithi142.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-12T12:16:14.850 INFO:teuthology.orchestra.run.smithi003.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-298-gb53b6d3e-1jammy [124 kB] 2024-09-12T12:16:14.853 INFO:teuthology.orchestra.run.smithi003.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-298-gb53b6d3e-1jammy [1,555 kB] 2024-09-12T12:16:14.901 INFO:teuthology.orchestra.run.smithi142.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-12T12:16:14.901 INFO:teuthology.orchestra.run.smithi142.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-12T12:16:14.903 INFO:teuthology.orchestra.run.smithi142.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-12T12:16:14.905 INFO:teuthology.orchestra.run.smithi003.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-298-gb53b6d3e-1jammy [6,143 kB] 2024-09-12T12:16:14.982 INFO:teuthology.orchestra.run.smithi142.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-12T12:16:14.987 INFO:teuthology.orchestra.run.smithi142.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-12T12:16:14.989 INFO:teuthology.orchestra.run.smithi142.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-12T12:16:15.006 INFO:teuthology.orchestra.run.smithi142.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-12T12:16:15.007 INFO:teuthology.orchestra.run.smithi142.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-12T12:16:15.008 INFO:teuthology.orchestra.run.smithi142.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-12T12:16:15.013 INFO:teuthology.orchestra.run.smithi142.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-12T12:16:15.064 INFO:teuthology.orchestra.run.smithi142.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-12T12:16:15.095 INFO:teuthology.orchestra.run.smithi003.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-298-gb53b6d3e-1jammy [22.8 MB] 2024-09-12T12:16:15.608 INFO:teuthology.orchestra.run.smithi142.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-298-gb53b6d3e-1jammy [14.1 kB] 2024-09-12T12:16:15.608 INFO:teuthology.orchestra.run.smithi142.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [877 kB] 2024-09-12T12:16:15.625 INFO:teuthology.orchestra.run.smithi142.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-298-gb53b6d3e-1jammy [2,462 kB] 2024-09-12T12:16:15.666 INFO:teuthology.orchestra.run.smithi142.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-298-gb53b6d3e-1jammy [794 kB] 2024-09-12T12:16:15.680 INFO:teuthology.orchestra.run.smithi142.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-298-gb53b6d3e-1jammy [152 kB] 2024-09-12T12:16:15.702 INFO:teuthology.orchestra.run.smithi142.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-298-gb53b6d3e-1jammy [2,284 kB] 2024-09-12T12:16:15.771 INFO:teuthology.orchestra.run.smithi142.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-298-gb53b6d3e-1jammy [8,625 kB] 2024-09-12T12:16:15.924 INFO:teuthology.orchestra.run.smithi003.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-298-gb53b6d3e-1jammy [14.1 kB] 2024-09-12T12:16:15.925 INFO:teuthology.orchestra.run.smithi003.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [877 kB] 2024-09-12T12:16:15.942 INFO:teuthology.orchestra.run.smithi003.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-298-gb53b6d3e-1jammy [2,462 kB] 2024-09-12T12:16:15.985 INFO:teuthology.orchestra.run.smithi003.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-298-gb53b6d3e-1jammy [794 kB] 2024-09-12T12:16:15.995 INFO:teuthology.orchestra.run.smithi003.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-298-gb53b6d3e-1jammy [152 kB] 2024-09-12T12:16:16.014 INFO:teuthology.orchestra.run.smithi142.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-298-gb53b6d3e-1jammy [14.2 kB] 2024-09-12T12:16:16.015 INFO:teuthology.orchestra.run.smithi142.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-298-gb53b6d3e-1jammy [51.4 MB] 2024-09-12T12:16:16.018 INFO:teuthology.orchestra.run.smithi003.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-298-gb53b6d3e-1jammy [2,284 kB] 2024-09-12T12:16:16.091 INFO:teuthology.orchestra.run.smithi003.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-298-gb53b6d3e-1jammy [8,625 kB] 2024-09-12T12:16:16.367 INFO:teuthology.orchestra.run.smithi003.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-298-gb53b6d3e-1jammy [14.2 kB] 2024-09-12T12:16:16.368 INFO:teuthology.orchestra.run.smithi003.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-298-gb53b6d3e-1jammy [51.4 MB] 2024-09-12T12:16:17.876 INFO:teuthology.orchestra.run.smithi142.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-298-gb53b6d3e-1jammy [130 kB] 2024-09-12T12:16:17.878 INFO:teuthology.orchestra.run.smithi142.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-298-gb53b6d3e-1jammy [237 kB] 2024-09-12T12:16:17.883 INFO:teuthology.orchestra.run.smithi142.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-298-gb53b6d3e-1jammy [34.0 kB] 2024-09-12T12:16:17.884 INFO:teuthology.orchestra.run.smithi142.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-298-gb53b6d3e-1jammy [23.5 kB] 2024-09-12T12:16:17.885 INFO:teuthology.orchestra.run.smithi142.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-298-gb53b6d3e-1jammy [39.0 kB] 2024-09-12T12:16:17.886 INFO:teuthology.orchestra.run.smithi142.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-298-gb53b6d3e-1jammy [13.6 MB] 2024-09-12T12:16:18.176 INFO:teuthology.orchestra.run.smithi142.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [91.5 kB] 2024-09-12T12:16:18.213 INFO:teuthology.orchestra.run.smithi003.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-298-gb53b6d3e-1jammy [130 kB] 2024-09-12T12:16:18.216 INFO:teuthology.orchestra.run.smithi003.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-298-gb53b6d3e-1jammy [237 kB] 2024-09-12T12:16:18.220 INFO:teuthology.orchestra.run.smithi003.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-298-gb53b6d3e-1jammy [34.0 kB] 2024-09-12T12:16:18.221 INFO:teuthology.orchestra.run.smithi003.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-298-gb53b6d3e-1jammy [23.5 kB] 2024-09-12T12:16:18.222 INFO:teuthology.orchestra.run.smithi003.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-298-gb53b6d3e-1jammy [39.0 kB] 2024-09-12T12:16:18.223 INFO:teuthology.orchestra.run.smithi003.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-298-gb53b6d3e-1jammy [13.6 MB] 2024-09-12T12:16:18.414 INFO:teuthology.orchestra.run.smithi003.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-298-gb53b6d3e-1jammy [91.5 kB] 2024-09-12T12:16:18.811 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 181 MB in 5s (34.5 MB/s) 2024-09-12T12:16:18.951 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-12T12:16:19.048 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-12T12:16:19.054 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-12T12:16:19.063 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 181 MB in 5s (33.1 MB/s) 2024-09-12T12:16:19.099 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-12T12:16:19.206 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-12T12:16:19.300 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-12T12:16:19.306 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-12T12:16:19.346 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-12T12:16:19.402 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-12T12:16:19.428 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-12T12:16:19.468 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T12:16:19.641 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-12T12:16:19.668 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-12T12:16:19.707 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T12:16:19.780 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cmd2. 2024-09-12T12:16:19.806 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-12T12:16:19.838 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-12T12:16:19.977 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cmd2. 2024-09-12T12:16:20.004 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-12T12:16:20.035 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-12T12:16:20.132 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-12T12:16:20.159 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-12T12:16:20.241 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T12:16:20.321 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-12T12:16:20.348 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-12T12:16:20.430 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T12:16:20.586 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-12T12:16:20.612 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-12T12:16:20.653 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T12:16:20.784 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-12T12:16:20.810 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-12T12:16:20.850 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T12:16:20.997 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-12T12:16:21.020 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-12T12:16:21.064 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T12:16:21.186 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-12T12:16:21.213 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-12T12:16:21.253 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T12:16:21.459 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-12T12:16:21.486 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T12:16:21.526 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:16:21.654 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-12T12:16:21.681 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T12:16:21.723 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:16:21.937 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-12T12:16:21.964 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T12:16:22.012 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:16:22.126 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-12T12:16:22.153 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T12:16:22.193 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:16:22.340 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-12T12:16:22.366 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T12:16:22.406 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:16:22.537 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-12T12:16:22.564 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-12T12:16:22.604 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:16:22.759 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-12T12:16:22.782 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-12T12:16:22.826 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T12:16:22.916 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-12T12:16:22.943 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-12T12:16:22.974 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T12:16:23.154 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librados2. 2024-09-12T12:16:23.180 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../010-librados2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:23.221 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:23.268 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librados2. 2024-09-12T12:16:23.295 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../010-librados2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:23.335 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:23.632 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librbd1. 2024-09-12T12:16:23.658 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../011-librbd1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:23.690 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:23.788 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librbd1. 2024-09-12T12:16:23.815 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../011-librbd1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:23.854 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:24.118 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libcephfs2. 2024-09-12T12:16:24.144 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:24.184 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:24.478 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rados. 2024-09-12T12:16:24.493 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../013-python3-rados_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:24.516 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs2. 2024-09-12T12:16:24.537 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:24.542 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:24.582 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:24.848 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-12T12:16:24.851 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rados. 2024-09-12T12:16:24.867 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../013-python3-rados_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:24.874 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:24.901 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:24.914 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:25.141 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cephfs. 2024-09-12T12:16:25.157 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:25.191 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:25.229 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-12T12:16:25.256 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:25.295 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:25.503 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-12T12:16:25.529 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:25.531 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cephfs. 2024-09-12T12:16:25.547 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:25.569 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:25.589 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:25.900 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-12T12:16:25.905 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-prettytable. 2024-09-12T12:16:25.927 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:25.932 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-12T12:16:25.968 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:25.972 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-12T12:16:26.241 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rbd. 2024-09-12T12:16:26.268 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:26.270 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-prettytable. 2024-09-12T12:16:26.286 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-12T12:16:26.308 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:26.320 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-12T12:16:26.547 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rbd. 2024-09-12T12:16:26.563 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:26.606 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:26.670 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-12T12:16:26.696 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-12T12:16:26.736 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T12:16:26.959 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-12T12:16:26.980 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-12T12:16:26.985 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-12T12:16:26.995 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-12T12:16:27.025 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T12:16:27.030 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T12:16:27.311 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-12T12:16:27.327 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-12T12:16:27.366 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-12T12:16:27.369 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T12:16:27.393 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-12T12:16:27.441 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T12:16:27.672 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-12T12:16:27.688 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-12T12:16:27.722 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T12:16:27.795 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua5.1. 2024-09-12T12:16:27.820 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-12T12:16:27.853 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-12T12:16:28.034 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package lua5.1. 2024-09-12T12:16:28.060 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-12T12:16:28.100 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-12T12:16:28.172 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua-any. 2024-09-12T12:16:28.199 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-12T12:16:28.239 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-12T12:16:28.411 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package lua-any. 2024-09-12T12:16:28.427 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-12T12:16:28.470 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-12T12:16:28.483 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package zip. 2024-09-12T12:16:28.498 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-12T12:16:28.542 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking zip (3.0-12build2) ... 2024-09-12T12:16:28.714 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package zip. 2024-09-12T12:16:28.730 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-12T12:16:28.772 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking zip (3.0-12build2) ... 2024-09-12T12:16:28.895 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package unzip. 2024-09-12T12:16:28.921 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-12T12:16:28.961 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-12T12:16:29.092 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package unzip. 2024-09-12T12:16:29.119 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-12T12:16:29.159 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-12T12:16:29.573 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package luarocks. 2024-09-12T12:16:29.600 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-12T12:16:29.640 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-12T12:16:29.880 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package luarocks. 2024-09-12T12:16:29.907 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-12T12:16:29.946 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-12T12:16:30.016 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librgw2. 2024-09-12T12:16:30.041 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../027-librgw2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:30.081 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:30.345 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package librgw2. 2024-09-12T12:16:30.373 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../027-librgw2_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:30.412 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:30.625 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rgw. 2024-09-12T12:16:30.640 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:30.683 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:30.969 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-12T12:16:30.972 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rgw. 2024-09-12T12:16:30.989 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:30.996 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-12T12:16:31.027 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T12:16:31.039 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:31.330 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libradosstriper1. 2024-09-12T12:16:31.334 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-12T12:16:31.357 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:31.361 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-12T12:16:31.392 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T12:16:31.397 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:31.703 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libradosstriper1. 2024-09-12T12:16:31.730 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:31.770 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:31.825 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-common. 2024-09-12T12:16:31.852 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../031-ceph-common_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:31.900 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:32.232 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-common. 2024-09-12T12:16:32.258 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../031-ceph-common_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:32.298 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:33.322 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-base. 2024-09-12T12:16:33.349 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../032-ceph-base_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:33.398 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:33.653 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-base. 2024-09-12T12:16:33.680 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../032-ceph-base_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:33.721 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:33.951 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-12T12:16:33.978 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-12T12:16:34.017 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-12T12:16:34.282 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-12T12:16:34.309 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-12T12:16:34.349 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-12T12:16:34.381 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cheroot. 2024-09-12T12:16:34.408 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-12T12:16:34.464 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T12:16:34.842 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-12T12:16:34.860 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cheroot. 2024-09-12T12:16:34.869 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-12T12:16:34.887 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-12T12:16:34.925 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-12T12:16:34.926 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T12:16:35.211 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-12T12:16:35.238 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-12T12:16:35.254 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-12T12:16:35.278 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-12T12:16:35.281 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-12T12:16:35.321 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-12T12:16:35.614 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-12T12:16:35.632 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-12T12:16:35.641 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-12T12:16:35.659 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-12T12:16:35.689 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-12T12:16:35.699 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-12T12:16:35.952 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-12T12:16:35.976 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-12T12:16:36.058 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-tempora. 2024-09-12T12:16:36.085 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-12T12:16:36.133 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-12T12:16:36.135 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-12T12:16:36.421 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-tempora. 2024-09-12T12:16:36.448 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-12T12:16:36.453 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-portend. 2024-09-12T12:16:36.479 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-12T12:16:36.487 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-12T12:16:36.519 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-12T12:16:36.791 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-portend. 2024-09-12T12:16:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-12T12:16:36.822 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-12T12:16:36.849 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-12T12:16:36.857 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-12T12:16:36.889 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-12T12:16:37.151 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-12T12:16:37.167 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-12T12:16:37.201 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-12T12:16:37.258 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-12T12:16:37.286 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-12T12:16:37.333 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-12T12:16:37.471 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-12T12:16:37.498 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-12T12:16:37.537 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-12T12:16:37.653 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-natsort. 2024-09-12T12:16:37.669 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-12T12:16:37.712 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-12T12:16:37.891 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-natsort. 2024-09-12T12:16:37.918 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-12T12:16:37.958 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-12T12:16:38.049 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-logutils. 2024-09-12T12:16:38.075 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-12T12:16:38.115 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-12T12:16:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-logutils. 2024-09-12T12:16:38.285 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-12T12:16:38.327 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-12T12:16:38.417 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-12T12:16:38.444 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-12T12:16:38.484 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-12T12:16:38.588 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-12T12:16:38.615 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-12T12:16:38.646 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-12T12:16:38.761 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-mako. 2024-09-12T12:16:38.777 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-12T12:16:38.811 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T12:16:38.933 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-mako. 2024-09-12T12:16:38.960 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-12T12:16:38.999 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T12:16:39.082 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-12T12:16:39.108 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-12T12:16:39.140 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-12T12:16:39.327 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-12T12:16:39.344 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-12T12:16:39.386 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-12T12:16:39.442 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-12T12:16:39.469 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-12T12:16:39.513 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-12T12:16:39.671 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-12T12:16:39.688 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-12T12:16:39.721 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-12T12:16:39.970 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-webob. 2024-09-12T12:16:39.996 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-12T12:16:40.025 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webob. 2024-09-12T12:16:40.037 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T12:16:40.052 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-12T12:16:40.108 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T12:16:40.432 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-waitress. 2024-09-12T12:16:40.458 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-12T12:16:40.461 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-waitress. 2024-09-12T12:16:40.488 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-12T12:16:40.510 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T12:16:40.531 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T12:16:40.834 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-tempita. 2024-09-12T12:16:40.847 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-tempita. 2024-09-12T12:16:40.861 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-12T12:16:40.874 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-12T12:16:40.901 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T12:16:40.914 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T12:16:41.208 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-paste. 2024-09-12T12:16:41.236 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-12T12:16:41.245 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-paste. 2024-09-12T12:16:41.272 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-12T12:16:41.274 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T12:16:41.321 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T12:16:41.594 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-12T12:16:41.611 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-12T12:16:41.648 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-12T12:16:41.652 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T12:16:41.664 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-12T12:16:41.707 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T12:16:41.930 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-12T12:16:41.957 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-12T12:16:41.997 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-12T12:16:42.009 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-12T12:16:42.037 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-12T12:16:42.076 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-12T12:16:42.325 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-webtest. 2024-09-12T12:16:42.353 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-12T12:16:42.353 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-webtest. 2024-09-12T12:16:42.369 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-12T12:16:42.392 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-12T12:16:42.404 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-12T12:16:42.648 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pecan. 2024-09-12T12:16:42.675 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-12T12:16:42.703 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pecan. 2024-09-12T12:16:42.715 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T12:16:42.730 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-12T12:16:42.769 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T12:16:43.070 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-certifi. 2024-09-12T12:16:43.096 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-12T12:16:43.135 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-certifi. 2024-09-12T12:16:43.136 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-12T12:16:43.164 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-12T12:16:43.201 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-12T12:16:43.464 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-idna. 2024-09-12T12:16:43.491 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-12T12:16:43.529 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-idna. 2024-09-12T12:16:43.530 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T12:16:43.556 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-12T12:16:43.595 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T12:16:43.850 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-urllib3. 2024-09-12T12:16:43.857 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-urllib3. 2024-09-12T12:16:43.877 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-12T12:16:43.884 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-12T12:16:43.917 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T12:16:44.082 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T12:16:44.270 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-requests. 2024-09-12T12:16:44.296 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-12T12:16:44.336 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T12:16:44.614 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-12T12:16:44.641 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-12T12:16:44.672 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T12:16:44.967 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-12T12:16:44.983 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:45.017 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:45.244 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-requests. 2024-09-12T12:16:45.271 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-12T12:16:45.310 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T12:16:45.380 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-12T12:16:45.407 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:45.438 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:45.621 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-12T12:16:45.648 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-12T12:16:45.688 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T12:16:45.733 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-12T12:16:45.759 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-12T12:16:45.799 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-12T12:16:45.982 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-12T12:16:46.009 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:46.049 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:46.102 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-openssl. 2024-09-12T12:16:46.128 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-12T12:16:46.227 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-12T12:16:46.429 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-12T12:16:46.456 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:46.495 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:46.748 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-12T12:16:46.775 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-12T12:16:46.814 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-12T12:16:47.126 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-openssl. 2024-09-12T12:16:47.153 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-12T12:16:47.192 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-12T12:16:47.355 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr. 2024-09-12T12:16:47.382 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:47.413 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:47.554 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr. 2024-09-12T12:16:47.581 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:47.620 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:47.808 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mon. 2024-09-12T12:16:47.835 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:47.874 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:47.965 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mon. 2024-09-12T12:16:47.992 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:48.023 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:48.488 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-osd. 2024-09-12T12:16:48.513 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:48.552 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:48.627 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-osd. 2024-09-12T12:16:48.654 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:48.693 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:49.581 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph. 2024-09-12T12:16:49.596 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../068-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:49.630 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:49.828 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph. 2024-09-12T12:16:49.855 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../068-ceph_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:49.896 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:50.150 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-fuse. 2024-09-12T12:16:50.177 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:50.216 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:50.241 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-fuse. 2024-09-12T12:16:50.268 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:50.307 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:50.603 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mds. 2024-09-12T12:16:50.629 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:50.669 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:50.685 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mds. 2024-09-12T12:16:50.713 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:50.752 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:51.081 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package cephadm. 2024-09-12T12:16:51.107 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../071-cephadm_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:51.148 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephadm. 2024-09-12T12:16:51.149 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:51.174 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../071-cephadm_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:16:51.213 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:51.408 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-12T12:16:51.423 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-12T12:16:51.458 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-12T12:16:51.499 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-12T12:16:51.526 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-12T12:16:51.565 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-12T12:16:51.786 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-12T12:16:51.801 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T12:16:51.844 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T12:16:51.869 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-12T12:16:51.896 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T12:16:51.935 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T12:16:52.891 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-babel. 2024-09-12T12:16:52.918 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T12:16:52.940 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-babel. 2024-09-12T12:16:52.957 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T12:16:52.968 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-12T12:16:52.998 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T12:16:53.268 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-jinja2. 2024-09-12T12:16:53.284 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-12T12:16:53.285 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jinja2. 2024-09-12T12:16:53.312 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-12T12:16:53.326 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T12:16:53.376 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T12:16:53.646 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-12T12:16:53.674 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:53.712 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:53.730 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-12T12:16:53.756 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:53.797 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:54.049 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-12T12:16:54.076 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-12T12:16:54.115 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-12T12:16:54.133 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-12T12:16:54.160 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-12T12:16:54.199 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-12T12:16:54.377 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-routes. 2024-09-12T12:16:54.404 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-12T12:16:54.435 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T12:16:54.502 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-routes. 2024-09-12T12:16:54.529 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-12T12:16:54.568 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T12:16:54.721 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-12T12:16:54.737 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:54.771 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:54.922 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-12T12:16:54.949 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:54.988 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:56.126 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-12T12:16:56.154 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-12T12:16:56.159 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-12T12:16:56.176 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-12T12:16:56.192 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T12:16:56.209 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T12:16:56.713 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-joblib. 2024-09-12T12:16:56.740 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-12T12:16:56.763 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-joblib. 2024-09-12T12:16:56.779 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T12:16:56.790 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-12T12:16:56.829 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T12:16:57.132 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-12T12:16:57.160 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-12T12:16:57.182 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-12T12:16:57.198 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-12T12:16:57.285 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-12T12:16:57.326 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-12T12:16:57.612 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-sklearn. 2024-09-12T12:16:57.621 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-sklearn. 2024-09-12T12:16:57.639 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-12T12:16:57.648 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-12T12:16:57.678 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T12:16:57.687 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T12:16:58.388 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-12T12:16:58.416 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:58.421 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-12T12:16:58.449 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:16:58.454 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:58.496 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:16:59.391 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-cachetools. 2024-09-12T12:16:59.419 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-12T12:16:59.449 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-12T12:16:59.532 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cachetools. 2024-09-12T12:16:59.559 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-12T12:16:59.599 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-12T12:16:59.727 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-12T12:16:59.757 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-12T12:16:59.793 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-12T12:16:59.893 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-12T12:16:59.921 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-12T12:16:59.961 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-12T12:17:00.139 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-12T12:17:00.153 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-12T12:17:00.196 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T12:17:00.516 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-rsa. 2024-09-12T12:17:00.544 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-12T12:17:00.564 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-12T12:17:00.583 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-12T12:17:00.592 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-12T12:17:00.630 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T12:17:00.920 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-google-auth. 2024-09-12T12:17:00.948 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-12T12:17:00.950 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rsa. 2024-09-12T12:17:00.981 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-12T12:17:00.986 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-12T12:17:01.016 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-12T12:17:01.306 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-12T12:17:01.328 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-google-auth. 2024-09-12T12:17:01.334 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-12T12:17:01.356 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-12T12:17:01.373 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T12:17:01.394 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-12T12:17:01.642 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-websocket. 2024-09-12T12:17:01.671 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-12T12:17:01.698 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-12T12:17:01.701 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-12T12:17:01.715 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-12T12:17:01.765 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T12:17:02.012 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-12T12:17:02.040 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-12T12:17:02.059 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-websocket. 2024-09-12T12:17:02.079 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T12:17:02.086 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-12T12:17:02.126 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-12T12:17:02.446 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-12T12:17:02.474 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-12T12:17:02.512 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T12:17:02.767 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-12T12:17:02.786 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:02.826 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:03.145 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-12T12:17:03.173 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-12T12:17:03.212 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T12:17:03.218 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-12T12:17:03.236 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:03.276 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:03.557 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-12T12:17:03.584 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T12:17:03.612 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-12T12:17:03.623 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T12:17:03.639 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-12T12:17:03.678 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T12:17:03.909 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package jq. 2024-09-12T12:17:03.925 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T12:17:03.959 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-12T12:17:04.082 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-12T12:17:04.109 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T12:17:04.156 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T12:17:04.178 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package socat. 2024-09-12T12:17:04.207 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-12T12:17:04.245 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-12T12:17:04.467 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package jq. 2024-09-12T12:17:04.483 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-12T12:17:04.534 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-12T12:17:04.556 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package xmlstarlet. 2024-09-12T12:17:04.573 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-12T12:17:04.614 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-12T12:17:04.795 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package socat. 2024-09-12T12:17:04.811 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-12T12:17:04.853 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-12T12:17:04.961 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-test. 2024-09-12T12:17:04.989 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../099-ceph-test_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:05.027 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:05.165 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package xmlstarlet. 2024-09-12T12:17:05.192 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-12T12:17:05.231 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-12T12:17:05.735 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-test. 2024-09-12T12:17:05.751 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../099-ceph-test_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:05.785 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:07.406 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ceph-volume. 2024-09-12T12:17:07.434 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:07.472 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:07.869 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-12T12:17:07.897 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:07.914 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-volume. 2024-09-12T12:17:07.930 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:07.936 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-mirror (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:07.972 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:08.246 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-colorama. 2024-09-12T12:17:08.274 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-12T12:17:08.301 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-12T12:17:08.312 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-12T12:17:08.329 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:08.368 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking cephfs-mirror (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:08.573 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-shell. 2024-09-12T12:17:08.602 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:08.640 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-shell (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:08.645 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-colorama. 2024-09-12T12:17:08.661 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-12T12:17:08.714 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-12T12:17:08.842 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package cephfs-top. 2024-09-12T12:17:08.859 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:08.892 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking cephfs-top (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:08.948 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package cephfs-shell. 2024-09-12T12:17:08.976 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:09.015 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking cephfs-shell (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:09.136 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-12T12:17:09.153 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:09.194 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:09.276 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package cephfs-top. 2024-09-12T12:17:09.292 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-298-gb53b6d3e-1jammy_all.deb ... 2024-09-12T12:17:09.334 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking cephfs-top (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:09.514 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-12T12:17:09.542 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-12T12:17:09.580 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T12:17:09.586 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-12T12:17:09.602 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:09.645 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:09.902 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-12T12:17:09.930 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-12T12:17:09.948 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-12T12:17:09.968 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-12T12:17:09.975 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-12T12:17:10.006 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T12:17:10.319 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-12T12:17:10.322 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package nvme-cli. 2024-09-12T12:17:10.346 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-12T12:17:10.350 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-12T12:17:10.380 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T12:17:10.385 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-12T12:17:10.731 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package nvme-cli. 2024-09-12T12:17:10.759 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-12T12:17:10.785 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package pkg-config. 2024-09-12T12:17:10.814 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-12T12:17:10.852 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T12:17:10.972 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T12:17:11.113 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-12T12:17:11.130 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-12T12:17:11.171 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-12T12:17:11.377 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package pkg-config. 2024-09-12T12:17:11.404 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-12T12:17:11.443 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T12:17:11.607 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-12T12:17:11.636 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-12T12:17:11.674 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-12T12:17:11.705 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-12T12:17:11.724 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-12T12:17:11.763 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-12T12:17:11.977 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pastescript. 2024-09-12T12:17:12.005 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-12T12:17:12.043 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-12T12:17:12.216 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-12T12:17:12.244 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-12T12:17:12.283 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-12T12:17:12.372 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pluggy. 2024-09-12T12:17:12.400 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-12T12:17:12.438 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-12T12:17:12.544 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pastescript. 2024-09-12T12:17:12.571 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-12T12:17:12.602 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-12T12:17:12.708 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-psutil. 2024-09-12T12:17:12.736 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-12T12:17:12.766 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-12T12:17:12.889 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pluggy. 2024-09-12T12:17:12.917 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-12T12:17:12.955 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-12T12:17:13.119 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-py. 2024-09-12T12:17:13.147 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-12T12:17:13.177 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-12T12:17:13.266 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-psutil. 2024-09-12T12:17:13.294 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-12T12:17:13.333 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-12T12:17:13.455 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pygments. 2024-09-12T12:17:13.472 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-12T12:17:13.505 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T12:17:13.653 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-py. 2024-09-12T12:17:13.680 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-12T12:17:13.719 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-12T12:17:13.934 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-12T12:17:13.963 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-12T12:17:14.000 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-12T12:17:14.058 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pygments. 2024-09-12T12:17:14.085 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-12T12:17:14.124 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T12:17:14.320 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-toml. 2024-09-12T12:17:14.348 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-12T12:17:14.386 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-12T12:17:14.578 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-12T12:17:14.606 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-12T12:17:14.644 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-12T12:17:14.673 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-pytest. 2024-09-12T12:17:14.701 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-12T12:17:14.739 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T12:17:14.955 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-toml. 2024-09-12T12:17:14.984 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-12T12:17:15.022 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-12T12:17:15.084 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package python3-simplejson. 2024-09-12T12:17:15.113 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-12T12:17:15.151 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-12T12:17:15.309 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pytest. 2024-09-12T12:17:15.336 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-12T12:17:15.375 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T12:17:15.462 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-12T12:17:15.490 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-12T12:17:15.537 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-12T12:17:15.695 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-simplejson. 2024-09-12T12:17:15.711 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-12T12:17:15.745 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-12T12:17:15.964 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-12T12:17:15.980 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-12T12:17:16.022 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-12T12:17:16.249 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package radosgw. 2024-09-12T12:17:16.277 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../122-radosgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:16.315 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:16.743 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package radosgw. 2024-09-12T12:17:16.771 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../122-radosgw_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:16.809 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:17.168 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package rbd-fuse. 2024-09-12T12:17:17.185 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:17.227 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:17.524 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-12T12:17:17.612 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package rbd-fuse. 2024-09-12T12:17:17.628 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-298-gb53b6d3e-1jammy_amd64.deb ... 2024-09-12T12:17:17.670 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:17.782 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T12:17:17.933 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T12:17:18.009 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-12T12:17:18.247 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-12T12:17:18.267 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-12T12:17:18.368 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-12T12:17:18.574 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-12T12:17:18.607 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-12T12:17:18.949 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-12T12:17:19.024 INFO:teuthology.orchestra.run.smithi003.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-12T12:17:19.093 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:19.386 INFO:teuthology.orchestra.run.smithi142.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-12T12:17:19.454 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:19.617 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user cephadm....done 2024-09-12T12:17:19.782 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T12:17:19.962 INFO:teuthology.orchestra.run.smithi142.stdout:Adding system user cephadm....done 2024-09-12T12:17:20.101 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-12T12:17:20.159 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-12T12:17:20.369 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-12T12:17:20.410 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-12T12:17:20.578 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-12T12:17:20.603 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-12T12:17:20.730 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-12T12:17:20.845 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-12T12:17:20.981 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-12T12:17:21.113 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T12:17:21.239 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-12T12:17:21.282 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-12T12:17:21.408 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-12T12:17:21.735 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T12:17:21.807 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-12T12:17:22.087 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-12T12:17:22.192 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-12T12:17:22.329 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-12T12:17:22.461 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-12T12:17:22.580 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-12T12:17:22.737 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-12T12:17:22.882 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-12T12:17:23.038 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-12T12:17:23.150 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-12T12:17:23.323 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-12T12:17:23.401 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-12T12:17:23.599 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-12T12:17:23.652 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-12T12:17:23.875 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-12T12:17:23.912 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:24.151 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-argparse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:24.154 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T12:17:24.272 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T12:17:24.398 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-12T12:17:24.402 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-12T12:17:24.537 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-12T12:17:24.663 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-12T12:17:24.666 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T12:17:24.792 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-12T12:17:24.922 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-12T12:17:25.026 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-12T12:17:25.048 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-12T12:17:25.316 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-12T12:17:25.369 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-12T12:17:25.407 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-12T12:17:25.471 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T12:17:25.597 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-12T12:17:25.704 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-12T12:17:25.757 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-12T12:17:25.830 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-12T12:17:25.965 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T12:17:25.972 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-12T12:17:26.091 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-12T12:17:26.373 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-12T12:17:26.524 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-12T12:17:26.684 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T12:17:26.809 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-12T12:17:26.952 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-12T12:17:27.069 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-12T12:17:27.253 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up zip (3.0-12build2) ... 2024-09-12T12:17:27.362 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up zip (3.0-12build2) ... 2024-09-12T12:17:27.379 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T12:17:27.488 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-12T12:17:28.131 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T12:17:28.148 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-12T12:17:28.398 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T12:17:28.441 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-12T12:17:28.524 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-12T12:17:28.550 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-12T12:17:28.650 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T12:17:28.651 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-12T12:17:28.935 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-12T12:17:28.944 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-12T12:17:29.319 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-12T12:17:29.379 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-12T12:17:29.554 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T12:17:29.638 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-12T12:17:29.913 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T12:17:30.040 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-12T12:17:30.232 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-12T12:17:30.366 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-12T12:17:30.541 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T12:17:30.684 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-12T12:17:30.884 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-12T12:17:31.160 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-12T12:17:31.219 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-12T12:17:31.269 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:31.470 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-12T12:17:31.570 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T12:17:31.596 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:31.880 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T12:17:31.914 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-12T12:17:32.249 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-12T12:17:32.994 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T12:17:33.229 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:17:33.363 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-12T12:17:33.571 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-12T12:17:33.639 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T12:17:33.765 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-12T12:17:33.856 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:17:33.891 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T12:17:34.016 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-12T12:17:34.184 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-12T12:17:34.300 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-12T12:17:34.401 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-12T12:17:34.427 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-12T12:17:34.502 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-12T12:17:34.686 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:17:34.812 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-12T12:17:34.813 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-12T12:17:35.088 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-12T12:17:35.122 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-12T12:17:35.373 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:17:35.374 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-12T12:17:35.499 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-12T12:17:35.641 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-12T12:17:35.825 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-12T12:17:35.892 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-12T12:17:36.126 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-12T12:17:36.170 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-12T12:17:36.394 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T12:17:36.453 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-12T12:17:36.520 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-12T12:17:36.729 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-12T12:17:36.796 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T12:17:36.922 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T12:17:36.988 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-12T12:17:37.207 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T12:17:37.273 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-12T12:17:37.399 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-12T12:17:37.508 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-12T12:17:37.717 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-12T12:17:37.767 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T12:17:37.868 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-12T12:17:37.868 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-12T12:17:37.969 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T12:17:38.058 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-12T12:17:38.177 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-12T12:17:38.304 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T12:17:38.405 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T12:17:38.495 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-12T12:17:38.763 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-12T12:17:38.782 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-12T12:17:38.914 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-12T12:17:39.041 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-12T12:17:39.065 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-12T12:17:39.159 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-12T12:17:39.171 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-12T12:17:39.401 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:17:39.417 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-12T12:17:39.518 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-12T12:17:39.552 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T12:17:39.870 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T12:17:39.911 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-12T12:17:40.171 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T12:17:40.196 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-12T12:17:40.322 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-12T12:17:40.464 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-12T12:17:40.590 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-12T12:17:40.676 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-12T12:17:40.741 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-12T12:17:40.944 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-12T12:17:41.067 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-12T12:17:41.203 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T12:17:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-12T12:17:41.454 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-12T12:17:41.653 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-12T12:17:41.856 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-12T12:17:41.864 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-12T12:17:42.149 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T12:17:42.316 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-12T12:17:42.477 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T12:17:42.583 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-12T12:17:42.628 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:42.745 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:42.846 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-12T12:17:42.868 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-12T12:17:43.158 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:43.245 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-12T12:17:43.284 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:43.401 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-12T12:17:43.538 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-12T12:17:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:43.787 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:43.888 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:43.907 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-12T12:17:44.058 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librados2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:44.085 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T12:17:44.085 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T12:17:44.209 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libsqlite3-mod-ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:44.360 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-12T12:17:44.581 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:44.699 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:44.713 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:44.825 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:44.881 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libradosstriper1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:44.951 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up cephfs-top (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:45.032 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-12T12:17:45.077 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T12:17:45.317 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librbd1 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:45.418 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-modules-core (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:45.519 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T12:17:45.741 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-12T12:17:46.204 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libcephfs-dev (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.330 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rados (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.330 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.431 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.456 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up librgw2 (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.541 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.581 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-top (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.667 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.699 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-12T12:17:46.793 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up cephfs-shell (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:46.919 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:47.470 INFO:teuthology.orchestra.run.smithi142.stdout:Adding group ceph....done 2024-09-12T12:17:47.855 INFO:teuthology.orchestra.run.smithi142.stdout:Adding system user ceph....done 2024-09-12T12:17:47.953 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rbd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:48.006 INFO:teuthology.orchestra.run.smithi142.stdout:Setting system user ceph properties....done 2024-09-12T12:17:48.019 INFO:teuthology.orchestra.run.smithi142.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-12T12:17:48.079 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up rbd-fuse (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:48.157 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-12T12:17:48.230 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-rgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:48.381 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up python3-cephfs (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:48.507 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-12T12:17:48.532 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-shell (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:48.700 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-common (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:48.970 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up cephfs-mirror (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:49.160 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:17:49.160 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:17:49.310 INFO:teuthology.orchestra.run.smithi003.stdout:Adding group ceph....done 2024-09-12T12:17:49.604 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:49.703 INFO:teuthology.orchestra.run.smithi003.stdout:Adding system user ceph....done 2024-09-12T12:17:49.713 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:49.830 INFO:teuthology.orchestra.run.smithi003.stdout:Setting system user ceph properties....done 2024-09-12T12:17:49.846 INFO:teuthology.orchestra.run.smithi003.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-12T12:17:49.993 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-12T12:17:50.155 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:17:50.155 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:17:50.342 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-12T12:17:50.616 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:50.835 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up cephfs-mirror (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:50.960 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-12T12:17:51.069 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:17:51.069 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:17:51.426 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:51.553 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-test (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:51.653 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T12:17:51.653 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T12:17:51.679 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up radosgw (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:52.111 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:52.150 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:17:52.150 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:17:52.328 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T12:17:52.328 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T12:17:52.623 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-base (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:52.828 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:52.940 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-12T12:17:53.065 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T12:17:53.065 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T12:17:53.433 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mds (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:53.540 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:53.682 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T12:17:53.682 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-12T12:17:53.801 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:53.952 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:54.159 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T12:17:54.160 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T12:17:54.202 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:54.449 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T12:17:54.449 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-12T12:17:54.628 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:54.754 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:54.880 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:54.911 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-osd (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:55.023 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:55.199 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T12:17:55.199 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-12T12:17:55.375 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-12T12:17:55.493 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-12T12:17:55.645 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-12T12:17:55.697 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-k8sevents (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:55.823 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:55.966 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mon (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:56.164 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T12:17:56.164 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-12T12:17:56.670 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-cephadm (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:56.729 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-12T12:17:56.921 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:57.138 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-mgr-dashboard (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:57.385 INFO:teuthology.orchestra.run.smithi142.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-12T12:17:57.389 DEBUG:teuthology.orchestra.run.smithi142:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-12T12:17:57.446 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-09-12T12:17:57.512 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ceph-volume (19.1.1-298-gb53b6d3e-1jammy) ... 2024-09-12T12:17:57.654 INFO:teuthology.orchestra.run.smithi142.stdout:Building dependency tree... 2024-09-12T12:17:57.655 INFO:teuthology.orchestra.run.smithi142.stdout:Reading state information... 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:17:57.854 INFO:teuthology.orchestra.run.smithi142.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:The following additional packages will be installed: 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout:Suggested packages: 2024-09-12T12:17:57.855 INFO:teuthology.orchestra.run.smithi142.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-12T12:17:57.856 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-12T12:17:57.856 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-12T12:17:57.897 INFO:teuthology.orchestra.run.smithi142.stdout:The following NEW packages will be installed: 2024-09-12T12:17:57.898 INFO:teuthology.orchestra.run.smithi142.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-12T12:17:57.898 INFO:teuthology.orchestra.run.smithi142.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-12T12:17:57.898 INFO:teuthology.orchestra.run.smithi142.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-12T12:17:57.899 INFO:teuthology.orchestra.run.smithi142.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-12T12:17:57.899 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-12T12:17:57.899 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-12T12:17:57.899 INFO:teuthology.orchestra.run.smithi142.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-12T12:17:57.900 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages will be upgraded: 2024-09-12T12:17:57.901 INFO:teuthology.orchestra.run.smithi142.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-12T12:17:57.975 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-12T12:17:58.067 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-12T12:17:58.082 INFO:teuthology.orchestra.run.smithi142.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-12T12:17:58.082 INFO:teuthology.orchestra.run.smithi142.stdout:Need to get 55.8 MB of archives. 2024-09-12T12:17:58.082 INFO:teuthology.orchestra.run.smithi142.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-12T12:17:58.082 INFO:teuthology.orchestra.run.smithi142.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-12T12:17:58.212 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-12T12:17:58.707 INFO:teuthology.orchestra.run.smithi142.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-12T12:17:58.725 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-12T12:17:58.848 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-12T12:17:58.849 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-12T12:17:58.850 INFO:teuthology.orchestra.run.smithi142.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-12T12:17:58.879 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-12T12:17:58.880 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-12T12:17:58.884 INFO:teuthology.orchestra.run.smithi142.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-12T12:17:58.886 INFO:teuthology.orchestra.run.smithi142.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-12T12:17:58.887 INFO:teuthology.orchestra.run.smithi142.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-12T12:17:58.890 INFO:teuthology.orchestra.run.smithi142.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-12T12:17:58.952 INFO:teuthology.orchestra.run.smithi142.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-12T12:17:58.952 INFO:teuthology.orchestra.run.smithi142.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-12T12:17:58.953 INFO:teuthology.orchestra.run.smithi142.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-12T12:17:58.953 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-12T12:17:59.034 INFO:teuthology.orchestra.run.smithi142.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-12T12:17:59.598 INFO:teuthology.orchestra.run.smithi003.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-12T12:17:59.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-12T12:17:59.663 INFO:teuthology.orchestra.run.smithi003.stdout:Reading package lists... 2024-09-12T12:17:59.821 INFO:teuthology.orchestra.run.smithi142.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-12T12:17:59.826 INFO:teuthology.orchestra.run.smithi142.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-12T12:17:59.826 INFO:teuthology.orchestra.run.smithi142.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-12T12:17:59.833 INFO:teuthology.orchestra.run.smithi142.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-12T12:17:59.833 INFO:teuthology.orchestra.run.smithi142.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-12T12:17:59.844 INFO:teuthology.orchestra.run.smithi142.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-12T12:17:59.848 INFO:teuthology.orchestra.run.smithi142.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-12T12:17:59.865 INFO:teuthology.orchestra.run.smithi142.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-12T12:17:59.865 INFO:teuthology.orchestra.run.smithi142.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-12T12:17:59.875 INFO:teuthology.orchestra.run.smithi003.stdout:Building dependency tree... 2024-09-12T12:17:59.876 INFO:teuthology.orchestra.run.smithi003.stdout:Reading state information... 2024-09-12T12:17:59.934 INFO:teuthology.orchestra.run.smithi142.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-12T12:17:59.950 INFO:teuthology.orchestra.run.smithi142.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-12T12:17:59.973 INFO:teuthology.orchestra.run.smithi142.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-12T12:18:00.042 INFO:teuthology.orchestra.run.smithi142.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-12T12:18:00.048 INFO:teuthology.orchestra.run.smithi142.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-12T12:18:00.049 INFO:teuthology.orchestra.run.smithi142.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-12T12:18:00.050 INFO:teuthology.orchestra.run.smithi142.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-12T12:18:00.051 INFO:teuthology.orchestra.run.smithi142.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-12T12:18:00.081 INFO:teuthology.orchestra.run.smithi142.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-12T12:18:00.084 INFO:teuthology.orchestra.run.smithi142.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-12T12:18:00.096 INFO:teuthology.orchestra.run.smithi003.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-12T12:18:00.096 INFO:teuthology.orchestra.run.smithi003.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-12T12:18:00.096 INFO:teuthology.orchestra.run.smithi003.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-12T12:18:00.096 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages were automatically installed and are no longer required: 2024-09-12T12:18:00.096 INFO:teuthology.orchestra.run.smithi003.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-12T12:18:00.096 INFO:teuthology.orchestra.run.smithi003.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-12T12:18:00.096 INFO:teuthology.orchestra.run.smithi003.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-12T12:18:00.097 INFO:teuthology.orchestra.run.smithi003.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-12T12:18:00.097 INFO:teuthology.orchestra.run.smithi003.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-12T12:18:00.097 INFO:teuthology.orchestra.run.smithi003.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-12T12:18:00.101 INFO:teuthology.orchestra.run.smithi003.stdout:The following additional packages will be installed: 2024-09-12T12:18:00.101 INFO:teuthology.orchestra.run.smithi003.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-12T12:18:00.101 INFO:teuthology.orchestra.run.smithi003.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-12T12:18:00.101 INFO:teuthology.orchestra.run.smithi003.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-12T12:18:00.101 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-12T12:18:00.101 INFO:teuthology.orchestra.run.smithi003.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-12T12:18:00.101 INFO:teuthology.orchestra.run.smithi003.stdout:Suggested packages: 2024-09-12T12:18:00.102 INFO:teuthology.orchestra.run.smithi003.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-12T12:18:00.102 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-12T12:18:00.102 INFO:teuthology.orchestra.run.smithi003.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-12T12:18:00.141 INFO:teuthology.orchestra.run.smithi003.stdout:The following NEW packages will be installed: 2024-09-12T12:18:00.142 INFO:teuthology.orchestra.run.smithi003.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-12T12:18:00.142 INFO:teuthology.orchestra.run.smithi003.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-12T12:18:00.143 INFO:teuthology.orchestra.run.smithi003.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-12T12:18:00.143 INFO:teuthology.orchestra.run.smithi003.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-12T12:18:00.143 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-12T12:18:00.143 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-12T12:18:00.143 INFO:teuthology.orchestra.run.smithi003.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-12T12:18:00.144 INFO:teuthology.orchestra.run.smithi003.stdout:The following packages will be upgraded: 2024-09-12T12:18:00.145 INFO:teuthology.orchestra.run.smithi003.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-12T12:18:00.196 INFO:teuthology.orchestra.run.smithi003.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-12T12:18:00.196 INFO:teuthology.orchestra.run.smithi003.stdout:Need to get 55.8 MB of archives. 2024-09-12T12:18:00.196 INFO:teuthology.orchestra.run.smithi003.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-12T12:18:00.196 INFO:teuthology.orchestra.run.smithi003.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-12T12:18:00.329 INFO:teuthology.orchestra.run.smithi003.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-12T12:18:00.336 INFO:teuthology.orchestra.run.smithi003.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-12T12:18:00.375 INFO:teuthology.orchestra.run.smithi003.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-12T12:18:00.376 INFO:teuthology.orchestra.run.smithi003.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-12T12:18:00.377 INFO:teuthology.orchestra.run.smithi003.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-12T12:18:00.388 INFO:teuthology.orchestra.run.smithi003.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-12T12:18:00.389 INFO:teuthology.orchestra.run.smithi003.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-12T12:18:00.392 INFO:teuthology.orchestra.run.smithi003.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-12T12:18:00.393 INFO:teuthology.orchestra.run.smithi003.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-12T12:18:00.394 INFO:teuthology.orchestra.run.smithi003.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-12T12:18:00.395 INFO:teuthology.orchestra.run.smithi003.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-12T12:18:00.396 INFO:teuthology.orchestra.run.smithi003.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-12T12:18:00.411 INFO:teuthology.orchestra.run.smithi003.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-12T12:18:00.411 INFO:teuthology.orchestra.run.smithi003.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-12T12:18:00.412 INFO:teuthology.orchestra.run.smithi003.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-12T12:18:00.640 INFO:teuthology.orchestra.run.smithi142.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-12T12:18:00.640 INFO:teuthology.orchestra.run.smithi142.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-12T12:18:00.641 INFO:teuthology.orchestra.run.smithi142.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-12T12:18:00.641 INFO:teuthology.orchestra.run.smithi142.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-12T12:18:00.641 INFO:teuthology.orchestra.run.smithi142.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-12T12:18:00.642 INFO:teuthology.orchestra.run.smithi142.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-12T12:18:00.670 INFO:teuthology.orchestra.run.smithi142.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-12T12:18:00.671 INFO:teuthology.orchestra.run.smithi142.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-12T12:18:00.733 INFO:teuthology.orchestra.run.smithi003.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-12T12:18:00.736 INFO:teuthology.orchestra.run.smithi003.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-12T12:18:00.736 INFO:teuthology.orchestra.run.smithi003.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-12T12:18:00.738 INFO:teuthology.orchestra.run.smithi003.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-12T12:18:00.739 INFO:teuthology.orchestra.run.smithi003.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-12T12:18:00.743 INFO:teuthology.orchestra.run.smithi003.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-12T12:18:00.744 INFO:teuthology.orchestra.run.smithi003.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-12T12:18:00.752 INFO:teuthology.orchestra.run.smithi003.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-12T12:18:00.752 INFO:teuthology.orchestra.run.smithi003.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-12T12:18:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-12T12:18:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-12T12:18:00.783 INFO:teuthology.orchestra.run.smithi003.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-12T12:18:00.802 INFO:teuthology.orchestra.run.smithi003.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-12T12:18:00.806 INFO:teuthology.orchestra.run.smithi003.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-12T12:18:00.806 INFO:teuthology.orchestra.run.smithi003.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-12T12:18:00.806 INFO:teuthology.orchestra.run.smithi003.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-12T12:18:00.807 INFO:teuthology.orchestra.run.smithi003.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-12T12:18:00.818 INFO:teuthology.orchestra.run.smithi003.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-12T12:18:00.820 INFO:teuthology.orchestra.run.smithi003.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-12T12:18:01.021 INFO:teuthology.orchestra.run.smithi003.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-12T12:18:01.022 INFO:teuthology.orchestra.run.smithi003.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-12T12:18:01.022 INFO:teuthology.orchestra.run.smithi003.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-12T12:18:01.022 INFO:teuthology.orchestra.run.smithi003.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-12T12:18:01.022 INFO:teuthology.orchestra.run.smithi003.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-12T12:18:01.022 INFO:teuthology.orchestra.run.smithi003.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-12T12:18:01.033 INFO:teuthology.orchestra.run.smithi003.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-12T12:18:01.034 INFO:teuthology.orchestra.run.smithi003.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-12T12:18:01.084 INFO:teuthology.orchestra.run.smithi142.stdout:Preconfiguring packages ... 2024-09-12T12:18:01.238 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 55.8 MB in 3s (20.2 MB/s) 2024-09-12T12:18:01.465 INFO:teuthology.orchestra.run.smithi003.stdout:Preconfiguring packages ... 2024-09-12T12:18:01.558 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173985 files and directories currently installed.) 2024-09-12T12:18:01.563 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-12T12:18:01.608 INFO:teuthology.orchestra.run.smithi003.stdout:Fetched 55.8 MB in 1s (63.2 MB/s) 2024-09-12T12:18:01.710 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-12T12:18:01.927 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173985 files and directories currently installed.) 2024-09-12T12:18:01.934 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-12T12:18:02.077 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-12T12:18:02.348 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-12T12:18:02.724 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-12T12:18:03.314 INFO:teuthology.orchestra.run.smithi142.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-12T12:18:03.668 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package flex. 2024-09-12T12:18:03.682 INFO:teuthology.orchestra.run.smithi003.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-12T12:18:03.773 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173985 files and directories currently installed.) 2024-09-12T12:18:03.778 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-12T12:18:03.824 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-12T12:18:03.992 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package flex. 2024-09-12T12:18:04.098 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173985 files and directories currently installed.) 2024-09-12T12:18:04.104 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-12T12:18:04.148 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-12T12:18:04.382 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-12T12:18:04.546 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-12T12:18:04.648 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-12T12:18:04.787 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-12T12:18:04.957 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-12T12:18:05.174 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-12T12:18:05.543 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174058 files and directories currently installed.) 2024-09-12T12:18:05.548 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-12T12:18:05.685 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174058 files and directories currently installed.) 2024-09-12T12:18:05.690 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-12T12:18:05.695 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-12T12:18:05.795 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-12T12:18:06.123 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-12T12:18:06.205 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-12T12:18:06.278 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-12T12:18:06.335 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-12T12:18:06.639 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-12T12:18:06.663 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-12T12:18:07.021 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package bison. 2024-09-12T12:18:07.095 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package bison. 2024-09-12T12:18:07.125 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174058 files and directories currently installed.) 2024-09-12T12:18:07.131 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-12T12:18:07.181 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-12T12:18:07.203 INFO:teuthology.orchestra.run.smithi003.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174058 files and directories currently installed.) 2024-09-12T12:18:07.209 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-12T12:18:07.264 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-12T12:18:07.582 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-12T12:18:07.609 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-12T12:18:07.631 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-12T12:18:07.647 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-12T12:18:07.648 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-12T12:18:07.680 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-12T12:18:07.993 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package dump. 2024-09-12T12:18:08.000 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package dump. 2024-09-12T12:18:08.020 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-12T12:18:08.028 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-12T12:18:08.059 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking dump (0.4b47-3) ... 2024-09-12T12:18:08.067 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking dump (0.4b47-3) ... 2024-09-12T12:18:08.321 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package indent. 2024-09-12T12:18:08.337 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:08.354 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package indent. 2024-09-12T12:18:08.371 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-12T12:18:08.371 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:08.412 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-12T12:18:08.623 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-12T12:18:08.639 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-12T12:18:08.682 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-12T12:18:08.699 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-12T12:18:08.716 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-12T12:18:08.757 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-12T12:18:09.044 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-12T12:18:09.069 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-12T12:18:09.071 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-12T12:18:09.098 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-12T12:18:09.110 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-12T12:18:09.136 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-12T12:18:09.363 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-12T12:18:09.379 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-12T12:18:09.413 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-12T12:18:09.422 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-12T12:18:09.450 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-12T12:18:09.489 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-12T12:18:09.724 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-12T12:18:09.752 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-12T12:18:09.791 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:09.792 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-12T12:18:09.821 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-12T12:18:09.850 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:10.052 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-12T12:18:10.068 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-12T12:18:10.069 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-12T12:18:10.083 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-12T12:18:10.110 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:10.129 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:10.381 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libjson-perl. 2024-09-12T12:18:10.387 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libjson-perl. 2024-09-12T12:18:10.398 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-12T12:18:10.404 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-12T12:18:10.439 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-12T12:18:10.445 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-12T12:18:10.751 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-12T12:18:10.765 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-12T12:18:10.779 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-12T12:18:10.793 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-12T12:18:10.817 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-12T12:18:10.832 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-12T12:18:12.435 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-12T12:18:12.462 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-12T12:18:12.487 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-12T12:18:12.493 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:12.515 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-12T12:18:12.545 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:12.745 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-12T12:18:12.773 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-12T12:18:12.812 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:12.823 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-12T12:18:12.839 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-12T12:18:12.881 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:13.131 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-12T12:18:13.159 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-12T12:18:13.198 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:13.200 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-12T12:18:13.229 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-12T12:18:13.267 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:13.559 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-12T12:18:13.570 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-12T12:18:13.587 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-12T12:18:13.587 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-12T12:18:13.620 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-12T12:18:13.625 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-12T12:18:13.928 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-12T12:18:13.956 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-12T12:18:13.995 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-12T12:18:14.331 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-12T12:18:14.359 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:14.393 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-12T12:18:14.397 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:14.422 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-12T12:18:14.460 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-12T12:18:14.692 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-12T12:18:14.719 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:14.754 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-12T12:18:14.771 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:14.775 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:14.804 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:15.078 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-12T12:18:15.106 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:15.144 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:15.148 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-12T12:18:15.177 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:15.215 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:15.405 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-12T12:18:15.421 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-12T12:18:15.463 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-12T12:18:15.518 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-12T12:18:15.534 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:15.576 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:15.812 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-12T12:18:15.828 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-12T12:18:15.870 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-12T12:18:15.917 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-12T12:18:15.944 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-12T12:18:15.983 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-12T12:18:16.386 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package modemmanager. 2024-09-12T12:18:16.407 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-12T12:18:16.414 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-12T12:18:16.435 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-12T12:18:16.461 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:16.473 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-12T12:18:16.850 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package modemmanager. 2024-09-12T12:18:16.878 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-12T12:18:16.927 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:17.143 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package network-manager. 2024-09-12T12:18:17.172 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-12T12:18:17.218 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-12T12:18:17.583 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package network-manager. 2024-09-12T12:18:17.612 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-12T12:18:17.659 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-12T12:18:17.850 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ppp. 2024-09-12T12:18:17.877 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-12T12:18:18.147 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-12T12:18:18.274 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package ppp. 2024-09-12T12:18:18.302 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-12T12:18:18.554 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package pptp-linux. 2024-09-12T12:18:18.578 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-12T12:18:18.582 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-12T12:18:18.621 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-12T12:18:18.873 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-12T12:18:18.889 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-12T12:18:18.903 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package pptp-linux. 2024-09-12T12:18:18.931 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-12T12:18:18.932 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-12T12:18:18.969 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-12T12:18:19.189 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-12T12:18:19.206 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-12T12:18:19.234 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-12T12:18:19.239 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-12T12:18:19.262 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-12T12:18:19.301 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking postgresql-client-common (238) ... 2024-09-12T12:18:19.571 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-12T12:18:19.599 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:19.629 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:19.702 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-12T12:18:19.730 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-12T12:18:19.760 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client-common (238) ... 2024-09-12T12:18:20.080 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-12T12:18:20.109 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:20.147 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:20.225 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package postgresql-common. 2024-09-12T12:18:20.253 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-12T12:18:20.312 INFO:teuthology.orchestra.run.smithi142.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-12T12:18:20.340 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking postgresql-common (238) ... 2024-09-12T12:18:20.743 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-common. 2024-09-12T12:18:20.772 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-12T12:18:20.828 INFO:teuthology.orchestra.run.smithi003.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-12T12:18:20.832 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package postgresql-14. 2024-09-12T12:18:20.858 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-common (238) ... 2024-09-12T12:18:20.860 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:20.901 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:21.317 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-14. 2024-09-12T12:18:21.345 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-12T12:18:21.388 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:22.151 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package postgresql. 2024-09-12T12:18:22.168 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-12T12:18:22.209 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking postgresql (14+238) ... 2024-09-12T12:18:22.470 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package postgresql-client. 2024-09-12T12:18:22.498 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-12T12:18:22.536 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking postgresql-client (14+238) ... 2024-09-12T12:18:22.611 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql. 2024-09-12T12:18:22.628 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-12T12:18:22.669 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql (14+238) ... 2024-09-12T12:18:22.780 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-12T12:18:22.796 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-12T12:18:22.838 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-12T12:18:22.913 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-client. 2024-09-12T12:18:22.930 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-12T12:18:22.971 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-client (14+238) ... 2024-09-12T12:18:23.090 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package tcl8.6. 2024-09-12T12:18:23.118 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-12T12:18:23.157 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-12T12:18:23.198 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-12T12:18:23.216 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-12T12:18:23.248 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-12T12:18:23.434 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package tcl. 2024-09-12T12:18:23.451 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-12T12:18:23.506 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-12T12:18:23.534 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package tcl8.6. 2024-09-12T12:18:23.563 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-12T12:18:23.600 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-12T12:18:23.854 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package wpasupplicant. 2024-09-12T12:18:23.882 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-12T12:18:23.886 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package tcl. 2024-09-12T12:18:23.903 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-12T12:18:23.920 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-12T12:18:23.949 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-12T12:18:24.283 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-12T12:18:24.297 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package wpasupplicant. 2024-09-12T12:18:24.300 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-12T12:18:24.326 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-12T12:18:24.332 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-12T12:18:24.364 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-12T12:18:24.686 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-12T12:18:24.713 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-12T12:18:24.743 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-12T12:18:24.760 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-12T12:18:24.772 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-12T12:18:24.810 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-12T12:18:25.056 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up postgresql-client-common (238) ... 2024-09-12T12:18:25.188 INFO:teuthology.orchestra.run.smithi003.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-12T12:18:25.217 INFO:teuthology.orchestra.run.smithi003.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-12T12:18:25.249 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-12T12:18:25.263 INFO:teuthology.orchestra.run.smithi003.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-12T12:18:25.350 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-12T12:18:25.451 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:25.560 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client-common (238) ... 2024-09-12T12:18:25.610 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-12T12:18:25.736 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-12T12:18:25.768 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-12T12:18:25.862 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:25.894 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-12T12:18:26.020 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:26.146 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-12T12:18:26.272 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-12T12:18:26.345 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-12T12:18:26.398 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:26.434 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-12T12:18:26.814 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-12T12:18:26.814 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-12T12:18:26.977 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-12T12:18:27.090 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-12T12:18:27.312 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-12T12:18:27.438 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-12T12:18:27.490 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-12T12:18:27.490 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-12T12:18:27.580 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-12T12:18:27.706 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-12T12:18:27.824 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-12T12:18:27.933 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:27.998 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-12T12:18:28.034 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-12T12:18:28.108 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-12T12:18:28.152 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-12T12:18:28.209 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-12T12:18:28.335 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-12T12:18:28.460 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-12T12:18:28.586 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:28.712 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-12T12:18:28.838 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-12T12:18:29.315 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-12T12:18:29.416 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up dump (0.4b47-3) ... 2024-09-12T12:18:29.427 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-12T12:18:29.501 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-12T12:18:29.553 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up dump (0.4b47-3) ... 2024-09-12T12:18:29.567 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:29.694 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-12T12:18:29.760 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-12T12:18:29.820 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-12T12:18:29.945 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-12T12:18:30.063 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-12T12:18:30.130 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:30.146 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-12T12:18:30.198 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-12T12:18:30.256 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-12T12:18:30.307 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-12T12:18:30.381 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-12T12:18:30.499 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-12T12:18:30.500 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-12T12:18:30.617 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-12T12:18:30.626 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:30.690 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-12T12:18:30.760 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-12T12:18:30.760 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-12T12:18:30.886 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-12T12:18:30.928 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up postgresql-client (14+238) ... 2024-09-12T12:18:31.054 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-12T12:18:31.062 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-12T12:18:31.188 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:31.314 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-12T12:18:31.482 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-client (14+238) ... 2024-09-12T12:18:31.492 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:31.492 INFO:teuthology.orchestra.run.smithi142.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-12T12:18:31.492 INFO:teuthology.orchestra.run.smithi142.stdout:which means they are currently configured by ifupdown: 2024-09-12T12:18:31.492 INFO:teuthology.orchestra.run.smithi142.stdout:- enp3s0f1 2024-09-12T12:18:31.493 INFO:teuthology.orchestra.run.smithi142.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-12T12:18:31.493 INFO:teuthology.orchestra.run.smithi142.stdout:remove their configuration from /etc/network/interfaces. 2024-09-12T12:18:31.493 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:31.600 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-12T12:18:31.610 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-12T12:18:31.939 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-12T12:18:32.026 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:32.026 INFO:teuthology.orchestra.run.smithi003.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-12T12:18:32.026 INFO:teuthology.orchestra.run.smithi003.stdout:which means they are currently configured by ifupdown: 2024-09-12T12:18:32.026 INFO:teuthology.orchestra.run.smithi003.stdout:- ens1f0 2024-09-12T12:18:32.026 INFO:teuthology.orchestra.run.smithi003.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-12T12:18:32.026 INFO:teuthology.orchestra.run.smithi003.stdout:remove their configuration from /etc/network/interfaces. 2024-09-12T12:18:32.026 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:32.158 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-12T12:18:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-12T12:18:32.506 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-12T12:18:32.756 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:32.862 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-12T12:18:32.866 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:32.967 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-12T12:18:33.093 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up postgresql-common (238) ... 2024-09-12T12:18:33.377 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-12T12:18:33.512 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-12T12:18:33.638 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-12T12:18:33.763 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-common (238) ... 2024-09-12T12:18:34.201 INFO:teuthology.orchestra.run.smithi142.stdout:Adding user postgres to group ssl-cert 2024-09-12T12:18:34.589 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:34.590 INFO:teuthology.orchestra.run.smithi142.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-12T12:18:34.709 INFO:teuthology.orchestra.run.smithi003.stdout:Adding user postgres to group ssl-cert 2024-09-12T12:18:34.751 INFO:teuthology.orchestra.run.smithi142.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-12T12:18:34.751 INFO:teuthology.orchestra.run.smithi142.stdout:Removing obsolete dictionary files: 2024-09-12T12:18:34.963 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:34.963 INFO:teuthology.orchestra.run.smithi003.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-12T12:18:35.170 INFO:teuthology.orchestra.run.smithi003.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-12T12:18:35.170 INFO:teuthology.orchestra.run.smithi003.stdout:Removing obsolete dictionary files: 2024-09-12T12:18:35.288 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-12T12:18:35.652 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:35.748 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-12T12:18:35.977 INFO:teuthology.orchestra.run.smithi142.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-12T12:18:36.006 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout:This user must also own the server process. 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout:The database cluster will be initialized with locale "en_US". 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout:The default text search configuration will be set to "english". 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout:Data page checksums are disabled. 2024-09-12T12:18:36.053 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:36.054 INFO:teuthology.orchestra.run.smithi142.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-12T12:18:36.054 INFO:teuthology.orchestra.run.smithi142.stdout:creating subdirectories ... ok 2024-09-12T12:18:36.054 INFO:teuthology.orchestra.run.smithi142.stdout:selecting dynamic shared memory implementation ... posix 2024-09-12T12:18:36.069 INFO:teuthology.orchestra.run.smithi142.stdout:selecting default max_connections ... 100 2024-09-12T12:18:36.086 INFO:teuthology.orchestra.run.smithi142.stdout:selecting default shared_buffers ... 128MB 2024-09-12T12:18:36.097 INFO:teuthology.orchestra.run.smithi142.stdout:selecting default time zone ... Etc/UTC 2024-09-12T12:18:36.097 INFO:teuthology.orchestra.run.smithi142.stdout:creating configuration files ... ok 2024-09-12T12:18:36.131 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-12T12:18:36.214 INFO:teuthology.orchestra.run.smithi142.stdout:running bootstrap script ... ok 2024-09-12T12:18:36.479 INFO:teuthology.orchestra.run.smithi003.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-12T12:18:36.491 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-12T12:18:36.543 INFO:teuthology.orchestra.run.smithi003.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-12T12:18:36.543 INFO:teuthology.orchestra.run.smithi003.stdout:This user must also own the server process. 2024-09-12T12:18:36.543 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:36.543 INFO:teuthology.orchestra.run.smithi003.stdout:The database cluster will be initialized with locale "en_US". 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout:The default text search configuration will be set to "english". 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout:Data page checksums are disabled. 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout:creating subdirectories ... ok 2024-09-12T12:18:36.544 INFO:teuthology.orchestra.run.smithi003.stdout:selecting dynamic shared memory implementation ... posix 2024-09-12T12:18:36.559 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default max_connections ... 100 2024-09-12T12:18:36.580 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default shared_buffers ... 128MB 2024-09-12T12:18:36.588 INFO:teuthology.orchestra.run.smithi003.stdout:selecting default time zone ... Etc/UTC 2024-09-12T12:18:36.589 INFO:teuthology.orchestra.run.smithi003.stdout:creating configuration files ... ok 2024-09-12T12:18:36.706 INFO:teuthology.orchestra.run.smithi003.stdout:running bootstrap script ... ok 2024-09-12T12:18:36.900 INFO:teuthology.orchestra.run.smithi142.stdout:performing post-bootstrap initialization ... ok 2024-09-12T12:18:37.361 INFO:teuthology.orchestra.run.smithi003.stdout:performing post-bootstrap initialization ... ok 2024-09-12T12:18:37.580 INFO:teuthology.orchestra.run.smithi142.stdout:syncing data to disk ... ok 2024-09-12T12:18:37.961 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-12T12:18:37.966 INFO:teuthology.orchestra.run.smithi003.stdout:syncing data to disk ... ok 2024-09-12T12:18:38.347 INFO:teuthology.orchestra.run.smithi003.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-12T12:18:40.709 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:40.842 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-12T12:18:40.968 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up postgresql (14+238) ... 2024-09-12T12:18:41.094 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:41.121 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:41.237 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:41.247 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-12T12:18:41.373 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up postgresql (14+238) ... 2024-09-12T12:18:41.475 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-12T12:18:41.475 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-12T12:18:41.490 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-12T12:18:41.591 INFO:teuthology.orchestra.run.smithi003.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-12T12:18:41.835 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-12T12:18:41.836 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-12T12:18:42.023 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-12T12:18:42.115 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-12T12:18:42.417 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-12T12:18:42.418 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-12T12:18:42.527 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-12T12:18:42.611 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-12T12:18:42.846 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-12T12:18:42.990 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-12T12:18:43.897 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-12T12:18:43.996 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-12T12:18:44.309 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-12T12:18:44.409 INFO:teuthology.orchestra.run.smithi003.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-12T12:18:44.504 INFO:teuthology.orchestra.run.smithi142.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-12T12:18:44.513 DEBUG:teuthology.parallel:result is None 2024-09-12T12:18:44.915 INFO:teuthology.orchestra.run.smithi003.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-12T12:18:44.920 DEBUG:teuthology.parallel:result is None 2024-09-12T12:18:44.924 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:18:45.078 DEBUG:teuthology.orchestra.run.smithi003:> dpkg-query -W -f '${Version}' ceph 2024-09-12T12:18:45.098 INFO:teuthology.orchestra.run.smithi003.stdout:19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:18:45.099 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:18:45.099 INFO:teuthology.task.install:The correct ceph version 19.1.1-298-gb53b6d3e-1jammy is installed. 2024-09-12T12:18:45.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:18:45.263 DEBUG:teuthology.orchestra.run.smithi142:> dpkg-query -W -f '${Version}' ceph 2024-09-12T12:18:45.282 INFO:teuthology.orchestra.run.smithi142.stdout:19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:18:45.282 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298-gb53b6d3e-1jammy 2024-09-12T12:18:45.282 INFO:teuthology.task.install:The correct ceph version 19.1.1-298-gb53b6d3e-1jammy is installed. 2024-09-12T12:18:45.284 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T12:18:45.284 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:45.284 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T12:18:45.299 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:45.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T12:18:45.340 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T12:18:45.340 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:45.340 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T12:18:45.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T12:18:45.411 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:45.411 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T12:18:45.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T12:18:45.479 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T12:18:45.479 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:45.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T12:18:45.493 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T12:18:45.549 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:45.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T12:18:45.561 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T12:18:45.615 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T12:18:45.615 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:45.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T12:18:45.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T12:18:45.683 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:45.683 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T12:18:45.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T12:18:45.752 INFO:teuthology.run_tasks:Running task ceph... 2024-09-12T12:18:45.831 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-12T12:18:45.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2024-09-12T12:18:45.834 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /var/log/ceph 2024-09-12T12:18:45.847 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-12T12:18:45.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T12:18:45.888 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T12:18:45.903 INFO:tasks.ceph:Creating extra log directories... 2024-09-12T12:18:45.903 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T12:18:45.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T12:18:45.963 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-12T12:18:45.963 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-12T12:18:45.963 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-12T12:18:45.963 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:45.963 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-12T12:18:46.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-12T12:18:46.080 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:46.080 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-12T12:18:46.086 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-12T12:18:46.164 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-12T12:18:46.164 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-12T12:18:46.164 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/sessionmap}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901713', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 481, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'sleep_before_teardown': 0, 'subset': '57/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAff60/SFZ9MgWP4VvK3P9pI+z79kgSgUPfuxUNpSMfaabZXNMv4GQRgY4GVTnvRukzP66vNtY4fN3abh9kgyac=', 'smithi142.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKvqt2M+a9vqwj94NVuvAgxnY7dJhwPVxuZOEmRdLN0vf7C2+QbjpcQVX9VpHhcc7jE2NUUWMNYG3lCBm1PT3wM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'client session with non-allowable root'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_sessionmap']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-11_21:24:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-12T12:18:46.165 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T12:18:46.167 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-12T12:18:46.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T12:18:46.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T12:18:46.265 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:46.266 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T12:18:46.275 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T12:18:46.275 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-09-12T12:18:46.323 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T12:18:46.324 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:46.324 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-09-12T12:18:46.324 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:46.324 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 12:12:25.464652422 +0000 2024-09-12T12:18:46.324 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 12:12:25.172655031 +0000 2024-09-12T12:18:46.324 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 12:12:25.172655031 +0000 2024-09-12T12:18:46.324 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-12T12:18:46.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T12:18:46.378 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T12:18:46.379 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T12:18:46.379 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000437168 s, 1.2 MB/s 2024-09-12T12:18:46.380 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T12:18:46.429 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-09-12T12:18:46.479 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T12:18:46.479 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:46.479 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-12T12:18:46.479 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:46.480 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 12:12:25.944648134 +0000 2024-09-12T12:18:46.480 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 12:12:25.684650457 +0000 2024-09-12T12:18:46.480 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 12:12:25.684650457 +0000 2024-09-12T12:18:46.480 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-12T12:18:46.480 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T12:18:46.533 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T12:18:46.533 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T12:18:46.533 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00034159 s, 1.5 MB/s 2024-09-12T12:18:46.534 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T12:18:46.584 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-09-12T12:18:46.631 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T12:18:46.631 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:46.631 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-09-12T12:18:46.632 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:46.632 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 12:12:26.464643490 +0000 2024-09-12T12:18:46.632 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 12:12:26.212645741 +0000 2024-09-12T12:18:46.632 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 12:12:26.212645741 +0000 2024-09-12T12:18:46.632 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-12T12:18:46.632 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T12:18:46.685 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T12:18:46.685 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T12:18:46.685 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00033596 s, 1.5 MB/s 2024-09-12T12:18:46.686 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T12:18:46.736 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-09-12T12:18:46.783 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T12:18:46.783 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:46.783 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-12T12:18:46.783 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:46.783 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 12:12:27.008638631 +0000 2024-09-12T12:18:46.784 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 12:12:26.724641168 +0000 2024-09-12T12:18:46.784 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 12:12:26.724641168 +0000 2024-09-12T12:18:46.784 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: - 2024-09-12T12:18:46.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T12:18:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T12:18:46.837 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T12:18:46.838 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.00039412 s, 1.3 MB/s 2024-09-12T12:18:46.839 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T12:18:46.888 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T12:18:46.888 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:46.889 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T12:18:46.894 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T12:18:46.894 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 869 Links: 1 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 12:12:22.227527358 +0000 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 12:12:21.971529645 +0000 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 12:12:21.971529645 +0000 2024-09-12T12:18:46.941 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-09-12T12:18:46.942 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T12:18:46.995 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T12:18:46.995 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T12:18:46.995 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000477582 s, 1.1 MB/s 2024-09-12T12:18:46.996 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T12:18:47.046 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 12:12:22.719522964 +0000 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 12:12:22.463525251 +0000 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 12:12:22.463525251 +0000 2024-09-12T12:18:47.093 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-09-12T12:18:47.093 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T12:18:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T12:18:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T12:18:47.146 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000443881 s, 1.2 MB/s 2024-09-12T12:18:47.148 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T12:18:47.198 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-09-12T12:18:47.245 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T12:18:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:47.246 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-09-12T12:18:47.246 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:47.246 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 12:12:23.195518713 +0000 2024-09-12T12:18:47.246 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 12:12:22.939520999 +0000 2024-09-12T12:18:47.246 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 12:12:22.939520999 +0000 2024-09-12T12:18:47.246 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-09-12T12:18:47.246 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T12:18:47.298 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T12:18:47.298 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T12:18:47.298 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000314343 s, 1.6 MB/s 2024-09-12T12:18:47.299 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T12:18:47.349 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-09-12T12:18:47.396 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T12:18:47.396 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:18:47.396 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-09-12T12:18:47.396 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:18:47.397 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-09-12 12:12:23.639514747 +0000 2024-09-12T12:18:47.397 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-09-12 12:12:23.395516926 +0000 2024-09-12T12:18:47.397 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-09-12 12:12:23.395516926 +0000 2024-09-12T12:18:47.397 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-09-12T12:18:47.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T12:18:47.450 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-09-12T12:18:47.451 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-09-12T12:18:47.451 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000386528 s, 1.3 MB/s 2024-09-12T12:18:47.452 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T12:18:47.501 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T12:18:47.534 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T12:18:47.535 INFO:tasks.ceph:Generating config... 2024-09-12T12:18:47.536 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-12T12:18:47.536 INFO:tasks.ceph:[client] debug client = 20 2024-09-12T12:18:47.537 INFO:tasks.ceph:[client] debug ms = 1 2024-09-12T12:18:47.537 INFO:tasks.ceph:[client] pid file = 2024-09-12T12:18:47.537 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-12T12:18:47.537 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-12T12:18:47.537 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-12T12:18:47.538 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-12T12:18:47.538 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-12T12:18:47.538 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-12T12:18:47.538 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-12T12:18:47.538 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-12T12:18:47.539 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-12T12:18:47.539 INFO:tasks.ceph:Setting up mon.a... 2024-09-12T12:18:47.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-12T12:18:47.582 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2024-09-12T12:18:47.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-12T12:18:47.675 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T12:18:47.734 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.b', '172.21.15.142'), ('mon.c', '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]')] 2024-09-12T12:18:47.734 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-12T12:18:47.735 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:47.735 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T12:18:47.781 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --add b 172.21.15.142 --addv c '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2024-09-12T12:18:47.870655+0000 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-12T12:18:47.870655+0000 2024-09-12T12:18:47.868 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 17 (quincy) 2024-09-12T12:18:47.869 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2024-09-12T12:18:47.869 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T12:18:47.869 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-09-12T12:18:47.869 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] mon.c 2024-09-12T12:18:47.869 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-12T12:18:47.870 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T12:18:47.875 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 252c40bb-cfe5-4c51-b4ad-5433711beaac... 2024-09-12T12:18:47.876 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:18:47.920 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:18:47.949 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T12:18:47.949 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-12T12:18:47.949 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:18:47.949 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-12T12:18:47.949 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.949 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-12T12:18:47.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-12T12:18:47.951 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-12T12:18:47.952 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-12T12:18:47.953 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-12T12:18:47.954 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:18:47.955 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-12T12:18:47.956 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:18:47.957 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:18:47.958 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-12T12:18:47.958 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:18:47.958 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-12T12:18:47.958 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-12T12:18:47.958 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:18:47.958 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-12T12:18:47.958 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-12T12:18:47.959 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-12T12:18:47.960 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-12T12:18:47.961 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:18:47.962 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:18:47.962 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-12T12:18:47.962 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:18:47.962 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:18:47.962 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-12T12:18:47.962 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:18:47.962 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-12T12:18:47.963 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-12T12:18:47.964 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-12T12:18:47.964 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-12T12:18:47.964 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:18:47.964 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:18:47.964 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:18:47.966 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-09-12T12:18:47.966 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-09-12T12:18:47.966 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:18:47.966 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-09-12T12:18:47.966 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.966 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-09-12T12:18:47.966 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:18:47.967 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:18:47.968 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-09-12T12:18:47.969 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:18:47.969 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-09-12T12:18:47.969 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:18:47.969 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:18:47.970 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-09-12T12:18:47.971 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-09-12T12:18:47.972 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.973 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:18:47.973 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-09-12T12:18:47.973 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.973 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-09-12T12:18:47.973 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-09-12T12:18:47.973 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-09-12T12:18:47.973 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:18:47.974 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:18:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.976 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on pool no app = False 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-09-12T12:18:47.977 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:18:47.978 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:18:47.979 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:18:47.979 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-12T12:18:47.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-12T12:18:48.055 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-12T12:18:48.055 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:48.055 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-12T12:18:48.060 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:48.060 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-12T12:18:48.111 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:18:48.111 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:48.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T12:18:48.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T12:18:48.175 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:18:48.176 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:18:48.223 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:18:48.223 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:48.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-12T12:18:48.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-12T12:18:48.247 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:18:48.247 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:18:48.292 INFO:tasks.ceph:Setting up mon nodes... 2024-09-12T12:18:48.292 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-12T12:18:48.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T12:18:48.337 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-12T12:18:48.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T12:18:48.384 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-12T12:18:48.396 INFO:tasks.ceph:Setting up mds nodes... 2024-09-12T12:18:48.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-12T12:18:48.440 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-12T12:18:48.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-12T12:18:48.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-12T12:18:48.586 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-12T12:18:48.589 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-12T12:18:48.643 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-12T12:18:48.688 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-12T12:18:48.691 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-12T12:18:48.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-12T12:18:48.783 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-12T12:18:48.786 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-12T12:18:48.844 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-12T12:18:48.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T12:18:48.882 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-12T12:18:48.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-12T12:18:48.968 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-12T12:18:48.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-12T12:18:49.014 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-12T12:18:49.026 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-12T12:18:49.102 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-12T12:18:49.114 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-12T12:18:49.114 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-12T12:18:49.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-12T12:18:49.129 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T12:18:49.129 INFO:tasks.ceph:role: osd.0 2024-09-12T12:18:49.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:18:49.129 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:18:49.191 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:18:49.655 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-12T12:18:49.657 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-12T12:18:49.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-12T12:18:49.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-12T12:18:49.812 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:18:49.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:18:49.814 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-12T12:18:49.871 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T12:18:49.871 INFO:tasks.ceph:role: osd.1 2024-09-12T12:18:49.871 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:18:49.871 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:18:49.936 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:18:50.382 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-12T12:18:50.384 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-12T12:18:50.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-12T12:18:50.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-12T12:18:50.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:18:50.518 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:18:50.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-12T12:18:50.575 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T12:18:50.575 INFO:tasks.ceph:role: osd.2 2024-09-12T12:18:50.575 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:18:50.575 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T12:18:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:18:50.638 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:18:50.638 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:18:50.638 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:18:50.638 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:18:50.638 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:18:50.639 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:18:50.639 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:18:50.639 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:18:50.639 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:18:51.093 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-12T12:18:51.096 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-12T12:18:51.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-12T12:18:51.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-12T12:18:51.254 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:18:51.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:18:51.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-12T12:18:51.316 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-12T12:18:51.316 INFO:tasks.ceph:role: osd.3 2024-09-12T12:18:51.316 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:18:51.316 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T12:18:51.378 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:18:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:18:51.379 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:18:51.875 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-09-12T12:18:51.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-09-12T12:18:51.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-12T12:18:52.019 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-12T12:18:52.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:18:52.031 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:18:52.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:18:52.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:52.146+0000 7fae57d0a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-12T12:18:52.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:52.150+0000 7fae57d0a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-12T12:18:52.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:52.150+0000 7fae57d0a7c0 -1 bdev(0x55faa7f7b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-12T12:18:52.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:52.150+0000 7fae57d0a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-12T12:18:55.306 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-12T12:18:55.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:18:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:55.398+0000 7f020b89c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-12T12:18:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:55.402+0000 7f020b89c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-12T12:18:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:55.402+0000 7f020b89c7c0 -1 bdev(0x56127b761800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-12T12:18:55.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:55.402+0000 7f020b89c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-12T12:18:58.793 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-12T12:18:58.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:18:58.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:58.926+0000 7f7b4ee367c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-12T12:18:58.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:58.930+0000 7f7b4ee367c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-12T12:18:58.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:58.930+0000 7f7b4ee367c0 -1 bdev(0x563181f07800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-12T12:18:58.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:18:58.930+0000 7f7b4ee367c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-12T12:19:02.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-12T12:19:02.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:19:02.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:02.478+0000 7f1ba5c277c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-12T12:19:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:02.482+0000 7f1ba5c277c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-12T12:19:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:02.482+0000 7f1ba5c277c0 -1 bdev(0x56180993d800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-12T12:19:02.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:02.482+0000 7f1ba5c277c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-12T12:19:05.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-12T12:19:05.688 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-12T12:19:05.703 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T12:19:05.704 INFO:tasks.ceph:role: osd.4 2024-09-12T12:19:05.704 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:19:05.704 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T12:19:05.767 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:19:05.767 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:19:05.768 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:19:06.218 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-09-12T12:19:06.220 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-09-12T12:19:06.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-12T12:19:06.335 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-12T12:19:06.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:19:06.347 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:19:06.348 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-12T12:19:06.404 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T12:19:06.404 INFO:tasks.ceph:role: osd.5 2024-09-12T12:19:06.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:19:06.404 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:19:06.469 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:19:06.470 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:19:06.936 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-09-12T12:19:06.938 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-09-12T12:19:06.938 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-12T12:19:07.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-12T12:19:07.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:19:07.107 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:19:07.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-12T12:19:07.163 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T12:19:07.164 INFO:tasks.ceph:role: osd.6 2024-09-12T12:19:07.164 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:19:07.164 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:19:07.228 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:19:07.229 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:19:07.666 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-09-12T12:19:07.669 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-09-12T12:19:07.669 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-12T12:19:07.813 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-12T12:19:07.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:19:07.824 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:19:07.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-12T12:19:07.880 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-12T12:19:07.880 INFO:tasks.ceph:role: osd.7 2024-09-12T12:19:07.880 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:19:07.880 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-12T12:19:07.949 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:19:07.950 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:19:08.370 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-09-12T12:19:08.372 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-09-12T12:19:08.372 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-12T12:19:08.474 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-12T12:19:08.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:19:08.486 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-09-12T12:19:08.487 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:19:08.606 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:08.604+0000 7f3c52e597c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-12T12:19:08.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:08.608+0000 7f3c52e597c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-12T12:19:08.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:08.608+0000 7f3c52e597c0 -1 bdev(0x5621aea9f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-12T12:19:08.611 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:08.608+0000 7f3c52e597c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-12T12:19:11.764 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-12T12:19:11.778 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:19:11.858 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:11.856+0000 7f83bc1547c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-12T12:19:11.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:11.856+0000 7f83bc1547c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-12T12:19:11.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:11.856+0000 7f83bc1547c0 -1 bdev(0x5610e05b3800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-12T12:19:11.859 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:11.856+0000 7f83bc1547c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-12T12:19:15.000 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-12T12:19:15.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:19:15.135 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:15.132+0000 7f9c9813e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-12T12:19:15.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:15.136+0000 7f9c9813e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-12T12:19:15.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:15.136+0000 7f9c9813e7c0 -1 bdev(0x56155115f800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-12T12:19:15.136 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:15.136+0000 7f9c9813e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-12T12:19:16.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:19:16.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:19:18.287 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-12T12:19:18.301 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:19:18.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:18.380+0000 7f020c9c07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-12T12:19:18.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:18.380+0000 7f020c9c07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-12T12:19:18.381 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:18.380+0000 7f020c9c07c0 -1 bdev(0x55cb6faed800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-12T12:19:18.382 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:18.380+0000 7f020c9c07c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-12T12:19:21.787 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-12T12:19:21.845 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-12T12:19:21.845 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:21.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-12T12:19:21.859 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:21.859 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-12T12:19:21.915 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:21.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-12T12:19:21.974 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:21.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-12T12:19:22.031 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:22.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-12T12:19:22.086 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:22.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-12T12:19:22.143 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:22.143 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-12T12:19:22.199 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.199 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-12T12:19:22.212 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.213 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-12T12:19:22.267 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-12T12:19:22.319 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-12T12:19:22.375 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-12T12:19:22.431 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-12T12:19:22.486 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.487 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-12T12:19:22.539 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:22.539 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-12T12:19:22.545 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:22.545 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-12T12:19:22.591 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.592 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-12T12:19:22.597 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:22.597 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-12T12:19:22.644 INFO:tasks.ceph:Adding keys to all mons... 2024-09-12T12:19:22.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T12:19:22.646 DEBUG:teuthology.orchestra.run.smithi142:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-12T12:19:22.689 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2024-09-12T12:19:22.689 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg== 2024-09-12T12:19:22.689 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:19:22.689 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg== 2024-09-12T12:19:22.689 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg== 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w== 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA== 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw== 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg== 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA== 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:19:22.690 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA== 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw== 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.4] 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA== 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.5] 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ== 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.6] 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ== 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.7] 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA== 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T12:19:22.691 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:19:22.692 INFO:teuthology.orchestra.run.smithi003.stdout:[client.2] 2024-09-12T12:19:22.692 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw== 2024-09-12T12:19:22.692 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2024-09-12T12:19:22.692 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw== 2024-09-12T12:19:22.692 INFO:teuthology.orchestra.run.smithi003.stdout:[client.3] 2024-09-12T12:19:22.692 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA== 2024-09-12T12:19:22.698 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.y] 2024-09-12T12:19:22.698 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg== 2024-09-12T12:19:22.698 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:19:22.698 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg== 2024-09-12T12:19:22.698 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:19:22.698 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg== 2024-09-12T12:19:22.698 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.0] 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w== 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.1] 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA== 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.2] 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw== 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.3] 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg== 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.x] 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA== 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA== 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw== 2024-09-12T12:19:22.699 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.4] 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA== 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.5] 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ== 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.6] 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ== 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.7] 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA== 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout:[client.0] 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout:[client.2] 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw== 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout:[client.1] 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw== 2024-09-12T12:19:22.700 INFO:teuthology.orchestra.run.smithi142.stdout:[client.3] 2024-09-12T12:19:22.701 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA== 2024-09-12T12:19:22.701 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T12:19:22.736 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T12:19:22.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:22.824 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:22.872 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:22.874 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:22.964 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:22.968 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T12:19:23.266 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T12:19:23.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:23.352 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:23.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:23.440 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-12T12:19:23.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.661 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.663 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.748 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:19:23.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:23.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:23.909 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:23.920 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:23.991 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:24.008 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:24.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:24.091 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:19:24.159 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-12T12:19:24.159 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-12T12:19:24.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T12:19:24.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-12T12:19:24.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-12T12:19:24.288 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T12:19:24.380 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-12T12:19:24.390 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-12T12:19:24.441 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-12T12:19:24.535 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-12T12:19:24.546 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:19:24.548 DEBUG:teuthology.orchestra.run.smithi142:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-12T12:19:24.591 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-12T12:19:24.591 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-12T12:19:24.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-12T12:19:24.596 INFO:tasks.ceph.mon.a:Started 2024-09-12T12:19:24.596 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-12T12:19:24.596 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-12T12:19:24.636 INFO:tasks.ceph.mon.b:Started 2024-09-12T12:19:24.637 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-12T12:19:24.637 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-12T12:19:24.639 INFO:tasks.ceph.mon.c:Started 2024-09-12T12:19:24.639 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-12T12:19:24.639 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-12T12:19:24.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-12T12:19:24.642 INFO:tasks.ceph.mgr.y:Started 2024-09-12T12:19:24.642 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-12T12:19:24.642 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-12T12:19:24.644 INFO:tasks.ceph.mgr.x:Started 2024-09-12T12:19:24.644 INFO:tasks.ceph:Setting crush tunables to default 2024-09-12T12:19:24.645 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2024-09-12T12:19:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffbdb8640 1 Processor -- start 2024-09-12T12:19:24.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffbdb8640 1 -- start start 2024-09-12T12:19:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffbdb8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffbdb8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffbdb8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efff40d1690 con 0x7efff40b5f70 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffbdb8640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7efff40d1970 con 0x7efff40b6b10 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffbdb8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efff40d23f0 con 0x7efff40b6350 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efff9353640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efff9353640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56332/0 (socket says 172.21.15.3:56332) 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 learned_addr learned my addr 172.21.15.3:0/3734061337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 msgr2=0x7efff406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 msgr2=0x7efff406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffa355640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffa355640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 msgr2=0x7efff40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-12T12:19:24.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffa355640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 msgr2=0x7efff40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-12T12:19:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffa355640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:19:24.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.762+0000 7efffa355640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:19:24.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 msgr2=0x7efff406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-12T12:19:24.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 msgr2=0x7efff406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-12T12:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efffa355640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efffa355640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 msgr2=0x7efff40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-12T12:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efffa355640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 msgr2=0x7efff40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-12T12:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efffa355640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:19:24.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.966+0000 7efffa355640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:19:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.998+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2266856576 0 0) 0x7efff40d1690 con 0x7efff40b5f70 2024-09-12T12:19:25.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:24.998+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effe0003530 con 0x7efff40b5f70 2024-09-12T12:19:25.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 249486840 0 0) 0x7effe0003530 con 0x7efff40b5f70 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 msgr2=0x7efff40cf000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.142:3300/0 conn(0x7efff40b6350 0x7efff40cf000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 msgr2=0x7efff406e780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 --2- 172.21.15.3:0/3734061337 >> v2:172.21.15.3:3300/0 conn(0x7efff40b6b10 0x7efff406e780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff40d26d0 con 0x7efff40b5f70 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7efff0001680 con 0x7efff40b5f70 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 >> v1:172.21.15.142:6789/0 conn(0x7efff40b5f70 legacy=0x7efff40d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0003a10 0x7effe0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe0004810 0x7effe0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effe00051d0 0x7effe000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efff40d1970 con 0x7effe0004810 2024-09-12T12:19:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efff40d23f0 con 0x7effe0003a10 2024-09-12T12:19:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7effe000c120 con 0x7effe00051d0 2024-09-12T12:19:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.006+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe0004810 0x7effe0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efff9b54640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0003a10 0x7effe0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efffa355640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effe00051d0 0x7effe000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effe00051d0 msgr2=0x7effe000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7effe00051d0 0x7effe000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0003a10 msgr2=0x7effe0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0003a10 0x7effe0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efff9353640 1 -- 172.21.15.3:0/3734061337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe0003530 con 0x7effe0004810 2024-09-12T12:19:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efffa355640 1 -- 172.21.15.3:0/3734061337 reap_dead start 2024-09-12T12:19:25.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.010+0000 7efff9b54640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0003a10 0x7effe0003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:25.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.014+0000 7efff9353640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe0004810 0x7effe0004c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effe400e900 tx=0x7effe401c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a13ca6eca85982 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe40b2070 con 0x7effe0004810 2024-09-12T12:19:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe40ad410 con 0x7effe0004810 2024-09-12T12:19:25.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efff8b52640 1 -- 172.21.15.3:0/3734061337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe400f040 con 0x7effe0004810 2024-09-12T12:19:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 -- 172.21.15.3:0/3734061337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe0004810 msgr2=0x7effe0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe0004810 0x7effe0004c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effe400e900 tx=0x7effe401c610 comp rx=0 tx=0).stop 2024-09-12T12:19:25.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 -- 172.21.15.3:0/3734061337 shutdown_connections 2024-09-12T12:19:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 --2- 172.21.15.3:0/3734061337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe0004810 0x7effe0004c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 -- 172.21.15.3:0/3734061337 >> 172.21.15.3:0/3734061337 conn(0x7efff409d880 msgr2=0x7efff408c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 -- 172.21.15.3:0/3734061337 shutdown_connections 2024-09-12T12:19:25.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 -- 172.21.15.3:0/3734061337 wait complete. 2024-09-12T12:19:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 Processor -- start 2024-09-12T12:19:25.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.018+0000 7efffbdb8640 1 -- start start 2024-09-12T12:19:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0004810 0x7efff40b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff41612c0 0x7efff4161740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efff40b7320 0x7efff40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efff415ea90 con 0x7efff41612c0 2024-09-12T12:19:25.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efff40b67e0 con 0x7effe0004810 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efff406e780 con 0x7efff40b7320 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff41612c0 0x7efff4161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9b54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0004810 0x7efff40b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9b54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0004810 0x7efff40b94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:52164/0 (socket says 172.21.15.3:52164) 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff41612c0 0x7efff4161740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56376/0 (socket says 172.21.15.3:56376) 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffa355640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efff40b7320 0x7efff40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9b54640 1 -- 172.21.15.3:0/3425363060 learned_addr learned my addr 172.21.15.3:0/3425363060 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 -- 172.21.15.3:0/3425363060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efff40b7320 msgr2=0x7efff40b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 --2- 172.21.15.3:0/3425363060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efff40b7320 0x7efff40b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 -- 172.21.15.3:0/3425363060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0004810 msgr2=0x7efff40b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 --2- 172.21.15.3:0/3425363060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0004810 0x7efff40b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 -- 172.21.15.3:0/3425363060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff41606b0 con 0x7efff41612c0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efff9353640 1 --2- 172.21.15.3:0/3425363060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff41612c0 0x7efff4161740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effe4009100 tx=0x7effe0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7effea7fc640 1 -- 172.21.15.3:0/3425363060 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe40b2040 con 0x7efff41612c0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7effea7fc640 1 -- 172.21.15.3:0/3425363060 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe40b8430 con 0x7efff41612c0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7effea7fc640 1 -- 172.21.15.3:0/3425363060 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe40b8820 con 0x7efff41612c0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff40c8270 con 0x7efff41612c0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff40c8830 con 0x7efff41612c0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7effea7fc640 1 -- 172.21.15.3:0/3425363060 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7effe40bf060 con 0x7efff41612c0 2024-09-12T12:19:25.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7effea7fc640 1 -- 172.21.15.3:0/3425363060 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7effe40c0940 con 0x7efff41612c0 2024-09-12T12:19:25.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.022+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff40d1690 con 0x7efff41612c0 2024-09-12T12:19:25.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.058+0000 7effea7fc640 1 -- 172.21.15.3:0/3425363060 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7effe40d72d0 con 0x7efff41612c0 2024-09-12T12:19:25.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.130+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7efff406e560 con 0x7efff41612c0 2024-09-12T12:19:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7effea7fc640 1 -- 172.21.15.3:0/3425363060 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7effe40c5070 con 0x7efff41612c0 2024-09-12T12:19:25.136 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-09-12T12:19:25.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff41612c0 msgr2=0x7efff4161740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 --2- 172.21.15.3:0/3425363060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff41612c0 0x7efff4161740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effe4009100 tx=0x7effe0005840 comp rx=0 tx=0).stop 2024-09-12T12:19:25.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 shutdown_connections 2024-09-12T12:19:25.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 --2- 172.21.15.3:0/3425363060 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efff40b7320 0x7efff40b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 --2- 172.21.15.3:0/3425363060 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efff41612c0 0x7efff4161740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 --2- 172.21.15.3:0/3425363060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effe0004810 0x7efff40b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 >> 172.21.15.3:0/3425363060 conn(0x7efff409d880 msgr2=0x7efff40d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:25.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 shutdown_connections 2024-09-12T12:19:25.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:25.134+0000 7efffbdb8640 1 -- 172.21.15.3:0/3425363060 wait complete. 2024-09-12T12:19:25.156 INFO:tasks.ceph:check_enable_crimson: False 2024-09-12T12:19:25.156 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-12T12:19:25.157 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:25.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-12T12:19:25.168 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:25.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-12T12:19:25.205 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:25.204+0000 7f70f0f4b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:19:25.215 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:25.214+0000 7f9893558180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:19:25.221 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:25.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-12T12:19:25.274 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:19:25.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-12T12:19:25.330 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:25.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-12T12:19:25.343 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:25.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-12T12:19:25.399 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:25.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-12T12:19:25.455 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:19:25.455 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-12T12:19:25.510 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new e8fd6b02-9ed1-4355-9614-bb1195121930 0 2024-09-12T12:19:25.670 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:25.670+0000 7f9893558180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:19:25.678 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46d5331640 1 Processor -- start 2024-09-12T12:19:25.678 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f70f0f4b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:19:25.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46d5331640 1 -- start start 2024-09-12T12:19:25.679 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46d5331640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f46d009b410 0x7f46d00d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46d5331640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46d00d0ea0 0x7f46d00b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46d5331640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d00b19a0 con 0x7f46d015c460 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46d5331640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f46d00d12d0 con 0x7f46d00d0ea0 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46d5331640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f46d009b8d0 con 0x7f46d009b410 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46ce7fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f46d009b410 0x7f46d00d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46ce7fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f46d009b410 0x7f46d00d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44932/0 (socket says 172.21.15.142:44932) 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46ce7fc640 1 -- 172.21.15.142:0/597821551 learned_addr learned my addr 172.21.15.142:0/597821551 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46cf7fe640 1 --2- 172.21.15.142:0/597821551 >> v2:172.21.15.3:3300/0 conn(0x7f46d00d0ea0 0x7f46d00b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46ce7fc640 1 -- 172.21.15.142:0/597821551 >> v2:172.21.15.3:3300/0 conn(0x7f46d00d0ea0 msgr2=0x7f46d00b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46ce7fc640 1 --2- 172.21.15.142:0/597821551 >> v2:172.21.15.3:3300/0 conn(0x7f46d00d0ea0 0x7f46d00b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46ce7fc640 1 -- 172.21.15.142:0/597821551 >> v1:172.21.15.3:6789/0 conn(0x7f46d015c460 legacy=0x7f46d00cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.680 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.676+0000 7f46ce7fc640 1 -- 172.21.15.142:0/597821551 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d00d0660 con 0x7f46d009b410 2024-09-12T12:19:25.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46ce7fc640 1 --2- 172.21.15.142:0/597821551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d009b410 0x7f46d00d0960 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46d0074480 tx=0x7f46b801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=198cf0ca81f8267 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:25.681 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46cdffb640 1 -- 172.21.15.142:0/597821551 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b80ab660 con 0x7f46d009b410 2024-09-12T12:19:25.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46cdffb640 1 -- 172.21.15.142:0/597821551 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b80b0070 con 0x7f46d009b410 2024-09-12T12:19:25.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46cdffb640 1 -- 172.21.15.142:0/597821551 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b80aba20 con 0x7f46d009b410 2024-09-12T12:19:25.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 -- 172.21.15.142:0/597821551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d009b410 msgr2=0x7f46d00d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 --2- 172.21.15.142:0/597821551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d009b410 0x7f46d00d0960 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46d0074480 tx=0x7f46b801c460 comp rx=0 tx=0).stop 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 -- 172.21.15.142:0/597821551 shutdown_connections 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 --2- 172.21.15.142:0/597821551 >> v2:172.21.15.3:3300/0 conn(0x7f46d00d0ea0 0x7f46d00b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 --2- 172.21.15.142:0/597821551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d009b410 0x7f46d00d0960 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 -- 172.21.15.142:0/597821551 >> 172.21.15.142:0/597821551 conn(0x7f46d00b3e70 msgr2=0x7f46d00b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 -- 172.21.15.142:0/597821551 shutdown_connections 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.680+0000 7f46d5331640 1 -- 172.21.15.142:0/597821551 wait complete. 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 Processor -- start 2024-09-12T12:19:25.683 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 -- start start 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46d009b410 0x7f46d00c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46d00d0ea0 0x7f46d0157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d015c460 0x7f46d00c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f46d00b3160 con 0x7f46d00d0ea0 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46d015c840 con 0x7f46d015c460 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46d00b32e0 con 0x7f46d009b410 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d015c460 0x7f46d00c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46ceffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46d009b410 0x7f46d00c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d015c460 0x7f46d00c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44942/0 (socket says 172.21.15.142:44942) 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 -- 172.21.15.142:0/992905322 learned_addr learned my addr 172.21.15.142:0/992905322 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46ceffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46d009b410 0x7f46d00c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:44388/0 (socket says 172.21.15.142:44388) 2024-09-12T12:19:25.686 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46ce7fc640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46d00d0ea0 0x7f46d0157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 -- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46d009b410 msgr2=0x7f46d00c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46d009b410 0x7f46d00c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 -- 172.21.15.142:0/992905322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46d00d0ea0 msgr2=0x7f46d0157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46d00d0ea0 0x7f46d0157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 -- 172.21.15.142:0/992905322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d01576d0 con 0x7f46d015c460 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46ceffd640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46d009b410 0x7f46d00c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46ce7fc640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46d00d0ea0 0x7f46d0157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46cf7fe640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d015c460 0x7f46d00c7b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f46c0007e70 tx=0x7f46c000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46c77fe640 1 -- 172.21.15.142:0/992905322 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0014620 con 0x7f46d015c460 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46c77fe640 1 -- 172.21.15.142:0/992905322 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c0019070 con 0x7f46d015c460 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d0070ad0 con 0x7f46d015c460 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46c77fe640 1 -- 172.21.15.142:0/992905322 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0014a20 con 0x7f46d015c460 2024-09-12T12:19:25.687 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d0070d90 con 0x7f46d015c460 2024-09-12T12:19:25.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d00b3670 con 0x7f46d015c460 2024-09-12T12:19:25.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46c77fe640 1 -- 172.21.15.142:0/992905322 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46c0014bc0 con 0x7f46d015c460 2024-09-12T12:19:25.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.684+0000 7f46c77fe640 1 -- 172.21.15.142:0/992905322 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f46c001d3c0 con 0x7f46d015c460 2024-09-12T12:19:25.689 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.688+0000 7f46c77fe640 1 -- 172.21.15.142:0/992905322 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f46d01576d0 con 0x7f46d015c460 2024-09-12T12:19:25.758 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.756+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e8fd6b02-9ed1-4355-9614-bb1195121930", "id": 0} v 0) -- 0x7f46d00cdce0 con 0x7f46d015c460 2024-09-12T12:19:25.765 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46c77fe640 1 -- 172.21.15.142:0/992905322 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8fd6b02-9ed1-4355-9614-bb1195121930", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f46c001e460 con 0x7f46d015c460 2024-09-12T12:19:25.765 INFO:teuthology.orchestra.run.smithi142.stdout:0 2024-09-12T12:19:25.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d015c460 msgr2=0x7f46d00c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d015c460 0x7f46d00c7b90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f46c0007e70 tx=0x7f46c000e4b0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 shutdown_connections 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46d015c460 0x7f46d00c7b90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46d00d0ea0 0x7f46d0157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 --2- 172.21.15.142:0/992905322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46d009b410 0x7f46d00c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 >> 172.21.15.142:0/992905322 conn(0x7f46d00b3e70 msgr2=0x7f46d0159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 shutdown_connections 2024-09-12T12:19:25.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.764+0000 7f46d5331640 1 -- 172.21.15.142:0/992905322 wait complete. 2024-09-12T12:19:25.783 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 3cf517e3-ffba-4602-a705-293666e71f8c 1 2024-09-12T12:19:25.877 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:25.874+0000 7f9893558180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:19:25.889 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:25.888+0000 7f70f0f4b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:19:25.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf52787640 1 Processor -- start 2024-09-12T12:19:25.905 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf52787640 1 -- start start 2024-09-12T12:19:25.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf52787640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf4c15b720 0x7fdf4c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf52787640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c0b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf52787640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4c0d0000 con 0x7fdf4c15b340 2024-09-12T12:19:25.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf52787640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdf4c0cc020 con 0x7fdf4c15b720 2024-09-12T12:19:25.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf52787640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c0cc300 con 0x7fdf4c0cfb70 2024-09-12T12:19:25.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf50d24640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c0b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf4b7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf4c15b720 0x7fdf4c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf4bfff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fdf4c15b340 0x7fdf4c15f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:39474/0 (socket says 172.21.15.142:39474) 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf4bfff640 1 -- 172.21.15.142:0/3877187491 learned_addr learned my addr 172.21.15.142:0/3877187491 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf50d24640 1 -- 172.21.15.142:0/3877187491 >> v2:172.21.15.142:3300/0 conn(0x7fdf4c15b720 msgr2=0x7fdf4c0b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf50d24640 1 --2- 172.21.15.142:0/3877187491 >> v2:172.21.15.142:3300/0 conn(0x7fdf4c15b720 0x7fdf4c0b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf50d24640 1 -- 172.21.15.142:0/3877187491 >> v1:172.21.15.142:6789/0 conn(0x7fdf4c15b340 legacy=0x7fdf4c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf50d24640 1 -- 172.21.15.142:0/3877187491 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c0cc5e0 con 0x7fdf4c0cfb70 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf4affd640 1 -- 172.21.15.142:0/3877187491 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828414473 0 0) 0x7fdf4c0d0000 con 0x7fdf4c15b340 2024-09-12T12:19:25.908 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.904+0000 7fdf4b7fe640 1 --2- 172.21.15.142:0/3877187491 >> v2:172.21.15.142:3300/0 conn(0x7fdf4c15b720 0x7fdf4c0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:25.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf50d24640 1 --2- 172.21.15.142:0/3877187491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c0b0b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009d90 tx=0x7fdf3c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cbd0fc7edbc43b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:25.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf4affd640 1 -- 172.21.15.142:0/3877187491 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0b2070 con 0x7fdf4c0cfb70 2024-09-12T12:19:25.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf4affd640 1 -- 172.21.15.142:0/3877187491 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c0a58e0 con 0x7fdf4c0cfb70 2024-09-12T12:19:25.910 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf4affd640 1 -- 172.21.15.142:0/3877187491 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0a5cb0 con 0x7fdf4c0cfb70 2024-09-12T12:19:25.910 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 -- 172.21.15.142:0/3877187491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 msgr2=0x7fdf4c0b0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.910 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 --2- 172.21.15.142:0/3877187491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c0b0b30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009d90 tx=0x7fdf3c01c8e0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.910 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 -- 172.21.15.142:0/3877187491 shutdown_connections 2024-09-12T12:19:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 --2- 172.21.15.142:0/3877187491 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c0b0b30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 --2- 172.21.15.142:0/3877187491 >> v2:172.21.15.142:3300/0 conn(0x7fdf4c15b720 0x7fdf4c0b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 -- 172.21.15.142:0/3877187491 >> 172.21.15.142:0/3877187491 conn(0x7fdf4c0b2d90 msgr2=0x7fdf4c0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 -- 172.21.15.142:0/3877187491 shutdown_connections 2024-09-12T12:19:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.908+0000 7fdf52787640 1 -- 172.21.15.142:0/3877187491 wait complete. 2024-09-12T12:19:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 Processor -- start 2024-09-12T12:19:25.911 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 -- start start 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c168ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf4c15b340 0x7fdf4c165910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf4c15b720 0x7fdf4c153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c0ce1c0 con 0x7fdf4c15b340 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c0ce040 con 0x7fdf4c15b720 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c0ce340 con 0x7fdf4c0cfb70 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf4c15b720 0x7fdf4c153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf4bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c168ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf4c15b720 0x7fdf4c153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44954/0 (socket says 172.21.15.142:44954) 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 -- 172.21.15.142:0/2797666354 learned_addr learned my addr 172.21.15.142:0/2797666354 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:25.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf4b7fe640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf4c15b340 0x7fdf4c165910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 -- 172.21.15.142:0/2797666354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 msgr2=0x7fdf4c168ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c168ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 -- 172.21.15.142:0/2797666354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf4c15b340 msgr2=0x7fdf4c165910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf4c15b340 0x7fdf4c165910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 -- 172.21.15.142:0/2797666354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c0b4620 con 0x7fdf4c15b720 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf50d24640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf4c15b720 0x7fdf4c153c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c01af90 tx=0x7fdf3c019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf48ff9640 1 -- 172.21.15.142:0/2797666354 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0bd020 con 0x7fdf4c15b720 2024-09-12T12:19:25.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf48ff9640 1 -- 172.21.15.142:0/2797666354 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c0b8670 con 0x7fdf4c15b720 2024-09-12T12:19:25.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c165110 con 0x7fdf4c15b720 2024-09-12T12:19:25.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf48ff9640 1 -- 172.21.15.142:0/2797666354 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0b2040 con 0x7fdf4c15b720 2024-09-12T12:19:25.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c16a310 con 0x7fdf4c15b720 2024-09-12T12:19:25.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf48ff9640 1 -- 172.21.15.142:0/2797666354 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf3c0c0020 con 0x7fdf4c15b720 2024-09-12T12:19:25.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf48ff9640 1 -- 172.21.15.142:0/2797666354 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fdf3c0adbe0 con 0x7fdf4c15b720 2024-09-12T12:19:25.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.912+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c0b0b30 con 0x7fdf4c15b720 2024-09-12T12:19:25.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.916+0000 7fdf48ff9640 1 -- 172.21.15.142:0/2797666354 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdf3c0c7030 con 0x7fdf4c15b720 2024-09-12T12:19:25.986 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.984+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3cf517e3-ffba-4602-a705-293666e71f8c", "id": 1} v 0) -- 0x7fdf4c1655c0 con 0x7fdf4c15b720 2024-09-12T12:19:25.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf48ff9640 1 -- 172.21.15.142:0/2797666354 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3cf517e3-ffba-4602-a705-293666e71f8c", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fdf3c016560 con 0x7fdf4c15b720 2024-09-12T12:19:25.992 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:19:25.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf4c15b720 msgr2=0x7fdf4c153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:25.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf4c15b720 0x7fdf4c153c60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c01af90 tx=0x7fdf3c019680 comp rx=0 tx=0).stop 2024-09-12T12:19:25.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 shutdown_connections 2024-09-12T12:19:25.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf4c15b720 0x7fdf4c153c60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf4c15b340 0x7fdf4c165910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 --2- 172.21.15.142:0/2797666354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf4c0cfb70 0x7fdf4c168ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:25.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 >> 172.21.15.142:0/2797666354 conn(0x7fdf4c0b2d90 msgr2=0x7fdf4c06c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:25.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 shutdown_connections 2024-09-12T12:19:25.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:25.992+0000 7fdf52787640 1 -- 172.21.15.142:0/2797666354 wait complete. 2024-09-12T12:19:26.010 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 1619cc70-8f41-4c98-bd54-f14a875ce4ca 2 2024-09-12T12:19:26.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7263f2a640 1 Processor -- start 2024-09-12T12:19:26.125 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7263f2a640 1 -- start start 2024-09-12T12:19:26.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7263f2a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c0b38b0 0x7f725c160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7263f2a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725c0d0310 con 0x7f725c06c3d0 2024-09-12T12:19:26.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7263f2a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725c0d05b0 con 0x7f725c15c500 2024-09-12T12:19:26.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7263f2a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f725c0d0890 con 0x7f725c0b38b0 2024-09-12T12:19:26.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7261cc6640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f725c15c500 0x7f725c06bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:39482/0 (socket says 172.21.15.142:39482) 2024-09-12T12:19:26.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f72624c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c0b38b0 0x7f725c160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.126 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7261cc6640 1 -- 172.21.15.142:0/850347814 learned_addr learned my addr 172.21.15.142:0/850347814 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-09-12T12:19:26.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f72624c7640 1 -- 172.21.15.142:0/850347814 >> v1:172.21.15.142:6789/0 conn(0x7f725c15c500 legacy=0x7f725c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f72624c7640 1 -- 172.21.15.142:0/850347814 >> v1:172.21.15.3:6789/0 conn(0x7f725c06c3d0 legacy=0x7f725c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:26.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f72624c7640 1 -- 172.21.15.142:0/850347814 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c0d0b70 con 0x7f725c0b38b0 2024-09-12T12:19:26.127 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.124+0000 7f7260cc4640 1 -- 172.21.15.142:0/850347814 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684577376 0 0) 0x7f725c0d05b0 con 0x7f725c15c500 2024-09-12T12:19:26.128 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f72624c7640 1 --2- 172.21.15.142:0/850347814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c0b38b0 0x7f725c160990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7254009dc0 tx=0x7f725401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b294cbbcc3476001 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7260cc4640 1 -- 172.21.15.142:0/850347814 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540b2070 con 0x7f725c0b38b0 2024-09-12T12:19:26.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7260cc4640 1 -- 172.21.15.142:0/850347814 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72540a5920 con 0x7f725c0b38b0 2024-09-12T12:19:26.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7260cc4640 1 -- 172.21.15.142:0/850347814 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540a5cf0 con 0x7f725c0b38b0 2024-09-12T12:19:26.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 -- 172.21.15.142:0/850347814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c0b38b0 msgr2=0x7f725c160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.129 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 --2- 172.21.15.142:0/850347814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c0b38b0 0x7f725c160990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7254009dc0 tx=0x7f725401c950 comp rx=0 tx=0).stop 2024-09-12T12:19:26.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 -- 172.21.15.142:0/850347814 shutdown_connections 2024-09-12T12:19:26.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 --2- 172.21.15.142:0/850347814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c0b38b0 0x7f725c160990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 -- 172.21.15.142:0/850347814 >> 172.21.15.142:0/850347814 conn(0x7f725c0b2d90 msgr2=0x7f725c0b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 -- 172.21.15.142:0/850347814 shutdown_connections 2024-09-12T12:19:26.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 -- 172.21.15.142:0/850347814 wait complete. 2024-09-12T12:19:26.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 Processor -- start 2024-09-12T12:19:26.130 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.128+0000 7f7263f2a640 1 -- start start 2024-09-12T12:19:26.132 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f725c06c3d0 0x7f725c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c0b38b0 0x7f725c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c15c500 0x7f725c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f725c0c9a00 con 0x7f725c0b38b0 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f725c0c9880 con 0x7f725c06c3d0 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f725c16a330 con 0x7f725c15c500 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c15c500 0x7f725c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7261cc6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f725c06c3d0 0x7f725c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c15c500 0x7f725c156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:44430/0 (socket says 172.21.15.142:44430) 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7261cc6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f725c06c3d0 0x7f725c1561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:44970/0 (socket says 172.21.15.142:44970) 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 -- 172.21.15.142:0/2079313817 learned_addr learned my addr 172.21.15.142:0/2079313817 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72614c5640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c0b38b0 0x7f725c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.133 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 -- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f725c06c3d0 msgr2=0x7f725c1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f725c06c3d0 0x7f725c1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 -- 172.21.15.142:0/2079313817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c0b38b0 msgr2=0x7f725c156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c0b38b0 0x7f725c156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 -- 172.21.15.142:0/2079313817 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c159360 con 0x7f725c15c500 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7261cc6640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f725c06c3d0 0x7f725c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72624c7640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c15c500 0x7f725c156db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f72540b0040 tx=0x7f72540196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72527fc640 1 -- 172.21.15.142:0/2079313817 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540bd020 con 0x7f725c15c500 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72527fc640 1 -- 172.21.15.142:0/2079313817 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72540ad990 con 0x7f725c15c500 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c159640 con 0x7f725c15c500 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72527fc640 1 -- 172.21.15.142:0/2079313817 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540b2040 con 0x7f725c15c500 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c159c00 con 0x7f725c15c500 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72527fc640 1 -- 172.21.15.142:0/2079313817 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f72540c0020 con 0x7f725c15c500 2024-09-12T12:19:26.134 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f72527fc640 1 -- 172.21.15.142:0/2079313817 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f72540b9840 con 0x7f725c15c500 2024-09-12T12:19:26.135 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.132+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c0b4090 con 0x7f725c15c500 2024-09-12T12:19:26.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.140+0000 7f72527fc640 1 -- 172.21.15.142:0/2079313817 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f72540c7030 con 0x7f725c15c500 2024-09-12T12:19:26.212 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.212+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1619cc70-8f41-4c98-bd54-f14a875ce4ca", "id": 2} v 0) -- 0x7f725c06dac0 con 0x7f725c15c500 2024-09-12T12:19:26.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f72527fc640 1 -- 172.21.15.142:0/2079313817 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1619cc70-8f41-4c98-bd54-f14a875ce4ca", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7254016540 con 0x7f725c15c500 2024-09-12T12:19:26.217 INFO:teuthology.orchestra.run.smithi142.stdout:2 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c15c500 msgr2=0x7f725c156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c15c500 0x7f725c156db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f72540b0040 tx=0x7f72540196f0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 shutdown_connections 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f725c15c500 0x7f725c156db0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f725c0b38b0 0x7f725c156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 --2- 172.21.15.142:0/2079313817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f725c06c3d0 0x7f725c1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 >> 172.21.15.142:0/2079313817 conn(0x7f725c0b2d90 msgr2=0x7f725c06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 shutdown_connections 2024-09-12T12:19:26.218 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.216+0000 7f7263f2a640 1 -- 172.21.15.142:0/2079313817 wait complete. 2024-09-12T12:19:26.233 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 837afb49-bced-4356-8bd9-d2641faf463c 3 2024-09-12T12:19:26.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.352+0000 7f3c67366640 1 Processor -- start 2024-09-12T12:19:26.354 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.352+0000 7f3c67366640 1 -- start start 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3c6015a890 0x7f3c600b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c600cee30 con 0x7f3c6015ac70 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c600b44a0 con 0x7f3c6006ef60 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3c600b4730 con 0x7f3c6015a890 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c65102640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3c6015a890 0x7f3c600b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c65903640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f3c6006ef60 0x7f3c600cf8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:39486/0 (socket says 172.21.15.142:39486) 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c65903640 1 -- 172.21.15.142:0/1067646635 learned_addr learned my addr 172.21.15.142:0/1067646635 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c65102640 1 -- 172.21.15.142:0/1067646635 >> v1:172.21.15.142:6789/0 conn(0x7f3c6006ef60 legacy=0x7f3c600cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c65102640 1 -- 172.21.15.142:0/1067646635 >> v1:172.21.15.3:6789/0 conn(0x7f3c6015ac70 legacy=0x7f3c6006e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:26.356 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c65102640 1 -- 172.21.15.142:0/1067646635 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c600b4a10 con 0x7f3c6015a890 2024-09-12T12:19:26.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c65102640 1 --2- 172.21.15.142:0/1067646635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c600b5980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0097c0 tx=0x7f3c5c01c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89445874a7078785 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c57fff640 1 -- 172.21.15.142:0/1067646635 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0b0070 con 0x7f3c6015a890 2024-09-12T12:19:26.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c57fff640 1 -- 172.21.15.142:0/1067646635 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c0ab600 con 0x7f3c6015a890 2024-09-12T12:19:26.358 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c57fff640 1 -- 172.21.15.142:0/1067646635 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0ab9d0 con 0x7f3c6015a890 2024-09-12T12:19:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- 172.21.15.142:0/1067646635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 msgr2=0x7f3c600b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 --2- 172.21.15.142:0/1067646635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c600b5980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c0097c0 tx=0x7f3c5c01c1f0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- 172.21.15.142:0/1067646635 shutdown_connections 2024-09-12T12:19:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 --2- 172.21.15.142:0/1067646635 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c600b5980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- 172.21.15.142:0/1067646635 >> 172.21.15.142:0/1067646635 conn(0x7f3c600b2c10 msgr2=0x7f3c600b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- 172.21.15.142:0/1067646635 shutdown_connections 2024-09-12T12:19:26.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.356+0000 7f3c67366640 1 -- 172.21.15.142:0/1067646635 wait complete. 2024-09-12T12:19:26.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 Processor -- start 2024-09-12T12:19:26.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 -- start start 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6006ef60 0x7f3c60141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c601431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6015ac70 0x7f3c6014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c600d0140 con 0x7f3c6015ac70 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3c600cffc0 con 0x7f3c6015a890 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3c600d02c0 con 0x7f3c6006ef60 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c64901640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c601431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6006ef60 0x7f3c60141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6006ef60 0x7f3c60141800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:44434/0 (socket says 172.21.15.142:44434) 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c64901640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c601431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45000/0 (socket says 172.21.15.142:45000) 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 -- 172.21.15.142:0/4024411670 learned_addr learned my addr 172.21.15.142:0/4024411670 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:26.363 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65903640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6015ac70 0x7f3c6014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 -- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 msgr2=0x7f3c601431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c601431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 -- 172.21.15.142:0/4024411670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6015ac70 msgr2=0x7f3c6014f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6015ac70 0x7f3c6014f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 -- 172.21.15.142:0/4024411670 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c60143c30 con 0x7f3c6006ef60 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65903640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6015ac70 0x7f3c6014f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c64901640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c601431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c65102640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6006ef60 0x7f3c60141800 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c018e00 tx=0x7f3c5c018f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c55ffb640 1 -- 172.21.15.142:0/4024411670 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0b8020 con 0x7f3c6006ef60 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c55ffb640 1 -- 172.21.15.142:0/4024411670 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5c0b4630 con 0x7f3c6006ef60 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c60143f10 con 0x7f3c6006ef60 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c55ffb640 1 -- 172.21.15.142:0/4024411670 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c0b0040 con 0x7f3c6006ef60 2024-09-12T12:19:26.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c601444d0 con 0x7f3c6006ef60 2024-09-12T12:19:26.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c55ffb640 1 -- 172.21.15.142:0/4024411670 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c5c0bb020 con 0x7f3c6006ef60 2024-09-12T12:19:26.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c55ffb640 1 -- 172.21.15.142:0/4024411670 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f3c5c0bdde0 con 0x7f3c6006ef60 2024-09-12T12:19:26.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.360+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c600d0680 con 0x7f3c6006ef60 2024-09-12T12:19:26.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.364+0000 7f3c55ffb640 1 -- 172.21.15.142:0/4024411670 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3c5c0c2030 con 0x7f3c6006ef60 2024-09-12T12:19:26.434 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.432+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "837afb49-bced-4356-8bd9-d2641faf463c", "id": 3} v 0) -- 0x7f3c60153850 con 0x7f3c6006ef60 2024-09-12T12:19:26.440 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c55ffb640 1 -- 172.21.15.142:0/4024411670 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "837afb49-bced-4356-8bd9-d2641faf463c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3c600cee30 con 0x7f3c6006ef60 2024-09-12T12:19:26.440 INFO:teuthology.orchestra.run.smithi142.stdout:3 2024-09-12T12:19:26.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6006ef60 msgr2=0x7f3c60141800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6006ef60 0x7f3c60141800 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c018e00 tx=0x7f3c5c018f90 comp rx=0 tx=0).stop 2024-09-12T12:19:26.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 shutdown_connections 2024-09-12T12:19:26.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6015ac70 0x7f3c6014f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.442 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015a890 0x7f3c601431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 --2- 172.21.15.142:0/4024411670 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6006ef60 0x7f3c60141800 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 >> 172.21.15.142:0/4024411670 conn(0x7f3c600b2c10 msgr2=0x7f3c60152450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 shutdown_connections 2024-09-12T12:19:26.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.440+0000 7f3c67366640 1 -- 172.21.15.142:0/4024411670 wait complete. 2024-09-12T12:19:26.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd 4 2024-09-12T12:19:26.460 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.458+0000 7f9893558180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:19:26.478 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:26.476+0000 7f70f0f4b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:19:26.516 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.514+0000 7f9893558180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:19:26.533 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:26.532+0000 7f70f0f4b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:19:26.570 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.568+0000 7f079e66d640 1 Processor -- start 2024-09-12T12:19:26.571 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.568+0000 7f079e66d640 1 -- start start 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f079815f0f0 0x7f079809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07980b48b0 0x7f079815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f079815bf50 con 0x7f079815f4d0 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f079815fd10 con 0x7f079815f0f0 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f079815c230 con 0x7f07980b48b0 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f079815f0f0 0x7f079809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079cc0a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07980b48b0 0x7f079815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f079815f0f0 0x7f079809aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45014/0 (socket says 172.21.15.142:45014) 2024-09-12T12:19:26.572 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 -- 172.21.15.142:0/3073432129 learned_addr learned my addr 172.21.15.142:0/3073432129 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:26.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 -- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07980b48b0 msgr2=0x7f079815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:26.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 --2- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07980b48b0 0x7f079815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 -- 172.21.15.142:0/3073432129 >> v1:172.21.15.3:6789/0 conn(0x7f079815f4d0 legacy=0x7f07980b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:26.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 -- 172.21.15.142:0/3073432129 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079815c510 con 0x7f079815f0f0 2024-09-12T12:19:26.573 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079cc0a640 1 --2- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07980b48b0 0x7f079815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:26.573 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.570+0000 7f9893558180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:19:26.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0797fff640 1 --2- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f0f0 0x7f079809aa70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f078c009870 tx=0x7f078c01c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f10e6c1a127cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0796ffd640 1 -- 172.21.15.142:0/3073432129 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0b0070 con 0x7f079815f0f0 2024-09-12T12:19:26.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0796ffd640 1 -- 172.21.15.142:0/3073432129 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078c0ab600 con 0x7f079815f0f0 2024-09-12T12:19:26.574 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f0796ffd640 1 -- 172.21.15.142:0/3073432129 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0ab9d0 con 0x7f079815f0f0 2024-09-12T12:19:26.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f0f0 msgr2=0x7f079809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 --2- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f0f0 0x7f079809aa70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f078c009870 tx=0x7f078c01c520 comp rx=0 tx=0).stop 2024-09-12T12:19:26.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- 172.21.15.142:0/3073432129 shutdown_connections 2024-09-12T12:19:26.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 --2- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07980b48b0 0x7f079815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.575 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 --2- 172.21.15.142:0/3073432129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f0f0 0x7f079809aa70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- 172.21.15.142:0/3073432129 >> 172.21.15.142:0/3073432129 conn(0x7f0798073410 msgr2=0x7f07980b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- 172.21.15.142:0/3073432129 shutdown_connections 2024-09-12T12:19:26.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.572+0000 7f079e66d640 1 -- 172.21.15.142:0/3073432129 wait complete. 2024-09-12T12:19:26.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 Processor -- start 2024-09-12T12:19:26.576 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 -- start start 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07980b48b0 0x7f07981568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f079815f0f0 0x7f079814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f4d0 0x7f079806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f079815e110 con 0x7f07980b48b0 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f079815df90 con 0x7f079815f4d0 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f079815e290 con 0x7f079815f0f0 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f07977fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f079815f0f0 0x7f079814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f4d0 0x7f079806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f4d0 0x7f079806f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45024/0 (socket says 172.21.15.142:45024) 2024-09-12T12:19:26.579 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 -- 172.21.15.142:0/118520993 learned_addr learned my addr 172.21.15.142:0/118520993 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f07977fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f079815f0f0 0x7f079814b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:44448/0 (socket says 172.21.15.142:44448) 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f0797fff640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07980b48b0 0x7f07981568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 -- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f079815f0f0 msgr2=0x7f079814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f079815f0f0 0x7f079814b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 -- 172.21.15.142:0/118520993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07980b48b0 msgr2=0x7f07981568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07980b48b0 0x7f07981568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 -- 172.21.15.142:0/118520993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07981f7920 con 0x7f079815f4d0 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f07977fe640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f079815f0f0 0x7f079814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079cc0a640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f4d0 0x7f079806f980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0790009f00 tx=0x7f079000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f0794ff9640 1 -- 172.21.15.142:0/118520993 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790018030 con 0x7f079815f4d0 2024-09-12T12:19:26.580 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f0794ff9640 1 -- 172.21.15.142:0/118520993 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079000ce50 con 0x7f079815f4d0 2024-09-12T12:19:26.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07981f7be0 con 0x7f079815f4d0 2024-09-12T12:19:26.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f0794ff9640 1 -- 172.21.15.142:0/118520993 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790003bc0 con 0x7f079815f4d0 2024-09-12T12:19:26.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07981f8480 con 0x7f079815f4d0 2024-09-12T12:19:26.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f0794ff9640 1 -- 172.21.15.142:0/118520993 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f079001b070 con 0x7f079815f4d0 2024-09-12T12:19:26.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f0794ff9640 1 -- 172.21.15.142:0/118520993 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f0790014990 con 0x7f079815f4d0 2024-09-12T12:19:26.581 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.576+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07981f7db0 con 0x7f079815f4d0 2024-09-12T12:19:26.582 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.580+0000 7f0794ff9640 1 -- 172.21.15.142:0/118520993 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0790010040 con 0x7f079815f4d0 2024-09-12T12:19:26.588 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:26.588+0000 7f70f0f4b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:19:26.651 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.648+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd", "id": 4} v 0) -- 0x7f079815cff0 con 0x7f079815f4d0 2024-09-12T12:19:26.652 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.650+0000 7f9893558180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:19:26.656 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f0794ff9640 1 -- 172.21.15.142:0/118520993 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f0790014c70 con 0x7f079815f4d0 2024-09-12T12:19:26.656 INFO:teuthology.orchestra.run.smithi142.stdout:4 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f4d0 msgr2=0x7f079806f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f4d0 0x7f079806f980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0790009f00 tx=0x7f079000c490 comp rx=0 tx=0).stop 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 shutdown_connections 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f079815f4d0 0x7f079806f980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f079815f0f0 0x7f079814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 --2- 172.21.15.142:0/118520993 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07980b48b0 0x7f07981568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 >> 172.21.15.142:0/118520993 conn(0x7f0798073410 msgr2=0x7f07980b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 shutdown_connections 2024-09-12T12:19:26.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.656+0000 7f079e66d640 1 -- 172.21.15.142:0/118520993 wait complete. 2024-09-12T12:19:26.667 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:26.664+0000 7f70f0f4b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:19:26.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3 5 2024-09-12T12:19:26.706 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.702+0000 7f9893558180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:19:26.720 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:26.720+0000 7f70f0f4b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:19:26.787 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.786+0000 7f9893558180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:19:26.799 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:26.796+0000 7f70f0f4b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:19:26.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.824+0000 7f1118ece640 1 Processor -- start 2024-09-12T12:19:26.827 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.824+0000 7f1118ece640 1 -- start start 2024-09-12T12:19:26.828 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f111406d7e0 0x7f111406dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11140cd050 0x7f11140cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11140ccb50 con 0x7f111415e7a0 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f111406e150 con 0x7f111406d7e0 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f111409b5b0 con 0x7f11140cd050 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11140cd050 0x7f11140cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11140cd050 0x7f11140cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45038/0 (socket says 172.21.15.142:45038) 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 -- 172.21.15.142:0/2172208862 learned_addr learned my addr 172.21.15.142:0/2172208862 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1111d9b640 1 --2- 172.21.15.142:0/2172208862 >> v2:172.21.15.3:3300/0 conn(0x7f111406d7e0 0x7f111406dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 -- 172.21.15.142:0/2172208862 >> v2:172.21.15.3:3300/0 conn(0x7f111406d7e0 msgr2=0x7f111406dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 --2- 172.21.15.142:0/2172208862 >> v2:172.21.15.3:3300/0 conn(0x7f111406d7e0 0x7f111406dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 -- 172.21.15.142:0/2172208862 >> v1:172.21.15.3:6789/0 conn(0x7f111415e7a0 legacy=0x7f111406cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:26.829 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 -- 172.21.15.142:0/2172208862 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111409b890 con 0x7f11140cd050 2024-09-12T12:19:26.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1112d9d640 1 --2- 172.21.15.142:0/2172208862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11140cd050 0x7f11140cd4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11140d2330 tx=0x7f110c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8e12eb0ac31a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f111159a640 1 -- 172.21.15.142:0/2172208862 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0a5770 con 0x7f11140cd050 2024-09-12T12:19:26.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f111159a640 1 -- 172.21.15.142:0/2172208862 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110c0a5910 con 0x7f11140cd050 2024-09-12T12:19:26.831 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f111159a640 1 -- 172.21.15.142:0/2172208862 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c0a5ce0 con 0x7f11140cd050 2024-09-12T12:19:26.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 -- 172.21.15.142:0/2172208862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11140cd050 msgr2=0x7f11140cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 --2- 172.21.15.142:0/2172208862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11140cd050 0x7f11140cd4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11140d2330 tx=0x7f110c01c890 comp rx=0 tx=0).stop 2024-09-12T12:19:26.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 -- 172.21.15.142:0/2172208862 shutdown_connections 2024-09-12T12:19:26.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 --2- 172.21.15.142:0/2172208862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11140cd050 0x7f11140cd4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 --2- 172.21.15.142:0/2172208862 >> v2:172.21.15.3:3300/0 conn(0x7f111406d7e0 0x7f111406dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.832 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.828+0000 7f1118ece640 1 -- 172.21.15.142:0/2172208862 >> 172.21.15.142:0/2172208862 conn(0x7f11140b3e70 msgr2=0x7f11140b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- 172.21.15.142:0/2172208862 shutdown_connections 2024-09-12T12:19:26.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- 172.21.15.142:0/2172208862 wait complete. 2024-09-12T12:19:26.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 Processor -- start 2024-09-12T12:19:26.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- start start 2024-09-12T12:19:26.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f111406d7e0 0x7f111414eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11140cd050 0x7f11141fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111415e7a0 0x7f11141571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:26.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f111416b2e0 con 0x7f111415e7a0 2024-09-12T12:19:26.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f111416b160 con 0x7f111406d7e0 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f111416b460 con 0x7f11140cd050 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11140cd050 0x7f11141fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f111259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f111406d7e0 0x7f111414eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11140cd050 0x7f11141fb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:44454/0 (socket says 172.21.15.142:44454) 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 -- 172.21.15.142:0/3058956138 learned_addr learned my addr 172.21.15.142:0/3058956138 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f111259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f111406d7e0 0x7f111414eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45042/0 (socket says 172.21.15.142:45042) 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1112d9d640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111415e7a0 0x7f11141571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 -- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f111406d7e0 msgr2=0x7f111414eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f111406d7e0 0x7f111414eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 -- 172.21.15.142:0/3058956138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111415e7a0 msgr2=0x7f11141571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111415e7a0 0x7f11141571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 -- 172.21.15.142:0/3058956138 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1114156260 con 0x7f11140cd050 2024-09-12T12:19:26.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f111259c640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f111406d7e0 0x7f111414eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1111d9b640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11140cd050 0x7f11141fb2e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f110400c820 tx=0x7f110400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1102ffd640 1 -- 172.21.15.142:0/3058956138 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104015070 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1102ffd640 1 -- 172.21.15.142:0/3058956138 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1104007a30 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1114156540 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1102ffd640 1 -- 172.21.15.142:0/3058956138 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104007e20 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f111414f230 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1102ffd640 1 -- 172.21.15.142:0/3058956138 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1104018070 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.832+0000 7f1102ffd640 1 -- 172.21.15.142:0/3058956138 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f1104011970 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.836+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111406dc10 con 0x7f11140cd050 2024-09-12T12:19:26.837 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.836+0000 7f1102ffd640 1 -- 172.21.15.142:0/3058956138 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1104011c90 con 0x7f11140cd050 2024-09-12T12:19:26.842 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.838+0000 7f9893558180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:19:26.854 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:26.852+0000 7f70f0f4b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:19:26.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.900+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3", "id": 5} v 0) -- 0x7f11140c8f50 con 0x7f11140cd050 2024-09-12T12:19:26.907 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1102ffd640 1 -- 172.21.15.142:0/3058956138 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f11040113f0 con 0x7f11140cd050 2024-09-12T12:19:26.908 INFO:teuthology.orchestra.run.smithi142.stdout:5 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11140cd050 msgr2=0x7f11141fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11140cd050 0x7f11141fb2e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f110400c820 tx=0x7f110400cce0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 shutdown_connections 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f111415e7a0 0x7f11141571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11140cd050 0x7f11141fb2e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 --2- 172.21.15.142:0/3058956138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f111406d7e0 0x7f111414eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 >> 172.21.15.142:0/3058956138 conn(0x7f11140b3e70 msgr2=0x7f11141545c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:26.909 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 shutdown_connections 2024-09-12T12:19:26.910 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:26.908+0000 7f1118ece640 1 -- 172.21.15.142:0/3058956138 wait complete. 2024-09-12T12:19:26.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new db6c35a4-95c7-4815-98ec-b9c3c6e99cd6 6 2024-09-12T12:19:27.000 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:26.998+0000 7f9893558180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:19:27.010 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:27.008+0000 7f70f0f4b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:19:27.064 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:27.062+0000 7f9893558180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:19:27.073 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:27.072+0000 7f70f0f4b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:19:27.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.084+0000 7fe412258640 1 Processor -- start 2024-09-12T12:19:27.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.084+0000 7fe412258640 1 -- start start 2024-09-12T12:19:27.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe40c15a890 0x7fe40c0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe40c15ac70 0x7fe40c06e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe40c0cee30 con 0x7fe40c06ed90 2024-09-12T12:19:27.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe40c0d08c0 con 0x7fe40c15a890 2024-09-12T12:19:27.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe40c06f1a0 con 0x7fe40c15ac70 2024-09-12T12:19:27.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe402ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe40c15ac70 0x7fe40c06e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.088 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe40bfff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe40c06ed90 0x7fe40c06bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:39492/0 (socket says 172.21.15.142:39492) 2024-09-12T12:19:27.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe40bfff640 1 -- 172.21.15.142:0/472611367 learned_addr learned my addr 172.21.15.142:0/472611367 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-09-12T12:19:27.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe40b7fe640 1 --2- 172.21.15.142:0/472611367 >> v2:172.21.15.3:3300/0 conn(0x7fe40c15a890 0x7fe40c0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe402ffd640 1 -- 172.21.15.142:0/472611367 >> v2:172.21.15.3:3300/0 conn(0x7fe40c15a890 msgr2=0x7fe40c0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:27.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe402ffd640 1 --2- 172.21.15.142:0/472611367 >> v2:172.21.15.3:3300/0 conn(0x7fe40c15a890 0x7fe40c0b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe402ffd640 1 -- 172.21.15.142:0/472611367 >> v1:172.21.15.142:6789/0 conn(0x7fe40c06ed90 legacy=0x7fe40c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.089 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe402ffd640 1 -- 172.21.15.142:0/472611367 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c0b44a0 con 0x7fe40c15ac70 2024-09-12T12:19:27.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe402ffd640 1 --2- 172.21.15.142:0/472611367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c15ac70 0x7fe40c06e850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe40c073bd0 tx=0x7fe3f401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0ec8f7592090ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:27.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe40affd640 1 -- 172.21.15.142:0/472611367 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40b0070 con 0x7fe40c15ac70 2024-09-12T12:19:27.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe40affd640 1 -- 172.21.15.142:0/472611367 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f40ab400 con 0x7fe40c15ac70 2024-09-12T12:19:27.090 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe40affd640 1 -- 172.21.15.142:0/472611367 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40ab7d0 con 0x7fe40c15ac70 2024-09-12T12:19:27.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- 172.21.15.142:0/472611367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c15ac70 msgr2=0x7fe40c06e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 --2- 172.21.15.142:0/472611367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c15ac70 0x7fe40c06e850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe40c073bd0 tx=0x7fe3f401c3f0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- 172.21.15.142:0/472611367 shutdown_connections 2024-09-12T12:19:27.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 --2- 172.21.15.142:0/472611367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c15ac70 0x7fe40c06e850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 --2- 172.21.15.142:0/472611367 >> v2:172.21.15.3:3300/0 conn(0x7fe40c15a890 0x7fe40c0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- 172.21.15.142:0/472611367 >> 172.21.15.142:0/472611367 conn(0x7fe40c0b2c10 msgr2=0x7fe40c0b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:27.091 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- 172.21.15.142:0/472611367 shutdown_connections 2024-09-12T12:19:27.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.088+0000 7fe412258640 1 -- 172.21.15.142:0/472611367 wait complete. 2024-09-12T12:19:27.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 Processor -- start 2024-09-12T12:19:27.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 -- start start 2024-09-12T12:19:27.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c06ed90 0x7fe40c149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c15a890 0x7fe40c144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.094 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe40c15ac70 0x7fe40c151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c0b5ec0 con 0x7fe40c15a890 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c06fe30 con 0x7fe40c06ed90 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe40c0b6040 con 0x7fe40c15ac70 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c06ed90 0x7fe40c149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c06ed90 0x7fe40c149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45060/0 (socket says 172.21.15.142:45060) 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe40c15ac70 0x7fe40c151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 -- 172.21.15.142:0/1458177632 learned_addr learned my addr 172.21.15.142:0/1458177632 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe402ffd640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c15a890 0x7fe40c144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 -- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe40c15ac70 msgr2=0x7fe40c151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe40c15ac70 0x7fe40c151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 -- 172.21.15.142:0/1458177632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c15a890 msgr2=0x7fe40c144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c15a890 0x7fe40c144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.095 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 -- 172.21.15.142:0/1458177632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c1504a0 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40bfff640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe40c15ac70 0x7fe40c151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe40b7fe640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c06ed90 0x7fe40c149150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00cdf0 tx=0x7fe3fc00a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe408ff9640 1 -- 172.21.15.142:0/1458177632 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc003ba0 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe408ff9640 1 -- 172.21.15.142:0/1458177632 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3fc003d60 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c147a50 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe408ff9640 1 -- 172.21.15.142:0/1458177632 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc004260 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c147fd0 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe408ff9640 1 -- 172.21.15.142:0/1458177632 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3fc01f440 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe408ff9640 1 -- 172.21.15.142:0/1458177632 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe3fc016bb0 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.092+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40c0b5980 con 0x7fe40c06ed90 2024-09-12T12:19:27.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.096+0000 7fe408ff9640 1 -- 172.21.15.142:0/1458177632 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe3fc004790 con 0x7fe40c06ed90 2024-09-12T12:19:27.165 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.164+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "db6c35a4-95c7-4815-98ec-b9c3c6e99cd6", "id": 6} v 0) -- 0x7fe40c146030 con 0x7fe40c06ed90 2024-09-12T12:19:27.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.168+0000 7fe408ff9640 1 -- 172.21.15.142:0/1458177632 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db6c35a4-95c7-4815-98ec-b9c3c6e99cd6", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe3fc0165c0 con 0x7fe40c06ed90 2024-09-12T12:19:27.170 INFO:teuthology.orchestra.run.smithi142.stdout:6 2024-09-12T12:19:27.171 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c06ed90 msgr2=0x7fe40c149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c06ed90 0x7fe40c149150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00cdf0 tx=0x7fe3fc00a660 comp rx=0 tx=0).stop 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 shutdown_connections 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe40c15ac70 0x7fe40c151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe40c15a890 0x7fe40c144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 --2- 172.21.15.142:0/1458177632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe40c06ed90 0x7fe40c149150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 >> 172.21.15.142:0/1458177632 conn(0x7fe40c0b2c10 msgr2=0x7fe40c146550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 shutdown_connections 2024-09-12T12:19:27.172 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.172+0000 7fe412258640 1 -- 172.21.15.142:0/1458177632 wait complete. 2024-09-12T12:19:27.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 7f348809-04cd-4d2b-abeb-06c3c1561d1e 7 2024-09-12T12:19:27.342 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2acde640 1 Processor -- start 2024-09-12T12:19:27.342 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2acde640 1 -- start start 2024-09-12T12:19:27.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2acde640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2406e850 0x7ffa2415c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2acde640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa2415df60 0x7ffa2415e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2acde640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa2415ce10 con 0x7ffa2406fb50 2024-09-12T12:19:27.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2acde640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2415d0f0 con 0x7ffa2415df60 2024-09-12T12:19:27.343 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2acde640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2415d3d0 con 0x7ffa2406e850 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa28a7a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2406e850 0x7ffa2415c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2927b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa2415df60 0x7ffa2415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa28a7a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2406e850 0x7ffa2415c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:44464/0 (socket says 172.21.15.142:44464) 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa2927b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa2415df60 0x7ffa2415e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:45068/0 (socket says 172.21.15.142:45068) 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.340+0000 7ffa28a7a640 1 -- 172.21.15.142:0/264463196 learned_addr learned my addr 172.21.15.142:0/264463196 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa28a7a640 1 -- 172.21.15.142:0/264463196 >> v2:172.21.15.142:3300/0 conn(0x7ffa2415df60 msgr2=0x7ffa2415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa28a7a640 1 --2- 172.21.15.142:0/264463196 >> v2:172.21.15.142:3300/0 conn(0x7ffa2415df60 0x7ffa2415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa28a7a640 1 -- 172.21.15.142:0/264463196 >> v1:172.21.15.3:6789/0 conn(0x7ffa2406fb50 legacy=0x7ffa2406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:27.344 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa28a7a640 1 -- 172.21.15.142:0/264463196 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2415d6b0 con 0x7ffa2406e850 2024-09-12T12:19:27.345 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa28a7a640 1 --2- 172.21.15.142:0/264463196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2406e850 0x7ffa2415c7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009e50 tx=0x7ffa1c01c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b20690d6d590e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:27.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa237fe640 1 -- 172.21.15.142:0/264463196 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0a5670 con 0x7ffa2406e850 2024-09-12T12:19:27.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa237fe640 1 -- 172.21.15.142:0/264463196 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c0a5810 con 0x7ffa2406e850 2024-09-12T12:19:27.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa237fe640 1 -- 172.21.15.142:0/264463196 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0a5be0 con 0x7ffa2406e850 2024-09-12T12:19:27.346 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 -- 172.21.15.142:0/264463196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2406e850 msgr2=0x7ffa2415c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 --2- 172.21.15.142:0/264463196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2406e850 0x7ffa2415c7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009e50 tx=0x7ffa1c01c9f0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 -- 172.21.15.142:0/264463196 shutdown_connections 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 --2- 172.21.15.142:0/264463196 >> v2:172.21.15.142:3300/0 conn(0x7ffa2415df60 0x7ffa2415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 --2- 172.21.15.142:0/264463196 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2406e850 0x7ffa2415c7e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 -- 172.21.15.142:0/264463196 >> 172.21.15.142:0/264463196 conn(0x7ffa240b2c10 msgr2=0x7ffa240b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 -- 172.21.15.142:0/264463196 shutdown_connections 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.344+0000 7ffa2acde640 1 -- 172.21.15.142:0/264463196 wait complete. 2024-09-12T12:19:27.347 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 Processor -- start 2024-09-12T12:19:27.348 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 -- start start 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2406e850 0x7ffa2414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa2406fb50 0x7ffa241591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2415df60 0x7ffa24159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241691b0 con 0x7ffa2406fb50 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa24169030 con 0x7ffa2406e850 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa24169330 con 0x7ffa2415df60 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2415df60 0x7ffa24159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.349 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa28a7a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2406e850 0x7ffa2414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2415df60 0x7ffa24159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:44472/0 (socket says 172.21.15.142:44472) 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 -- 172.21.15.142:0/3579217237 learned_addr learned my addr 172.21.15.142:0/3579217237 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 -- 172.21.15.142:0/3579217237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2406e850 msgr2=0x7ffa2414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2406e850 0x7ffa2414f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 -- 172.21.15.142:0/3579217237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa2406fb50 msgr2=0x7ffa241591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa23fff640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa2406fb50 0x7ffa241591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa2406fb50 0x7ffa241591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 -- 172.21.15.142:0/3579217237 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa24154660 con 0x7ffa2415df60 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2927b640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2415df60 0x7ffa24159700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400e7f0 tx=0x7ffa1400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:27.350 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa217fa640 1 -- 172.21.15.142:0/3579217237 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14017070 con 0x7ffa2415df60 2024-09-12T12:19:27.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa217fa640 1 -- 172.21.15.142:0/3579217237 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1400c940 con 0x7ffa2415df60 2024-09-12T12:19:27.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa217fa640 1 -- 172.21.15.142:0/3579217237 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400eea0 con 0x7ffa2415df60 2024-09-12T12:19:27.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa240ad980 con 0x7ffa2415df60 2024-09-12T12:19:27.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa240adf00 con 0x7ffa2415df60 2024-09-12T12:19:27.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa217fa640 1 -- 172.21.15.142:0/3579217237 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa1401a070 con 0x7ffa2415df60 2024-09-12T12:19:27.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa217fa640 1 -- 172.21.15.142:0/3579217237 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ffa14013ad0 con 0x7ffa2415df60 2024-09-12T12:19:27.351 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.348+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2415c7e0 con 0x7ffa2415df60 2024-09-12T12:19:27.352 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.352+0000 7ffa217fa640 1 -- 172.21.15.142:0/3579217237 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ffa14006000 con 0x7ffa2415df60 2024-09-12T12:19:27.451 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.452+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7f348809-04cd-4d2b-abeb-06c3c1561d1e", "id": 7} v 0) -- 0x7ffa240b2040 con 0x7ffa2415df60 2024-09-12T12:19:27.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.456+0000 7ffa217fa640 1 -- 172.21.15.142:0/3579217237 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7f348809-04cd-4d2b-abeb-06c3c1561d1e", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ffa1400cdb0 con 0x7ffa2415df60 2024-09-12T12:19:27.458 INFO:teuthology.orchestra.run.smithi142.stdout:7 2024-09-12T12:19:27.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2415df60 msgr2=0x7ffa24159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2415df60 0x7ffa24159700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400e7f0 tx=0x7ffa1400ecb0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 shutdown_connections 2024-09-12T12:19:27.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa2415df60 0x7ffa24159700 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa2406fb50 0x7ffa241591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 --2- 172.21.15.142:0/3579217237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2406e850 0x7ffa2414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 >> 172.21.15.142:0/3579217237 conn(0x7ffa240b2c10 msgr2=0x7ffa240b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:27.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 shutdown_connections 2024-09-12T12:19:27.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:19:27.460+0000 7ffa2acde640 1 -- 172.21.15.142:0/3579217237 wait complete. 2024-09-12T12:19:27.483 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-12T12:19:27.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-12T12:19:27.485 INFO:tasks.ceph.osd.0:Started 2024-09-12T12:19:27.485 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-12T12:19:27.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-12T12:19:27.487 INFO:tasks.ceph.osd.1:Started 2024-09-12T12:19:27.487 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-12T12:19:27.488 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-12T12:19:27.489 INFO:tasks.ceph.osd.2:Started 2024-09-12T12:19:27.490 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-12T12:19:27.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-12T12:19:27.492 INFO:tasks.ceph.osd.3:Started 2024-09-12T12:19:27.492 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-12T12:19:27.492 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-12T12:19:27.494 INFO:tasks.ceph.osd.4:Started 2024-09-12T12:19:27.494 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-12T12:19:27.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-12T12:19:27.496 INFO:tasks.ceph.osd.5:Started 2024-09-12T12:19:27.496 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-12T12:19:27.496 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-12T12:19:27.497 INFO:tasks.ceph.osd.6:Started 2024-09-12T12:19:27.498 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-12T12:19:27.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-12T12:19:27.500 INFO:tasks.ceph.osd.7:Started 2024-09-12T12:19:27.500 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f1970ec4640 1 Processor -- start 2024-09-12T12:19:27.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f1970ec4640 1 -- start start 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f1970ec4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f196c0cfbe0 0x7f196c1695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f1970ec4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196c0d31a0 con 0x7f196c15e5d0 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f1970ec4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196c0d3480 con 0x7f196c0d5050 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f1970ec4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f196c0b1060 con 0x7f196c0cfbe0 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f196a59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f196c15e5d0 0x7f196c0ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39604/0 (socket says 172.21.15.3:39604) 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.630+0000 7f196a59c640 1 -- 172.21.15.3:0/348204634 learned_addr learned my addr 172.21.15.3:0/348204634 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196ad9d640 1 --2- 172.21.15.3:0/348204634 >> v2:172.21.15.142:3300/0 conn(0x7f196c0cfbe0 0x7f196c1695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196959a640 1 -- 172.21.15.3:0/348204634 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973552332 0 0) 0x7f196c0d31a0 con 0x7f196c15e5d0 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196959a640 1 -- 172.21.15.3:0/348204634 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1958003530 con 0x7f196c15e5d0 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196959a640 1 -- 172.21.15.3:0/348204634 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819237015 0 0) 0x7f196c0d3480 con 0x7f196c0d5050 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196959a640 1 -- 172.21.15.3:0/348204634 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f196c0d31a0 con 0x7f196c0d5050 2024-09-12T12:19:27.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196ad9d640 1 -- 172.21.15.3:0/348204634 >> v1:172.21.15.142:6789/0 conn(0x7f196c0d5050 legacy=0x7f196c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196ad9d640 1 -- 172.21.15.3:0/348204634 >> v1:172.21.15.3:6789/0 conn(0x7f196c15e5d0 legacy=0x7f196c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196ad9d640 1 -- 172.21.15.3:0/348204634 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c0d3760 con 0x7f196c0cfbe0 2024-09-12T12:19:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196ad9d640 1 --2- 172.21.15.3:0/348204634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 0x7f196c1695b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f195c0097c0 tx=0x7f195c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf68128435bde3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196959a640 1 -- 172.21.15.3:0/348204634 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0b0070 con 0x7f196c0cfbe0 2024-09-12T12:19:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196959a640 1 -- 172.21.15.3:0/348204634 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195c0ab600 con 0x7f196c0cfbe0 2024-09-12T12:19:27.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f196959a640 1 -- 172.21.15.3:0/348204634 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0ab9d0 con 0x7f196c0cfbe0 2024-09-12T12:19:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f1970ec4640 1 -- 172.21.15.3:0/348204634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 msgr2=0x7f196c1695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f1970ec4640 1 --2- 172.21.15.3:0/348204634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 0x7f196c1695b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f195c0097c0 tx=0x7f195c01c410 comp rx=0 tx=0).stop 2024-09-12T12:19:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f1970ec4640 1 -- 172.21.15.3:0/348204634 shutdown_connections 2024-09-12T12:19:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f1970ec4640 1 --2- 172.21.15.3:0/348204634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 0x7f196c1695b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f1970ec4640 1 -- 172.21.15.3:0/348204634 >> 172.21.15.3:0/348204634 conn(0x7f196c08c000 msgr2=0x7f196c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:27.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f1970ec4640 1 -- 172.21.15.3:0/348204634 shutdown_connections 2024-09-12T12:19:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.634+0000 7f1970ec4640 1 -- 172.21.15.3:0/348204634 wait complete. 2024-09-12T12:19:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 Processor -- start 2024-09-12T12:19:27.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 -- start start 2024-09-12T12:19:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 0x7f196c159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196c0d5050 0x7f196c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f196c15e5d0 0x7f196c0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f196c16cab0 con 0x7f196c15e5d0 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f196c16c930 con 0x7f196c0cfbe0 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f196c16cc30 con 0x7f196c0d5050 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f196c15e5d0 0x7f196c0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f196c15e5d0 0x7f196c0c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56432/0 (socket says 172.21.15.3:56432) 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 -- 172.21.15.3:0/2958219411 learned_addr learned my addr 172.21.15.3:0/2958219411 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196a59c640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 0x7f196c159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1969d9b640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196c0d5050 0x7f196c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 -- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196c0d5050 msgr2=0x7f196c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196c0d5050 0x7f196c159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 -- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 msgr2=0x7f196c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 0x7f196c159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 -- 172.21.15.3:0/2958219411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c0c9fb0 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f196ad9d640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f196c15e5d0 0x7f196c0c9130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f195c0097c0 tx=0x7f195c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1962ffd640 1 -- 172.21.15.3:0/2958219411 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0b8020 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1962ffd640 1 -- 172.21.15.3:0/2958219411 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195c0b4610 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c0ca290 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1962ffd640 1 -- 172.21.15.3:0/2958219411 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0b0040 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c0ca850 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1962ffd640 1 -- 172.21.15.3:0/2958219411 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f195c0bb020 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1962ffd640 1 -- 172.21.15.3:0/2958219411 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f195c0aaa70 con 0x7f196c15e5d0 2024-09-12T12:19:27.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.638+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c0cf4d0 con 0x7f196c15e5d0 2024-09-12T12:19:27.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.642+0000 7f1962ffd640 1 -- 172.21.15.3:0/2958219411 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f195c0c2030 con 0x7f196c15e5d0 2024-09-12T12:19:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.698+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f196c0ba490 con 0x7f196c15e5d0 2024-09-12T12:19:27.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1962ffd640 1 -- 172.21.15.3:0/2958219411 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f195c016090 con 0x7f196c15e5d0 2024-09-12T12:19:27.703 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:27.703 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:27.455538+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f196c15e5d0 msgr2=0x7f196c0c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f196c15e5d0 0x7f196c0c9130 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f195c0097c0 tx=0x7f195c0191b0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 shutdown_connections 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f196c15e5d0 0x7f196c0c9130 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f196c0d5050 0x7f196c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 --2- 172.21.15.3:0/2958219411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196c0cfbe0 0x7f196c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 >> 172.21.15.3:0/2958219411 conn(0x7f196c08c000 msgr2=0x7f196c0b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 shutdown_connections 2024-09-12T12:19:27.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:27.702+0000 7f1970ec4640 1 -- 172.21.15.3:0/2958219411 wait complete. 2024-09-12T12:19:27.717 INFO:tasks.ceph.mgr.y.smithi003.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-12T12:19:27.717 INFO:tasks.ceph.mgr.y.smithi003.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-12T12:19:27.717 INFO:tasks.ceph.mgr.y.smithi003.stderr: from numpy import show_config as show_numpy_config 2024-09-12T12:19:27.719 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-12T12:19:27.719 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-12T12:19:27.727 INFO:tasks.ceph.mgr.x.smithi142.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-12T12:19:27.727 INFO:tasks.ceph.mgr.x.smithi142.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-12T12:19:27.728 INFO:tasks.ceph.mgr.x.smithi142.stderr: from numpy import show_config as show_numpy_config 2024-09-12T12:19:27.761 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:27.758+0000 7f9893558180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:19:27.791 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:27.788+0000 7f70f0f4b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:19:27.900 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:27.898+0000 7f9893558180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:19:27.938 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:27.936+0000 7f70f0f4b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:19:28.057 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:28.054+0000 7f9893558180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:19:28.104 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:28.104+0000 7f70f0f4b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:19:28.123 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:28.122+0000 7f9893558180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:19:28.165 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:28.164+0000 7f70f0f4b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:19:28.264 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:28.262+0000 7f9893558180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:19:28.301 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:28.300+0000 7f70f0f4b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:19:28.527 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:28.638 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.634+0000 7f0f00609640 1 Processor -- start 2024-09-12T12:19:28.638 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.634+0000 7f0f00609640 1 -- start start 2024-09-12T12:19:28.639 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0f00609640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef8145e90 0x7f0ef815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:28.639 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0f00609640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ef80b6ef0 0x7f0ef806bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:28.639 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0f00609640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ef806c910 con 0x7f0ef806c100 2024-09-12T12:19:28.639 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0f00609640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef815e920 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.639 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0f00609640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef8146270 con 0x7f0ef8145e90 2024-09-12T12:19:28.639 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efdba4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ef80b6ef0 0x7f0ef806bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efeba6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ef806c100 0x7f0ef815ffa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33446/0 (socket says 172.21.15.3:33446) 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efeba6640 1 -- 172.21.15.3:0/1945831857 learned_addr learned my addr 172.21.15.3:0/1945831857 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efe3a5640 1 --2- 172.21.15.3:0/1945831857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef8145e90 0x7f0ef815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efdba4640 1 -- 172.21.15.3:0/1945831857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef8145e90 msgr2=0x7f0ef815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efdba4640 1 --2- 172.21.15.3:0/1945831857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef8145e90 0x7f0ef815c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efdba4640 1 -- 172.21.15.3:0/1945831857 >> v1:172.21.15.3:6789/0 conn(0x7f0ef806c100 legacy=0x7f0ef815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efdba4640 1 -- 172.21.15.3:0/1945831857 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef815dd10 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.640 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efd3a3640 1 -- 172.21.15.3:0/1945831857 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3261206901 0 0) 0x7f0ef00014a0 con 0x7f0ef806c100 2024-09-12T12:19:28.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.638+0000 7f0efdba4640 1 --2- 172.21.15.3:0/1945831857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef806bbc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40097c0 tx=0x7f0ef401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b36e9117725be50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:28.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efd3a3640 1 -- 172.21.15.3:0/1945831857 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40b0070 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.642 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efd3a3640 1 -- 172.21.15.3:0/1945831857 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef40ab600 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.643 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efd3a3640 1 -- 172.21.15.3:0/1945831857 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40ab9d0 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.643 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- 172.21.15.3:0/1945831857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 msgr2=0x7f0ef806bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:28.643 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 --2- 172.21.15.3:0/1945831857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef806bbc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40097c0 tx=0x7f0ef401c4a0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- 172.21.15.3:0/1945831857 shutdown_connections 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 --2- 172.21.15.3:0/1945831857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef806bbc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 --2- 172.21.15.3:0/1945831857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef8145e90 0x7f0ef815c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- 172.21.15.3:0/1945831857 >> 172.21.15.3:0/1945831857 conn(0x7f0ef809aa40 msgr2=0x7f0ef80b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- 172.21.15.3:0/1945831857 shutdown_connections 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- 172.21.15.3:0/1945831857 wait complete. 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 Processor -- start 2024-09-12T12:19:28.644 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- start start 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef806c100 0x7f0ef8070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef8138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ef8145e90 0x7f0ef8141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef815a5f0 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef806c570 con 0x7f0ef8145e90 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0f00609640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef815a770 con 0x7f0ef806c100 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef8138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef8138dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60264/0 (socket says 172.21.15.3:60264) 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 -- 172.21.15.3:0/696169955 learned_addr learned my addr 172.21.15.3:0/696169955 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efe3a5640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef806c100 0x7f0ef8070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:28.647 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efeba6640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ef8145e90 0x7f0ef8141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 -- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef806c100 msgr2=0x7f0ef8070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef806c100 0x7f0ef8070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 -- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ef8145e90 msgr2=0x7f0ef8141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ef8145e90 0x7f0ef8141530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 -- 172.21.15.3:0/696169955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef8138010 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efe3a5640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef806c100 0x7f0ef8070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efeba6640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ef8145e90 0x7f0ef8141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0efdba4640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef8138dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ef401a0d0 tx=0x7f0ef401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0eeeffd640 1 -- 172.21.15.3:0/696169955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40b0050 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0eeeffd640 1 -- 172.21.15.3:0/696169955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef40bc070 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.642+0000 7f0eeeffd640 1 -- 172.21.15.3:0/696169955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40b5830 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.646+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef81382f0 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.646+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef8070ff0 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.648 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.646+0000 7f0eeeffd640 1 -- 172.21.15.3:0/696169955 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ef40b59d0 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.649 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.646+0000 7f0eeeffd640 1 -- 172.21.15.3:0/696169955 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0ef40b7450 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.649 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.646+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef80758f0 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.649 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.646+0000 7f0eeeffd640 1 -- 172.21.15.3:0/696169955 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0ef4016090 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.707 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ef815bac0 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.709 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0eeeffd640 1 -- 172.21.15.3:0/696169955 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0ef40c1460 con 0x7f0ef80b6ef0 2024-09-12T12:19:28.709 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-12T12:19:28.709 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":10,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:27.455538+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:28.710 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 msgr2=0x7f0ef8138dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:28.710 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef8138dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ef401a0d0 tx=0x7f0ef401ace0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.710 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 shutdown_connections 2024-09-12T12:19:28.710 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ef8145e90 0x7f0ef8141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.711 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ef80b6ef0 0x7f0ef8138dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.711 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 --2- 172.21.15.3:0/696169955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ef806c100 0x7f0ef8070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:28.711 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 >> 172.21.15.3:0/696169955 conn(0x7f0ef809aa40 msgr2=0x7f0ef815a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:28.711 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 shutdown_connections 2024-09-12T12:19:28.711 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:28.706+0000 7f0f00609640 1 -- 172.21.15.3:0/696169955 wait complete. 2024-09-12T12:19:28.724 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T12:19:28.845 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-09-12T12:19:28.844+0000 7f1eb2ffd7c0 -1 Falling back to public interface 2024-09-12T12:19:28.917 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:28.914+0000 7f9893558180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:19:28.933 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:28.932+0000 7f70f0f4b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:19:28.996 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:28.994+0000 7f9893558180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:19:29.009 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:29.008+0000 7f70f0f4b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:19:29.101 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-09-12T12:19:29.100+0000 7f27bd3db7c0 -1 Falling back to public interface 2024-09-12T12:19:29.128 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-12T12:19:29.126+0000 7f3f650807c0 -1 Falling back to public interface 2024-09-12T12:19:29.132 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-09-12T12:19:29.132+0000 7fdc88d877c0 -1 Falling back to public interface 2024-09-12T12:19:29.135 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-12T12:19:29.134+0000 7fa4adfc17c0 -1 Falling back to public interface 2024-09-12T12:19:29.151 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-12T12:19:29.150+0000 7fec3f4017c0 -1 Falling back to public interface 2024-09-12T12:19:29.184 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-09-12T12:19:29.184+0000 7fcbc59387c0 -1 Falling back to public interface 2024-09-12T12:19:29.188 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:29.186+0000 7f9893558180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:19:29.192 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:29.192+0000 7f70f0f4b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:19:29.265 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:29.262+0000 7f9893558180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:19:29.266 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:29.264+0000 7f70f0f4b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:19:29.372 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-12T12:19:29.370+0000 7f93ae9ad7c0 -1 Falling back to public interface 2024-09-12T12:19:29.557 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:29.556+0000 7f70f0f4b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:19:29.574 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:29.570+0000 7f9893558180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:19:29.618 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:29.616+0000 7f70f0f4b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:19:29.638 INFO:tasks.ceph.mgr.y.smithi003.stderr:2024-09-12T12:19:29.634+0000 7f9893558180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:19:30.660 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:30.660+0000 7f7097b80640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-12T12:19:30.661 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-09-12T12:19:30.660+0000 7f7097b80640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T12:19:33.290 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-09-12T12:19:33.288+0000 7f27bd3db7c0 -1 osd.6 0 log_to_monitors true 2024-09-12T12:19:33.365 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-12T12:19:33.361+0000 7f93ae9ad7c0 -1 osd.2 0 log_to_monitors true 2024-09-12T12:19:33.369 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-09-12T12:19:33.368+0000 7fdc88d877c0 -1 osd.5 0 log_to_monitors true 2024-09-12T12:19:33.438 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-09-12T12:19:33.436+0000 7f1eb2ffd7c0 -1 osd.4 0 log_to_monitors true 2024-09-12T12:19:33.535 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-09-12T12:19:33.536+0000 7fcbc59387c0 -1 osd.7 0 log_to_monitors true 2024-09-12T12:19:33.580 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-12T12:19:33.577+0000 7f3f650807c0 -1 osd.1 0 log_to_monitors true 2024-09-12T12:19:33.783 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-12T12:19:33.781+0000 7fec3f4017c0 -1 osd.0 0 log_to_monitors true 2024-09-12T12:19:33.788 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-12T12:19:33.785+0000 7fa4adfc17c0 -1 osd.3 0 log_to_monitors true 2024-09-12T12:19:35.532 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:35.685 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efda57ae640 1 Processor -- start 2024-09-12T12:19:35.685 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efda57ae640 1 -- start start 2024-09-12T12:19:35.686 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efda57ae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efda00cfa60 0x7efda00af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:35.686 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efda57ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda015fe10 0x7efda01602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:35.686 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efda57ae640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda00b5bf0 con 0x7efda0159c80 2024-09-12T12:19:35.687 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efda57ae640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efda00cfed0 con 0x7efda00cfa60 2024-09-12T12:19:35.687 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efda57ae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efda00b5e60 con 0x7efda015fe10 2024-09-12T12:19:35.687 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efd9e7fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efda00cfa60 0x7efda00af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:35.687 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efd9e7fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efda00cfa60 0x7efda00af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:55356/0 (socket says 172.21.15.3:55356) 2024-09-12T12:19:35.687 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.685+0000 7efd9e7fc640 1 -- 172.21.15.3:0/3066804878 learned_addr learned my addr 172.21.15.3:0/3066804878 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:35.687 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.689+0000 7efd9f7fe640 1 --2- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda015fe10 0x7efda01602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:35.690 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.689+0000 7efd9e7fc640 1 -- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda015fe10 msgr2=0x7efda01602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:35.690 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.689+0000 7efd9e7fc640 1 --2- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda015fe10 0x7efda01602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.690 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.689+0000 7efd9e7fc640 1 -- 172.21.15.3:0/3066804878 >> v1:172.21.15.142:6789/0 conn(0x7efda0159c80 legacy=0x7efda00ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:35.690 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.689+0000 7efd9e7fc640 1 -- 172.21.15.3:0/3066804878 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda00b6140 con 0x7efda00cfa60 2024-09-12T12:19:35.690 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.689+0000 7efd9e7fc640 1 --2- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda00cfa60 0x7efda00af7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd94009870 tx=0x7efd9401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=417c9b6eb18fc336 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:35.694 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.693+0000 7efd9dffb640 1 -- 172.21.15.3:0/3066804878 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940ab650 con 0x7efda00cfa60 2024-09-12T12:19:35.694 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.693+0000 7efd9dffb640 1 -- 172.21.15.3:0/3066804878 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd940b0070 con 0x7efda00cfa60 2024-09-12T12:19:35.694 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.693+0000 7efd9dffb640 1 -- 172.21.15.3:0/3066804878 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd940aba10 con 0x7efda00cfa60 2024-09-12T12:19:35.695 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda00cfa60 msgr2=0x7efda00af7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:35.696 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 --2- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda00cfa60 0x7efda00af7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd94009870 tx=0x7efd9401c560 comp rx=0 tx=0).stop 2024-09-12T12:19:35.696 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- 172.21.15.3:0/3066804878 shutdown_connections 2024-09-12T12:19:35.696 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 --2- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda015fe10 0x7efda01602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.696 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 --2- 172.21.15.3:0/3066804878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda00cfa60 0x7efda00af7b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.696 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- 172.21.15.3:0/3066804878 >> 172.21.15.3:0/3066804878 conn(0x7efda0073430 msgr2=0x7efda0073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:35.696 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- 172.21.15.3:0/3066804878 shutdown_connections 2024-09-12T12:19:35.696 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- 172.21.15.3:0/3066804878 wait complete. 2024-09-12T12:19:35.697 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 Processor -- start 2024-09-12T12:19:35.698 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- start start 2024-09-12T12:19:35.699 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efda00cfa60 0x7efda01fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:35.699 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda0159c80 0x7efda0158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:35.699 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda015fe10 0x7efda0154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:35.699 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efda00cef00 con 0x7efda00cfa60 2024-09-12T12:19:35.699 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efda00ced80 con 0x7efda015fe10 2024-09-12T12:19:35.699 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efda57ae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efda00cf080 con 0x7efda0159c80 2024-09-12T12:19:35.699 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.697+0000 7efd9effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efda00cfa60 0x7efda01fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:35.700 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efda00cfa60 0x7efda01fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60282/0 (socket says 172.21.15.3:60282) 2024-09-12T12:19:35.700 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 -- 172.21.15.3:0/1684818406 learned_addr learned my addr 172.21.15.3:0/1684818406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:35.700 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9f7fe640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda015fe10 0x7efda0154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:35.700 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9e7fc640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda0159c80 0x7efda0158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 -- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda0159c80 msgr2=0x7efda0158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda0159c80 0x7efda0158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 -- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda015fe10 msgr2=0x7efda0154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda015fe10 0x7efda0154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 -- 172.21.15.3:0/1684818406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda00b4ef0 con 0x7efda00cfa60 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd9effd640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efda00cfa60 0x7efda01fb8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c500 tx=0x7efd9000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd877fe640 1 -- 172.21.15.3:0/1684818406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90016020 con 0x7efda00cfa60 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd877fe640 1 -- 172.21.15.3:0/1684818406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90005220 con 0x7efda00cfa60 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda00b51d0 con 0x7efda00cfa60 2024-09-12T12:19:35.701 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd877fe640 1 -- 172.21.15.3:0/1684818406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90007030 con 0x7efda00cfa60 2024-09-12T12:19:35.702 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda01fbe00 con 0x7efda00cfa60 2024-09-12T12:19:35.702 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efd877fe640 1 -- 172.21.15.3:0/1684818406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7efd900071d0 con 0x7efda00cfa60 2024-09-12T12:19:35.703 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.701+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd800052a0 con 0x7efda00cfa60 2024-09-12T12:19:35.703 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.705+0000 7efd877fe640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd8c067830 0x7efd8c069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:35.703 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.705+0000 7efd877fe640 1 -- 172.21.15.3:0/1684818406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2199+0+0 (secure 0 0 0) 0x7efd90084790 con 0x7efda00cfa60 2024-09-12T12:19:35.703 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.705+0000 7efd9e7fc640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd8c067830 0x7efd8c069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:35.704 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.705+0000 7efd9e7fc640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd8c067830 0x7efd8c069cf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd940af000 tx=0x7efd940a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:35.709 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.709+0000 7efd877fe640 1 -- 172.21.15.3:0/1684818406 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efd90020020 con 0x7efda00cfa60 2024-09-12T12:19:35.967 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.965+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd80005590 con 0x7efda00cfa60 2024-09-12T12:19:35.968 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.969+0000 7efd877fe640 1 -- 172.21.15.3:0/1684818406 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7efd900074c0 con 0x7efda00cfa60 2024-09-12T12:19:35.968 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-12T12:19:35.968 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":13,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:35.669474+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:35.971 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd8c067830 msgr2=0x7efd8c069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:35.971 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd8c067830 0x7efd8c069cf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efd940af000 tx=0x7efd940a7000 comp rx=0 tx=0).stop 2024-09-12T12:19:35.971 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efda00cfa60 msgr2=0x7efda01fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efda00cfa60 0x7efda01fb8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efd9000c500 tx=0x7efd9000c9c0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 shutdown_connections 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd8c067830 0x7efd8c069cf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda015fe10 0x7efda0154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda0159c80 0x7efda0158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 --2- 172.21.15.3:0/1684818406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efda00cfa60 0x7efda01fb8c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 >> 172.21.15.3:0/1684818406 conn(0x7efda0073430 msgr2=0x7efda0075130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:35.972 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 shutdown_connections 2024-09-12T12:19:35.973 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:35.973+0000 7efda57ae640 1 -- 172.21.15.3:0/1684818406 wait complete. 2024-09-12T12:19:35.994 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-12T12:19:36.147 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-12T12:19:36.145+0000 7f3f60806640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T12:19:36.193 INFO:tasks.ceph.osd.1.smithi003.stderr:2024-09-12T12:19:36.193+0000 7f3f58e4d640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:36.235 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-12T12:19:36.237+0000 7f93ab168640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T12:19:36.271 INFO:tasks.ceph.osd.2.smithi003.stderr:2024-09-12T12:19:36.269+0000 7f93a277a640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:36.817 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-09-12T12:19:36.816+0000 7fdc8450d640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T12:19:36.835 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-09-12T12:19:36.832+0000 7fcbc20f3640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T12:19:36.840 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-09-12T12:19:36.840+0000 7fdc7cb54640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:36.860 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-09-12T12:19:36.860+0000 7fcbb9705640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:36.860 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-09-12T12:19:36.860+0000 7f1eae783640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T12:19:36.865 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-09-12T12:19:36.864+0000 7f27b8b61640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T12:19:36.904 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-09-12T12:19:36.904+0000 7f27b11a8640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:36.911 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-09-12T12:19:36.912+0000 7f1ea6dca640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:37.130 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-12T12:19:37.129+0000 7fec3bbbc640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T12:19:37.150 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-12T12:19:37.149+0000 7fa4a9747640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T12:19:37.156 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-09-12T12:19:37.157+0000 7fec331ce640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:37.168 INFO:tasks.ceph.osd.3.smithi003.stderr:2024-09-12T12:19:37.169+0000 7fa4a1d8e640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:19:42.801 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:42.925 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38192c2640 1 Processor -- start 2024-09-12T12:19:42.925 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38192c2640 1 -- start start 2024-09-12T12:19:42.926 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38192c2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f381415b140 0x7f38140af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38192c2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b0430 0x7f38140b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38192c2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38140afd80 con 0x7f38140b5540 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38192c2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f38140b59b0 con 0x7f381415b140 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38192c2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f38140b0060 con 0x7f38140b0430 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f3812ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f381415b140 0x7f38140af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f3812ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f381415b140 0x7f38140af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53724/0 (socket says 172.21.15.3:53724) 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f3812ffd640 1 -- 172.21.15.3:0/1689411865 learned_addr learned my addr 172.21.15.3:0/1689411865 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f38127fc640 1 --2- 172.21.15.3:0/1689411865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b0430 0x7f38140b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f3812ffd640 1 -- 172.21.15.3:0/1689411865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b0430 msgr2=0x7f38140b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f3812ffd640 1 --2- 172.21.15.3:0/1689411865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b0430 0x7f38140b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f3812ffd640 1 -- 172.21.15.3:0/1689411865 >> v1:172.21.15.142:6789/0 conn(0x7f38140b5540 legacy=0x7f381415edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:42.927 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.924+0000 7f3812ffd640 1 -- 172.21.15.3:0/1689411865 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38140b0910 con 0x7f381415b140 2024-09-12T12:19:42.929 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f3812ffd640 1 --2- 172.21.15.3:0/1689411865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381415b140 0x7f38140af4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91dc7c1bfd7d5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:42.929 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f3811ffb640 1 -- 172.21.15.3:0/1689411865 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c0a57b0 con 0x7f381415b140 2024-09-12T12:19:42.929 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f3811ffb640 1 -- 172.21.15.3:0/1689411865 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380c0a5950 con 0x7f381415b140 2024-09-12T12:19:42.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f3811ffb640 1 -- 172.21.15.3:0/1689411865 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c0a5d20 con 0x7f381415b140 2024-09-12T12:19:42.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 -- 172.21.15.3:0/1689411865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381415b140 msgr2=0x7f38140af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:42.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 --2- 172.21.15.3:0/1689411865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381415b140 0x7f38140af4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c01c930 comp rx=0 tx=0).stop 2024-09-12T12:19:42.930 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 -- 172.21.15.3:0/1689411865 shutdown_connections 2024-09-12T12:19:42.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 --2- 172.21.15.3:0/1689411865 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b0430 0x7f38140b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:42.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 --2- 172.21.15.3:0/1689411865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f381415b140 0x7f38140af4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:42.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 -- 172.21.15.3:0/1689411865 >> 172.21.15.3:0/1689411865 conn(0x7f38140b6a00 msgr2=0x7f381409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:42.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 -- 172.21.15.3:0/1689411865 shutdown_connections 2024-09-12T12:19:42.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 -- 172.21.15.3:0/1689411865 wait complete. 2024-09-12T12:19:42.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 Processor -- start 2024-09-12T12:19:42.931 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.928+0000 7f38192c2640 1 -- start start 2024-09-12T12:19:42.933 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38140b0430 0x7f38140d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b5540 0x7f3814148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381415b140 0x7f3814145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f381415b6a0 con 0x7f38140b0430 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f381415b520 con 0x7f381415b140 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3814168100 con 0x7f38140b5540 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38140b0430 0x7f38140d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38140b0430 0x7f38140d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53736/0 (socket says 172.21.15.3:53736) 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 -- 172.21.15.3:0/2281113695 learned_addr learned my addr 172.21.15.3:0/2281113695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38127fc640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b5540 0x7f3814148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:42.934 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38137fe640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381415b140 0x7f3814145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 -- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b5540 msgr2=0x7f3814148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b5540 0x7f3814148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 -- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381415b140 msgr2=0x7f3814145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381415b140 0x7f3814145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 -- 172.21.15.3:0/2281113695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38140d2190 con 0x7f38140b0430 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f3812ffd640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38140b0430 0x7f38140d1c50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f380c01afe0 tx=0x7f380c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f380b7fe640 1 -- 172.21.15.3:0/2281113695 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c0bc020 con 0x7f38140b0430 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f380b7fe640 1 -- 172.21.15.3:0/2281113695 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380c0b8670 con 0x7f38140b0430 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381414b3d0 con 0x7f38140b0430 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f380b7fe640 1 -- 172.21.15.3:0/2281113695 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c0b8a60 con 0x7f38140b0430 2024-09-12T12:19:42.935 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381414b990 con 0x7f38140b0430 2024-09-12T12:19:42.936 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38140b5000 con 0x7f38140b0430 2024-09-12T12:19:42.936 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f380b7fe640 1 -- 172.21.15.3:0/2281113695 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f380c0bf020 con 0x7f38140b0430 2024-09-12T12:19:42.936 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f380b7fe640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37f40678c0 0x7f37f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:42.936 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f380b7fe640 1 -- 172.21.15.3:0/2281113695 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f380c120a80 con 0x7f38140b0430 2024-09-12T12:19:42.937 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.932+0000 7f38127fc640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37f40678c0 0x7f37f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:42.937 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.936+0000 7f38127fc640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37f40678c0 0x7f37f4069d80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3814145100 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:42.939 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:42.940+0000 7f380b7fe640 1 -- 172.21.15.3:0/2281113695 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f380c0c6030 con 0x7f38140b0430 2024-09-12T12:19:43.107 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.104+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3814073660 con 0x7f38140b0430 2024-09-12T12:19:43.108 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f380b7fe640 1 -- 172.21.15.3:0/2281113695 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12836 (secure 0 0 0) 0x7f380c0165e0 con 0x7f38140b0430 2024-09-12T12:19:43.109 INFO:teuthology.misc.health.smithi003.stdout: 2024-09-12T12:19:43.109 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":17,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:40.187978+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:43.111 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37f40678c0 msgr2=0x7f37f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.111 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37f40678c0 0x7f37f4069d80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3814145100 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38140b0430 msgr2=0x7f38140d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38140b0430 0x7f38140d1c50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f380c01afe0 tx=0x7f380c0196d0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 shutdown_connections 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37f40678c0 0x7f37f4069d80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381415b140 0x7f3814145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38140b5540 0x7f3814148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 --2- 172.21.15.3:0/2281113695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38140b0430 0x7f38140d1c50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 >> 172.21.15.3:0/2281113695 conn(0x7f38140b6a00 msgr2=0x7f381415d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 shutdown_connections 2024-09-12T12:19:43.112 INFO:teuthology.misc.health.smithi003.stderr:2024-09-12T12:19:43.108+0000 7f38192c2640 1 -- 172.21.15.3:0/2281113695 wait complete. 2024-09-12T12:19:43.127 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-12T12:19:43.127 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-12T12:19:43.128 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:19:43.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:19:43.130 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:19:43.130 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:19:43.131 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:19:43.133 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:19:43.133 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:19:43.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:19:43.135 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:19:43.136 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:19:43.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:19:43.138 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:19:43.138 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-12T12:19:43.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:19:43.203 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:19:43.203 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:19:43.220 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:19:43.223 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:19:43.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9d022d640 1 Processor -- start 2024-09-12T12:19:43.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9d022d640 1 -- start start 2024-09-12T12:19:43.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9d022d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9c815ea80 0x7fe9c809ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:43.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9d022d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe9c80d09a0 0x7fe9c80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:43.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9d022d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c809b390 con 0x7fe9c809aa70 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9d022d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe9c815f0a0 con 0x7fe9c815ea80 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9d022d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe9c815f380 con 0x7fe9c80d09a0 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9cdfc9640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe9c809aa70 0x7fe9c8162c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41410/0 (socket says 172.21.15.3:41410) 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9cdfc9640 1 -- 172.21.15.3:0/901758208 learned_addr learned my addr 172.21.15.3:0/901758208 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9cd7c8640 1 --2- 172.21.15.3:0/901758208 >> v2:172.21.15.3:3300/0 conn(0x7fe9c815ea80 0x7fe9c809ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9ccfc7640 1 -- 172.21.15.3:0/901758208 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072881752 0 0) 0x7fe9c809b390 con 0x7fe9c809aa70 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9ccfc7640 1 -- 172.21.15.3:0/901758208 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9b4003530 con 0x7fe9c809aa70 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9cd7c8640 1 -- 172.21.15.3:0/901758208 >> v2:172.21.15.142:3300/0 conn(0x7fe9c80d09a0 msgr2=0x7fe9c80d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9cd7c8640 1 --2- 172.21.15.3:0/901758208 >> v2:172.21.15.142:3300/0 conn(0x7fe9c80d09a0 0x7fe9c80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9cd7c8640 1 -- 172.21.15.3:0/901758208 >> v1:172.21.15.3:6789/0 conn(0x7fe9c809aa70 legacy=0x7fe9c8162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.256+0000 7fe9cd7c8640 1 -- 172.21.15.3:0/901758208 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c815f660 con 0x7fe9c815ea80 2024-09-12T12:19:43.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9cd7c8640 1 --2- 172.21.15.3:0/901758208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c809ae50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80740d0 tx=0x7fe9b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d60b5485b97ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9ccfc7640 1 -- 172.21.15.3:0/901758208 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80aa600 con 0x7fe9c815ea80 2024-09-12T12:19:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9ccfc7640 1 -- 172.21.15.3:0/901758208 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b80af070 con 0x7fe9c815ea80 2024-09-12T12:19:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9ccfc7640 1 -- 172.21.15.3:0/901758208 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80aa9c0 con 0x7fe9c815ea80 2024-09-12T12:19:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- 172.21.15.3:0/901758208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 msgr2=0x7fe9c809ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 --2- 172.21.15.3:0/901758208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c809ae50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80740d0 tx=0x7fe9b801c480 comp rx=0 tx=0).stop 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- 172.21.15.3:0/901758208 shutdown_connections 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 --2- 172.21.15.3:0/901758208 >> v2:172.21.15.142:3300/0 conn(0x7fe9c80d09a0 0x7fe9c80d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 --2- 172.21.15.3:0/901758208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c809ae50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- 172.21.15.3:0/901758208 >> 172.21.15.3:0/901758208 conn(0x7fe9c80d23a0 msgr2=0x7fe9c80d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- 172.21.15.3:0/901758208 shutdown_connections 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- 172.21.15.3:0/901758208 wait complete. 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 Processor -- start 2024-09-12T12:19:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- start start 2024-09-12T12:19:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9c809aa70 0x7fe9c815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c80d09a0 0x7fe9c80b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c80b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c81601b0 con 0x7fe9c815ea80 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8160030 con 0x7fe9c80d09a0 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9d022d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c816bd80 con 0x7fe9c809aa70 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9ce7ca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c80b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9cdfc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9c809aa70 0x7fe9c815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9ce7ca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c80b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53796/0 (socket says 172.21.15.3:53796) 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9ce7ca640 1 -- 172.21.15.3:0/3740492533 learned_addr learned my addr 172.21.15.3:0/3740492533 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.260+0000 7fe9cd7c8640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c80d09a0 0x7fe9c80b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9ce7ca640 1 -- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9c809aa70 msgr2=0x7fe9c815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9ce7ca640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9c809aa70 0x7fe9c815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9ce7ca640 1 -- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c80d09a0 msgr2=0x7fe9c80b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9ce7ca640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c80d09a0 0x7fe9c80b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9ce7ca640 1 -- 172.21.15.3:0/3740492533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c80c84d0 con 0x7fe9c815ea80 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9ce7ca640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c80b6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b970 tx=0x7fe9c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9be7fc640 1 -- 172.21.15.3:0/3740492533 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0019070 con 0x7fe9c815ea80 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9be7fc640 1 -- 172.21.15.3:0/3740492533 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c000cb60 con 0x7fe9c815ea80 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c80c87b0 con 0x7fe9c815ea80 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9be7fc640 1 -- 172.21.15.3:0/3740492533 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0014440 con 0x7fe9c815ea80 2024-09-12T12:19:43.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c80c62f0 con 0x7fe9c815ea80 2024-09-12T12:19:43.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c80c5e00 con 0x7fe9c815ea80 2024-09-12T12:19:43.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9be7fc640 1 -- 172.21.15.3:0/3740492533 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe9c001c070 con 0x7fe9c815ea80 2024-09-12T12:19:43.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9be7fc640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9ac0678c0 0x7fe9ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:43.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9be7fc640 1 -- 172.21.15.3:0/3740492533 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe9c00856f0 con 0x7fe9c815ea80 2024-09-12T12:19:43.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9cdfc9640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9ac0678c0 0x7fe9ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:43.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.264+0000 7fe9cdfc9640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9ac0678c0 0x7fe9ac069d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4007fd0 tx=0x7fe9c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:43.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.268+0000 7fe9be7fc640 1 -- 172.21.15.3:0/3740492533 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe9c0055eb0 con 0x7fe9c815ea80 2024-09-12T12:19:43.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.468+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9c80d0e10 con 0x7fe9c815ea80 2024-09-12T12:19:43.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.468+0000 7fe9be7fc640 1 -- 172.21.15.3:0/3740492533 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1428 (secure 0 0 0) 0x7fe9c005a1b0 con 0x7fe9c815ea80 2024-09-12T12:19:43.472 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:43.472 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"btime":"2024-09-12T12:19:43:230041+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T12:19:43.473 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-09-12T12:19:43.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9ac0678c0 msgr2=0x7fe9ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9ac0678c0 0x7fe9ac069d80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4007fd0 tx=0x7fe9c4009040 comp rx=0 tx=0).stop 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 msgr2=0x7fe9c80b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c80b6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b970 tx=0x7fe9c000be30 comp rx=0 tx=0).stop 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 shutdown_connections 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9ac0678c0 0x7fe9ac069d80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9c815ea80 0x7fe9c80b6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9c80d09a0 0x7fe9c80b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 --2- 172.21.15.3:0/3740492533 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9c809aa70 0x7fe9c815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 >> 172.21.15.3:0/3740492533 conn(0x7fe9c80d23a0 msgr2=0x7fe9c80701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 shutdown_connections 2024-09-12T12:19:43.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:43.472+0000 7fe9d022d640 1 -- 172.21.15.3:0/3740492533 wait complete. 2024-09-12T12:19:45.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:19:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.624+0000 7f346917c640 1 Processor -- start 2024-09-12T12:19:45.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.624+0000 7f346917c640 1 -- start start 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346917c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f346406f6e0 0x7f3464072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346917c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f346415f150 con 0x7f346415b2d0 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346917c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f346415b6b0 con 0x7f34640cc300 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346917c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f34640b3790 con 0x7f346406f6e0 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346259c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f346406f6e0 0x7f3464072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f3462d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f346415b2d0 0x7f34640b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41424/0 (socket says 172.21.15.3:41424) 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f3462d9d640 1 -- 172.21.15.3:0/1858620314 learned_addr learned my addr 172.21.15.3:0/1858620314 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:45.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346259c640 1 -- 172.21.15.3:0/1858620314 >> v1:172.21.15.142:6789/0 conn(0x7f34640cc300 legacy=0x7f34640aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346259c640 1 -- 172.21.15.3:0/1858620314 >> v1:172.21.15.3:6789/0 conn(0x7f346415b2d0 legacy=0x7f34640b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346259c640 1 -- 172.21.15.3:0/1858620314 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346415ee70 con 0x7f346406f6e0 2024-09-12T12:19:45.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f346259c640 1 --2- 172.21.15.3:0/1858620314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f3464072f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d90 tx=0x7f344c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9428ad59a4b3407 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.628+0000 7f3461d9b640 1 -- 172.21.15.3:0/1858620314 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c0a6700 con 0x7f346406f6e0 2024-09-12T12:19:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3461d9b640 1 -- 172.21.15.3:0/1858620314 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344c0a68a0 con 0x7f346406f6e0 2024-09-12T12:19:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3461d9b640 1 -- 172.21.15.3:0/1858620314 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c0a6c70 con 0x7f346406f6e0 2024-09-12T12:19:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- 172.21.15.3:0/1858620314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 msgr2=0x7f3464072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 --2- 172.21.15.3:0/1858620314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f3464072f30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d90 tx=0x7f344c01c930 comp rx=0 tx=0).stop 2024-09-12T12:19:45.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- 172.21.15.3:0/1858620314 shutdown_connections 2024-09-12T12:19:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 --2- 172.21.15.3:0/1858620314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f3464072f30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- 172.21.15.3:0/1858620314 >> 172.21.15.3:0/1858620314 conn(0x7f34640b1d80 msgr2=0x7f34640b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- 172.21.15.3:0/1858620314 shutdown_connections 2024-09-12T12:19:45.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- 172.21.15.3:0/1858620314 wait complete. 2024-09-12T12:19:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 Processor -- start 2024-09-12T12:19:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- start start 2024-09-12T12:19:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f346409db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34640cc300 0x7f3464153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f346415b2d0 0x7f346409be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f346406d8b0 con 0x7f346406f6e0 2024-09-12T12:19:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3464073880 con 0x7f34640cc300 2024-09-12T12:19:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f346406ede0 con 0x7f346415b2d0 2024-09-12T12:19:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f346409db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f346409db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53816/0 (socket says 172.21.15.3:53816) 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 -- 172.21.15.3:0/4099474509 learned_addr learned my addr 172.21.15.3:0/4099474509 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346359e640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f346415b2d0 0x7f346409be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346259c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34640cc300 0x7f3464153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 -- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f346415b2d0 msgr2=0x7f346409be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f346415b2d0 0x7f346409be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 -- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34640cc300 msgr2=0x7f3464153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34640cc300 0x7f3464153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 -- 172.21.15.3:0/4099474509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34640d0000 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346359e640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f346415b2d0 0x7f346409be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346259c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34640cc300 0x7f3464153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f3462d9d640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f346409db50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f345400e730 tx=0x7f345400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f345b7fe640 1 -- 172.21.15.3:0/4099474509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454019070 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f345b7fe640 1 -- 172.21.15.3:0/4099474509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3454007cd0 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346409c7d0 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f345b7fe640 1 -- 172.21.15.3:0/4099474509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345400c880 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.632+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346409cd90 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.636+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346406f110 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.636+0000 7f345b7fe640 1 -- 172.21.15.3:0/4099474509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f345401c070 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.636+0000 7f345b7fe640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34440678c0 0x7f3444069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.636+0000 7f345b7fe640 1 -- 172.21.15.3:0/4099474509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3454086df0 con 0x7f346406f6e0 2024-09-12T12:19:45.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.636+0000 7f346259c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34440678c0 0x7f3444069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.636+0000 7f346259c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34440678c0 0x7f3444069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f344c0af000 tx=0x7f344c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:45.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.640+0000 7f345b7fe640 1 -- 172.21.15.3:0/4099474509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3454057c60 con 0x7f346406f6e0 2024-09-12T12:19:45.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f346406cd80 con 0x7f346406f6e0 2024-09-12T12:19:45.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f345b7fe640 1 -- 172.21.15.3:0/4099474509 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3897 (secure 0 0 0) 0x7f3454057560 con 0x7f346406f6e0 2024-09-12T12:19:45.842 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:45.842 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"btime":"2024-09-12T12:19:44:231173+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-12T12:19:45.842 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34440678c0 msgr2=0x7f3444069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34440678c0 0x7f3444069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f344c0af000 tx=0x7f344c0a7000 comp rx=0 tx=0).stop 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 msgr2=0x7f346409db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f346409db50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f345400e730 tx=0x7f345400ebf0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 shutdown_connections 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34440678c0 0x7f3444069d80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f346415b2d0 0x7f346409be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34640cc300 0x7f3464153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 --2- 172.21.15.3:0/4099474509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f346406f6e0 0x7f346409db50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 >> 172.21.15.3:0/4099474509 conn(0x7f34640b1d80 msgr2=0x7f34640b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.840+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 shutdown_connections 2024-09-12T12:19:45.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.844+0000 7f346917c640 1 -- 172.21.15.3:0/4099474509 wait complete. 2024-09-12T12:19:45.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T12:19:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb3b846640 1 Processor -- start 2024-09-12T12:19:45.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb3b846640 1 -- start start 2024-09-12T12:19:45.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb3b846640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb34159bd0 0x7feb3406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb3b846640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feb3415d2d0 0x7feb340ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb3b846640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb3406fbb0 con 0x7feb34159fb0 2024-09-12T12:19:45.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb3b846640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7feb3406f010 con 0x7feb34159bd0 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb3b846640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7feb3415d760 con 0x7feb3415d2d0 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb395e2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb34159bd0 0x7feb3406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb395e2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb34159bd0 0x7feb3406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53820/0 (socket says 172.21.15.3:53820) 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb395e2640 1 -- 172.21.15.3:0/1193701804 learned_addr learned my addr 172.21.15.3:0/1193701804 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb39de3640 1 --2- 172.21.15.3:0/1193701804 >> v2:172.21.15.142:3300/0 conn(0x7feb3415d2d0 0x7feb340ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb395e2640 1 -- 172.21.15.3:0/1193701804 >> v2:172.21.15.142:3300/0 conn(0x7feb3415d2d0 msgr2=0x7feb340ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb395e2640 1 --2- 172.21.15.3:0/1193701804 >> v2:172.21.15.142:3300/0 conn(0x7feb3415d2d0 0x7feb340ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb395e2640 1 -- 172.21.15.3:0/1193701804 >> v1:172.21.15.142:6789/0 conn(0x7feb34159fb0 legacy=0x7feb3406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:45.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.976+0000 7feb395e2640 1 -- 172.21.15.3:0/1193701804 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3406e400 con 0x7feb34159bd0 2024-09-12T12:19:45.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb395e2640 1 --2- 172.21.15.3:0/1193701804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb3406dd40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feb3406ef80 tx=0x7feb3001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96aa1a9718c95abc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:45.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb2bfff640 1 -- 172.21.15.3:0/1193701804 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300a5750 con 0x7feb34159bd0 2024-09-12T12:19:45.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb2bfff640 1 -- 172.21.15.3:0/1193701804 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb300a58f0 con 0x7feb34159bd0 2024-09-12T12:19:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb2bfff640 1 -- 172.21.15.3:0/1193701804 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300a5cc0 con 0x7feb34159bd0 2024-09-12T12:19:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 -- 172.21.15.3:0/1193701804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 msgr2=0x7feb3406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 --2- 172.21.15.3:0/1193701804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb3406dd40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feb3406ef80 tx=0x7feb3001c980 comp rx=0 tx=0).stop 2024-09-12T12:19:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 -- 172.21.15.3:0/1193701804 shutdown_connections 2024-09-12T12:19:45.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 --2- 172.21.15.3:0/1193701804 >> v2:172.21.15.142:3300/0 conn(0x7feb3415d2d0 0x7feb340ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 --2- 172.21.15.3:0/1193701804 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb3406dd40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 -- 172.21.15.3:0/1193701804 >> 172.21.15.3:0/1193701804 conn(0x7feb340b1d80 msgr2=0x7feb340b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 -- 172.21.15.3:0/1193701804 shutdown_connections 2024-09-12T12:19:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 -- 172.21.15.3:0/1193701804 wait complete. 2024-09-12T12:19:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 Processor -- start 2024-09-12T12:19:45.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.980+0000 7feb3b846640 1 -- start start 2024-09-12T12:19:45.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb340cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb34159fb0 0x7feb341f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb3415d2d0 0x7feb3409d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb34169360 con 0x7feb34159bd0 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb341691e0 con 0x7feb3415d2d0 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb341694e0 con 0x7feb34159fb0 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb340cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb340cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53824/0 (socket says 172.21.15.3:53824) 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 -- 172.21.15.3:0/3661463406 learned_addr learned my addr 172.21.15.3:0/3661463406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 -- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb34159fb0 msgr2=0x7feb341f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb34159fb0 0x7feb341f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 -- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb3415d2d0 msgr2=0x7feb3409d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb3415d2d0 0x7feb3409d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 -- 172.21.15.3:0/3661463406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb340cfe10 con 0x7feb34159bd0 2024-09-12T12:19:45.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb395e2640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb340cf8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb300af040 tx=0x7feb30019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb29ffb640 1 -- 172.21.15.3:0/3661463406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300b8670 con 0x7feb34159bd0 2024-09-12T12:19:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb29ffb640 1 -- 172.21.15.3:0/3661463406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb300be070 con 0x7feb34159bd0 2024-09-12T12:19:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341f98a0 con 0x7feb34159bd0 2024-09-12T12:19:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb29ffb640 1 -- 172.21.15.3:0/3661463406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300b8a90 con 0x7feb34159bd0 2024-09-12T12:19:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb340cdfd0 con 0x7feb34159bd0 2024-09-12T12:19:45.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb340ad060 con 0x7feb34159bd0 2024-09-12T12:19:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb29ffb640 1 -- 172.21.15.3:0/3661463406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feb300b8c30 con 0x7feb34159bd0 2024-09-12T12:19:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb29ffb640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb180678c0 0x7feb18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb29ffb640 1 -- 172.21.15.3:0/3661463406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7feb30120c80 con 0x7feb34159bd0 2024-09-12T12:19:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.984+0000 7feb38de1640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb180678c0 0x7feb18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:45.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.988+0000 7feb38de1640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb180678c0 0x7feb18069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb3415f600 tx=0x7feb240079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:45.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:45.988+0000 7feb29ffb640 1 -- 172.21.15.3:0/3661463406 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feb300165a0 con 0x7feb34159bd0 2024-09-12T12:19:46.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.152+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feb340b4520 con 0x7feb34159bd0 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb29ffb640 1 -- 172.21.15.3:0/3661463406 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7feb300f1b10 con 0x7feb34159bd0 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:19:46.156 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T12:19:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb180678c0 msgr2=0x7feb18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb180678c0 0x7feb18069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feb3415f600 tx=0x7feb240079c0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 msgr2=0x7feb340cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb340cf8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feb300af040 tx=0x7feb30019720 comp rx=0 tx=0).stop 2024-09-12T12:19:46.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 shutdown_connections 2024-09-12T12:19:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb180678c0 0x7feb18069d80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb3415d2d0 0x7feb3409d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb34159fb0 0x7feb341f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 --2- 172.21.15.3:0/3661463406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb34159bd0 0x7feb340cf8d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 >> 172.21.15.3:0/3661463406 conn(0x7feb340b1d80 msgr2=0x7feb3415fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 shutdown_connections 2024-09-12T12:19:46.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.156+0000 7feb3b846640 1 -- 172.21.15.3:0/3661463406 wait complete. 2024-09-12T12:19:46.174 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:19:46.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:19:46.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:19:46.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:19:46.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 Processor -- start 2024-09-12T12:19:46.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 -- start start 2024-09-12T12:19:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41015e580 0x7fc41006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41006fe30 0x7fc41006ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc4100afec0 0x7fc410162090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc41015e960 con 0x7fc41006fe30 2024-09-12T12:19:46.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc4100aee20 con 0x7fc4100afec0 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.288+0000 7fc4180a0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4100af100 con 0x7fc41015e580 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41006fe30 0x7fc41006ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc41006fe30 0x7fc41006ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53830/0 (socket says 172.21.15.3:53830) 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 -- 172.21.15.3:0/2124902572 learned_addr learned my addr 172.21.15.3:0/2124902572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41663d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc4100afec0 0x7fc410162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc415e3c640 1 --2- 172.21.15.3:0/2124902572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41015e580 0x7fc41006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 -- 172.21.15.3:0/2124902572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41015e580 msgr2=0x7fc41006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 --2- 172.21.15.3:0/2124902572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41015e580 0x7fc41006e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 -- 172.21.15.3:0/2124902572 >> v2:172.21.15.142:3300/0 conn(0x7fc4100afec0 msgr2=0x7fc410162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:46.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 --2- 172.21.15.3:0/2124902572 >> v2:172.21.15.142:3300/0 conn(0x7fc4100afec0 0x7fc410162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 -- 172.21.15.3:0/2124902572 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4100af3e0 con 0x7fc41006fe30 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc41563b640 1 --2- 172.21.15.3:0/2124902572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41006fe30 0x7fc41006ed00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc400009870 tx=0x7fc40001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76faebf3cb47d2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc414e3a640 1 -- 172.21.15.3:0/2124902572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000aa610 con 0x7fc41006fe30 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc414e3a640 1 -- 172.21.15.3:0/2124902572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4000af070 con 0x7fc41006fe30 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc414e3a640 1 -- 172.21.15.3:0/2124902572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000aa9d0 con 0x7fc41006fe30 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 -- 172.21.15.3:0/2124902572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41006fe30 msgr2=0x7fc41006ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 --2- 172.21.15.3:0/2124902572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41006fe30 0x7fc41006ed00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc400009870 tx=0x7fc40001c4f0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 -- 172.21.15.3:0/2124902572 shutdown_connections 2024-09-12T12:19:46.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 --2- 172.21.15.3:0/2124902572 >> v2:172.21.15.142:3300/0 conn(0x7fc4100afec0 0x7fc410162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 --2- 172.21.15.3:0/2124902572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41006fe30 0x7fc41006ed00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 --2- 172.21.15.3:0/2124902572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41015e580 0x7fc41006e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 -- 172.21.15.3:0/2124902572 >> 172.21.15.3:0/2124902572 conn(0x7fc4100b29e0 msgr2=0x7fc4100b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 -- 172.21.15.3:0/2124902572 shutdown_connections 2024-09-12T12:19:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.292+0000 7fc4180a0640 1 -- 172.21.15.3:0/2124902572 wait complete. 2024-09-12T12:19:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 Processor -- start 2024-09-12T12:19:46.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 -- start start 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41006fe30 0x7fc4101fa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4100afec0 0x7fc4100cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e580 0x7fc41014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc4100b3b10 con 0x7fc41015e580 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4100aead0 con 0x7fc4100afec0 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc4100b3c90 con 0x7fc41006fe30 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e580 0x7fc41014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e580 0x7fc41014dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53846/0 (socket says 172.21.15.3:53846) 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 -- 172.21.15.3:0/558082758 learned_addr learned my addr 172.21.15.3:0/558082758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:46.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41563b640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4100afec0 0x7fc4100cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc415e3c640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41006fe30 0x7fc4101fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 -- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41006fe30 msgr2=0x7fc4101fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41006fe30 0x7fc4101fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 -- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4100afec0 msgr2=0x7fc4100cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4100afec0 0x7fc4100cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 -- 172.21.15.3:0/558082758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4100ce4e0 con 0x7fc41015e580 2024-09-12T12:19:46.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc415e3c640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41006fe30 0x7fc4101fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc41663d640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e580 0x7fc41014dd80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc40800efc0 tx=0x7fc40800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4067fc640 1 -- 172.21.15.3:0/558082758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408018020 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4067fc640 1 -- 172.21.15.3:0/558082758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc408014480 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41014e800 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4067fc640 1 -- 172.21.15.3:0/558082758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408014870 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4100cd7a0 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc410162090 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4067fc640 1 -- 172.21.15.3:0/558082758 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc40801b020 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4067fc640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc3f40678f0 0x7fc3f4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.296+0000 7fc4067fc640 1 -- 172.21.15.3:0/558082758 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc408086700 con 0x7fc41015e580 2024-09-12T12:19:46.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.300+0000 7fc415e3c640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc3f40678f0 0x7fc3f4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.300+0000 7fc415e3c640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc3f40678f0 0x7fc3f4069db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0099c0 tx=0x7fc40c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:46.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.304+0000 7fc4067fc640 1 -- 172.21.15.3:0/558082758 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc408022030 con 0x7fc41015e580 2024-09-12T12:19:46.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.464+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc41015a720 con 0x7fc41015e580 2024-09-12T12:19:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4067fc640 1 -- 172.21.15.3:0/558082758 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fc408010040 con 0x7fc41015e580 2024-09-12T12:19:46.717 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc3f40678f0 msgr2=0x7fc3f4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc3f40678f0 0x7fc3f4069db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0099c0 tx=0x7fc40c008040 comp rx=0 tx=0).stop 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e580 msgr2=0x7fc41014dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e580 0x7fc41014dd80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc40800efc0 tx=0x7fc40800c490 comp rx=0 tx=0).stop 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 shutdown_connections 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc3f40678f0 0x7fc3f4069db0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc41015e580 0x7fc41014dd80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4100afec0 0x7fc4100cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 --2- 172.21.15.3:0/558082758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc41006fe30 0x7fc4101fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 >> 172.21.15.3:0/558082758 conn(0x7fc4100b29e0 msgr2=0x7fc4100b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 shutdown_connections 2024-09-12T12:19:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.716+0000 7fc4180a0640 1 -- 172.21.15.3:0/558082758 wait complete. 2024-09-12T12:19:46.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:19:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3e5f7640 1 Processor -- start 2024-09-12T12:19:46.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3e5f7640 1 -- start start 2024-09-12T12:19:46.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3e5f7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7a3815aae0 0x7f7a3806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3e5f7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a3806ece0 0x7f7a380ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3e5f7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a3815aec0 con 0x7f7a3806fa00 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3e5f7640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7a380b3dc0 con 0x7f7a3806ece0 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3e5f7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7a380b40a0 con 0x7f7a3815aae0 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3cb94640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a3806ece0 0x7f7a380ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3cb94640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7a3806ece0 0x7f7a380ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53852/0 (socket says 172.21.15.3:53852) 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3cb94640 1 -- 172.21.15.3:0/3654476298 learned_addr learned my addr 172.21.15.3:0/3654476298 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:46.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a37fff640 1 --2- 172.21.15.3:0/3654476298 >> v2:172.21.15.142:3300/0 conn(0x7f7a3815aae0 0x7f7a3806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3cb94640 1 -- 172.21.15.3:0/3654476298 >> v2:172.21.15.142:3300/0 conn(0x7f7a3815aae0 msgr2=0x7f7a3806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3cb94640 1 --2- 172.21.15.3:0/3654476298 >> v2:172.21.15.142:3300/0 conn(0x7f7a3815aae0 0x7f7a3806f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3cb94640 1 -- 172.21.15.3:0/3654476298 >> v1:172.21.15.142:6789/0 conn(0x7f7a3806fa00 legacy=0x7f7a3806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:46.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.920+0000 7f7a3cb94640 1 -- 172.21.15.3:0/3654476298 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a380b4380 con 0x7f7a3806ece0 2024-09-12T12:19:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3cb94640 1 --2- 172.21.15.3:0/3654476298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806ece0 0x7f7a380ad5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a380d1100 tx=0x7f7a3001eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347eb9530577c423 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:46.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a36ffd640 1 -- 172.21.15.3:0/3654476298 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000b2d0 con 0x7f7a3806ece0 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a36ffd640 1 -- 172.21.15.3:0/3654476298 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3000b470 con 0x7f7a3806ece0 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a36ffd640 1 -- 172.21.15.3:0/3654476298 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a300a7b60 con 0x7f7a3806ece0 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/3654476298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806ece0 msgr2=0x7f7a380ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/3654476298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806ece0 0x7f7a380ad5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a380d1100 tx=0x7f7a3001eaf0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/3654476298 shutdown_connections 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/3654476298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806ece0 0x7f7a380ad5b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/3654476298 >> v2:172.21.15.142:3300/0 conn(0x7f7a3815aae0 0x7f7a3806f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/3654476298 >> 172.21.15.3:0/3654476298 conn(0x7f7a380b1d80 msgr2=0x7f7a380b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:46.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/3654476298 shutdown_connections 2024-09-12T12:19:46.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/3654476298 wait complete. 2024-09-12T12:19:46.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 Processor -- start 2024-09-12T12:19:46.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- start start 2024-09-12T12:19:46.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a3806ece0 0x7f7a381451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806fa00 0x7f7a38154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a3815aae0 0x7f7a3815cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380adc70 con 0x7f7a3806fa00 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a380adaf0 con 0x7f7a3806ece0 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3e5f7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7a380addf0 con 0x7f7a3815aae0 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806fa00 0x7f7a38154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806fa00 0x7f7a38154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53862/0 (socket says 172.21.15.3:53862) 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 -- 172.21.15.3:0/2727996658 learned_addr learned my addr 172.21.15.3:0/2727996658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a37fff640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a3806ece0 0x7f7a381451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a3cb94640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a3815aae0 0x7f7a3815cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 -- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a3815aae0 msgr2=0x7f7a3815cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a3815aae0 0x7f7a3815cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 -- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a3806ece0 msgr2=0x7f7a381451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a3806ece0 0x7f7a381451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:46.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a377fe640 1 -- 172.21.15.3:0/2727996658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a381456e0 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.924+0000 7f7a37fff640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a3806ece0 0x7f7a381451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a377fe640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806fa00 0x7f7a38154220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00c410 tx=0x7f7a2c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a34ff9640 1 -- 172.21.15.3:0/2727996658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011790 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a34ff9640 1 -- 172.21.15.3:0/2727996658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2c017070 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a34ff9640 1 -- 172.21.15.3:0/2727996658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011b70 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3815da80 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3815c100 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a380ad5b0 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a34ff9640 1 -- 172.21.15.3:0/2727996658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a2c011d10 con 0x7f7a3806fa00 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a34ff9640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7a240678c0 0x7f7a24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:46.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a34ff9640 1 -- 172.21.15.3:0/2727996658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f7a2c085830 con 0x7f7a3806fa00 2024-09-12T12:19:46.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a37fff640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7a240678c0 0x7f7a24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:46.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.928+0000 7f7a37fff640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7a240678c0 0x7f7a24069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097a0 tx=0x7f7a28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:46.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:46.932+0000 7f7a34ff9640 1 -- 172.21.15.3:0/2727996658 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7a2c021020 con 0x7f7a3806fa00 2024-09-12T12:19:47.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.092+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7a380cc620 con 0x7f7a3806fa00 2024-09-12T12:19:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.720+0000 7f7a34ff9640 1 -- 172.21.15.3:0/2727996658 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f7a2c056420 con 0x7f7a3806fa00 2024-09-12T12:19:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7a240678c0 msgr2=0x7f7a24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7a240678c0 0x7f7a24069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097a0 tx=0x7f7a28008040 comp rx=0 tx=0).stop 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806fa00 msgr2=0x7f7a38154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806fa00 0x7f7a38154220 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00c410 tx=0x7f7a2c00c8d0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 shutdown_connections 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7a240678c0 0x7f7a24069d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7a3815aae0 0x7f7a3815cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a3806fa00 0x7f7a38154220 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 --2- 172.21.15.3:0/2727996658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a3806ece0 0x7f7a381451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 >> 172.21.15.3:0/2727996658 conn(0x7f7a380b1d80 msgr2=0x7f7a3806e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 shutdown_connections 2024-09-12T12:19:47.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.724+0000 7f7a3e5f7640 1 -- 172.21.15.3:0/2727996658 wait complete. 2024-09-12T12:19:47.759 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:19:47.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f99a4b4a640 1 Processor -- start 2024-09-12T12:19:47.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f99a4b4a640 1 -- start start 2024-09-12T12:19:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f99a4b4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 0x7f99a006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f99a4b4a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a015aec0 con 0x7f99a006ece0 2024-09-12T12:19:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f99a4b4a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a00b3dc0 con 0x7f99a006fa00 2024-09-12T12:19:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f99a4b4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f99a00b40a0 con 0x7f99a015aae0 2024-09-12T12:19:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f99a006ece0 0x7f99a00ad5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:41432/0 (socket says 172.21.15.3:41432) 2024-09-12T12:19:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 -- 172.21.15.3:0/4267159633 learned_addr learned my addr 172.21.15.3:0/4267159633 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999e59c640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 0x7f99a006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278402025 0 0) 0x7f99a015aec0 con 0x7f99a006ece0 2024-09-12T12:19:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9988003530 con 0x7f99a006ece0 2024-09-12T12:19:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3674388677 0 0) 0x7f9988003530 con 0x7f99a006ece0 2024-09-12T12:19:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 msgr2=0x7f99a006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 0x7f99a006f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 >> v1:172.21.15.142:6789/0 conn(0x7f99a006fa00 legacy=0x7f99a006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a00b4380 con 0x7f99a006ece0 2024-09-12T12:19:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f9998001a00 con 0x7f99a006ece0 2024-09-12T12:19:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 >> v1:172.21.15.3:6789/0 conn(0x7f99a006ece0 legacy=0x7f99a00ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 0x7f99880039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9988003f70 0x7f9988004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988004d50 0x7f99880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99a00b40a0 con 0x7f9988004d50 2024-09-12T12:19:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f998800bb60 con 0x7f9988003f70 2024-09-12T12:19:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f998800bd60 con 0x7f99a015aae0 2024-09-12T12:19:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988004d50 0x7f99880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999e59c640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 0x7f99880039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999dd9b640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9988003f70 0x7f9988004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 -- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 msgr2=0x7f99880039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:47.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a015aae0 0x7f99880039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 -- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9988003f70 msgr2=0x7f9988004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:47.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9988003f70 0x7f9988004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.884+0000 7f999ed9d640 1 -- 172.21.15.3:0/4267159633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9988003530 con 0x7f9988004d50 2024-09-12T12:19:47.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999ed9d640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988004d50 0x7f99880051d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99a006fff0 tx=0x7f999801fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cbe9ddaf9ff646 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:47.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999800a480 con 0x7f9988004d50 2024-09-12T12:19:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99980aa6b0 con 0x7f9988004d50 2024-09-12T12:19:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999d59a640 1 -- 172.21.15.3:0/4267159633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99980aaae0 con 0x7f9988004d50 2024-09-12T12:19:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- 172.21.15.3:0/4267159633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988004d50 msgr2=0x7f99880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 --2- 172.21.15.3:0/4267159633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9988004d50 0x7f99880051d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99a006fff0 tx=0x7f999801fa80 comp rx=0 tx=0).stop 2024-09-12T12:19:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999ed9d640 1 -- 172.21.15.3:0/4267159633 reap_dead start 2024-09-12T12:19:47.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- 172.21.15.3:0/4267159633 shutdown_connections 2024-09-12T12:19:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- 172.21.15.3:0/4267159633 >> 172.21.15.3:0/4267159633 conn(0x7f99a00b1d80 msgr2=0x7f99a00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- 172.21.15.3:0/4267159633 shutdown_connections 2024-09-12T12:19:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- 172.21.15.3:0/4267159633 wait complete. 2024-09-12T12:19:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 Processor -- start 2024-09-12T12:19:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- start start 2024-09-12T12:19:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0099880 0x7f99a00726e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99a0168900 0x7f99a016ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a0140540 0x7f99a01409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99a00b3a90 con 0x7f99a0168900 2024-09-12T12:19:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99a00b4940 con 0x7f99a0099880 2024-09-12T12:19:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f99a4b4a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f99a00b4e60 con 0x7f99a0140540 2024-09-12T12:19:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99a0168900 0x7f99a016ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99a0168900 0x7f99a016ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53888/0 (socket says 172.21.15.3:53888) 2024-09-12T12:19:47.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.888+0000 7f999dd9b640 1 -- 172.21.15.3:0/3742938780 learned_addr learned my addr 172.21.15.3:0/3742938780 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:47.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999e59c640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0099880 0x7f99a00726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999ed9d640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a0140540 0x7f99a01409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999dd9b640 1 -- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a0140540 msgr2=0x7f99a01409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:47.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999dd9b640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a0140540 0x7f99a01409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999dd9b640 1 -- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0099880 msgr2=0x7f99a00726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999dd9b640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0099880 0x7f99a00726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999dd9b640 1 -- 172.21.15.3:0/3742938780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a016a140 con 0x7f99a0168900 2024-09-12T12:19:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999ed9d640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a0140540 0x7f99a01409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999e59c640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0099880 0x7f99a00726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:47.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999dd9b640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99a0168900 0x7f99a016ac00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f999400e550 tx=0x7f999400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f998effd640 1 -- 172.21.15.3:0/3742938780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940052e0 con 0x7f99a0168900 2024-09-12T12:19:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f998effd640 1 -- 172.21.15.3:0/3742938780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9994018070 con 0x7f99a0168900 2024-09-12T12:19:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a0168ff0 con 0x7f99a0168900 2024-09-12T12:19:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f998effd640 1 -- 172.21.15.3:0/3742938780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994003820 con 0x7f99a0168900 2024-09-12T12:19:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a016a400 con 0x7f99a0168900 2024-09-12T12:19:47.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a006f4c0 con 0x7f99a0168900 2024-09-12T12:19:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f998effd640 1 -- 172.21.15.3:0/3742938780 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99940039c0 con 0x7f99a0168900 2024-09-12T12:19:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f998effd640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99800678f0 0x7f9980069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f998effd640 1 -- 172.21.15.3:0/3742938780 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f9994085480 con 0x7f99a0168900 2024-09-12T12:19:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999e59c640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99800678f0 0x7f9980069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.892+0000 7f999e59c640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99800678f0 0x7f9980069db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9990009a20 tx=0x7f9990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:47.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:47.896+0000 7f998effd640 1 -- 172.21.15.3:0/3742938780 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f99940560f0 con 0x7f99a0168900 2024-09-12T12:19:48.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.096+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f99a00cc620 con 0x7f99a0168900 2024-09-12T12:19:48.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.736+0000 7f998effd640 1 -- 172.21.15.3:0/3742938780 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f9994005480 con 0x7f99a0168900 2024-09-12T12:19:48.738 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T12:19:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99800678f0 msgr2=0x7f9980069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99800678f0 0x7f9980069db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9990009a20 tx=0x7f9990008040 comp rx=0 tx=0).stop 2024-09-12T12:19:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99a0168900 msgr2=0x7f99a016ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99a0168900 0x7f99a016ac00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f999400e550 tx=0x7f999400ea10 comp rx=0 tx=0).stop 2024-09-12T12:19:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 shutdown_connections 2024-09-12T12:19:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99800678f0 0x7f9980069db0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99a0140540 0x7f99a01409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99a0168900 0x7f99a016ac00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 --2- 172.21.15.3:0/3742938780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99a0099880 0x7f99a00726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 >> 172.21.15.3:0/3742938780 conn(0x7f99a00b1d80 msgr2=0x7f99a00750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 shutdown_connections 2024-09-12T12:19:48.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.740+0000 7f99a4b4a640 1 -- 172.21.15.3:0/3742938780 wait complete. 2024-09-12T12:19:48.777 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:48.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.892+0000 7fc6e7dae640 1 Processor -- start 2024-09-12T12:19:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.892+0000 7fc6e7dae640 1 -- start start 2024-09-12T12:19:48.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e7dae640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc6e00cfbe0 0x7fc6e01695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e7dae640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e00d31a0 con 0x7fc6e00d5050 2024-09-12T12:19:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e7dae640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e00d3480 con 0x7fc6e015e5d0 2024-09-12T12:19:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e7dae640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e00b1060 con 0x7fc6e00cfbe0 2024-09-12T12:19:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e5349640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc6e00d5050 0x7fc6e00cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45248/0 (socket says 172.21.15.3:45248) 2024-09-12T12:19:48.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e5349640 1 -- 172.21.15.3:0/2899178713 learned_addr learned my addr 172.21.15.3:0/2899178713 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e634b640 1 --2- 172.21.15.3:0/2899178713 >> v2:172.21.15.142:3300/0 conn(0x7fc6e00cfbe0 0x7fc6e01695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078747003 0 0) 0x7fc6e00d31a0 con 0x7fc6e00d5050 2024-09-12T12:19:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6cc003530 con 0x7fc6e00d5050 2024-09-12T12:19:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2398523114 0 0) 0x7fc6cc003530 con 0x7fc6e00d5050 2024-09-12T12:19:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 >> v2:172.21.15.142:3300/0 conn(0x7fc6e00cfbe0 msgr2=0x7fc6e01695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:48.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 --2- 172.21.15.3:0/2899178713 >> v2:172.21.15.142:3300/0 conn(0x7fc6e00cfbe0 0x7fc6e01695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 >> v1:172.21.15.142:6789/0 conn(0x7fc6e015e5d0 legacy=0x7fc6e00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e00d3760 con 0x7fc6e00d5050 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fc6d4001680 con 0x7fc6e00d5050 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 >> v1:172.21.15.3:6789/0 conn(0x7fc6e00d5050 legacy=0x7fc6e00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0039c0 0x7fc6cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc004810 0x7fc6cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6cc0051d0 0x7fc6cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e00b1060 con 0x7fc6cc0051d0 2024-09-12T12:19:48.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc00c170 con 0x7fc6cc0039c0 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6cc00c370 con 0x7fc6cc004810 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e634b640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6cc0051d0 0x7fc6cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e5b4a640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0039c0 0x7fc6cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e5349640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc004810 0x7fc6cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e634b640 1 -- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc004810 msgr2=0x7fc6cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e634b640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc004810 0x7fc6cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e634b640 1 -- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0039c0 msgr2=0x7fc6cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e634b640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0039c0 0x7fc6cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e634b640 1 -- 172.21.15.3:0/2899178713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc003530 con 0x7fc6cc0051d0 2024-09-12T12:19:48.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.896+0000 7fc6e5349640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc004810 0x7fc6cc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:48.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e634b640 1 -- 172.21.15.3:0/2899178713 reap_dead start 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e634b640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6cc0051d0 0x7fc6cc00b9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009fc0 tx=0x7fc6d801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfbd1fb5d3f16a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8009c10 con 0x7fc6cc0051d0 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d80a5d10 con 0x7fc6cc0051d0 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e4b48640 1 -- 172.21.15.3:0/2899178713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4001d50 con 0x7fc6cc0051d0 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 -- 172.21.15.3:0/2899178713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6cc0051d0 msgr2=0x7fc6cc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6cc0051d0 0x7fc6cc00b9d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009fc0 tx=0x7fc6d801ccc0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 -- 172.21.15.3:0/2899178713 shutdown_connections 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/2899178713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6cc0051d0 0x7fc6cc00b9d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 -- 172.21.15.3:0/2899178713 >> 172.21.15.3:0/2899178713 conn(0x7fc6e008c000 msgr2=0x7fc6e00d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 -- 172.21.15.3:0/2899178713 shutdown_connections 2024-09-12T12:19:48.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 -- 172.21.15.3:0/2899178713 wait complete. 2024-09-12T12:19:48.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 Processor -- start 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 -- start start 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0051d0 0x7fc6e00cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.900+0000 7fc6e7dae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e00b3a70 0x7fc6e0159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e7dae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e00b8930 0x7fc6e00b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e7dae640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e00cf4d0 con 0x7fc6e00b3a70 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e7dae640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e015ebc0 con 0x7fc6cc0051d0 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e7dae640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e00cf650 con 0x7fc6e00b8930 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e00b3a70 0x7fc6e0159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e00b3a70 0x7fc6e0159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60808/0 (socket says 172.21.15.3:60808) 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 -- 172.21.15.3:0/436512426 learned_addr learned my addr 172.21.15.3:0/436512426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5b4a640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0051d0 0x7fc6e00cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e634b640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e00b8930 0x7fc6e00b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 -- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e00b8930 msgr2=0x7fc6e00b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e00b8930 0x7fc6e00b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 -- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0051d0 msgr2=0x7fc6e00cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0051d0 0x7fc6e00cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 -- 172.21.15.3:0/436512426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e00cc5d0 con 0x7fc6e00b3a70 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e634b640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e00b8930 0x7fc6e00b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5349640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e00b3a70 0x7fc6e0159ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4010b70 tx=0x7fc6d400e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6d27fc640 1 -- 172.21.15.3:0/436512426 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d401b070 con 0x7fc6e00b3a70 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6d27fc640 1 -- 172.21.15.3:0/436512426 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d4011040 con 0x7fc6e00b3a70 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6d27fc640 1 -- 172.21.15.3:0/436512426 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4010e70 con 0x7fc6e00b3a70 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e00cc8b0 con 0x7fc6e00b3a70 2024-09-12T12:19:48.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e00b51a0 con 0x7fc6e00b3a70 2024-09-12T12:19:48.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e00ceb60 con 0x7fc6e00b3a70 2024-09-12T12:19:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6d27fc640 1 -- 172.21.15.3:0/436512426 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc6d40143a0 con 0x7fc6e00b3a70 2024-09-12T12:19:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6d27fc640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c40678c0 0x7fc6c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6d27fc640 1 -- 172.21.15.3:0/436512426 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc6d4088360 con 0x7fc6e00b3a70 2024-09-12T12:19:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.904+0000 7fc6e5b4a640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c40678c0 0x7fc6c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:48.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.908+0000 7fc6e5b4a640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c40678c0 0x7fc6c4069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc000c10 tx=0x7fc6dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:48.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:48.908+0000 7fc6d27fc640 1 -- 172.21.15.3:0/436512426 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc6d40588e0 con 0x7fc6e00b3a70 2024-09-12T12:19:49.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.072+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6e00b9be0 con 0x7fc6e00b3a70 2024-09-12T12:19:49.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.076+0000 7fc6d27fc640 1 -- 172.21.15.3:0/436512426 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16618 (secure 0 0 0) 0x7fc6d405cbe0 con 0x7fc6e00b3a70 2024-09-12T12:19:49.078 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:49.079 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:48.098672+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:49.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c40678c0 msgr2=0x7fc6c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c40678c0 0x7fc6c4069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc000c10 tx=0x7fc6dc009040 comp rx=0 tx=0).stop 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e00b3a70 msgr2=0x7fc6e0159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e00b3a70 0x7fc6e0159ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4010b70 tx=0x7fc6d400e4d0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 shutdown_connections 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c40678c0 0x7fc6c4069d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6e00b8930 0x7fc6e00b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6e00b3a70 0x7fc6e0159ad0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 --2- 172.21.15.3:0/436512426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc0051d0 0x7fc6e00cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 >> 172.21.15.3:0/436512426 conn(0x7fc6e008c000 msgr2=0x7fc6e006e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 shutdown_connections 2024-09-12T12:19:49.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.080+0000 7fc6e7dae640 1 -- 172.21.15.3:0/436512426 wait complete. 2024-09-12T12:19:49.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:49.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.216+0000 7f9997ae0640 1 Processor -- start 2024-09-12T12:19:49.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.216+0000 7f9997ae0640 1 -- start start 2024-09-12T12:19:49.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f9997ae0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9990159bd0 0x7f999006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f9997ae0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9990159fb0 0x7f999015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f9997ae0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999006e400 con 0x7f999006f4a0 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f9997ae0640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f999006f010 con 0x7f9990159bd0 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f9997ae0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f999006e6e0 con 0x7f9990159fb0 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9990159bd0 0x7f999006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9990159bd0 0x7f999006dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60822/0 (socket says 172.21.15.3:60822) 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 -- 172.21.15.3:0/1543986035 learned_addr learned my addr 172.21.15.3:0/1543986035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999507b640 1 --2- 172.21.15.3:0/1543986035 >> v2:172.21.15.142:3300/0 conn(0x7f9990159fb0 0x7f999015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 -- 172.21.15.3:0/1543986035 >> v2:172.21.15.142:3300/0 conn(0x7f9990159fb0 msgr2=0x7f999015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 --2- 172.21.15.3:0/1543986035 >> v2:172.21.15.142:3300/0 conn(0x7f9990159fb0 0x7f999015d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 -- 172.21.15.3:0/1543986035 >> v1:172.21.15.142:6789/0 conn(0x7f999006f4a0 legacy=0x7f99900ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:49.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 -- 172.21.15.3:0/1543986035 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999006e9c0 con 0x7f9990159bd0 2024-09-12T12:19:49.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999587c640 1 --2- 172.21.15.3:0/1543986035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159bd0 0x7f999006dd40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f999015c850 tx=0x7f998001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16f31f1d0b65b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.220+0000 7f999487a640 1 -- 172.21.15.3:0/1543986035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800ab5f0 con 0x7f9990159bd0 2024-09-12T12:19:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999487a640 1 -- 172.21.15.3:0/1543986035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99800b0070 con 0x7f9990159bd0 2024-09-12T12:19:49.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999487a640 1 -- 172.21.15.3:0/1543986035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99800ab9b0 con 0x7f9990159bd0 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- 172.21.15.3:0/1543986035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159bd0 msgr2=0x7f999006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 --2- 172.21.15.3:0/1543986035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159bd0 0x7f999006dd40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f999015c850 tx=0x7f998001c4e0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- 172.21.15.3:0/1543986035 shutdown_connections 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 --2- 172.21.15.3:0/1543986035 >> v2:172.21.15.142:3300/0 conn(0x7f9990159fb0 0x7f999015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 --2- 172.21.15.3:0/1543986035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159bd0 0x7f999006dd40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- 172.21.15.3:0/1543986035 >> 172.21.15.3:0/1543986035 conn(0x7f99900b1d80 msgr2=0x7f99900b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- 172.21.15.3:0/1543986035 shutdown_connections 2024-09-12T12:19:49.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- 172.21.15.3:0/1543986035 wait complete. 2024-09-12T12:19:49.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 Processor -- start 2024-09-12T12:19:49.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- start start 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f999006f4a0 0x7f99901452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9990159bd0 0x7f99900b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159fb0 0x7f99900b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99901691e0 con 0x7f9990159fb0 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99900b3aa0 con 0x7f9990159bd0 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f9997ae0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9990169360 con 0x7f999006f4a0 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159fb0 0x7f99900b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159fb0 0x7f99900b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60834/0 (socket says 172.21.15.3:60834) 2024-09-12T12:19:49.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 -- 172.21.15.3:0/3999328245 learned_addr learned my addr 172.21.15.3:0/3999328245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:49.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999587c640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f999006f4a0 0x7f99901452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999507b640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9990159bd0 0x7f99900b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 -- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f999006f4a0 msgr2=0x7f99901452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f999006f4a0 0x7f99901452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 -- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9990159bd0 msgr2=0x7f99900b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9990159bd0 0x7f99900b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 -- 172.21.15.3:0/3999328245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9990154270 con 0x7f9990159fb0 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999587c640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f999006f4a0 0x7f99901452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999507b640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9990159bd0 0x7f99900b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.224+0000 7f999607d640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159fb0 0x7f99900b0bc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f998800c7b0 tx=0x7f998800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f99867fc640 1 -- 172.21.15.3:0/3999328245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988004b70 con 0x7f9990159fb0 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f99867fc640 1 -- 172.21.15.3:0/3999328245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9988004d10 con 0x7f9990159fb0 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99900cef20 con 0x7f9990159fb0 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f99867fc640 1 -- 172.21.15.3:0/3999328245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998800aa90 con 0x7f9990159fb0 2024-09-12T12:19:49.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99900cf4e0 con 0x7f9990159fb0 2024-09-12T12:19:49.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9990154440 con 0x7f9990159fb0 2024-09-12T12:19:49.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f99867fc640 1 -- 172.21.15.3:0/3999328245 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f998800ac30 con 0x7f9990159fb0 2024-09-12T12:19:49.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f99867fc640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99740678c0 0x7f9974069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f99867fc640 1 -- 172.21.15.3:0/3999328245 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9988085e40 con 0x7f9990159fb0 2024-09-12T12:19:49.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f999587c640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99740678c0 0x7f9974069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.228+0000 7f999587c640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99740678c0 0x7f9974069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99800af000 tx=0x7f99800a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:49.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.232+0000 7f99867fc640 1 -- 172.21.15.3:0/3999328245 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9988020070 con 0x7f9990159fb0 2024-09-12T12:19:49.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.400+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9990157280 con 0x7f9990159fb0 2024-09-12T12:19:49.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.400+0000 7f99867fc640 1 -- 172.21.15.3:0/3999328245 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16618 (secure 0 0 0) 0x7f9988056a10 con 0x7f9990159fb0 2024-09-12T12:19:49.404 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:49.404 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":20,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:48.098672+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99740678c0 msgr2=0x7f9974069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99740678c0 0x7f9974069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99800af000 tx=0x7f99800a7000 comp rx=0 tx=0).stop 2024-09-12T12:19:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159fb0 msgr2=0x7f99900b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159fb0 0x7f99900b0bc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f998800c7b0 tx=0x7f998800cc70 comp rx=0 tx=0).stop 2024-09-12T12:19:49.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 shutdown_connections 2024-09-12T12:19:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99740678c0 0x7f9974069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9990159fb0 0x7f99900b0bc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9990159bd0 0x7f99900b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 --2- 172.21.15.3:0/3999328245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f999006f4a0 0x7f99901452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 >> 172.21.15.3:0/3999328245 conn(0x7f99900b1d80 msgr2=0x7f999015fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 shutdown_connections 2024-09-12T12:19:49.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.404+0000 7f9997ae0640 1 -- 172.21.15.3:0/3999328245 wait complete. 2024-09-12T12:19:49.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:19:49.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 Processor -- start 2024-09-12T12:19:49.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 -- start start 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f570415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f570406f150 0x7f57040ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f570406f5b0 0x7f57040ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f570415d830 con 0x7f570406f5b0 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f570406e0b0 con 0x7f5704159dd0 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570aa7f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f570406e390 con 0x7f570406f150 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f570406f5b0 0x7f57040ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f570406f5b0 0x7f57040ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60846/0 (socket says 172.21.15.3:60846) 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 -- 172.21.15.3:0/233923942 learned_addr learned my addr 172.21.15.3:0/233923942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570881b640 1 --2- 172.21.15.3:0/233923942 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f570415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f5703fff640 1 --2- 172.21.15.3:0/233923942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f570406f150 0x7f57040ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 -- 172.21.15.3:0/233923942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f570406f150 msgr2=0x7f57040ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 --2- 172.21.15.3:0/233923942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f570406f150 0x7f57040ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 -- 172.21.15.3:0/233923942 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 msgr2=0x7f570415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 --2- 172.21.15.3:0/233923942 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f570415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570901c640 1 -- 172.21.15.3:0/233923942 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570406e670 con 0x7f570406f5b0 2024-09-12T12:19:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f5703fff640 1 --2- 172.21.15.3:0/233923942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f570406f150 0x7f57040ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:49.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.552+0000 7f570881b640 1 --2- 172.21.15.3:0/233923942 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f570415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570901c640 1 --2- 172.21.15.3:0/233923942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f57040ad5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f57040702b0 tx=0x7f56f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c85f408aaae7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f57037fe640 1 -- 172.21.15.3:0/233923942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f40aa610 con 0x7f570406f5b0 2024-09-12T12:19:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f57037fe640 1 -- 172.21.15.3:0/233923942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f40af070 con 0x7f570406f5b0 2024-09-12T12:19:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f57037fe640 1 -- 172.21.15.3:0/233923942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f40aa9d0 con 0x7f570406f5b0 2024-09-12T12:19:49.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 -- 172.21.15.3:0/233923942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 msgr2=0x7f57040ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 --2- 172.21.15.3:0/233923942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f57040ad5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f57040702b0 tx=0x7f56f401c4f0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 -- 172.21.15.3:0/233923942 shutdown_connections 2024-09-12T12:19:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 --2- 172.21.15.3:0/233923942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f57040ad5a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 --2- 172.21.15.3:0/233923942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f570406f150 0x7f57040ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 --2- 172.21.15.3:0/233923942 >> v2:172.21.15.142:3300/0 conn(0x7f5704159dd0 0x7f570415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 -- 172.21.15.3:0/233923942 >> 172.21.15.3:0/233923942 conn(0x7f57040b1d80 msgr2=0x7f57040b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:49.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 -- 172.21.15.3:0/233923942 shutdown_connections 2024-09-12T12:19:49.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 -- 172.21.15.3:0/233923942 wait complete. 2024-09-12T12:19:49.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 Processor -- start 2024-09-12T12:19:49.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.556+0000 7f570aa7f640 1 -- start start 2024-09-12T12:19:49.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406f150 0x7f5704158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f5704158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5704159dd0 0x7f5704094990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f57040b2a40 con 0x7f570406f5b0 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f570415a1b0 con 0x7f570406f150 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f57040b2bc0 con 0x7f5704159dd0 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f5704158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f5704158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60854/0 (socket says 172.21.15.3:60854) 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 -- 172.21.15.3:0/1319701377 learned_addr learned my addr 172.21.15.3:0/1319701377 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 -- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5704159dd0 msgr2=0x7f5704094990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5704159dd0 0x7f5704094990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 -- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406f150 msgr2=0x7f5704158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406f150 0x7f5704158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:49.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 -- 172.21.15.3:0/1319701377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5704099730 con 0x7f570406f5b0 2024-09-12T12:19:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f5703fff640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f5704158fd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c7f0 tx=0x7f56ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f57017fa640 1 -- 172.21.15.3:0/1319701377 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec015070 con 0x7f570406f5b0 2024-09-12T12:19:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f57017fa640 1 -- 172.21.15.3:0/1319701377 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec007820 con 0x7f570406f5b0 2024-09-12T12:19:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5704099a10 con 0x7f570406f5b0 2024-09-12T12:19:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f57017fa640 1 -- 172.21.15.3:0/1319701377 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec007c10 con 0x7f570406f5b0 2024-09-12T12:19:49.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570415dbb0 con 0x7f570406f5b0 2024-09-12T12:19:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570415d2f0 con 0x7f570406f5b0 2024-09-12T12:19:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f57017fa640 1 -- 172.21.15.3:0/1319701377 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f56ec018070 con 0x7f570406f5b0 2024-09-12T12:19:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f57017fa640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f56e40678c0 0x7f56e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f57017fa640 1 -- 172.21.15.3:0/1319701377 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56ec085e40 con 0x7f570406f5b0 2024-09-12T12:19:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.560+0000 7f570881b640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f56e40678c0 0x7f56e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:49.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.564+0000 7f570881b640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f56e40678c0 0x7f56e4069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56fc008ea0 tx=0x7f56fc0083d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:49.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.564+0000 7f57017fa640 1 -- 172.21.15.3:0/1319701377 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f56ec0568e0 con 0x7f570406f5b0 2024-09-12T12:19:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:49.768+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5704075a10 con 0x7f570406f5b0 2024-09-12T12:19:50.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.740+0000 7f57017fa640 1 -- 172.21.15.3:0/1319701377 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f56ec0561e0 con 0x7f570406f5b0 2024-09-12T12:19:50.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f56e40678c0 msgr2=0x7f56e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f56e40678c0 0x7f56e4069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56fc008ea0 tx=0x7f56fc0083d0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 msgr2=0x7f5704158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f5704158fd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c7f0 tx=0x7f56ec00ccb0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 shutdown_connections 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f56e40678c0 0x7f56e4069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5704159dd0 0x7f5704094990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f570406f5b0 0x7f5704158fd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 --2- 172.21.15.3:0/1319701377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f570406f150 0x7f5704158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 >> 172.21.15.3:0/1319701377 conn(0x7f57040b1d80 msgr2=0x7f5704075290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 shutdown_connections 2024-09-12T12:19:50.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.744+0000 7f570aa7f640 1 -- 172.21.15.3:0/1319701377 wait complete. 2024-09-12T12:19:50.768 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T12:19:50.769 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T12:19:50.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97adb07640 1 Processor -- start 2024-09-12T12:19:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97adb07640 1 -- start start 2024-09-12T12:19:50.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97adb07640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f97a80b6b10 0x7f97a806e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97adb07640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97a80b6350 0x7f97a80cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97adb07640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a80d1690 con 0x7f97a80b5f70 2024-09-12T12:19:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97adb07640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f97a80d1970 con 0x7f97a80b6350 2024-09-12T12:19:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97adb07640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f97a80d23f0 con 0x7f97a80b6b10 2024-09-12T12:19:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f97a80b6350 0x7f97a80cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a77fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f97a80b5f70 0x7f97a80d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45262/0 (socket says 172.21.15.3:45262) 2024-09-12T12:19:50.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a77fe640 1 -- 172.21.15.3:0/2232396248 learned_addr learned my addr 172.21.15.3:0/2232396248 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:50.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a6ffd640 1 --2- 172.21.15.3:0/2232396248 >> v2:172.21.15.142:3300/0 conn(0x7f97a80b6b10 0x7f97a806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a7fff640 1 -- 172.21.15.3:0/2232396248 >> v2:172.21.15.142:3300/0 conn(0x7f97a80b6b10 msgr2=0x7f97a806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a7fff640 1 --2- 172.21.15.3:0/2232396248 >> v2:172.21.15.142:3300/0 conn(0x7f97a80b6b10 0x7f97a806e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a7fff640 1 -- 172.21.15.3:0/2232396248 >> v1:172.21.15.3:6789/0 conn(0x7f97a80b5f70 legacy=0x7f97a80d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a7fff640 1 -- 172.21.15.3:0/2232396248 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80d26d0 con 0x7f97a80b6350 2024-09-12T12:19:50.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.900+0000 7f97a67fc640 1 -- 172.21.15.3:0/2232396248 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1366045746 0 0) 0x7f97a80d1690 con 0x7f97a80b5f70 2024-09-12T12:19:50.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97a7fff640 1 --2- 172.21.15.3:0/2232396248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a80cf000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97a808dd60 tx=0x7f97a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb3e43063c7974c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97a67fc640 1 -- 172.21.15.3:0/2232396248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a00b1070 con 0x7f97a80b6350 2024-09-12T12:19:50.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97a67fc640 1 -- 172.21.15.3:0/2232396248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a00ac5f0 con 0x7f97a80b6350 2024-09-12T12:19:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97a67fc640 1 -- 172.21.15.3:0/2232396248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a00ac9c0 con 0x7f97a80b6350 2024-09-12T12:19:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 -- 172.21.15.3:0/2232396248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 msgr2=0x7f97a80cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 --2- 172.21.15.3:0/2232396248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a80cf000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97a808dd60 tx=0x7f97a001c480 comp rx=0 tx=0).stop 2024-09-12T12:19:50.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 -- 172.21.15.3:0/2232396248 shutdown_connections 2024-09-12T12:19:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 --2- 172.21.15.3:0/2232396248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a80cf000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 --2- 172.21.15.3:0/2232396248 >> v2:172.21.15.142:3300/0 conn(0x7f97a80b6b10 0x7f97a806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 -- 172.21.15.3:0/2232396248 >> 172.21.15.3:0/2232396248 conn(0x7f97a809d880 msgr2=0x7f97a808c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 -- 172.21.15.3:0/2232396248 shutdown_connections 2024-09-12T12:19:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 -- 172.21.15.3:0/2232396248 wait complete. 2024-09-12T12:19:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 Processor -- start 2024-09-12T12:19:50.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.904+0000 7f97adb07640 1 -- start start 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97a80b5f70 0x7f97a80b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a80b6b10 0x7f97a815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97a816b720 con 0x7f97a80b6350 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97a816b5a0 con 0x7f97a80b6b10 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f97a816b8a0 con 0x7f97a80b5f70 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a815cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60878/0 (socket says 172.21.15.3:60878) 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 -- 172.21.15.3:0/3561706408 learned_addr learned my addr 172.21.15.3:0/3561706408 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a77fe640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97a80b5f70 0x7f97a80b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a7fff640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a80b6b10 0x7f97a815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 -- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97a80b5f70 msgr2=0x7f97a80b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97a80b5f70 0x7f97a80b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 -- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a80b6b10 msgr2=0x7f97a815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a80b6b10 0x7f97a815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 -- 172.21.15.3:0/3561706408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a80b8310 con 0x7f97a80b6350 2024-09-12T12:19:50.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a7fff640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a80b6b10 0x7f97a815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a6ffd640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a815cff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9790007ef0 tx=0x7f979000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f979ffff640 1 -- 172.21.15.3:0/3561706408 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790014620 con 0x7f97a80b6350 2024-09-12T12:19:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f979ffff640 1 -- 172.21.15.3:0/3561706408 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9790019070 con 0x7f97a80b6350 2024-09-12T12:19:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a814ce00 con 0x7f97a80b6350 2024-09-12T12:19:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f979ffff640 1 -- 172.21.15.3:0/3561706408 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790014a00 con 0x7f97a80b6350 2024-09-12T12:19:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a814d3c0 con 0x7f97a80b6350 2024-09-12T12:19:50.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a806e560 con 0x7f97a80b6350 2024-09-12T12:19:50.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f979ffff640 1 -- 172.21.15.3:0/3561706408 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9790014ba0 con 0x7f97a80b6350 2024-09-12T12:19:50.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f979ffff640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f97880678c0 0x7f9788069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f979ffff640 1 -- 172.21.15.3:0/3561706408 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f97900865d0 con 0x7f97a80b6350 2024-09-12T12:19:50.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.908+0000 7f97a77fe640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f97880678c0 0x7f9788069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.912+0000 7f97a77fe640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f97880678c0 0x7f9788069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9798000f70 tx=0x7f9798007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:50.912+0000 7f979ffff640 1 -- 172.21.15.3:0/3561706408 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9790057110 con 0x7f97a80b6350 2024-09-12T12:19:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.108+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f97a80d1c50 con 0x7f97a80b6350 2024-09-12T12:19:51.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.792+0000 7f979ffff640 1 -- 172.21.15.3:0/3561706408 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f9790056a10 con 0x7f97a80b6350 2024-09-12T12:19:51.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f97880678c0 msgr2=0x7f9788069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f97880678c0 0x7f9788069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9798000f70 tx=0x7f9798007c10 comp rx=0 tx=0).stop 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 msgr2=0x7f97a815cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a815cff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9790007ef0 tx=0x7f979000e4b0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 shutdown_connections 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f97880678c0 0x7f9788069d80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a80b6b10 0x7f97a815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97a80b6350 0x7f97a815cff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 --2- 172.21.15.3:0/3561706408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f97a80b5f70 0x7f97a80b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 >> 172.21.15.3:0/3561706408 conn(0x7f97a809d880 msgr2=0x7f97a806d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 shutdown_connections 2024-09-12T12:19:51.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.796+0000 7f97adb07640 1 -- 172.21.15.3:0/3561706408 wait complete. 2024-09-12T12:19:51.819 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T12:19:51.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:19:51.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f861df7a640 1 Processor -- start 2024-09-12T12:19:51.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f861df7a640 1 -- start start 2024-09-12T12:19:51.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f861df7a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159fb0 0x7f861806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:51.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f861df7a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f861815d2d0 0x7f86180ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:51.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f861df7a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f861806fbb0 con 0x7f8618159bd0 2024-09-12T12:19:51.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f861df7a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f861806f010 con 0x7f861815d2d0 2024-09-12T12:19:51.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f861df7a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f861806e400 con 0x7f8618159fb0 2024-09-12T12:19:51.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159fb0 0x7f861806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8617fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f861815d2d0 0x7f86180ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159fb0 0x7f861806dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:48978/0 (socket says 172.21.15.3:48978) 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8616ffd640 1 -- 172.21.15.3:0/920946459 learned_addr learned my addr 172.21.15.3:0/920946459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8617fff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f861815d2d0 0x7f86180ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:46996/0 (socket says 172.21.15.3:46996) 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f86177fe640 1 --1- 172.21.15.3:0/920946459 >> v1:172.21.15.142:6789/0 conn(0x7f8618159bd0 0x7f861806f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:58294/0 (socket says 172.21.15.3:58294) 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8617fff640 1 -- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159fb0 msgr2=0x7f861806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8617fff640 1 --2- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159fb0 0x7f861806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8617fff640 1 -- 172.21.15.3:0/920946459 >> v1:172.21.15.142:6789/0 conn(0x7f8618159bd0 legacy=0x7f861806f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8617fff640 1 -- 172.21.15.3:0/920946459 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861806e6e0 con 0x7f861815d2d0 2024-09-12T12:19:51.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.948+0000 7f8616ffd640 1 --2- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159fb0 0x7f861806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:51.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f8617fff640 1 --2- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 0x7f86180ad060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8610009d90 tx=0x7f861001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a3f93b09f190ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:51.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f86167fc640 1 -- 172.21.15.3:0/920946459 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100b2070 con 0x7f861815d2d0 2024-09-12T12:19:51.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f86167fc640 1 -- 172.21.15.3:0/920946459 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86100a5910 con 0x7f861815d2d0 2024-09-12T12:19:51.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f86167fc640 1 -- 172.21.15.3:0/920946459 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100a5ce0 con 0x7f861815d2d0 2024-09-12T12:19:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 -- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 msgr2=0x7f86180ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 --2- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 0x7f86180ad060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8610009d90 tx=0x7f861001c940 comp rx=0 tx=0).stop 2024-09-12T12:19:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 -- 172.21.15.3:0/920946459 shutdown_connections 2024-09-12T12:19:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 --2- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159fb0 0x7f861806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 --2- 172.21.15.3:0/920946459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 0x7f86180ad060 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 -- 172.21.15.3:0/920946459 >> 172.21.15.3:0/920946459 conn(0x7f86180b1d80 msgr2=0x7f86180b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:51.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 -- 172.21.15.3:0/920946459 shutdown_connections 2024-09-12T12:19:51.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 -- 172.21.15.3:0/920946459 wait complete. 2024-09-12T12:19:51.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 Processor -- start 2024-09-12T12:19:51.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.952+0000 7f861df7a640 1 -- start start 2024-09-12T12:19:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159bd0 0x7f86181590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8618159fb0 0x7f8618145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 0x7f86180b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:51.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8618169360 con 0x7f8618159fb0 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86181691e0 con 0x7f861815d2d0 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f86181694e0 con 0x7f8618159bd0 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8618159fb0 0x7f8618145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8618159fb0 0x7f8618145220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60892/0 (socket says 172.21.15.3:60892) 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 -- 172.21.15.3:0/2263878928 learned_addr learned my addr 172.21.15.3:0/2263878928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f86177fe640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159bd0 0x7f86181590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8617fff640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 0x7f86180b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 -- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159bd0 msgr2=0x7f86181590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159bd0 0x7f86181590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 -- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 msgr2=0x7f86180b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 0x7f86180b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 -- 172.21.15.3:0/2263878928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86180b11c0 con 0x7f8618159fb0 2024-09-12T12:19:51.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f86177fe640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159bd0 0x7f86181590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f8616ffd640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8618159fb0 0x7f8618145220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f860c00e9a0 tx=0x7f860c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f85fffff640 1 -- 172.21.15.3:0/2263878928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c017070 con 0x7f8618159fb0 2024-09-12T12:19:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f85fffff640 1 -- 172.21.15.3:0/2263878928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c00ca00 con 0x7f8618159fb0 2024-09-12T12:19:51.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f85fffff640 1 -- 172.21.15.3:0/2263878928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c00cdb0 con 0x7f8618159fb0 2024-09-12T12:19:51.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86180b14a0 con 0x7f8618159fb0 2024-09-12T12:19:51.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8618071e40 con 0x7f8618159fb0 2024-09-12T12:19:51.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86180ad060 con 0x7f8618159fb0 2024-09-12T12:19:51.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f85fffff640 1 -- 172.21.15.3:0/2263878928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f860c01a070 con 0x7f8618159fb0 2024-09-12T12:19:51.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f85fffff640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86040678c0 0x7f8604069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:51.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.956+0000 7f85fffff640 1 -- 172.21.15.3:0/2263878928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f860c085ec0 con 0x7f8618159fb0 2024-09-12T12:19:51.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.960+0000 7f86177fe640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86040678c0 0x7f8604069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:51.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.960+0000 7f86177fe640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86040678c0 0x7f8604069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8608007fd0 tx=0x7f8608009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:51.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:51.964+0000 7f85fffff640 1 -- 172.21.15.3:0/2263878928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f860c056a90 con 0x7f8618159fb0 2024-09-12T12:19:52.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.160+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86180b2b00 con 0x7f8618159fb0 2024-09-12T12:19:52.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f85fffff640 1 -- 172.21.15.3:0/2263878928 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f860c056390 con 0x7f8618159fb0 2024-09-12T12:19:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:52.165 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:19:52.165 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:19:52.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86040678c0 msgr2=0x7f8604069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86040678c0 0x7f8604069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8608007fd0 tx=0x7f8608009040 comp rx=0 tx=0).stop 2024-09-12T12:19:52.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8618159fb0 msgr2=0x7f8618145220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8618159fb0 0x7f8618145220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f860c00e9a0 tx=0x7f860c00ee60 comp rx=0 tx=0).stop 2024-09-12T12:19:52.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 shutdown_connections 2024-09-12T12:19:52.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86040678c0 0x7f8604069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f861815d2d0 0x7f86180b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8618159fb0 0x7f8618145220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 --2- 172.21.15.3:0/2263878928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8618159bd0 0x7f86181590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 >> 172.21.15.3:0/2263878928 conn(0x7f86180b1d80 msgr2=0x7f861815b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:52.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 shutdown_connections 2024-09-12T12:19:52.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.164+0000 7f861df7a640 1 -- 172.21.15.3:0/2263878928 wait complete. 2024-09-12T12:19:52.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f798134c640 1 Processor -- start 2024-09-12T12:19:52.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f798134c640 1 -- start start 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f798134c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f797c159fb0 0x7f797c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f798134c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f797c15d2d0 0x7f797c0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f798134c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f797c06fbb0 con 0x7f797c159bd0 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f798134c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f797c06f010 con 0x7f797c159fb0 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f798134c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f797c15d760 con 0x7f797c15d2d0 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f797affd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f797c159bd0 0x7f797c06f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45270/0 (socket says 172.21.15.3:45270) 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f797affd640 1 -- 172.21.15.3:0/4072864330 learned_addr learned my addr 172.21.15.3:0/4072864330 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f7973fff640 1 --2- 172.21.15.3:0/4072864330 >> v2:172.21.15.3:3300/0 conn(0x7f797c159fb0 0x7f797c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f797b7fe640 1 --2- 172.21.15.3:0/4072864330 >> v2:172.21.15.142:3300/0 conn(0x7f797c15d2d0 0x7f797c0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f797a7fc640 1 -- 172.21.15.3:0/4072864330 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288395176 0 0) 0x7f797c06fbb0 con 0x7f797c159bd0 2024-09-12T12:19:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f797a7fc640 1 -- 172.21.15.3:0/4072864330 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7968003530 con 0x7f797c159bd0 2024-09-12T12:19:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f7973fff640 1 -- 172.21.15.3:0/4072864330 >> v2:172.21.15.142:3300/0 conn(0x7f797c15d2d0 msgr2=0x7f797c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f7973fff640 1 --2- 172.21.15.3:0/4072864330 >> v2:172.21.15.142:3300/0 conn(0x7f797c15d2d0 0x7f797c0ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f7973fff640 1 -- 172.21.15.3:0/4072864330 >> v1:172.21.15.3:6789/0 conn(0x7f797c159bd0 legacy=0x7f797c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.312+0000 7f7973fff640 1 -- 172.21.15.3:0/4072864330 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c06e400 con 0x7f797c159fb0 2024-09-12T12:19:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f7973fff640 1 --2- 172.21.15.3:0/4072864330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159fb0 0x7f797c06dd40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f797c06ef80 tx=0x7f796401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a28f91a0750ac00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797a7fc640 1 -- 172.21.15.3:0/4072864330 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79640b2070 con 0x7f797c159fb0 2024-09-12T12:19:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797a7fc640 1 -- 172.21.15.3:0/4072864330 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79640a9070 con 0x7f797c159fb0 2024-09-12T12:19:52.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797a7fc640 1 -- 172.21.15.3:0/4072864330 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79640ad880 con 0x7f797c159fb0 2024-09-12T12:19:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- 172.21.15.3:0/4072864330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159fb0 msgr2=0x7f797c06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 --2- 172.21.15.3:0/4072864330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159fb0 0x7f797c06dd40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f797c06ef80 tx=0x7f796401c450 comp rx=0 tx=0).stop 2024-09-12T12:19:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- 172.21.15.3:0/4072864330 shutdown_connections 2024-09-12T12:19:52.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 --2- 172.21.15.3:0/4072864330 >> v2:172.21.15.142:3300/0 conn(0x7f797c15d2d0 0x7f797c0ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 --2- 172.21.15.3:0/4072864330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159fb0 0x7f797c06dd40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- 172.21.15.3:0/4072864330 >> 172.21.15.3:0/4072864330 conn(0x7f797c0b1d80 msgr2=0x7f797c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- 172.21.15.3:0/4072864330 shutdown_connections 2024-09-12T12:19:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- 172.21.15.3:0/4072864330 wait complete. 2024-09-12T12:19:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 Processor -- start 2024-09-12T12:19:52.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- start start 2024-09-12T12:19:52.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159bd0 0x7f797c145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f797c159fb0 0x7f797c0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f797c15d2d0 0x7f797c15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f797c169360 con 0x7f797c159bd0 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1691e0 con 0x7f797c15d2d0 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f797c1694e0 con 0x7f797c159fb0 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159bd0 0x7f797c145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159bd0 0x7f797c145870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60920/0 (socket says 172.21.15.3:60920) 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 -- 172.21.15.3:0/127758467 learned_addr learned my addr 172.21.15.3:0/127758467 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 -- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f797c159fb0 msgr2=0x7f797c0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f797c159fb0 0x7f797c0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 -- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f797c15d2d0 msgr2=0x7f797c15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f797c15d2d0 0x7f797c15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 -- 172.21.15.3:0/127758467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c0500c0 con 0x7f797c159bd0 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f797affd640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159bd0 0x7f797c145870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c450 tx=0x7f796c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:52.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f79737fe640 1 -- 172.21.15.3:0/127758467 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c017070 con 0x7f797c159bd0 2024-09-12T12:19:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f79737fe640 1 -- 172.21.15.3:0/127758467 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796c005cf0 con 0x7f797c159bd0 2024-09-12T12:19:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c154aa0 con 0x7f797c159bd0 2024-09-12T12:19:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.316+0000 7f79737fe640 1 -- 172.21.15.3:0/127758467 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c007290 con 0x7f797c159bd0 2024-09-12T12:19:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.320+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c155060 con 0x7f797c159bd0 2024-09-12T12:19:52.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.320+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c0ad060 con 0x7f797c159bd0 2024-09-12T12:19:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.320+0000 7f79737fe640 1 -- 172.21.15.3:0/127758467 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f796c01a070 con 0x7f797c159bd0 2024-09-12T12:19:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.320+0000 7f79737fe640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f795c067910 0x7f795c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.320+0000 7f79737fe640 1 -- 172.21.15.3:0/127758467 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f796c085100 con 0x7f797c159bd0 2024-09-12T12:19:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.320+0000 7f7973fff640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f795c067910 0x7f795c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.320+0000 7f7973fff640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f795c067910 0x7f795c069dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79640af000 tx=0x7f7964004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:52.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.324+0000 7f79737fe640 1 -- 172.21.15.3:0/127758467 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f796c055c20 con 0x7f797c159bd0 2024-09-12T12:19:52.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.488+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f797c0b2b00 con 0x7f797c159bd0 2024-09-12T12:19:52.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f79737fe640 1 -- 172.21.15.3:0/127758467 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f796c055520 con 0x7f797c159bd0 2024-09-12T12:19:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:52.494 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:49.738546+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f795c067910 msgr2=0x7f795c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f795c067910 0x7f795c069dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79640af000 tx=0x7f7964004740 comp rx=0 tx=0).stop 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159bd0 msgr2=0x7f797c145870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159bd0 0x7f797c145870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c450 tx=0x7f796c00c910 comp rx=0 tx=0).stop 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 shutdown_connections 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f795c067910 0x7f795c069dd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f797c15d2d0 0x7f797c15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f797c159fb0 0x7f797c0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.492+0000 7f798134c640 1 --2- 172.21.15.3:0/127758467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f797c159bd0 0x7f797c145870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.496+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 >> 172.21.15.3:0/127758467 conn(0x7f797c0b1d80 msgr2=0x7f797c1529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.496+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 shutdown_connections 2024-09-12T12:19:52.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.496+0000 7f798134c640 1 -- 172.21.15.3:0/127758467 wait complete. 2024-09-12T12:19:52.513 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:19:52.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:19:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.624+0000 7fc1c98e8640 1 Processor -- start 2024-09-12T12:19:52.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.624+0000 7fc1c98e8640 1 -- start start 2024-09-12T12:19:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c98e8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1c415e4b0 0x7fc1c40ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c98e8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1c40b42b0 0x7fc1c4071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c98e8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c40ad5a0 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c98e8640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c406e810 con 0x7fc1c40b42b0 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c98e8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c406dee0 con 0x7fc1c415e4b0 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1c40b42b0 0x7fc1c4071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1c40b42b0 0x7fc1c4071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60928/0 (socket says 172.21.15.3:60928) 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 -- 172.21.15.3:0/3947739553 learned_addr learned my addr 172.21.15.3:0/3947739553 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c2ffd640 1 --2- 172.21.15.3:0/3947739553 >> v2:172.21.15.142:3300/0 conn(0x7fc1c415e4b0 0x7fc1c40ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 -- 172.21.15.3:0/3947739553 >> v2:172.21.15.142:3300/0 conn(0x7fc1c415e4b0 msgr2=0x7fc1c40ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 --2- 172.21.15.3:0/3947739553 >> v2:172.21.15.142:3300/0 conn(0x7fc1c415e4b0 0x7fc1c40ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 -- 172.21.15.3:0/3947739553 >> v1:172.21.15.142:6789/0 conn(0x7fc1c40b4bd0 legacy=0x7fc1c406eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 -- 172.21.15.3:0/3947739553 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c406e1c0 con 0x7fc1c40b42b0 2024-09-12T12:19:52.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c27fc640 1 --2- 172.21.15.3:0/3947739553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b42b0 0x7fc1c4071da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4071350 tx=0x7fc1b801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5de3e96ac47d255 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c1ffb640 1 -- 172.21.15.3:0/3947739553 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80b0070 con 0x7fc1c40b42b0 2024-09-12T12:19:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c1ffb640 1 -- 172.21.15.3:0/3947739553 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b80ab600 con 0x7fc1c40b42b0 2024-09-12T12:19:52.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.628+0000 7fc1c1ffb640 1 -- 172.21.15.3:0/3947739553 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80ab9d0 con 0x7fc1c40b42b0 2024-09-12T12:19:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/3947739553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b42b0 msgr2=0x7fc1c4071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/3947739553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b42b0 0x7fc1c4071da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4071350 tx=0x7fc1b801c490 comp rx=0 tx=0).stop 2024-09-12T12:19:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/3947739553 shutdown_connections 2024-09-12T12:19:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/3947739553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b42b0 0x7fc1c4071da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/3947739553 >> v2:172.21.15.142:3300/0 conn(0x7fc1c415e4b0 0x7fc1c40ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/3947739553 >> 172.21.15.3:0/3947739553 conn(0x7fc1c40b1d80 msgr2=0x7fc1c40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/3947739553 shutdown_connections 2024-09-12T12:19:52.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/3947739553 wait complete. 2024-09-12T12:19:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 Processor -- start 2024-09-12T12:19:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- start start 2024-09-12T12:19:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b4bd0 0x7fc1c41455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c415e4b0 0x7fc1c4160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c4161480 0x7fc1c4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40b3390 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c406f7c0 con 0x7fc1c415e4b0 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c416ba00 con 0x7fc1c4161480 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b4bd0 0x7fc1c41455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b4bd0 0x7fc1c41455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60934/0 (socket says 172.21.15.3:60934) 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 -- 172.21.15.3:0/2470220371 learned_addr learned my addr 172.21.15.3:0/2470220371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c27fc640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c415e4b0 0x7fc1c4160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 -- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c4161480 msgr2=0x7fc1c4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c37fe640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c4161480 0x7fc1c4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c4161480 0x7fc1c4157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 -- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c415e4b0 msgr2=0x7fc1c4160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c415e4b0 0x7fc1c4160f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 -- 172.21.15.3:0/2470220371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c4145b00 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c2ffd640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b4bd0 0x7fc1c41455c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400d620 tx=0x7fc1b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1abfff640 1 -- 172.21.15.3:0/2470220371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400b620 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1abfff640 1 -- 172.21.15.3:0/2470220371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b4019070 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c4161a50 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1abfff640 1 -- 172.21.15.3:0/2470220371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400b9e0 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c4155540 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.632+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c4071da0 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.636+0000 7fc1abfff640 1 -- 172.21.15.3:0/2470220371 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc1b400bb80 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.636+0000 7fc1abfff640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1b00678c0 0x7fc1b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:52.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.636+0000 7fc1abfff640 1 -- 172.21.15.3:0/2470220371 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc1b4086180 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.636+0000 7fc1c27fc640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1b00678c0 0x7fc1b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:52.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.636+0000 7fc1c27fc640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1b00678c0 0x7fc1b0069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ad000 tx=0x7fc1b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:52.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.640+0000 7fc1abfff640 1 -- 172.21.15.3:0/2470220371 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1b4056c20 con 0x7fc1c40b4bd0 2024-09-12T12:19:52.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.800+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1c40af380 con 0x7fc1b00678c0 2024-09-12T12:19:52.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.816+0000 7fc1abfff640 1 -- 172.21.15.3:0/2470220371 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240914 (secure 0 0 0) 0x7fc1c40af380 con 0x7fc1b00678c0 2024-09-12T12:19:52.823 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:19:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1b00678c0 msgr2=0x7fc1b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1b00678c0 0x7fc1b0069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ad000 tx=0x7fc1b8008040 comp rx=0 tx=0).stop 2024-09-12T12:19:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b4bd0 msgr2=0x7fc1c41455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b4bd0 0x7fc1c41455c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400d620 tx=0x7fc1b400dae0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 shutdown_connections 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1b00678c0 0x7fc1b0069d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1c4161480 0x7fc1c4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1c415e4b0 0x7fc1c4160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 --2- 172.21.15.3:0/2470220371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1c40b4bd0 0x7fc1c41455c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 >> 172.21.15.3:0/2470220371 conn(0x7fc1c40b1d80 msgr2=0x7fc1c40b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 shutdown_connections 2024-09-12T12:19:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:52.820+0000 7fc1c98e8640 1 -- 172.21.15.3:0/2470220371 wait complete. 2024-09-12T12:19:52.840 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T12:19:51.656235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218776,"kb_used_data":2548,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755944,"statfs":{"total":773094113280,"available":772870086656,"internally_reserved":0,"allocated":2609152,"data_stored":1623275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.934925"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.740554+0000","last_change":"2024-09-12T12:19:47.740554+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.740554+0000","last_undegraded":"2024-09-12T12:19:47.740554+0000","last_fullsized":"2024-09-12T12:19:47.740554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739346+0000","last_change":"2024-09-12T12:19:47.739346+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739346+0000","last_undegraded":"2024-09-12T12:19:47.739346+0000","last_fullsized":"2024-09-12T12:19:47.739346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739200+0000","last_change":"2024-09-12T12:19:47.739200+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739200+0000","last_undegraded":"2024-09-12T12:19:47.739200+0000","last_fullsized":"2024-09-12T12:19:47.739200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738680+0000","last_change":"2024-09-12T12:19:47.738680+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738680+0000","last_undegraded":"2024-09-12T12:19:47.738680+0000","last_fullsized":"2024-09-12T12:19:47.738680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735706+0000","last_change":"2024-09-12T12:19:47.735706+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735706+0000","last_undegraded":"2024-09-12T12:19:47.735706+0000","last_fullsized":"2024-09-12T12:19:47.735706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761667+0000","last_change":"2024-09-12T12:19:47.757834+0000","last_active":"2024-09-12T12:19:47.761667+0000","last_peered":"2024-09-12T12:19:47.761667+0000","last_clean":"2024-09-12T12:19:47.761667+0000","last_became_active":"2024-09-12T12:19:47.757145+0000","last_became_peered":"2024-09-12T12:19:47.757145+0000","last_unstale":"2024-09-12T12:19:47.761667+0000","last_undegraded":"2024-09-12T12:19:47.761667+0000","last_fullsized":"2024-09-12T12:19:47.761667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:59.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731060+0000","last_change":"2024-09-12T12:19:46.731060+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731060+0000","last_undegraded":"2024-09-12T12:19:46.731060+0000","last_fullsized":"2024-09-12T12:19:46.731060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761691+0000","last_change":"2024-09-12T12:19:47.758883+0000","last_active":"2024-09-12T12:19:47.761691+0000","last_peered":"2024-09-12T12:19:47.761691+0000","last_clean":"2024-09-12T12:19:47.761691+0000","last_became_active":"2024-09-12T12:19:47.758396+0000","last_became_peered":"2024-09-12T12:19:47.758396+0000","last_unstale":"2024-09-12T12:19:47.761691+0000","last_undegraded":"2024-09-12T12:19:47.761691+0000","last_fullsized":"2024-09-12T12:19:47.761691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:42.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738678+0000","last_change":"2024-09-12T12:19:47.738678+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738678+0000","last_undegraded":"2024-09-12T12:19:47.738678+0000","last_fullsized":"2024-09-12T12:19:47.738678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729394+0000","last_change":"2024-09-12T12:19:46.729394+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729394+0000","last_undegraded":"2024-09-12T12:19:46.729394+0000","last_fullsized":"2024-09-12T12:19:46.729394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.732973+0000","last_change":"2024-09-12T12:19:46.732973+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.732973+0000","last_undegraded":"2024-09-12T12:19:46.732973+0000","last_fullsized":"2024-09-12T12:19:46.732973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729401+0000","last_change":"2024-09-12T12:19:46.729401+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729401+0000","last_undegraded":"2024-09-12T12:19:46.729401+0000","last_fullsized":"2024-09-12T12:19:46.729401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.728277+0000","last_change":"2024-09-12T12:19:46.728277+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.728277+0000","last_undegraded":"2024-09-12T12:19:46.728277+0000","last_fullsized":"2024-09-12T12:19:46.728277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731424+0000","last_change":"2024-09-12T12:19:46.731424+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731424+0000","last_undegraded":"2024-09-12T12:19:46.731424+0000","last_fullsized":"2024-09-12T12:19:46.731424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726355+0000","last_change":"2024-09-12T12:19:46.726355+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726355+0000","last_undegraded":"2024-09-12T12:19:46.726355+0000","last_fullsized":"2024-09-12T12:19:46.726355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731339+0000","last_change":"2024-09-12T12:19:46.731339+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731339+0000","last_undegraded":"2024-09-12T12:19:46.731339+0000","last_fullsized":"2024-09-12T12:19:46.731339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.732689+0000","last_change":"2024-09-12T12:19:46.732689+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.732689+0000","last_undegraded":"2024-09-12T12:19:46.732689+0000","last_fullsized":"2024-09-12T12:19:46.732689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.734356+0000","last_change":"2024-09-12T12:19:46.734356+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.734356+0000","last_undegraded":"2024-09-12T12:19:46.734356+0000","last_fullsized":"2024-09-12T12:19:46.734356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.733700+0000","last_change":"2024-09-12T12:19:46.733700+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.733700+0000","last_undegraded":"2024-09-12T12:19:46.733700+0000","last_fullsized":"2024-09-12T12:19:46.733700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.735680+0000","last_change":"2024-09-12T12:19:46.735680+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.735680+0000","last_undegraded":"2024-09-12T12:19:46.735680+0000","last_fullsized":"2024-09-12T12:19:46.735680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.727437+0000","last_change":"2024-09-12T12:19:46.727437+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.727437+0000","last_undegraded":"2024-09-12T12:19:46.727437+0000","last_fullsized":"2024-09-12T12:19:46.727437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.736700+0000","last_change":"2024-09-12T12:19:46.736700+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.736700+0000","last_undegraded":"2024-09-12T12:19:46.736700+0000","last_fullsized":"2024-09-12T12:19:46.736700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-12T12:19:46.728343+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:19:46.728343+0000","last_peered":"2024-09-12T12:19:46.728343+0000","last_clean":"2024-09-12T12:19:46.728343+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:19:46.728343+0000","last_undegraded":"2024-09-12T12:19:46.728343+0000","last_fullsized":"2024-09-12T12:19:46.728343+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.730089+0000","last_change":"2024-09-12T12:19:46.730089+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.730089+0000","last_undegraded":"2024-09-12T12:19:46.730089+0000","last_fullsized":"2024-09-12T12:19:46.730089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739193+0000","last_change":"2024-09-12T12:19:47.739193+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739193+0000","last_undegraded":"2024-09-12T12:19:47.739193+0000","last_fullsized":"2024-09-12T12:19:47.739193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723642+0000","last_change":"2024-09-12T12:19:46.723642+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723642+0000","last_undegraded":"2024-09-12T12:19:46.723642+0000","last_fullsized":"2024-09-12T12:19:46.723642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.734967+0000","last_change":"2024-09-12T12:19:46.734967+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.734967+0000","last_undegraded":"2024-09-12T12:19:46.734967+0000","last_fullsized":"2024-09-12T12:19:46.734967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729087+0000","last_change":"2024-09-12T12:19:46.729087+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729087+0000","last_undegraded":"2024-09-12T12:19:46.729087+0000","last_fullsized":"2024-09-12T12:19:46.729087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726297+0000","last_change":"2024-09-12T12:19:46.726297+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726297+0000","last_undegraded":"2024-09-12T12:19:46.726297+0000","last_fullsized":"2024-09-12T12:19:46.726297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723726+0000","last_change":"2024-09-12T12:19:46.723726+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723726+0000","last_undegraded":"2024-09-12T12:19:46.723726+0000","last_fullsized":"2024-09-12T12:19:46.723726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726266+0000","last_change":"2024-09-12T12:19:46.726266+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726266+0000","last_undegraded":"2024-09-12T12:19:46.726266+0000","last_fullsized":"2024-09-12T12:19:46.726266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735219+0000","last_change":"2024-09-12T12:19:47.735219+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735219+0000","last_undegraded":"2024-09-12T12:19:47.735219+0000","last_fullsized":"2024-09-12T12:19:47.735219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731477+0000","last_change":"2024-09-12T12:19:46.731477+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731477+0000","last_undegraded":"2024-09-12T12:19:46.731477+0000","last_fullsized":"2024-09-12T12:19:46.731477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726910+0000","last_change":"2024-09-12T12:19:46.726910+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726910+0000","last_undegraded":"2024-09-12T12:19:46.726910+0000","last_fullsized":"2024-09-12T12:19:46.726910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.727078+0000","last_change":"2024-09-12T12:19:46.727078+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.727078+0000","last_undegraded":"2024-09-12T12:19:46.727078+0000","last_fullsized":"2024-09-12T12:19:46.727078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735412+0000","last_change":"2024-09-12T12:19:47.735412+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735412+0000","last_undegraded":"2024-09-12T12:19:47.735412+0000","last_fullsized":"2024-09-12T12:19:47.735412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726299+0000","last_change":"2024-09-12T12:19:46.726299+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726299+0000","last_undegraded":"2024-09-12T12:19:46.726299+0000","last_fullsized":"2024-09-12T12:19:46.726299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.736208+0000","last_change":"2024-09-12T12:19:47.736208+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.736208+0000","last_undegraded":"2024-09-12T12:19:47.736208+0000","last_fullsized":"2024-09-12T12:19:47.736208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723874+0000","last_change":"2024-09-12T12:19:46.723874+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723874+0000","last_undegraded":"2024-09-12T12:19:46.723874+0000","last_fullsized":"2024-09-12T12:19:46.723874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723551+0000","last_change":"2024-09-12T12:19:46.723551+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723551+0000","last_undegraded":"2024-09-12T12:19:46.723551+0000","last_fullsized":"2024-09-12T12:19:46.723551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.725466+0000","last_change":"2024-09-12T12:19:46.725466+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.725466+0000","last_undegraded":"2024-09-12T12:19:46.725466+0000","last_fullsized":"2024-09-12T12:19:46.725466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761902+0000","last_change":"2024-09-12T12:19:47.759563+0000","last_active":"2024-09-12T12:19:47.761902+0000","last_peered":"2024-09-12T12:19:47.761902+0000","last_clean":"2024-09-12T12:19:47.761902+0000","last_became_active":"2024-09-12T12:19:47.759012+0000","last_became_peered":"2024-09-12T12:19:47.759012+0000","last_unstale":"2024-09-12T12:19:47.761902+0000","last_undegraded":"2024-09-12T12:19:47.761902+0000","last_fullsized":"2024-09-12T12:19:47.761902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735760+0000","last_change":"2024-09-12T12:19:47.735760+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735760+0000","last_undegraded":"2024-09-12T12:19:47.735760+0000","last_fullsized":"2024-09-12T12:19:47.735760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726850+0000","last_change":"2024-09-12T12:19:46.726850+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726850+0000","last_undegraded":"2024-09-12T12:19:46.726850+0000","last_fullsized":"2024-09-12T12:19:46.726850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":60808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":644901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":644901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:19:52.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:19:53.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 Processor -- start 2024-09-12T12:19:53.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 -- start start 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6b90159bd0 0x7f6b9006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b90159fb0 0x7f6b9015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b9006f4a0 0x7f6b900ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6b9006f010 con 0x7f6b90159fb0 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6b9006e400 con 0x7f6b90159bd0 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b96e08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9006e6e0 con 0x7f6b9006f4a0 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b90159fb0 0x7f6b9015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6b90159fb0 0x7f6b9015d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60950/0 (socket says 172.21.15.3:60950) 2024-09-12T12:19:53.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 -- 172.21.15.3:0/1403267688 learned_addr learned my addr 172.21.15.3:0/1403267688 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b94ba4640 1 --2- 172.21.15.3:0/1403267688 >> v2:172.21.15.142:3300/0 conn(0x7f6b90159bd0 0x7f6b9006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 -- 172.21.15.3:0/1403267688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b9006f4a0 msgr2=0x7f6b900ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b953a5640 1 --2- 172.21.15.3:0/1403267688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b9006f4a0 0x7f6b900ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 --2- 172.21.15.3:0/1403267688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b9006f4a0 0x7f6b900ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 -- 172.21.15.3:0/1403267688 >> v2:172.21.15.142:3300/0 conn(0x7f6b90159bd0 msgr2=0x7f6b9006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 --2- 172.21.15.3:0/1403267688 >> v2:172.21.15.142:3300/0 conn(0x7f6b90159bd0 0x7f6b9006dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.008+0000 7f6b8ffff640 1 -- 172.21.15.3:0/1403267688 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9006e9c0 con 0x7f6b90159fb0 2024-09-12T12:19:53.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b8ffff640 1 --2- 172.21.15.3:0/1403267688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b90159fb0 0x7f6b9015d2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b90071320 tx=0x7f6b8401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3e5c699354670c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b8f7fe640 1 -- 172.21.15.3:0/1403267688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840b0070 con 0x7f6b90159fb0 2024-09-12T12:19:53.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b8f7fe640 1 -- 172.21.15.3:0/1403267688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b840ab600 con 0x7f6b90159fb0 2024-09-12T12:19:53.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b8f7fe640 1 -- 172.21.15.3:0/1403267688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840ab9d0 con 0x7f6b90159fb0 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 -- 172.21.15.3:0/1403267688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b90159fb0 msgr2=0x7f6b9015d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 --2- 172.21.15.3:0/1403267688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b90159fb0 0x7f6b9015d2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b90071320 tx=0x7f6b8401c520 comp rx=0 tx=0).stop 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 -- 172.21.15.3:0/1403267688 shutdown_connections 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 --2- 172.21.15.3:0/1403267688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b9006f4a0 0x7f6b900ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 --2- 172.21.15.3:0/1403267688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b90159fb0 0x7f6b9015d2d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 --2- 172.21.15.3:0/1403267688 >> v2:172.21.15.142:3300/0 conn(0x7f6b90159bd0 0x7f6b9006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 -- 172.21.15.3:0/1403267688 >> 172.21.15.3:0/1403267688 conn(0x7f6b900b1d80 msgr2=0x7f6b900b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 -- 172.21.15.3:0/1403267688 shutdown_connections 2024-09-12T12:19:53.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 -- 172.21.15.3:0/1403267688 wait complete. 2024-09-12T12:19:53.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 Processor -- start 2024-09-12T12:19:53.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.012+0000 7f6b96e08640 1 -- start start 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b9006f4a0 0x7f6b900cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90159bd0 0x7f6b90099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90159fb0 0x7f6b90073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9015fe40 con 0x7f6b9006f4a0 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6b900b3aa0 con 0x7f6b90159bd0 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9015ffc0 con 0x7f6b90159fb0 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b9006f4a0 0x7f6b900cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b9006f4a0 0x7f6b900cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60966/0 (socket says 172.21.15.3:60966) 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 -- 172.21.15.3:0/2636856759 learned_addr learned my addr 172.21.15.3:0/2636856759 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8ffff640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90159bd0 0x7f6b90099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b953a5640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90159fb0 0x7f6b90073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 -- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90159fb0 msgr2=0x7f6b90073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90159fb0 0x7f6b90073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 -- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90159bd0 msgr2=0x7f6b90099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90159bd0 0x7f6b90099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 -- 172.21.15.3:0/2636856759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b90145470 con 0x7f6b9006f4a0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b94ba4640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b9006f4a0 0x7f6b900cf9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c7f0 tx=0x7f6b8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8d7fa640 1 -- 172.21.15.3:0/2636856759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80015070 con 0x7f6b9006f4a0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8d7fa640 1 -- 172.21.15.3:0/2636856759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b80004240 con 0x7f6b9006f4a0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b90145730 con 0x7f6b9006f4a0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8d7fa640 1 -- 172.21.15.3:0/2636856759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80003620 con 0x7f6b9006f4a0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b90099e60 con 0x7f6b9006f4a0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b900ad060 con 0x7f6b9006f4a0 2024-09-12T12:19:53.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8d7fa640 1 -- 172.21.15.3:0/2636856759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6b800037c0 con 0x7f6b9006f4a0 2024-09-12T12:19:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8d7fa640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6b7c0678c0 0x7f6b7c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8d7fa640 1 -- 172.21.15.3:0/2636856759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6b80085a50 con 0x7f6b9006f4a0 2024-09-12T12:19:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.016+0000 7f6b8ffff640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6b7c0678c0 0x7f6b7c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.020+0000 7f6b8ffff640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6b7c0678c0 0x7f6b7c069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b840ad000 tx=0x7f6b84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.020+0000 7f6b8d7fa640 1 -- 172.21.15.3:0/2636856759 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6b80056620 con 0x7f6b9006f4a0 2024-09-12T12:19:53.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.176+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b900b2b00 con 0x7f6b7c0678c0 2024-09-12T12:19:53.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.196+0000 7f6b8d7fa640 1 -- 172.21.15.3:0/2636856759 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240914 (secure 0 0 0) 0x7f6b900b2b00 con 0x7f6b7c0678c0 2024-09-12T12:19:53.198 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:53.200 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6b7c0678c0 msgr2=0x7f6b7c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6b7c0678c0 0x7f6b7c069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b840ad000 tx=0x7f6b84008040 comp rx=0 tx=0).stop 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b9006f4a0 msgr2=0x7f6b900cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b9006f4a0 0x7f6b900cf9d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c7f0 tx=0x7f6b8000ccb0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 shutdown_connections 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6b7c0678c0 0x7f6b7c069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6b90159fb0 0x7f6b90073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6b90159bd0 0x7f6b90099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 --2- 172.21.15.3:0/2636856759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6b9006f4a0 0x7f6b900cf9d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 >> 172.21.15.3:0/2636856759 conn(0x7f6b900b1d80 msgr2=0x7f6b900750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 shutdown_connections 2024-09-12T12:19:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.200+0000 7f6b96e08640 1 -- 172.21.15.3:0/2636856759 wait complete. 2024-09-12T12:19:53.216 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T12:19:51.656235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218776,"kb_used_data":2548,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755944,"statfs":{"total":773094113280,"available":772870086656,"internally_reserved":0,"allocated":2609152,"data_stored":1623275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.934925"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.740554+0000","last_change":"2024-09-12T12:19:47.740554+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.740554+0000","last_undegraded":"2024-09-12T12:19:47.740554+0000","last_fullsized":"2024-09-12T12:19:47.740554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739346+0000","last_change":"2024-09-12T12:19:47.739346+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739346+0000","last_undegraded":"2024-09-12T12:19:47.739346+0000","last_fullsized":"2024-09-12T12:19:47.739346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739200+0000","last_change":"2024-09-12T12:19:47.739200+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739200+0000","last_undegraded":"2024-09-12T12:19:47.739200+0000","last_fullsized":"2024-09-12T12:19:47.739200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738680+0000","last_change":"2024-09-12T12:19:47.738680+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738680+0000","last_undegraded":"2024-09-12T12:19:47.738680+0000","last_fullsized":"2024-09-12T12:19:47.738680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735706+0000","last_change":"2024-09-12T12:19:47.735706+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735706+0000","last_undegraded":"2024-09-12T12:19:47.735706+0000","last_fullsized":"2024-09-12T12:19:47.735706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761667+0000","last_change":"2024-09-12T12:19:47.757834+0000","last_active":"2024-09-12T12:19:47.761667+0000","last_peered":"2024-09-12T12:19:47.761667+0000","last_clean":"2024-09-12T12:19:47.761667+0000","last_became_active":"2024-09-12T12:19:47.757145+0000","last_became_peered":"2024-09-12T12:19:47.757145+0000","last_unstale":"2024-09-12T12:19:47.761667+0000","last_undegraded":"2024-09-12T12:19:47.761667+0000","last_fullsized":"2024-09-12T12:19:47.761667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:59.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731060+0000","last_change":"2024-09-12T12:19:46.731060+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731060+0000","last_undegraded":"2024-09-12T12:19:46.731060+0000","last_fullsized":"2024-09-12T12:19:46.731060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761691+0000","last_change":"2024-09-12T12:19:47.758883+0000","last_active":"2024-09-12T12:19:47.761691+0000","last_peered":"2024-09-12T12:19:47.761691+0000","last_clean":"2024-09-12T12:19:47.761691+0000","last_became_active":"2024-09-12T12:19:47.758396+0000","last_became_peered":"2024-09-12T12:19:47.758396+0000","last_unstale":"2024-09-12T12:19:47.761691+0000","last_undegraded":"2024-09-12T12:19:47.761691+0000","last_fullsized":"2024-09-12T12:19:47.761691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:42.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738678+0000","last_change":"2024-09-12T12:19:47.738678+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738678+0000","last_undegraded":"2024-09-12T12:19:47.738678+0000","last_fullsized":"2024-09-12T12:19:47.738678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729394+0000","last_change":"2024-09-12T12:19:46.729394+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729394+0000","last_undegraded":"2024-09-12T12:19:46.729394+0000","last_fullsized":"2024-09-12T12:19:46.729394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.732973+0000","last_change":"2024-09-12T12:19:46.732973+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.732973+0000","last_undegraded":"2024-09-12T12:19:46.732973+0000","last_fullsized":"2024-09-12T12:19:46.732973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729401+0000","last_change":"2024-09-12T12:19:46.729401+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729401+0000","last_undegraded":"2024-09-12T12:19:46.729401+0000","last_fullsized":"2024-09-12T12:19:46.729401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.728277+0000","last_change":"2024-09-12T12:19:46.728277+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.728277+0000","last_undegraded":"2024-09-12T12:19:46.728277+0000","last_fullsized":"2024-09-12T12:19:46.728277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731424+0000","last_change":"2024-09-12T12:19:46.731424+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731424+0000","last_undegraded":"2024-09-12T12:19:46.731424+0000","last_fullsized":"2024-09-12T12:19:46.731424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726355+0000","last_change":"2024-09-12T12:19:46.726355+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726355+0000","last_undegraded":"2024-09-12T12:19:46.726355+0000","last_fullsized":"2024-09-12T12:19:46.726355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731339+0000","last_change":"2024-09-12T12:19:46.731339+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731339+0000","last_undegraded":"2024-09-12T12:19:46.731339+0000","last_fullsized":"2024-09-12T12:19:46.731339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.732689+0000","last_change":"2024-09-12T12:19:46.732689+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.732689+0000","last_undegraded":"2024-09-12T12:19:46.732689+0000","last_fullsized":"2024-09-12T12:19:46.732689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.734356+0000","last_change":"2024-09-12T12:19:46.734356+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.734356+0000","last_undegraded":"2024-09-12T12:19:46.734356+0000","last_fullsized":"2024-09-12T12:19:46.734356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.733700+0000","last_change":"2024-09-12T12:19:46.733700+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.733700+0000","last_undegraded":"2024-09-12T12:19:46.733700+0000","last_fullsized":"2024-09-12T12:19:46.733700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.735680+0000","last_change":"2024-09-12T12:19:46.735680+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.735680+0000","last_undegraded":"2024-09-12T12:19:46.735680+0000","last_fullsized":"2024-09-12T12:19:46.735680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.727437+0000","last_change":"2024-09-12T12:19:46.727437+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.727437+0000","last_undegraded":"2024-09-12T12:19:46.727437+0000","last_fullsized":"2024-09-12T12:19:46.727437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.736700+0000","last_change":"2024-09-12T12:19:46.736700+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.736700+0000","last_undegraded":"2024-09-12T12:19:46.736700+0000","last_fullsized":"2024-09-12T12:19:46.736700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-12T12:19:46.728343+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:19:46.728343+0000","last_peered":"2024-09-12T12:19:46.728343+0000","last_clean":"2024-09-12T12:19:46.728343+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:19:46.728343+0000","last_undegraded":"2024-09-12T12:19:46.728343+0000","last_fullsized":"2024-09-12T12:19:46.728343+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.730089+0000","last_change":"2024-09-12T12:19:46.730089+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.730089+0000","last_undegraded":"2024-09-12T12:19:46.730089+0000","last_fullsized":"2024-09-12T12:19:46.730089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739193+0000","last_change":"2024-09-12T12:19:47.739193+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739193+0000","last_undegraded":"2024-09-12T12:19:47.739193+0000","last_fullsized":"2024-09-12T12:19:47.739193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723642+0000","last_change":"2024-09-12T12:19:46.723642+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723642+0000","last_undegraded":"2024-09-12T12:19:46.723642+0000","last_fullsized":"2024-09-12T12:19:46.723642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.734967+0000","last_change":"2024-09-12T12:19:46.734967+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.734967+0000","last_undegraded":"2024-09-12T12:19:46.734967+0000","last_fullsized":"2024-09-12T12:19:46.734967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729087+0000","last_change":"2024-09-12T12:19:46.729087+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729087+0000","last_undegraded":"2024-09-12T12:19:46.729087+0000","last_fullsized":"2024-09-12T12:19:46.729087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726297+0000","last_change":"2024-09-12T12:19:46.726297+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726297+0000","last_undegraded":"2024-09-12T12:19:46.726297+0000","last_fullsized":"2024-09-12T12:19:46.726297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723726+0000","last_change":"2024-09-12T12:19:46.723726+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723726+0000","last_undegraded":"2024-09-12T12:19:46.723726+0000","last_fullsized":"2024-09-12T12:19:46.723726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726266+0000","last_change":"2024-09-12T12:19:46.726266+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726266+0000","last_undegraded":"2024-09-12T12:19:46.726266+0000","last_fullsized":"2024-09-12T12:19:46.726266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735219+0000","last_change":"2024-09-12T12:19:47.735219+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735219+0000","last_undegraded":"2024-09-12T12:19:47.735219+0000","last_fullsized":"2024-09-12T12:19:47.735219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731477+0000","last_change":"2024-09-12T12:19:46.731477+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731477+0000","last_undegraded":"2024-09-12T12:19:46.731477+0000","last_fullsized":"2024-09-12T12:19:46.731477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726910+0000","last_change":"2024-09-12T12:19:46.726910+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726910+0000","last_undegraded":"2024-09-12T12:19:46.726910+0000","last_fullsized":"2024-09-12T12:19:46.726910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.727078+0000","last_change":"2024-09-12T12:19:46.727078+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.727078+0000","last_undegraded":"2024-09-12T12:19:46.727078+0000","last_fullsized":"2024-09-12T12:19:46.727078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735412+0000","last_change":"2024-09-12T12:19:47.735412+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735412+0000","last_undegraded":"2024-09-12T12:19:47.735412+0000","last_fullsized":"2024-09-12T12:19:47.735412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726299+0000","last_change":"2024-09-12T12:19:46.726299+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726299+0000","last_undegraded":"2024-09-12T12:19:46.726299+0000","last_fullsized":"2024-09-12T12:19:46.726299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.736208+0000","last_change":"2024-09-12T12:19:47.736208+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.736208+0000","last_undegraded":"2024-09-12T12:19:47.736208+0000","last_fullsized":"2024-09-12T12:19:47.736208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723874+0000","last_change":"2024-09-12T12:19:46.723874+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723874+0000","last_undegraded":"2024-09-12T12:19:46.723874+0000","last_fullsized":"2024-09-12T12:19:46.723874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723551+0000","last_change":"2024-09-12T12:19:46.723551+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723551+0000","last_undegraded":"2024-09-12T12:19:46.723551+0000","last_fullsized":"2024-09-12T12:19:46.723551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.725466+0000","last_change":"2024-09-12T12:19:46.725466+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.725466+0000","last_undegraded":"2024-09-12T12:19:46.725466+0000","last_fullsized":"2024-09-12T12:19:46.725466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761902+0000","last_change":"2024-09-12T12:19:47.759563+0000","last_active":"2024-09-12T12:19:47.761902+0000","last_peered":"2024-09-12T12:19:47.761902+0000","last_clean":"2024-09-12T12:19:47.761902+0000","last_became_active":"2024-09-12T12:19:47.759012+0000","last_became_peered":"2024-09-12T12:19:47.759012+0000","last_unstale":"2024-09-12T12:19:47.761902+0000","last_undegraded":"2024-09-12T12:19:47.761902+0000","last_fullsized":"2024-09-12T12:19:47.761902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735760+0000","last_change":"2024-09-12T12:19:47.735760+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735760+0000","last_undegraded":"2024-09-12T12:19:47.735760+0000","last_fullsized":"2024-09-12T12:19:47.735760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726850+0000","last_change":"2024-09-12T12:19:46.726850+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726850+0000","last_undegraded":"2024-09-12T12:19:46.726850+0000","last_fullsized":"2024-09-12T12:19:46.726850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":60808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":644901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":644901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:19:53.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f7580224640 1 Processor -- start 2024-09-12T12:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f7580224640 1 -- start start 2024-09-12T12:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f7580224640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 0x7f7578072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f7580224640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f75780cc300 0x7f7578073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f7580224640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f757815f150 con 0x7f757815b2d0 2024-09-12T12:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f7580224640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f75780b3790 con 0x7f75780cc300 2024-09-12T12:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f7580224640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75780ae920 con 0x7f757806f6e0 2024-09-12T12:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f757815b2d0 0x7f75780b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45284/0 (socket says 172.21.15.3:45284) 2024-09-12T12:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 -- 172.21.15.3:0/3236365741 learned_addr learned my addr 172.21.15.3:0/3236365741 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757e7c1640 1 --2- 172.21.15.3:0/3236365741 >> v2:172.21.15.142:3300/0 conn(0x7f75780cc300 0x7f7578073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757d7bf640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 0x7f7578072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537219917 0 0) 0x7f757815f150 con 0x7f757815b2d0 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7564003530 con 0x7f757815b2d0 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 394815769 0 0) 0x7f7564003530 con 0x7f757815b2d0 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 msgr2=0x7f7578072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 0x7f7578072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 >> v2:172.21.15.142:3300/0 conn(0x7f75780cc300 msgr2=0x7f7578073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 --2- 172.21.15.3:0/3236365741 >> v2:172.21.15.142:3300/0 conn(0x7f75780cc300 0x7f7578073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757815b6b0 con 0x7f757815b2d0 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f7574001680 con 0x7f757815b2d0 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 >> v1:172.21.15.3:6789/0 conn(0x7f757815b2d0 legacy=0x7f75780b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75640039c0 0x7f7564003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7564004810 0x7f7564004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 0x7f75640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f75780b3790 con 0x7f75640039c0 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75780ae920 con 0x7f7564004810 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f756400b9d0 con 0x7f757806f6e0 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75640039c0 0x7f7564003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757d7bf640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7564004810 0x7f7564004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757e7c1640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 0x7f75640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 -- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 msgr2=0x7f75640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 0x7f75640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 -- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7564004810 msgr2=0x7f7564004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7564004810 0x7f7564004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757dfc0640 1 -- 172.21.15.3:0/3236365741 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7564003530 con 0x7f75640039c0 2024-09-12T12:19:53.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.380+0000 7f757e7c1640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757806f6e0 0x7f75640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f757dfc0640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75640039c0 0x7f7564003de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f757400c240 tx=0x7f757401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67d322850df082e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574007d60 con 0x7f75640039c0 2024-09-12T12:19:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740a9020 con 0x7f75640039c0 2024-09-12T12:19:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f757cfbe640 1 -- 172.21.15.3:0/3236365741 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574005ae0 con 0x7f75640039c0 2024-09-12T12:19:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f7580224640 1 -- 172.21.15.3:0/3236365741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75640039c0 msgr2=0x7f7564003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f7580224640 1 --2- 172.21.15.3:0/3236365741 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f75640039c0 0x7f7564003de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f757400c240 tx=0x7f757401cc90 comp rx=0 tx=0).stop 2024-09-12T12:19:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f757e7c1640 1 -- 172.21.15.3:0/3236365741 reap_dead start 2024-09-12T12:19:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f7580224640 1 -- 172.21.15.3:0/3236365741 shutdown_connections 2024-09-12T12:19:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f7580224640 1 -- 172.21.15.3:0/3236365741 >> 172.21.15.3:0/3236365741 conn(0x7f75780b1d80 msgr2=0x7f75780b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f7580224640 1 -- 172.21.15.3:0/3236365741 shutdown_connections 2024-09-12T12:19:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.384+0000 7f7580224640 1 -- 172.21.15.3:0/3236365741 wait complete. 2024-09-12T12:19:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 Processor -- start 2024-09-12T12:19:53.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 -- start start 2024-09-12T12:19:53.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7578146810 0x7f7578154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757814e670 0x7f75781508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75780b0740 0x7f75780b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f757806d8d0 con 0x7f7578146810 2024-09-12T12:19:53.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75780ae5b0 con 0x7f75780b0740 2024-09-12T12:19:53.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f75780af5a0 con 0x7f757814e670 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7578146810 0x7f7578154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7578146810 0x7f7578154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60976/0 (socket says 172.21.15.3:60976) 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 -- 172.21.15.3:0/3544466029 learned_addr learned my addr 172.21.15.3:0/3544466029 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757d7bf640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757814e670 0x7f75781508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757e7c1640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75780b0740 0x7f75780b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 -- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757814e670 msgr2=0x7f75781508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757814e670 0x7f75781508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 -- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75780b0740 msgr2=0x7f75780b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75780b0740 0x7f75780b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 -- 172.21.15.3:0/3544466029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7578154810 con 0x7f7578146810 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757d7bf640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757814e670 0x7f75781508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:53.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757e7c1640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75780b0740 0x7f75780b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:53.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f757dfc0640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7578146810 0x7f7578154e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75740ab040 tx=0x7f7564005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f756e7fc640 1 -- 172.21.15.3:0/3544466029 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574004ab0 con 0x7f7578146810 2024-09-12T12:19:53.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f756e7fc640 1 -- 172.21.15.3:0/3544466029 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740b2070 con 0x7f7578146810 2024-09-12T12:19:53.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757814fe10 con 0x7f7578146810 2024-09-12T12:19:53.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f756e7fc640 1 -- 172.21.15.3:0/3544466029 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740070d0 con 0x7f7578146810 2024-09-12T12:19:53.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781503d0 con 0x7f7578146810 2024-09-12T12:19:53.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7578073450 con 0x7f7578146810 2024-09-12T12:19:53.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f756e7fc640 1 -- 172.21.15.3:0/3544466029 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7574007270 con 0x7f7578146810 2024-09-12T12:19:53.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.388+0000 7f756e7fc640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f755c0678f0 0x7f755c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.392+0000 7f756e7fc640 1 -- 172.21.15.3:0/3544466029 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f757411f450 con 0x7f7578146810 2024-09-12T12:19:53.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.392+0000 7f757d7bf640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f755c0678f0 0x7f755c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.392+0000 7f757d7bf640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f755c0678f0 0x7f755c069db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f757814eab0 tx=0x7f7568008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.392+0000 7f756e7fc640 1 -- 172.21.15.3:0/3544466029 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f75740169d0 con 0x7f7578146810 2024-09-12T12:19:53.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.552+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75780d0cb0 con 0x7f755c0678f0 2024-09-12T12:19:53.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.572+0000 7f756e7fc640 1 -- 172.21.15.3:0/3544466029 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240914 (secure 0 0 0) 0x7f75780d0cb0 con 0x7f755c0678f0 2024-09-12T12:19:53.575 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:53.577 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:19:53.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f755c0678f0 msgr2=0x7f755c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f755c0678f0 0x7f755c069db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f757814eab0 tx=0x7f7568008040 comp rx=0 tx=0).stop 2024-09-12T12:19:53.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7578146810 msgr2=0x7f7578154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7578146810 0x7f7578154e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75740ab040 tx=0x7f7564005840 comp rx=0 tx=0).stop 2024-09-12T12:19:53.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 shutdown_connections 2024-09-12T12:19:53.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f755c0678f0 0x7f755c069db0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75780b0740 0x7f75780b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f757814e670 0x7f75781508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 --2- 172.21.15.3:0/3544466029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7578146810 0x7f7578154e70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 >> 172.21.15.3:0/3544466029 conn(0x7f75780b1d80 msgr2=0x7f7578155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:53.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 shutdown_connections 2024-09-12T12:19:53.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.576+0000 7f7580224640 1 -- 172.21.15.3:0/3544466029 wait complete. 2024-09-12T12:19:53.593 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-12T12:19:51.656235+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218776,"kb_used_data":2548,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755944,"statfs":{"total":773094113280,"available":772870086656,"internally_reserved":0,"allocated":2609152,"data_stored":1623275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.934925"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.740554+0000","last_change":"2024-09-12T12:19:47.740554+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.740554+0000","last_undegraded":"2024-09-12T12:19:47.740554+0000","last_fullsized":"2024-09-12T12:19:47.740554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739346+0000","last_change":"2024-09-12T12:19:47.739346+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739346+0000","last_undegraded":"2024-09-12T12:19:47.739346+0000","last_fullsized":"2024-09-12T12:19:47.739346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739200+0000","last_change":"2024-09-12T12:19:47.739200+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739200+0000","last_undegraded":"2024-09-12T12:19:47.739200+0000","last_fullsized":"2024-09-12T12:19:47.739200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738680+0000","last_change":"2024-09-12T12:19:47.738680+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738680+0000","last_undegraded":"2024-09-12T12:19:47.738680+0000","last_fullsized":"2024-09-12T12:19:47.738680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735706+0000","last_change":"2024-09-12T12:19:47.735706+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735706+0000","last_undegraded":"2024-09-12T12:19:47.735706+0000","last_fullsized":"2024-09-12T12:19:47.735706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761667+0000","last_change":"2024-09-12T12:19:47.757834+0000","last_active":"2024-09-12T12:19:47.761667+0000","last_peered":"2024-09-12T12:19:47.761667+0000","last_clean":"2024-09-12T12:19:47.761667+0000","last_became_active":"2024-09-12T12:19:47.757145+0000","last_became_peered":"2024-09-12T12:19:47.757145+0000","last_unstale":"2024-09-12T12:19:47.761667+0000","last_undegraded":"2024-09-12T12:19:47.761667+0000","last_fullsized":"2024-09-12T12:19:47.761667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:59.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731060+0000","last_change":"2024-09-12T12:19:46.731060+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731060+0000","last_undegraded":"2024-09-12T12:19:46.731060+0000","last_fullsized":"2024-09-12T12:19:46.731060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761691+0000","last_change":"2024-09-12T12:19:47.758883+0000","last_active":"2024-09-12T12:19:47.761691+0000","last_peered":"2024-09-12T12:19:47.761691+0000","last_clean":"2024-09-12T12:19:47.761691+0000","last_became_active":"2024-09-12T12:19:47.758396+0000","last_became_peered":"2024-09-12T12:19:47.758396+0000","last_unstale":"2024-09-12T12:19:47.761691+0000","last_undegraded":"2024-09-12T12:19:47.761691+0000","last_fullsized":"2024-09-12T12:19:47.761691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:42.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738678+0000","last_change":"2024-09-12T12:19:47.738678+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738678+0000","last_undegraded":"2024-09-12T12:19:47.738678+0000","last_fullsized":"2024-09-12T12:19:47.738678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729394+0000","last_change":"2024-09-12T12:19:46.729394+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729394+0000","last_undegraded":"2024-09-12T12:19:46.729394+0000","last_fullsized":"2024-09-12T12:19:46.729394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.732973+0000","last_change":"2024-09-12T12:19:46.732973+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.732973+0000","last_undegraded":"2024-09-12T12:19:46.732973+0000","last_fullsized":"2024-09-12T12:19:46.732973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729401+0000","last_change":"2024-09-12T12:19:46.729401+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729401+0000","last_undegraded":"2024-09-12T12:19:46.729401+0000","last_fullsized":"2024-09-12T12:19:46.729401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.728277+0000","last_change":"2024-09-12T12:19:46.728277+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.728277+0000","last_undegraded":"2024-09-12T12:19:46.728277+0000","last_fullsized":"2024-09-12T12:19:46.728277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731424+0000","last_change":"2024-09-12T12:19:46.731424+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731424+0000","last_undegraded":"2024-09-12T12:19:46.731424+0000","last_fullsized":"2024-09-12T12:19:46.731424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726355+0000","last_change":"2024-09-12T12:19:46.726355+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726355+0000","last_undegraded":"2024-09-12T12:19:46.726355+0000","last_fullsized":"2024-09-12T12:19:46.726355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731339+0000","last_change":"2024-09-12T12:19:46.731339+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731339+0000","last_undegraded":"2024-09-12T12:19:46.731339+0000","last_fullsized":"2024-09-12T12:19:46.731339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.732689+0000","last_change":"2024-09-12T12:19:46.732689+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.732689+0000","last_undegraded":"2024-09-12T12:19:46.732689+0000","last_fullsized":"2024-09-12T12:19:46.732689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.734356+0000","last_change":"2024-09-12T12:19:46.734356+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.734356+0000","last_undegraded":"2024-09-12T12:19:46.734356+0000","last_fullsized":"2024-09-12T12:19:46.734356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.733700+0000","last_change":"2024-09-12T12:19:46.733700+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.733700+0000","last_undegraded":"2024-09-12T12:19:46.733700+0000","last_fullsized":"2024-09-12T12:19:46.733700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.735680+0000","last_change":"2024-09-12T12:19:46.735680+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.735680+0000","last_undegraded":"2024-09-12T12:19:46.735680+0000","last_fullsized":"2024-09-12T12:19:46.735680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.727437+0000","last_change":"2024-09-12T12:19:46.727437+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.727437+0000","last_undegraded":"2024-09-12T12:19:46.727437+0000","last_fullsized":"2024-09-12T12:19:46.727437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.736700+0000","last_change":"2024-09-12T12:19:46.736700+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.736700+0000","last_undegraded":"2024-09-12T12:19:46.736700+0000","last_fullsized":"2024-09-12T12:19:46.736700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-12T12:19:46.728343+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:19:46.728343+0000","last_peered":"2024-09-12T12:19:46.728343+0000","last_clean":"2024-09-12T12:19:46.728343+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:19:46.728343+0000","last_undegraded":"2024-09-12T12:19:46.728343+0000","last_fullsized":"2024-09-12T12:19:46.728343+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.730089+0000","last_change":"2024-09-12T12:19:46.730089+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.730089+0000","last_undegraded":"2024-09-12T12:19:46.730089+0000","last_fullsized":"2024-09-12T12:19:46.730089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739193+0000","last_change":"2024-09-12T12:19:47.739193+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739193+0000","last_undegraded":"2024-09-12T12:19:47.739193+0000","last_fullsized":"2024-09-12T12:19:47.739193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723642+0000","last_change":"2024-09-12T12:19:46.723642+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723642+0000","last_undegraded":"2024-09-12T12:19:46.723642+0000","last_fullsized":"2024-09-12T12:19:46.723642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.734967+0000","last_change":"2024-09-12T12:19:46.734967+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.734967+0000","last_undegraded":"2024-09-12T12:19:46.734967+0000","last_fullsized":"2024-09-12T12:19:46.734967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.729087+0000","last_change":"2024-09-12T12:19:46.729087+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.729087+0000","last_undegraded":"2024-09-12T12:19:46.729087+0000","last_fullsized":"2024-09-12T12:19:46.729087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726297+0000","last_change":"2024-09-12T12:19:46.726297+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726297+0000","last_undegraded":"2024-09-12T12:19:46.726297+0000","last_fullsized":"2024-09-12T12:19:46.726297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723726+0000","last_change":"2024-09-12T12:19:46.723726+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723726+0000","last_undegraded":"2024-09-12T12:19:46.723726+0000","last_fullsized":"2024-09-12T12:19:46.723726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726266+0000","last_change":"2024-09-12T12:19:46.726266+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726266+0000","last_undegraded":"2024-09-12T12:19:46.726266+0000","last_fullsized":"2024-09-12T12:19:46.726266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735219+0000","last_change":"2024-09-12T12:19:47.735219+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735219+0000","last_undegraded":"2024-09-12T12:19:47.735219+0000","last_fullsized":"2024-09-12T12:19:47.735219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.731477+0000","last_change":"2024-09-12T12:19:46.731477+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.731477+0000","last_undegraded":"2024-09-12T12:19:46.731477+0000","last_fullsized":"2024-09-12T12:19:46.731477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726910+0000","last_change":"2024-09-12T12:19:46.726910+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726910+0000","last_undegraded":"2024-09-12T12:19:46.726910+0000","last_fullsized":"2024-09-12T12:19:46.726910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.727078+0000","last_change":"2024-09-12T12:19:46.727078+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.727078+0000","last_undegraded":"2024-09-12T12:19:46.727078+0000","last_fullsized":"2024-09-12T12:19:46.727078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735412+0000","last_change":"2024-09-12T12:19:47.735412+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735412+0000","last_undegraded":"2024-09-12T12:19:47.735412+0000","last_fullsized":"2024-09-12T12:19:47.735412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726299+0000","last_change":"2024-09-12T12:19:46.726299+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726299+0000","last_undegraded":"2024-09-12T12:19:46.726299+0000","last_fullsized":"2024-09-12T12:19:46.726299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.736208+0000","last_change":"2024-09-12T12:19:47.736208+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.736208+0000","last_undegraded":"2024-09-12T12:19:47.736208+0000","last_fullsized":"2024-09-12T12:19:47.736208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723874+0000","last_change":"2024-09-12T12:19:46.723874+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723874+0000","last_undegraded":"2024-09-12T12:19:46.723874+0000","last_fullsized":"2024-09-12T12:19:46.723874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.723551+0000","last_change":"2024-09-12T12:19:46.723551+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.723551+0000","last_undegraded":"2024-09-12T12:19:46.723551+0000","last_fullsized":"2024-09-12T12:19:46.723551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.725466+0000","last_change":"2024-09-12T12:19:46.725466+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.725466+0000","last_undegraded":"2024-09-12T12:19:46.725466+0000","last_fullsized":"2024-09-12T12:19:46.725466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:19:47.718118+0000","last_change":"2024-09-12T12:19:47.718118+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.718118+0000","last_undegraded":"2024-09-12T12:19:47.718118+0000","last_fullsized":"2024-09-12T12:19:47.718118+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761902+0000","last_change":"2024-09-12T12:19:47.759563+0000","last_active":"2024-09-12T12:19:47.761902+0000","last_peered":"2024-09-12T12:19:47.761902+0000","last_clean":"2024-09-12T12:19:47.761902+0000","last_became_active":"2024-09-12T12:19:47.759012+0000","last_became_peered":"2024-09-12T12:19:47.759012+0000","last_unstale":"2024-09-12T12:19:47.761902+0000","last_undegraded":"2024-09-12T12:19:47.761902+0000","last_fullsized":"2024-09-12T12:19:47.761902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735760+0000","last_change":"2024-09-12T12:19:47.735760+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735760+0000","last_undegraded":"2024-09-12T12:19:47.735760+0000","last_fullsized":"2024-09-12T12:19:47.735760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-12T12:19:46.726850+0000","last_change":"2024-09-12T12:19:46.726850+0000","last_active":"2024-09-12T12:19:46.707792+0000","last_peered":"2024-09-12T12:19:46.707792+0000","last_clean":"2024-09-12T12:19:46.707792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:46.726850+0000","last_undegraded":"2024-09-12T12:19:46.726850+0000","last_fullsized":"2024-09-12T12:19:46.726850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":60808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":644901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":54533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":770048,"data_stored":644901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:19:53.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.740554+0000', 'last_change': '2024-09-12T12:19:47.740554+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.740554+0000', 'last_undegraded': '2024-09-12T12:19:47.740554+0000', 'last_fullsized': '2024-09-12T12:19:47.740554+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-12T12:19:53.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.739346+0000', 'last_change': '2024-09-12T12:19:47.739346+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.739346+0000', 'last_undegraded': '2024-09-12T12:19:47.739346+0000', 'last_fullsized': '2024-09-12T12:19:47.739346+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-12T12:19:53.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-12T12:19:53.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.739200+0000', 'last_change': '2024-09-12T12:19:47.739200+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.739200+0000', 'last_undegraded': '2024-09-12T12:19:47.739200+0000', 'last_fullsized': '2024-09-12T12:19:47.739200+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.738680+0000', 'last_change': '2024-09-12T12:19:47.738680+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.738680+0000', 'last_undegraded': '2024-09-12T12:19:47.738680+0000', 'last_fullsized': '2024-09-12T12:19:47.738680+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.735706+0000', 'last_change': '2024-09-12T12:19:47.735706+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.735706+0000', 'last_undegraded': '2024-09-12T12:19:47.735706+0000', 'last_fullsized': '2024-09-12T12:19:47.735706+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-12T12:19:53.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.731060+0000', 'last_change': '2024-09-12T12:19:46.731060+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.731060+0000', 'last_undegraded': '2024-09-12T12:19:46.731060+0000', 'last_fullsized': '2024-09-12T12:19:46.731060+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-12T12:19:53.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.738678+0000', 'last_change': '2024-09-12T12:19:47.738678+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.738678+0000', 'last_undegraded': '2024-09-12T12:19:47.738678+0000', 'last_fullsized': '2024-09-12T12:19:47.738678+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.729394+0000', 'last_change': '2024-09-12T12:19:46.729394+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.729394+0000', 'last_undegraded': '2024-09-12T12:19:46.729394+0000', 'last_fullsized': '2024-09-12T12:19:46.729394+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.732973+0000', 'last_change': '2024-09-12T12:19:46.732973+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.732973+0000', 'last_undegraded': '2024-09-12T12:19:46.732973+0000', 'last_fullsized': '2024-09-12T12:19:46.732973+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.729401+0000', 'last_change': '2024-09-12T12:19:46.729401+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.729401+0000', 'last_undegraded': '2024-09-12T12:19:46.729401+0000', 'last_fullsized': '2024-09-12T12:19:46.729401+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:19:53.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.728277+0000', 'last_change': '2024-09-12T12:19:46.728277+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.728277+0000', 'last_undegraded': '2024-09-12T12:19:46.728277+0000', 'last_fullsized': '2024-09-12T12:19:46.728277+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.731424+0000', 'last_change': '2024-09-12T12:19:46.731424+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.731424+0000', 'last_undegraded': '2024-09-12T12:19:46.731424+0000', 'last_fullsized': '2024-09-12T12:19:46.731424+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-12T12:19:53.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.726355+0000', 'last_change': '2024-09-12T12:19:46.726355+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.726355+0000', 'last_undegraded': '2024-09-12T12:19:46.726355+0000', 'last_fullsized': '2024-09-12T12:19:46.726355+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.731339+0000', 'last_change': '2024-09-12T12:19:46.731339+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.731339+0000', 'last_undegraded': '2024-09-12T12:19:46.731339+0000', 'last_fullsized': '2024-09-12T12:19:46.731339+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-12T12:19:53.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.732689+0000', 'last_change': '2024-09-12T12:19:46.732689+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.732689+0000', 'last_undegraded': '2024-09-12T12:19:46.732689+0000', 'last_fullsized': '2024-09-12T12:19:46.732689+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.734356+0000', 'last_change': '2024-09-12T12:19:46.734356+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.734356+0000', 'last_undegraded': '2024-09-12T12:19:46.734356+0000', 'last_fullsized': '2024-09-12T12:19:46.734356+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-12T12:19:53.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.733700+0000', 'last_change': '2024-09-12T12:19:46.733700+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.733700+0000', 'last_undegraded': '2024-09-12T12:19:46.733700+0000', 'last_fullsized': '2024-09-12T12:19:46.733700+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.735680+0000', 'last_change': '2024-09-12T12:19:46.735680+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.735680+0000', 'last_undegraded': '2024-09-12T12:19:46.735680+0000', 'last_fullsized': '2024-09-12T12:19:46.735680+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.727437+0000', 'last_change': '2024-09-12T12:19:46.727437+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.727437+0000', 'last_undegraded': '2024-09-12T12:19:46.727437+0000', 'last_fullsized': '2024-09-12T12:19:46.727437+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-12T12:19:53.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.736700+0000', 'last_change': '2024-09-12T12:19:46.736700+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.736700+0000', 'last_undegraded': '2024-09-12T12:19:46.736700+0000', 'last_fullsized': '2024-09-12T12:19:46.736700+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.730089+0000', 'last_change': '2024-09-12T12:19:46.730089+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.730089+0000', 'last_undegraded': '2024-09-12T12:19:46.730089+0000', 'last_fullsized': '2024-09-12T12:19:46.730089+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.739193+0000', 'last_change': '2024-09-12T12:19:47.739193+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.739193+0000', 'last_undegraded': '2024-09-12T12:19:47.739193+0000', 'last_fullsized': '2024-09-12T12:19:47.739193+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.723642+0000', 'last_change': '2024-09-12T12:19:46.723642+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.723642+0000', 'last_undegraded': '2024-09-12T12:19:46.723642+0000', 'last_fullsized': '2024-09-12T12:19:46.723642+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:19:53.610 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.734967+0000', 'last_change': '2024-09-12T12:19:46.734967+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.734967+0000', 'last_undegraded': '2024-09-12T12:19:46.734967+0000', 'last_fullsized': '2024-09-12T12:19:46.734967+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.729087+0000', 'last_change': '2024-09-12T12:19:46.729087+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.729087+0000', 'last_undegraded': '2024-09-12T12:19:46.729087+0000', 'last_fullsized': '2024-09-12T12:19:46.729087+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-12T12:19:53.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.726297+0000', 'last_change': '2024-09-12T12:19:46.726297+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.726297+0000', 'last_undegraded': '2024-09-12T12:19:46.726297+0000', 'last_fullsized': '2024-09-12T12:19:46.726297+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.723726+0000', 'last_change': '2024-09-12T12:19:46.723726+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.723726+0000', 'last_undegraded': '2024-09-12T12:19:46.723726+0000', 'last_fullsized': '2024-09-12T12:19:46.723726+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.726266+0000', 'last_change': '2024-09-12T12:19:46.726266+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.726266+0000', 'last_undegraded': '2024-09-12T12:19:46.726266+0000', 'last_fullsized': '2024-09-12T12:19:46.726266+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.735219+0000', 'last_change': '2024-09-12T12:19:47.735219+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.735219+0000', 'last_undegraded': '2024-09-12T12:19:47.735219+0000', 'last_fullsized': '2024-09-12T12:19:47.735219+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.612 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.731477+0000', 'last_change': '2024-09-12T12:19:46.731477+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.731477+0000', 'last_undegraded': '2024-09-12T12:19:46.731477+0000', 'last_fullsized': '2024-09-12T12:19:46.731477+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.726910+0000', 'last_change': '2024-09-12T12:19:46.726910+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.726910+0000', 'last_undegraded': '2024-09-12T12:19:46.726910+0000', 'last_fullsized': '2024-09-12T12:19:46.726910+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:PG 2.9 is not active+clean 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.727078+0000', 'last_change': '2024-09-12T12:19:46.727078+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.727078+0000', 'last_undegraded': '2024-09-12T12:19:46.727078+0000', 'last_fullsized': '2024-09-12T12:19:46.727078+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:19:53.613 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-12T12:19:53.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.735412+0000', 'last_change': '2024-09-12T12:19:47.735412+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.735412+0000', 'last_undegraded': '2024-09-12T12:19:47.735412+0000', 'last_fullsized': '2024-09-12T12:19:47.735412+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.614 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-12T12:19:53.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.726299+0000', 'last_change': '2024-09-12T12:19:46.726299+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.726299+0000', 'last_undegraded': '2024-09-12T12:19:46.726299+0000', 'last_fullsized': '2024-09-12T12:19:46.726299+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:19:53.614 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-12T12:19:53.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.736208+0000', 'last_change': '2024-09-12T12:19:47.736208+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.736208+0000', 'last_undegraded': '2024-09-12T12:19:47.736208+0000', 'last_fullsized': '2024-09-12T12:19:47.736208+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.614 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-12T12:19:53.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.723874+0000', 'last_change': '2024-09-12T12:19:46.723874+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.723874+0000', 'last_undegraded': '2024-09-12T12:19:46.723874+0000', 'last_fullsized': '2024-09-12T12:19:46.723874+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.723551+0000', 'last_change': '2024-09-12T12:19:46.723551+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.723551+0000', 'last_undegraded': '2024-09-12T12:19:46.723551+0000', 'last_fullsized': '2024-09-12T12:19:46.723551+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.615 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-12T12:19:53.616 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.725466+0000', 'last_change': '2024-09-12T12:19:46.725466+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.725466+0000', 'last_undegraded': '2024-09-12T12:19:46.725466+0000', 'last_fullsized': '2024-09-12T12:19:46.725466+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:19:53.616 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-12T12:19:53.616 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:19:47.718118+0000', 'last_change': '2024-09-12T12:19:47.718118+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.718118+0000', 'last_undegraded': '2024-09-12T12:19:47.718118+0000', 'last_fullsized': '2024-09-12T12:19:47.718118+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:19:53.616 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-12T12:19:53.616 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:47.735760+0000', 'last_change': '2024-09-12T12:19:47.735760+0000', 'last_active': '2024-09-12T12:19:47.718118+0000', 'last_peered': '2024-09-12T12:19:47.718118+0000', 'last_clean': '2024-09-12T12:19:47.718118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:47.735760+0000', 'last_undegraded': '2024-09-12T12:19:47.735760+0000', 'last_fullsized': '2024-09-12T12:19:47.735760+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:47.718118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:19:53.616 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-12T12:19:53.616 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:19:46.726850+0000', 'last_change': '2024-09-12T12:19:46.726850+0000', 'last_active': '2024-09-12T12:19:46.707792+0000', 'last_peered': '2024-09-12T12:19:46.707792+0000', 'last_clean': '2024-09-12T12:19:46.707792+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:19:46.726850+0000', 'last_undegraded': '2024-09-12T12:19:46.726850+0000', 'last_fullsized': '2024-09-12T12:19:46.726850+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'last_clean_scrub_stamp': '2024-09-12T12:19:46.707792+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:19:53.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T12:19:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.756+0000 7fc126d04640 1 Processor -- start 2024-09-12T12:19:53.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.756+0000 7fc126d04640 1 -- start start 2024-09-12T12:19:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc126d04640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc120159fb0 0x7fc12006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc126d04640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015d2d0 0x7fc1200ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc126d04640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12006fbb0 con 0x7fc120159bd0 2024-09-12T12:19:53.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc126d04640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc12006f010 con 0x7fc120159fb0 2024-09-12T12:19:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc126d04640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc12015d760 con 0x7fc12015d2d0 2024-09-12T12:19:53.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc120159fb0 0x7fc12006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc124aa0640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc120159bd0 0x7fc12006f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45294/0 (socket says 172.21.15.3:45294) 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc124aa0640 1 -- 172.21.15.3:0/564267316 learned_addr learned my addr 172.21.15.3:0/564267316 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc1252a1640 1 --2- 172.21.15.3:0/564267316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015d2d0 0x7fc1200ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11ffff640 1 -- 172.21.15.3:0/564267316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015d2d0 msgr2=0x7fc1200ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11ffff640 1 --2- 172.21.15.3:0/564267316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015d2d0 0x7fc1200ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11ffff640 1 -- 172.21.15.3:0/564267316 >> v1:172.21.15.3:6789/0 conn(0x7fc120159bd0 legacy=0x7fc12006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11ffff640 1 -- 172.21.15.3:0/564267316 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12006e400 con 0x7fc120159fb0 2024-09-12T12:19:53.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc1252a1640 1 --2- 172.21.15.3:0/564267316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015d2d0 0x7fc1200ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:53.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11ffff640 1 --2- 172.21.15.3:0/564267316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc12006dd40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097c0 tx=0x7fc11401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecaa09a36c2f467 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11f7fe640 1 -- 172.21.15.3:0/564267316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140b2070 con 0x7fc120159fb0 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11f7fe640 1 -- 172.21.15.3:0/564267316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140ad680 con 0x7fc120159fb0 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.760+0000 7fc11f7fe640 1 -- 172.21.15.3:0/564267316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140ada50 con 0x7fc120159fb0 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- 172.21.15.3:0/564267316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 msgr2=0x7fc12006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 --2- 172.21.15.3:0/564267316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc12006dd40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1140097c0 tx=0x7fc11401c470 comp rx=0 tx=0).stop 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- 172.21.15.3:0/564267316 shutdown_connections 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 --2- 172.21.15.3:0/564267316 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc12015d2d0 0x7fc1200ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 --2- 172.21.15.3:0/564267316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc12006dd40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- 172.21.15.3:0/564267316 >> 172.21.15.3:0/564267316 conn(0x7fc1200b1d80 msgr2=0x7fc1200b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- 172.21.15.3:0/564267316 shutdown_connections 2024-09-12T12:19:53.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- 172.21.15.3:0/564267316 wait complete. 2024-09-12T12:19:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 Processor -- start 2024-09-12T12:19:53.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- start start 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc120159bd0 0x7fc1201458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc1200aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc12015d2d0 0x7fc120099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc120169360 con 0x7fc120159fb0 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1201691e0 con 0x7fc12015d2d0 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc126d04640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1201694e0 con 0x7fc120159bd0 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc1200aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc1200aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60980/0 (socket says 172.21.15.3:60980) 2024-09-12T12:19:53.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 -- 172.21.15.3:0/2861049556 learned_addr learned my addr 172.21.15.3:0/2861049556 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc124aa0640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc120159bd0 0x7fc1201458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 -- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc120159bd0 msgr2=0x7fc1201458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc120159bd0 0x7fc1201458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc1252a1640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc12015d2d0 0x7fc120099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 -- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc12015d2d0 msgr2=0x7fc120099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc12015d2d0 0x7fc120099940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc11ffff640 1 -- 172.21.15.3:0/2861049556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1200b0450 con 0x7fc120159fb0 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc124aa0640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc120159bd0 0x7fc1201458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.764+0000 7fc1252a1640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc12015d2d0 0x7fc120099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc11ffff640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc1200aff10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc11401a0a0 tx=0x7fc11401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc11d7fa640 1 -- 172.21.15.3:0/2861049556 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140b2040 con 0x7fc120159fb0 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc11d7fa640 1 -- 172.21.15.3:0/2861049556 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140bd070 con 0x7fc120159fb0 2024-09-12T12:19:53.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1200b06d0 con 0x7fc120159fb0 2024-09-12T12:19:53.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc11d7fa640 1 -- 172.21.15.3:0/2861049556 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140b8450 con 0x7fc120159fb0 2024-09-12T12:19:53.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc120150b50 con 0x7fc120159fb0 2024-09-12T12:19:53.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1200ad060 con 0x7fc120159fb0 2024-09-12T12:19:53.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc11d7fa640 1 -- 172.21.15.3:0/2861049556 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc1140b85f0 con 0x7fc120159fb0 2024-09-12T12:19:53.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc11d7fa640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc10c0678c0 0x7fc10c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc11d7fa640 1 -- 172.21.15.3:0/2861049556 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc11411ffe0 con 0x7fc120159fb0 2024-09-12T12:19:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc124aa0640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc10c0678c0 0x7fc10c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:53.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.768+0000 7fc124aa0640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc10c0678c0 0x7fc10c069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc1201f8a20 tx=0x7fc110009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:53.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:53.772+0000 7fc11d7fa640 1 -- 172.21.15.3:0/2861049556 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1200b0450 con 0x7fc120159fb0 2024-09-12T12:19:54.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.000+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc1200cc8f0 con 0x7fc120159fb0 2024-09-12T12:19:54.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc11d7fa640 1 -- 172.21.15.3:0/2861049556 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1139 (secure 0 0 0) 0x7fc114016090 con 0x7fc120159fb0 2024-09-12T12:19:54.004 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:54.004 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1726143578,"num_in_osds":8,"osd_in_since":1726143567,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":86},{"state_name":"creating+peering","count":11}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":224391168,"bytes_avail":772869722112,"bytes_total":773094113280,"inactive_pgs_ratio":0.11340206116437912,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4306}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-12T12:19:24.976131+0000","services":{}},"progress_events":{}} 2024-09-12T12:19:54.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc10c0678c0 msgr2=0x7fc10c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc10c0678c0 0x7fc10c069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc1201f8a20 tx=0x7fc110009040 comp rx=0 tx=0).stop 2024-09-12T12:19:54.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 msgr2=0x7fc1200aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc1200aff10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc11401a0a0 tx=0x7fc11401acb0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 shutdown_connections 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc10c0678c0 0x7fc10c069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc12015d2d0 0x7fc120099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc120159fb0 0x7fc1200aff10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 --2- 172.21.15.3:0/2861049556 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc120159bd0 0x7fc1201458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 >> 172.21.15.3:0/2861049556 conn(0x7fc1200b1d80 msgr2=0x7fc1200b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 shutdown_connections 2024-09-12T12:19:54.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.004+0000 7fc126d04640 1 -- 172.21.15.3:0/2861049556 wait complete. 2024-09-12T12:19:54.024 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T12:19:54.024 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:19:54.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.140+0000 7fdb85593640 1 Processor -- start 2024-09-12T12:19:54.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb85593640 1 -- start start 2024-09-12T12:19:54.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb85593640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb800703d0 0x7fdb8009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:54.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb85593640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb800721e0 con 0x7fdb800cd4a0 2024-09-12T12:19:54.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb85593640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb800ce7c0 con 0x7fdb8009b350 2024-09-12T12:19:54.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb85593640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdb800ccd50 con 0x7fdb800703d0 2024-09-12T12:19:54.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb7effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb800703d0 0x7fdb8009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:54.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb7effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdb800703d0 0x7fdb8009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60982/0 (socket says 172.21.15.3:60982) 2024-09-12T12:19:54.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb7effd640 1 -- 172.21.15.3:0/4242622069 learned_addr learned my addr 172.21.15.3:0/4242622069 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:54.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb7effd640 1 -- 172.21.15.3:0/4242622069 >> v1:172.21.15.142:6789/0 conn(0x7fdb8009b350 legacy=0x7fdb8009b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:54.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb7effd640 1 -- 172.21.15.3:0/4242622069 >> v1:172.21.15.3:6789/0 conn(0x7fdb800cd4a0 legacy=0x7fdb800cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.144+0000 7fdb7effd640 1 -- 172.21.15.3:0/4242622069 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb800ceaa0 con 0x7fdb800703d0 2024-09-12T12:19:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb7effd640 1 --2- 172.21.15.3:0/4242622069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8009aa70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009870 tx=0x7fdb7001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05464ca67fc2838 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb7e7fc640 1 -- 172.21.15.3:0/4242622069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700ab430 con 0x7fdb800703d0 2024-09-12T12:19:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb7e7fc640 1 -- 172.21.15.3:0/4242622069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb700b0070 con 0x7fdb800703d0 2024-09-12T12:19:54.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb7e7fc640 1 -- 172.21.15.3:0/4242622069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700ab7f0 con 0x7fdb800703d0 2024-09-12T12:19:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 -- 172.21.15.3:0/4242622069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 msgr2=0x7fdb8009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 --2- 172.21.15.3:0/4242622069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8009aa70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009870 tx=0x7fdb7001c520 comp rx=0 tx=0).stop 2024-09-12T12:19:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 -- 172.21.15.3:0/4242622069 shutdown_connections 2024-09-12T12:19:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 --2- 172.21.15.3:0/4242622069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8009aa70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 -- 172.21.15.3:0/4242622069 >> 172.21.15.3:0/4242622069 conn(0x7fdb800b3e60 msgr2=0x7fdb800b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 -- 172.21.15.3:0/4242622069 shutdown_connections 2024-09-12T12:19:54.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 -- 172.21.15.3:0/4242622069 wait complete. 2024-09-12T12:19:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 Processor -- start 2024-09-12T12:19:54.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.148+0000 7fdb85593640 1 -- start start 2024-09-12T12:19:54.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:54.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb8009b350 0x7fdb800b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb800cd4a0 0x7fdb80152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80071140 con 0x7fdb800703d0 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80070fc0 con 0x7fdb800cd4a0 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdb800712c0 con 0x7fdb8009b350 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8014c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60990/0 (socket says 172.21.15.3:60990) 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 -- 172.21.15.3:0/234647469 learned_addr learned my addr 172.21.15.3:0/234647469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 -- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb8009b350 msgr2=0x7fdb800b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb767fc640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb8009b350 0x7fdb800b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7f7fe640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb800cd4a0 0x7fdb80152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:54.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb8009b350 0x7fdb800b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 -- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb800cd4a0 msgr2=0x7fdb80152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb800cd4a0 0x7fdb80152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 -- 172.21.15.3:0/234647469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8015eaa0 con 0x7fdb800703d0 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb767fc640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb8009b350 0x7fdb800b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb7effd640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8014c470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb700ad040 tx=0x7fdb7001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb77fff640 1 -- 172.21.15.3:0/234647469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700b4800 con 0x7fdb800703d0 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb77fff640 1 -- 172.21.15.3:0/234647469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb700b49c0 con 0x7fdb800703d0 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb77fff640 1 -- 172.21.15.3:0/234647469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb700b4db0 con 0x7fdb800703d0 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8015ed80 con 0x7fdb800703d0 2024-09-12T12:19:54.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8015f340 con 0x7fdb800703d0 2024-09-12T12:19:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8009aa70 con 0x7fdb800703d0 2024-09-12T12:19:54.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb77fff640 1 -- 172.21.15.3:0/234647469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdb700b3380 con 0x7fdb800703d0 2024-09-12T12:19:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb77fff640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdb600678f0 0x7fdb60069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb77fff640 1 -- 172.21.15.3:0/234647469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdb700a8030 con 0x7fdb800703d0 2024-09-12T12:19:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.152+0000 7fdb767fc640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdb600678f0 0x7fdb60069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:54.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.156+0000 7fdb767fc640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdb600678f0 0x7fdb60069db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb800d1690 tx=0x7fdb68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:54.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.156+0000 7fdb77fff640 1 -- 172.21.15.3:0/234647469 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdb70016110 con 0x7fdb800703d0 2024-09-12T12:19:54.244 INFO:tasks.ceph.mds.d.smithi142.stderr:2024-09-12T12:19:54.243+0000 7f1933365640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:19:54.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.324+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb8015f6b0 con 0x7fdb600678f0 2024-09-12T12:19:54.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.340+0000 7fdb77fff640 1 -- 172.21.15.3:0/234647469 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251048 (secure 0 0 0) 0x7fdb680554c0 con 0x7fdb600678f0 2024-09-12T12:19:54.346 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:54.348 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:19:54.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdb600678f0 msgr2=0x7fdb60069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdb600678f0 0x7fdb60069db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb800d1690 tx=0x7fdb68009040 comp rx=0 tx=0).stop 2024-09-12T12:19:54.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 msgr2=0x7fdb8014c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8014c470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb700ad040 tx=0x7fdb7001ad60 comp rx=0 tx=0).stop 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 shutdown_connections 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdb600678f0 0x7fdb60069db0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb800cd4a0 0x7fdb80152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdb8009b350 0x7fdb800b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 --2- 172.21.15.3:0/234647469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdb800703d0 0x7fdb8014c470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.344+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 >> 172.21.15.3:0/234647469 conn(0x7fdb800b3e60 msgr2=0x7fdb800cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.348+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 shutdown_connections 2024-09-12T12:19:54.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:54.348+0000 7fdb85593640 1 -- 172.21.15.3:0/234647469 wait complete. 2024-09-12T12:19:54.364 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-12T12:19:53.656850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219132,"kb_used_data":2852,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755588,"statfs":{"total":773094113280,"available":772869722112,"internally_reserved":0,"allocated":2920448,"data_stored":1761933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002196"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995369+0000","last_change":"2024-09-12T12:19:48.781313+0000","last_active":"2024-09-12T12:19:49.995369+0000","last_peered":"2024-09-12T12:19:49.995369+0000","last_clean":"2024-09-12T12:19:49.995369+0000","last_became_active":"2024-09-12T12:19:48.780836+0000","last_became_peered":"2024-09-12T12:19:48.780836+0000","last_unstale":"2024-09-12T12:19:49.995369+0000","last_undegraded":"2024-09-12T12:19:49.995369+0000","last_fullsized":"2024-09-12T12:19:49.995369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754520+0000","last_change":"2024-09-12T12:19:48.759206+0000","last_active":"2024-09-12T12:19:49.754520+0000","last_peered":"2024-09-12T12:19:49.754520+0000","last_clean":"2024-09-12T12:19:49.754520+0000","last_became_active":"2024-09-12T12:19:48.758450+0000","last_became_peered":"2024-09-12T12:19:48.758450+0000","last_unstale":"2024-09-12T12:19:49.754520+0000","last_undegraded":"2024-09-12T12:19:49.754520+0000","last_fullsized":"2024-09-12T12:19:49.754520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992848+0000","last_change":"2024-09-12T12:19:48.769260+0000","last_active":"2024-09-12T12:19:49.992848+0000","last_peered":"2024-09-12T12:19:49.992848+0000","last_clean":"2024-09-12T12:19:49.992848+0000","last_became_active":"2024-09-12T12:19:48.768869+0000","last_became_peered":"2024-09-12T12:19:48.768869+0000","last_unstale":"2024-09-12T12:19:49.992848+0000","last_undegraded":"2024-09-12T12:19:49.992848+0000","last_fullsized":"2024-09-12T12:19:49.992848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:41:14.637521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994736+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994736+0000","last_peered":"2024-09-12T12:19:49.994736+0000","last_clean":"2024-09-12T12:19:49.994736+0000","last_became_active":"2024-09-12T12:19:48.779065+0000","last_became_peered":"2024-09-12T12:19:48.779065+0000","last_unstale":"2024-09-12T12:19:49.994736+0000","last_undegraded":"2024-09-12T12:19:49.994736+0000","last_fullsized":"2024-09-12T12:19:49.994736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993392+0000","last_change":"2024-09-12T12:19:48.766342+0000","last_active":"2024-09-12T12:19:49.993392+0000","last_peered":"2024-09-12T12:19:49.993392+0000","last_clean":"2024-09-12T12:19:49.993392+0000","last_became_active":"2024-09-12T12:19:48.766063+0000","last_became_peered":"2024-09-12T12:19:48.766063+0000","last_unstale":"2024-09-12T12:19:49.993392+0000","last_undegraded":"2024-09-12T12:19:49.993392+0000","last_fullsized":"2024-09-12T12:19:49.993392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993183+0000","last_change":"2024-09-12T12:19:48.769354+0000","last_active":"2024-09-12T12:19:49.993183+0000","last_peered":"2024-09-12T12:19:49.993183+0000","last_clean":"2024-09-12T12:19:49.993183+0000","last_became_active":"2024-09-12T12:19:48.769084+0000","last_became_peered":"2024-09-12T12:19:48.769084+0000","last_unstale":"2024-09-12T12:19:49.993183+0000","last_undegraded":"2024-09-12T12:19:49.993183+0000","last_fullsized":"2024-09-12T12:19:49.993183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:56.236676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992999+0000","last_change":"2024-09-12T12:19:48.773715+0000","last_active":"2024-09-12T12:19:49.992999+0000","last_peered":"2024-09-12T12:19:49.992999+0000","last_clean":"2024-09-12T12:19:49.992999+0000","last_became_active":"2024-09-12T12:19:48.773419+0000","last_became_peered":"2024-09-12T12:19:48.773419+0000","last_unstale":"2024-09-12T12:19:49.992999+0000","last_undegraded":"2024-09-12T12:19:49.992999+0000","last_fullsized":"2024-09-12T12:19:49.992999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.740554+0000","last_change":"2024-09-12T12:19:47.740554+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.740554+0000","last_undegraded":"2024-09-12T12:19:47.740554+0000","last_fullsized":"2024-09-12T12:19:47.740554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995405+0000","last_change":"2024-09-12T12:19:48.775263+0000","last_active":"2024-09-12T12:19:49.995405+0000","last_peered":"2024-09-12T12:19:49.995405+0000","last_clean":"2024-09-12T12:19:49.995405+0000","last_became_active":"2024-09-12T12:19:48.774506+0000","last_became_peered":"2024-09-12T12:19:48.774506+0000","last_unstale":"2024-09-12T12:19:49.995405+0000","last_undegraded":"2024-09-12T12:19:49.995405+0000","last_fullsized":"2024-09-12T12:19:49.995405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:38:46.549546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759178+0000","last_change":"2024-09-12T12:19:48.778949+0000","last_active":"2024-09-12T12:19:49.759178+0000","last_peered":"2024-09-12T12:19:49.759178+0000","last_clean":"2024-09-12T12:19:49.759178+0000","last_became_active":"2024-09-12T12:19:48.778672+0000","last_became_peered":"2024-09-12T12:19:48.778672+0000","last_unstale":"2024-09-12T12:19:49.759178+0000","last_undegraded":"2024-09-12T12:19:49.759178+0000","last_fullsized":"2024-09-12T12:19:49.759178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758789+0000","last_change":"2024-09-12T12:19:48.779161+0000","last_active":"2024-09-12T12:19:49.758789+0000","last_peered":"2024-09-12T12:19:49.758789+0000","last_clean":"2024-09-12T12:19:49.758789+0000","last_became_active":"2024-09-12T12:19:48.778765+0000","last_became_peered":"2024-09-12T12:19:48.778765+0000","last_unstale":"2024-09-12T12:19:49.758789+0000","last_undegraded":"2024-09-12T12:19:49.758789+0000","last_fullsized":"2024-09-12T12:19:49.758789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749404+0000","last_change":"2024-09-12T12:19:48.762583+0000","last_active":"2024-09-12T12:19:49.749404+0000","last_peered":"2024-09-12T12:19:49.749404+0000","last_clean":"2024-09-12T12:19:49.749404+0000","last_became_active":"2024-09-12T12:19:48.762315+0000","last_became_peered":"2024-09-12T12:19:48.762315+0000","last_unstale":"2024-09-12T12:19:49.749404+0000","last_undegraded":"2024-09-12T12:19:49.749404+0000","last_fullsized":"2024-09-12T12:19:49.749404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739346+0000","last_change":"2024-09-12T12:19:47.739346+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739346+0000","last_undegraded":"2024-09-12T12:19:47.739346+0000","last_fullsized":"2024-09-12T12:19:47.739346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993137+0000","last_change":"2024-09-12T12:19:48.763027+0000","last_active":"2024-09-12T12:19:49.993137+0000","last_peered":"2024-09-12T12:19:49.993137+0000","last_clean":"2024-09-12T12:19:49.993137+0000","last_became_active":"2024-09-12T12:19:48.762544+0000","last_became_peered":"2024-09-12T12:19:48.762544+0000","last_unstale":"2024-09-12T12:19:49.993137+0000","last_undegraded":"2024-09-12T12:19:49.993137+0000","last_fullsized":"2024-09-12T12:19:49.993137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:44.728800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753861+0000","last_change":"2024-09-12T12:19:48.770278+0000","last_active":"2024-09-12T12:19:49.753861+0000","last_peered":"2024-09-12T12:19:49.753861+0000","last_clean":"2024-09-12T12:19:49.753861+0000","last_became_active":"2024-09-12T12:19:48.769983+0000","last_became_peered":"2024-09-12T12:19:48.769983+0000","last_unstale":"2024-09-12T12:19:49.753861+0000","last_undegraded":"2024-09-12T12:19:49.753861+0000","last_fullsized":"2024-09-12T12:19:49.753861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992463+0000","last_change":"2024-09-12T12:19:48.769537+0000","last_active":"2024-09-12T12:19:49.992463+0000","last_peered":"2024-09-12T12:19:49.992463+0000","last_clean":"2024-09-12T12:19:49.992463+0000","last_became_active":"2024-09-12T12:19:48.769243+0000","last_became_peered":"2024-09-12T12:19:48.769243+0000","last_unstale":"2024-09-12T12:19:49.992463+0000","last_undegraded":"2024-09-12T12:19:49.992463+0000","last_fullsized":"2024-09-12T12:19:49.992463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995255+0000","last_change":"2024-09-12T12:19:48.766440+0000","last_active":"2024-09-12T12:19:49.995255+0000","last_peered":"2024-09-12T12:19:49.995255+0000","last_clean":"2024-09-12T12:19:49.995255+0000","last_became_active":"2024-09-12T12:19:48.766110+0000","last_became_peered":"2024-09-12T12:19:48.766110+0000","last_unstale":"2024-09-12T12:19:49.995255+0000","last_undegraded":"2024-09-12T12:19:49.995255+0000","last_fullsized":"2024-09-12T12:19:49.995255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992782+0000","last_change":"2024-09-12T12:19:48.780118+0000","last_active":"2024-09-12T12:19:49.992782+0000","last_peered":"2024-09-12T12:19:49.992782+0000","last_clean":"2024-09-12T12:19:49.992782+0000","last_became_active":"2024-09-12T12:19:48.779839+0000","last_became_peered":"2024-09-12T12:19:48.779839+0000","last_unstale":"2024-09-12T12:19:49.992782+0000","last_undegraded":"2024-09-12T12:19:49.992782+0000","last_fullsized":"2024-09-12T12:19:49.992782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993776+0000","last_change":"2024-09-12T12:19:48.766221+0000","last_active":"2024-09-12T12:19:49.993776+0000","last_peered":"2024-09-12T12:19:49.993776+0000","last_clean":"2024-09-12T12:19:49.993776+0000","last_became_active":"2024-09-12T12:19:48.765549+0000","last_became_peered":"2024-09-12T12:19:48.765549+0000","last_unstale":"2024-09-12T12:19:49.993776+0000","last_undegraded":"2024-09-12T12:19:49.993776+0000","last_fullsized":"2024-09-12T12:19:49.993776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758800+0000","last_change":"2024-09-12T12:19:48.764597+0000","last_active":"2024-09-12T12:19:49.758800+0000","last_peered":"2024-09-12T12:19:49.758800+0000","last_clean":"2024-09-12T12:19:49.758800+0000","last_became_active":"2024-09-12T12:19:48.764377+0000","last_became_peered":"2024-09-12T12:19:48.764377+0000","last_unstale":"2024-09-12T12:19:49.758800+0000","last_undegraded":"2024-09-12T12:19:49.758800+0000","last_fullsized":"2024-09-12T12:19:49.758800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992940+0000","last_change":"2024-09-12T12:19:48.763523+0000","last_active":"2024-09-12T12:19:49.992940+0000","last_peered":"2024-09-12T12:19:49.992940+0000","last_clean":"2024-09-12T12:19:49.992940+0000","last_became_active":"2024-09-12T12:19:48.762889+0000","last_became_peered":"2024-09-12T12:19:48.762889+0000","last_unstale":"2024-09-12T12:19:49.992940+0000","last_undegraded":"2024-09-12T12:19:49.992940+0000","last_fullsized":"2024-09-12T12:19:49.992940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:04:18.599614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993579+0000","last_change":"2024-09-12T12:19:48.760812+0000","last_active":"2024-09-12T12:19:49.993579+0000","last_peered":"2024-09-12T12:19:49.993579+0000","last_clean":"2024-09-12T12:19:49.993579+0000","last_became_active":"2024-09-12T12:19:48.760551+0000","last_became_peered":"2024-09-12T12:19:48.760551+0000","last_unstale":"2024-09-12T12:19:49.993579+0000","last_undegraded":"2024-09-12T12:19:49.993579+0000","last_fullsized":"2024-09-12T12:19:49.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739200+0000","last_change":"2024-09-12T12:19:47.739200+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739200+0000","last_undegraded":"2024-09-12T12:19:47.739200+0000","last_fullsized":"2024-09-12T12:19:47.739200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992589+0000","last_change":"2024-09-12T12:19:48.780825+0000","last_active":"2024-09-12T12:19:49.992589+0000","last_peered":"2024-09-12T12:19:49.992589+0000","last_clean":"2024-09-12T12:19:49.992589+0000","last_became_active":"2024-09-12T12:19:48.780593+0000","last_became_peered":"2024-09-12T12:19:48.780593+0000","last_unstale":"2024-09-12T12:19:49.992589+0000","last_undegraded":"2024-09-12T12:19:49.992589+0000","last_fullsized":"2024-09-12T12:19:49.992589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:50.202989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738680+0000","last_change":"2024-09-12T12:19:47.738680+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738680+0000","last_undegraded":"2024-09-12T12:19:47.738680+0000","last_fullsized":"2024-09-12T12:19:47.738680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759372+0000","last_change":"2024-09-12T12:19:48.774953+0000","last_active":"2024-09-12T12:19:49.759372+0000","last_peered":"2024-09-12T12:19:49.759372+0000","last_clean":"2024-09-12T12:19:49.759372+0000","last_became_active":"2024-09-12T12:19:48.774474+0000","last_became_peered":"2024-09-12T12:19:48.774474+0000","last_unstale":"2024-09-12T12:19:49.759372+0000","last_undegraded":"2024-09-12T12:19:49.759372+0000","last_fullsized":"2024-09-12T12:19:49.759372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754066+0000","last_change":"2024-09-12T12:19:48.771914+0000","last_active":"2024-09-12T12:19:49.754066+0000","last_peered":"2024-09-12T12:19:49.754066+0000","last_clean":"2024-09-12T12:19:49.754066+0000","last_became_active":"2024-09-12T12:19:48.771637+0000","last_became_peered":"2024-09-12T12:19:48.771637+0000","last_unstale":"2024-09-12T12:19:49.754066+0000","last_undegraded":"2024-09-12T12:19:49.754066+0000","last_fullsized":"2024-09-12T12:19:49.754066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735706+0000","last_change":"2024-09-12T12:19:47.735706+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735706+0000","last_undegraded":"2024-09-12T12:19:47.735706+0000","last_fullsized":"2024-09-12T12:19:47.735706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993036+0000","last_change":"2024-09-12T12:19:48.759011+0000","last_active":"2024-09-12T12:19:49.993036+0000","last_peered":"2024-09-12T12:19:49.993036+0000","last_clean":"2024-09-12T12:19:49.993036+0000","last_became_active":"2024-09-12T12:19:48.758267+0000","last_became_peered":"2024-09-12T12:19:48.758267+0000","last_unstale":"2024-09-12T12:19:49.993036+0000","last_undegraded":"2024-09-12T12:19:49.993036+0000","last_fullsized":"2024-09-12T12:19:49.993036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:45.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993953+0000","last_change":"2024-09-12T12:19:48.779776+0000","last_active":"2024-09-12T12:19:49.993953+0000","last_peered":"2024-09-12T12:19:49.993953+0000","last_clean":"2024-09-12T12:19:49.993953+0000","last_became_active":"2024-09-12T12:19:48.779500+0000","last_became_peered":"2024-09-12T12:19:48.779500+0000","last_unstale":"2024-09-12T12:19:49.993953+0000","last_undegraded":"2024-09-12T12:19:49.993953+0000","last_fullsized":"2024-09-12T12:19:49.993953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994016+0000","last_change":"2024-09-12T12:19:48.775150+0000","last_active":"2024-09-12T12:19:49.994016+0000","last_peered":"2024-09-12T12:19:49.994016+0000","last_clean":"2024-09-12T12:19:49.994016+0000","last_became_active":"2024-09-12T12:19:48.774841+0000","last_became_peered":"2024-09-12T12:19:48.774841+0000","last_unstale":"2024-09-12T12:19:49.994016+0000","last_undegraded":"2024-09-12T12:19:49.994016+0000","last_fullsized":"2024-09-12T12:19:49.994016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758353+0000","last_change":"2024-09-12T12:19:48.761408+0000","last_active":"2024-09-12T12:19:49.758353+0000","last_peered":"2024-09-12T12:19:49.758353+0000","last_clean":"2024-09-12T12:19:49.758353+0000","last_became_active":"2024-09-12T12:19:48.761091+0000","last_became_peered":"2024-09-12T12:19:48.761091+0000","last_unstale":"2024-09-12T12:19:49.758353+0000","last_undegraded":"2024-09-12T12:19:49.758353+0000","last_fullsized":"2024-09-12T12:19:49.758353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:51.034838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748611+0000","last_change":"2024-09-12T12:19:48.764078+0000","last_active":"2024-09-12T12:19:49.748611+0000","last_peered":"2024-09-12T12:19:49.748611+0000","last_clean":"2024-09-12T12:19:49.748611+0000","last_became_active":"2024-09-12T12:19:48.763831+0000","last_became_peered":"2024-09-12T12:19:48.763831+0000","last_unstale":"2024-09-12T12:19:49.748611+0000","last_undegraded":"2024-09-12T12:19:49.748611+0000","last_fullsized":"2024-09-12T12:19:49.748611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:06.350696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761667+0000","last_change":"2024-09-12T12:19:47.757834+0000","last_active":"2024-09-12T12:19:47.761667+0000","last_peered":"2024-09-12T12:19:47.761667+0000","last_clean":"2024-09-12T12:19:47.761667+0000","last_became_active":"2024-09-12T12:19:47.757145+0000","last_became_peered":"2024-09-12T12:19:47.757145+0000","last_unstale":"2024-09-12T12:19:47.761667+0000","last_undegraded":"2024-09-12T12:19:47.761667+0000","last_fullsized":"2024-09-12T12:19:47.761667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:59.832560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993352+0000","last_change":"2024-09-12T12:19:48.761558+0000","last_active":"2024-09-12T12:19:49.993352+0000","last_peered":"2024-09-12T12:19:49.993352+0000","last_clean":"2024-09-12T12:19:49.993352+0000","last_became_active":"2024-09-12T12:19:48.761230+0000","last_became_peered":"2024-09-12T12:19:48.761230+0000","last_unstale":"2024-09-12T12:19:49.993352+0000","last_undegraded":"2024-09-12T12:19:49.993352+0000","last_fullsized":"2024-09-12T12:19:49.993352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:40.506967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758441+0000","last_change":"2024-09-12T12:19:47.758747+0000","last_active":"2024-09-12T12:19:49.758441+0000","last_peered":"2024-09-12T12:19:49.758441+0000","last_clean":"2024-09-12T12:19:49.758441+0000","last_became_active":"2024-09-12T12:19:47.757870+0000","last_became_peered":"2024-09-12T12:19:47.757870+0000","last_unstale":"2024-09-12T12:19:49.758441+0000","last_undegraded":"2024-09-12T12:19:49.758441+0000","last_fullsized":"2024-09-12T12:19:49.758441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761691+0000","last_change":"2024-09-12T12:19:47.758883+0000","last_active":"2024-09-12T12:19:47.761691+0000","last_peered":"2024-09-12T12:19:47.761691+0000","last_clean":"2024-09-12T12:19:47.761691+0000","last_became_active":"2024-09-12T12:19:47.758396+0000","last_became_peered":"2024-09-12T12:19:47.758396+0000","last_unstale":"2024-09-12T12:19:47.761691+0000","last_undegraded":"2024-09-12T12:19:47.761691+0000","last_fullsized":"2024-09-12T12:19:47.761691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:42.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.738678+0000","last_change":"2024-09-12T12:19:47.738678+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.738678+0000","last_undegraded":"2024-09-12T12:19:47.738678+0000","last_fullsized":"2024-09-12T12:19:47.738678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992602+0000","last_change":"2024-09-12T12:19:47.740584+0000","last_active":"2024-09-12T12:19:49.992602+0000","last_peered":"2024-09-12T12:19:49.992602+0000","last_clean":"2024-09-12T12:19:49.992602+0000","last_became_active":"2024-09-12T12:19:47.739985+0000","last_became_peered":"2024-09-12T12:19:47.739985+0000","last_unstale":"2024-09-12T12:19:49.992602+0000","last_undegraded":"2024-09-12T12:19:49.992602+0000","last_fullsized":"2024-09-12T12:19:49.992602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753847+0000","last_change":"2024-09-12T12:19:48.771435+0000","last_active":"2024-09-12T12:19:49.753847+0000","last_peered":"2024-09-12T12:19:49.753847+0000","last_clean":"2024-09-12T12:19:49.753847+0000","last_became_active":"2024-09-12T12:19:48.771108+0000","last_became_peered":"2024-09-12T12:19:48.771108+0000","last_unstale":"2024-09-12T12:19:49.753847+0000","last_undegraded":"2024-09-12T12:19:49.753847+0000","last_fullsized":"2024-09-12T12:19:49.753847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758579+0000","last_change":"2024-09-12T12:19:47.760876+0000","last_active":"2024-09-12T12:19:49.758579+0000","last_peered":"2024-09-12T12:19:49.758579+0000","last_clean":"2024-09-12T12:19:49.758579+0000","last_became_active":"2024-09-12T12:19:47.760036+0000","last_became_peered":"2024-09-12T12:19:47.760036+0000","last_unstale":"2024-09-12T12:19:49.758579+0000","last_undegraded":"2024-09-12T12:19:49.758579+0000","last_fullsized":"2024-09-12T12:19:49.758579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993540+0000","last_change":"2024-09-12T12:19:48.765500+0000","last_active":"2024-09-12T12:19:49.993540+0000","last_peered":"2024-09-12T12:19:49.993540+0000","last_clean":"2024-09-12T12:19:49.993540+0000","last_became_active":"2024-09-12T12:19:48.765191+0000","last_became_peered":"2024-09-12T12:19:48.765191+0000","last_unstale":"2024-09-12T12:19:49.993540+0000","last_undegraded":"2024-09-12T12:19:49.993540+0000","last_fullsized":"2024-09-12T12:19:49.993540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993676+0000","last_change":"2024-09-12T12:19:47.761042+0000","last_active":"2024-09-12T12:19:49.993676+0000","last_peered":"2024-09-12T12:19:49.993676+0000","last_clean":"2024-09-12T12:19:49.993676+0000","last_became_active":"2024-09-12T12:19:47.760261+0000","last_became_peered":"2024-09-12T12:19:47.760261+0000","last_unstale":"2024-09-12T12:19:49.993676+0000","last_undegraded":"2024-09-12T12:19:49.993676+0000","last_fullsized":"2024-09-12T12:19:49.993676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993444+0000","last_change":"2024-09-12T12:19:48.769338+0000","last_active":"2024-09-12T12:19:49.993444+0000","last_peered":"2024-09-12T12:19:49.993444+0000","last_clean":"2024-09-12T12:19:49.993444+0000","last_became_active":"2024-09-12T12:19:48.768955+0000","last_became_peered":"2024-09-12T12:19:48.768955+0000","last_unstale":"2024-09-12T12:19:49.993444+0000","last_undegraded":"2024-09-12T12:19:49.993444+0000","last_fullsized":"2024-09-12T12:19:49.993444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.656049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754127+0000","last_change":"2024-09-12T12:19:47.750999+0000","last_active":"2024-09-12T12:19:49.754127+0000","last_peered":"2024-09-12T12:19:49.754127+0000","last_clean":"2024-09-12T12:19:49.754127+0000","last_became_active":"2024-09-12T12:19:47.750658+0000","last_became_peered":"2024-09-12T12:19:47.750658+0000","last_unstale":"2024-09-12T12:19:49.754127+0000","last_undegraded":"2024-09-12T12:19:49.754127+0000","last_fullsized":"2024-09-12T12:19:49.754127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749115+0000","last_change":"2024-09-12T12:19:48.767953+0000","last_active":"2024-09-12T12:19:49.749115+0000","last_peered":"2024-09-12T12:19:49.749115+0000","last_clean":"2024-09-12T12:19:49.749115+0000","last_became_active":"2024-09-12T12:19:48.767595+0000","last_became_peered":"2024-09-12T12:19:48.767595+0000","last_unstale":"2024-09-12T12:19:49.749115+0000","last_undegraded":"2024-09-12T12:19:49.749115+0000","last_fullsized":"2024-09-12T12:19:49.749115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758590+0000","last_change":"2024-09-12T12:19:47.760548+0000","last_active":"2024-09-12T12:19:49.758590+0000","last_peered":"2024-09-12T12:19:49.758590+0000","last_clean":"2024-09-12T12:19:49.758590+0000","last_became_active":"2024-09-12T12:19:47.760033+0000","last_became_peered":"2024-09-12T12:19:47.760033+0000","last_unstale":"2024-09-12T12:19:49.758590+0000","last_undegraded":"2024-09-12T12:19:49.758590+0000","last_fullsized":"2024-09-12T12:19:49.758590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994749+0000","last_change":"2024-09-12T12:19:48.773829+0000","last_active":"2024-09-12T12:19:49.994749+0000","last_peered":"2024-09-12T12:19:49.994749+0000","last_clean":"2024-09-12T12:19:49.994749+0000","last_became_active":"2024-09-12T12:19:48.773359+0000","last_became_peered":"2024-09-12T12:19:48.773359+0000","last_unstale":"2024-09-12T12:19:49.994749+0000","last_undegraded":"2024-09-12T12:19:49.994749+0000","last_fullsized":"2024-09-12T12:19:49.994749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993173+0000","last_change":"2024-09-12T12:19:47.757426+0000","last_active":"2024-09-12T12:19:49.993173+0000","last_peered":"2024-09-12T12:19:49.993173+0000","last_clean":"2024-09-12T12:19:49.993173+0000","last_became_active":"2024-09-12T12:19:47.756782+0000","last_became_peered":"2024-09-12T12:19:47.756782+0000","last_unstale":"2024-09-12T12:19:49.993173+0000","last_undegraded":"2024-09-12T12:19:49.993173+0000","last_fullsized":"2024-09-12T12:19:49.993173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992472+0000","last_change":"2024-09-12T12:19:48.759150+0000","last_active":"2024-09-12T12:19:49.992472+0000","last_peered":"2024-09-12T12:19:49.992472+0000","last_clean":"2024-09-12T12:19:49.992472+0000","last_became_active":"2024-09-12T12:19:48.758654+0000","last_became_peered":"2024-09-12T12:19:48.758654+0000","last_unstale":"2024-09-12T12:19:49.992472+0000","last_undegraded":"2024-09-12T12:19:49.992472+0000","last_fullsized":"2024-09-12T12:19:49.992472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:37.527639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994079+0000","last_change":"2024-09-12T12:19:47.747437+0000","last_active":"2024-09-12T12:19:49.994079+0000","last_peered":"2024-09-12T12:19:49.994079+0000","last_clean":"2024-09-12T12:19:49.994079+0000","last_became_active":"2024-09-12T12:19:47.746926+0000","last_became_peered":"2024-09-12T12:19:47.746926+0000","last_unstale":"2024-09-12T12:19:49.994079+0000","last_undegraded":"2024-09-12T12:19:49.994079+0000","last_fullsized":"2024-09-12T12:19:49.994079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993790+0000","last_change":"2024-09-12T12:19:48.775580+0000","last_active":"2024-09-12T12:19:49.993790+0000","last_peered":"2024-09-12T12:19:49.993790+0000","last_clean":"2024-09-12T12:19:49.993790+0000","last_became_active":"2024-09-12T12:19:48.767021+0000","last_became_peered":"2024-09-12T12:19:48.767021+0000","last_unstale":"2024-09-12T12:19:49.993790+0000","last_undegraded":"2024-09-12T12:19:49.993790+0000","last_fullsized":"2024-09-12T12:19:49.993790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758162+0000","last_change":"2024-09-12T12:19:47.754802+0000","last_active":"2024-09-12T12:19:49.758162+0000","last_peered":"2024-09-12T12:19:49.758162+0000","last_clean":"2024-09-12T12:19:49.758162+0000","last_became_active":"2024-09-12T12:19:47.754242+0000","last_became_peered":"2024-09-12T12:19:47.754242+0000","last_unstale":"2024-09-12T12:19:49.758162+0000","last_undegraded":"2024-09-12T12:19:49.758162+0000","last_fullsized":"2024-09-12T12:19:49.758162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758450+0000","last_change":"2024-09-12T12:19:48.761340+0000","last_active":"2024-09-12T12:19:49.758450+0000","last_peered":"2024-09-12T12:19:49.758450+0000","last_clean":"2024-09-12T12:19:49.758450+0000","last_became_active":"2024-09-12T12:19:48.761037+0000","last_became_peered":"2024-09-12T12:19:48.761037+0000","last_unstale":"2024-09-12T12:19:49.758450+0000","last_undegraded":"2024-09-12T12:19:49.758450+0000","last_fullsized":"2024-09-12T12:19:49.758450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992730+0000","last_change":"2024-09-12T12:19:47.761479+0000","last_active":"2024-09-12T12:19:49.992730+0000","last_peered":"2024-09-12T12:19:49.992730+0000","last_clean":"2024-09-12T12:19:49.992730+0000","last_became_active":"2024-09-12T12:19:47.760812+0000","last_became_peered":"2024-09-12T12:19:47.760812+0000","last_unstale":"2024-09-12T12:19:49.992730+0000","last_undegraded":"2024-09-12T12:19:49.992730+0000","last_fullsized":"2024-09-12T12:19:49.992730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993407+0000","last_change":"2024-09-12T12:19:48.772189+0000","last_active":"2024-09-12T12:19:49.993407+0000","last_peered":"2024-09-12T12:19:49.993407+0000","last_clean":"2024-09-12T12:19:49.993407+0000","last_became_active":"2024-09-12T12:19:48.765342+0000","last_became_peered":"2024-09-12T12:19:48.765342+0000","last_unstale":"2024-09-12T12:19:49.993407+0000","last_undegraded":"2024-09-12T12:19:49.993407+0000","last_fullsized":"2024-09-12T12:19:49.993407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749324+0000","last_change":"2024-09-12T12:19:47.754107+0000","last_active":"2024-09-12T12:19:49.749324+0000","last_peered":"2024-09-12T12:19:49.749324+0000","last_clean":"2024-09-12T12:19:49.749324+0000","last_became_active":"2024-09-12T12:19:47.753433+0000","last_became_peered":"2024-09-12T12:19:47.753433+0000","last_unstale":"2024-09-12T12:19:49.749324+0000","last_undegraded":"2024-09-12T12:19:49.749324+0000","last_fullsized":"2024-09-12T12:19:49.749324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:28.270374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995095+0000","last_change":"2024-09-12T12:19:48.770240+0000","last_active":"2024-09-12T12:19:49.995095+0000","last_peered":"2024-09-12T12:19:49.995095+0000","last_clean":"2024-09-12T12:19:49.995095+0000","last_became_active":"2024-09-12T12:19:48.766786+0000","last_became_peered":"2024-09-12T12:19:48.766786+0000","last_unstale":"2024-09-12T12:19:49.995095+0000","last_undegraded":"2024-09-12T12:19:49.995095+0000","last_fullsized":"2024-09-12T12:19:49.995095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992870+0000","last_change":"2024-09-12T12:19:47.757967+0000","last_active":"2024-09-12T12:19:49.992870+0000","last_peered":"2024-09-12T12:19:49.992870+0000","last_clean":"2024-09-12T12:19:49.992870+0000","last_became_active":"2024-09-12T12:19:47.757370+0000","last_became_peered":"2024-09-12T12:19:47.757370+0000","last_unstale":"2024-09-12T12:19:49.992870+0000","last_undegraded":"2024-09-12T12:19:49.992870+0000","last_fullsized":"2024-09-12T12:19:49.992870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995730+0000","last_change":"2024-09-12T12:19:48.766283+0000","last_active":"2024-09-12T12:19:49.995730+0000","last_peered":"2024-09-12T12:19:49.995730+0000","last_clean":"2024-09-12T12:19:49.995730+0000","last_became_active":"2024-09-12T12:19:48.765694+0000","last_became_peered":"2024-09-12T12:19:48.765694+0000","last_unstale":"2024-09-12T12:19:49.995730+0000","last_undegraded":"2024-09-12T12:19:49.995730+0000","last_fullsized":"2024-09-12T12:19:49.995730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:42:42.831767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754237+0000","last_change":"2024-09-12T12:19:47.751839+0000","last_active":"2024-09-12T12:19:49.754237+0000","last_peered":"2024-09-12T12:19:49.754237+0000","last_clean":"2024-09-12T12:19:49.754237+0000","last_became_active":"2024-09-12T12:19:47.751482+0000","last_became_peered":"2024-09-12T12:19:47.751482+0000","last_unstale":"2024-09-12T12:19:49.754237+0000","last_undegraded":"2024-09-12T12:19:49.754237+0000","last_fullsized":"2024-09-12T12:19:49.754237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995497+0000","last_change":"2024-09-12T12:19:48.768636+0000","last_active":"2024-09-12T12:19:49.995497+0000","last_peered":"2024-09-12T12:19:49.995497+0000","last_clean":"2024-09-12T12:19:49.995497+0000","last_became_active":"2024-09-12T12:19:48.768342+0000","last_became_peered":"2024-09-12T12:19:48.768342+0000","last_unstale":"2024-09-12T12:19:49.995497+0000","last_undegraded":"2024-09-12T12:19:49.995497+0000","last_fullsized":"2024-09-12T12:19:49.995497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993248+0000","last_change":"2024-09-12T12:19:47.755829+0000","last_active":"2024-09-12T12:19:49.993248+0000","last_peered":"2024-09-12T12:19:49.993248+0000","last_clean":"2024-09-12T12:19:49.993248+0000","last_became_active":"2024-09-12T12:19:47.748443+0000","last_became_peered":"2024-09-12T12:19:47.748443+0000","last_unstale":"2024-09-12T12:19:49.993248+0000","last_undegraded":"2024-09-12T12:19:49.993248+0000","last_fullsized":"2024-09-12T12:19:49.993248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993070+0000","last_change":"2024-09-12T12:19:48.775968+0000","last_active":"2024-09-12T12:19:49.993070+0000","last_peered":"2024-09-12T12:19:49.993070+0000","last_clean":"2024-09-12T12:19:49.993070+0000","last_became_active":"2024-09-12T12:19:48.775731+0000","last_became_peered":"2024-09-12T12:19:48.775731+0000","last_unstale":"2024-09-12T12:19:49.993070+0000","last_undegraded":"2024-09-12T12:19:49.993070+0000","last_fullsized":"2024-09-12T12:19:49.993070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992710+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:19:49.992710+0000","last_peered":"2024-09-12T12:19:49.992710+0000","last_clean":"2024-09-12T12:19:49.992710+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:19:49.992710+0000","last_undegraded":"2024-09-12T12:19:49.992710+0000","last_fullsized":"2024-09-12T12:19:49.992710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993779+0000","last_change":"2024-09-12T12:19:47.757991+0000","last_active":"2024-09-12T12:19:49.993779+0000","last_peered":"2024-09-12T12:19:49.993779+0000","last_clean":"2024-09-12T12:19:49.993779+0000","last_became_active":"2024-09-12T12:19:47.757562+0000","last_became_peered":"2024-09-12T12:19:47.757562+0000","last_unstale":"2024-09-12T12:19:49.993779+0000","last_undegraded":"2024-09-12T12:19:49.993779+0000","last_fullsized":"2024-09-12T12:19:49.993779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.739193+0000","last_change":"2024-09-12T12:19:47.739193+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.739193+0000","last_undegraded":"2024-09-12T12:19:47.739193+0000","last_fullsized":"2024-09-12T12:19:47.739193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992441+0000","last_change":"2024-09-12T12:19:47.761015+0000","last_active":"2024-09-12T12:19:49.992441+0000","last_peered":"2024-09-12T12:19:49.992441+0000","last_clean":"2024-09-12T12:19:49.992441+0000","last_became_active":"2024-09-12T12:19:47.760494+0000","last_became_peered":"2024-09-12T12:19:47.760494+0000","last_unstale":"2024-09-12T12:19:49.992441+0000","last_undegraded":"2024-09-12T12:19:49.992441+0000","last_fullsized":"2024-09-12T12:19:49.992441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992375+0000","last_change":"2024-09-12T12:19:48.765122+0000","last_active":"2024-09-12T12:19:49.992375+0000","last_peered":"2024-09-12T12:19:49.992375+0000","last_clean":"2024-09-12T12:19:49.992375+0000","last_became_active":"2024-09-12T12:19:48.764670+0000","last_became_peered":"2024-09-12T12:19:48.764670+0000","last_unstale":"2024-09-12T12:19:49.992375+0000","last_undegraded":"2024-09-12T12:19:49.992375+0000","last_fullsized":"2024-09-12T12:19:49.992375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:57:01.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993998+0000","last_change":"2024-09-12T12:19:47.758292+0000","last_active":"2024-09-12T12:19:49.993998+0000","last_peered":"2024-09-12T12:19:49.993998+0000","last_clean":"2024-09-12T12:19:49.993998+0000","last_became_active":"2024-09-12T12:19:47.757643+0000","last_became_peered":"2024-09-12T12:19:47.757643+0000","last_unstale":"2024-09-12T12:19:49.993998+0000","last_undegraded":"2024-09-12T12:19:49.993998+0000","last_fullsized":"2024-09-12T12:19:49.993998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753888+0000","last_change":"2024-09-12T12:19:48.763042+0000","last_active":"2024-09-12T12:19:49.753888+0000","last_peered":"2024-09-12T12:19:49.753888+0000","last_clean":"2024-09-12T12:19:49.753888+0000","last_became_active":"2024-09-12T12:19:48.762214+0000","last_became_peered":"2024-09-12T12:19:48.762214+0000","last_unstale":"2024-09-12T12:19:49.753888+0000","last_undegraded":"2024-09-12T12:19:49.753888+0000","last_fullsized":"2024-09-12T12:19:49.753888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994326+0000","last_change":"2024-09-12T12:19:47.752251+0000","last_active":"2024-09-12T12:19:49.994326+0000","last_peered":"2024-09-12T12:19:49.994326+0000","last_clean":"2024-09-12T12:19:49.994326+0000","last_became_active":"2024-09-12T12:19:47.751530+0000","last_became_peered":"2024-09-12T12:19:47.751530+0000","last_unstale":"2024-09-12T12:19:49.994326+0000","last_undegraded":"2024-09-12T12:19:49.994326+0000","last_fullsized":"2024-09-12T12:19:49.994326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749074+0000","last_change":"2024-09-12T12:19:48.762555+0000","last_active":"2024-09-12T12:19:49.749074+0000","last_peered":"2024-09-12T12:19:49.749074+0000","last_clean":"2024-09-12T12:19:49.749074+0000","last_became_active":"2024-09-12T12:19:48.760885+0000","last_became_peered":"2024-09-12T12:19:48.760885+0000","last_unstale":"2024-09-12T12:19:49.749074+0000","last_undegraded":"2024-09-12T12:19:49.749074+0000","last_fullsized":"2024-09-12T12:19:49.749074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:16.559958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753838+0000","last_change":"2024-09-12T12:19:47.758162+0000","last_active":"2024-09-12T12:19:49.753838+0000","last_peered":"2024-09-12T12:19:49.753838+0000","last_clean":"2024-09-12T12:19:49.753838+0000","last_became_active":"2024-09-12T12:19:47.757816+0000","last_became_peered":"2024-09-12T12:19:47.757816+0000","last_unstale":"2024-09-12T12:19:49.753838+0000","last_undegraded":"2024-09-12T12:19:49.753838+0000","last_fullsized":"2024-09-12T12:19:49.753838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994340+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994340+0000","last_peered":"2024-09-12T12:19:49.994340+0000","last_clean":"2024-09-12T12:19:49.994340+0000","last_became_active":"2024-09-12T12:19:48.779234+0000","last_became_peered":"2024-09-12T12:19:48.779234+0000","last_unstale":"2024-09-12T12:19:49.994340+0000","last_undegraded":"2024-09-12T12:19:49.994340+0000","last_fullsized":"2024-09-12T12:19:49.994340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993920+0000","last_change":"2024-09-12T12:19:47.765924+0000","last_active":"2024-09-12T12:19:49.993920+0000","last_peered":"2024-09-12T12:19:49.993920+0000","last_clean":"2024-09-12T12:19:49.993920+0000","last_became_active":"2024-09-12T12:19:47.765244+0000","last_became_peered":"2024-09-12T12:19:47.765244+0000","last_unstale":"2024-09-12T12:19:49.993920+0000","last_undegraded":"2024-09-12T12:19:49.993920+0000","last_fullsized":"2024-09-12T12:19:49.993920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754093+0000","last_change":"2024-09-12T12:19:48.762307+0000","last_active":"2024-09-12T12:19:49.754093+0000","last_peered":"2024-09-12T12:19:49.754093+0000","last_clean":"2024-09-12T12:19:49.754093+0000","last_became_active":"2024-09-12T12:19:48.761728+0000","last_became_peered":"2024-09-12T12:19:48.761728+0000","last_unstale":"2024-09-12T12:19:49.754093+0000","last_undegraded":"2024-09-12T12:19:49.754093+0000","last_fullsized":"2024-09-12T12:19:49.754093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754345+0000","last_change":"2024-09-12T12:19:47.760726+0000","last_active":"2024-09-12T12:19:49.754345+0000","last_peered":"2024-09-12T12:19:49.754345+0000","last_clean":"2024-09-12T12:19:49.754345+0000","last_became_active":"2024-09-12T12:19:47.760059+0000","last_became_peered":"2024-09-12T12:19:47.760059+0000","last_unstale":"2024-09-12T12:19:49.754345+0000","last_undegraded":"2024-09-12T12:19:49.754345+0000","last_fullsized":"2024-09-12T12:19:49.754345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735219+0000","last_change":"2024-09-12T12:19:47.735219+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735219+0000","last_undegraded":"2024-09-12T12:19:47.735219+0000","last_fullsized":"2024-09-12T12:19:47.735219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758743+0000","last_change":"2024-09-12T12:19:47.765527+0000","last_active":"2024-09-12T12:19:49.758743+0000","last_peered":"2024-09-12T12:19:49.758743+0000","last_clean":"2024-09-12T12:19:49.758743+0000","last_became_active":"2024-09-12T12:19:47.764743+0000","last_became_peered":"2024-09-12T12:19:47.764743+0000","last_unstale":"2024-09-12T12:19:49.758743+0000","last_undegraded":"2024-09-12T12:19:49.758743+0000","last_fullsized":"2024-09-12T12:19:49.758743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992737+0000","last_change":"2024-09-12T12:19:48.761030+0000","last_active":"2024-09-12T12:19:49.992737+0000","last_peered":"2024-09-12T12:19:49.992737+0000","last_clean":"2024-09-12T12:19:49.992737+0000","last_became_active":"2024-09-12T12:19:48.760740+0000","last_became_peered":"2024-09-12T12:19:48.760740+0000","last_unstale":"2024-09-12T12:19:49.992737+0000","last_undegraded":"2024-09-12T12:19:49.992737+0000","last_fullsized":"2024-09-12T12:19:49.992737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:12.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992155+0000","last_change":"2024-09-12T12:19:47.764445+0000","last_active":"2024-09-12T12:19:49.992155+0000","last_peered":"2024-09-12T12:19:49.992155+0000","last_clean":"2024-09-12T12:19:49.992155+0000","last_became_active":"2024-09-12T12:19:47.763777+0000","last_became_peered":"2024-09-12T12:19:47.763777+0000","last_unstale":"2024-09-12T12:19:49.992155+0000","last_undegraded":"2024-09-12T12:19:49.992155+0000","last_fullsized":"2024-09-12T12:19:49.992155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748601+0000","last_change":"2024-09-12T12:19:48.755118+0000","last_active":"2024-09-12T12:19:49.748601+0000","last_peered":"2024-09-12T12:19:49.748601+0000","last_clean":"2024-09-12T12:19:49.748601+0000","last_became_active":"2024-09-12T12:19:48.754819+0000","last_became_peered":"2024-09-12T12:19:48.754819+0000","last_unstale":"2024-09-12T12:19:49.748601+0000","last_undegraded":"2024-09-12T12:19:49.748601+0000","last_fullsized":"2024-09-12T12:19:49.748601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:33.261675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753992+0000","last_change":"2024-09-12T12:19:47.755131+0000","last_active":"2024-09-12T12:19:49.753992+0000","last_peered":"2024-09-12T12:19:49.753992+0000","last_clean":"2024-09-12T12:19:49.753992+0000","last_became_active":"2024-09-12T12:19:47.754580+0000","last_became_peered":"2024-09-12T12:19:47.754580+0000","last_unstale":"2024-09-12T12:19:49.753992+0000","last_undegraded":"2024-09-12T12:19:49.753992+0000","last_fullsized":"2024-09-12T12:19:49.753992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735412+0000","last_change":"2024-09-12T12:19:47.735412+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735412+0000","last_undegraded":"2024-09-12T12:19:47.735412+0000","last_fullsized":"2024-09-12T12:19:47.735412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753795+0000","last_change":"2024-09-12T12:19:47.751410+0000","last_active":"2024-09-12T12:19:49.753795+0000","last_peered":"2024-09-12T12:19:49.753795+0000","last_clean":"2024-09-12T12:19:49.753795+0000","last_became_active":"2024-09-12T12:19:47.750752+0000","last_became_peered":"2024-09-12T12:19:47.750752+0000","last_unstale":"2024-09-12T12:19:49.753795+0000","last_undegraded":"2024-09-12T12:19:49.753795+0000","last_fullsized":"2024-09-12T12:19:49.753795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.736208+0000","last_change":"2024-09-12T12:19:47.736208+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.736208+0000","last_undegraded":"2024-09-12T12:19:47.736208+0000","last_fullsized":"2024-09-12T12:19:47.736208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993986+0000","last_change":"2024-09-12T12:19:47.768458+0000","last_active":"2024-09-12T12:19:49.993986+0000","last_peered":"2024-09-12T12:19:49.993986+0000","last_clean":"2024-09-12T12:19:49.993986+0000","last_became_active":"2024-09-12T12:19:47.767666+0000","last_became_peered":"2024-09-12T12:19:47.767666+0000","last_unstale":"2024-09-12T12:19:49.993986+0000","last_undegraded":"2024-09-12T12:19:49.993986+0000","last_fullsized":"2024-09-12T12:19:49.993986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758358+0000","last_change":"2024-09-12T12:19:48.760279+0000","last_active":"2024-09-12T12:19:49.758358+0000","last_peered":"2024-09-12T12:19:49.758358+0000","last_clean":"2024-09-12T12:19:49.758358+0000","last_became_active":"2024-09-12T12:19:48.759972+0000","last_became_peered":"2024-09-12T12:19:48.759972+0000","last_unstale":"2024-09-12T12:19:49.758358+0000","last_undegraded":"2024-09-12T12:19:49.758358+0000","last_fullsized":"2024-09-12T12:19:49.758358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993934+0000","last_change":"2024-09-12T12:19:48.765639+0000","last_active":"2024-09-12T12:19:49.993934+0000","last_peered":"2024-09-12T12:19:49.993934+0000","last_clean":"2024-09-12T12:19:49.993934+0000","last_became_active":"2024-09-12T12:19:48.765358+0000","last_became_peered":"2024-09-12T12:19:48.765358+0000","last_unstale":"2024-09-12T12:19:49.993934+0000","last_undegraded":"2024-09-12T12:19:49.993934+0000","last_fullsized":"2024-09-12T12:19:49.993934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994200+0000","last_change":"2024-09-12T12:19:47.763682+0000","last_active":"2024-09-12T12:19:49.994200+0000","last_peered":"2024-09-12T12:19:49.994200+0000","last_clean":"2024-09-12T12:19:49.994200+0000","last_became_active":"2024-09-12T12:19:47.762792+0000","last_became_peered":"2024-09-12T12:19:47.762792+0000","last_unstale":"2024-09-12T12:19:49.994200+0000","last_undegraded":"2024-09-12T12:19:49.994200+0000","last_fullsized":"2024-09-12T12:19:49.994200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995111+0000","last_change":"2024-09-12T12:19:48.765714+0000","last_active":"2024-09-12T12:19:49.995111+0000","last_peered":"2024-09-12T12:19:49.995111+0000","last_clean":"2024-09-12T12:19:49.995111+0000","last_became_active":"2024-09-12T12:19:48.765396+0000","last_became_peered":"2024-09-12T12:19:48.765396+0000","last_unstale":"2024-09-12T12:19:49.995111+0000","last_undegraded":"2024-09-12T12:19:49.995111+0000","last_fullsized":"2024-09-12T12:19:49.995111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993218+0000","last_change":"2024-09-12T12:19:47.770906+0000","last_active":"2024-09-12T12:19:49.993218+0000","last_peered":"2024-09-12T12:19:49.993218+0000","last_clean":"2024-09-12T12:19:49.993218+0000","last_became_active":"2024-09-12T12:19:47.770451+0000","last_became_peered":"2024-09-12T12:19:47.770451+0000","last_unstale":"2024-09-12T12:19:49.993218+0000","last_undegraded":"2024-09-12T12:19:49.993218+0000","last_fullsized":"2024-09-12T12:19:49.993218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993423+0000","last_change":"2024-09-12T12:19:48.774830+0000","last_active":"2024-09-12T12:19:49.993423+0000","last_peered":"2024-09-12T12:19:49.993423+0000","last_clean":"2024-09-12T12:19:49.993423+0000","last_became_active":"2024-09-12T12:19:48.774146+0000","last_became_peered":"2024-09-12T12:19:48.774146+0000","last_unstale":"2024-09-12T12:19:49.993423+0000","last_undegraded":"2024-09-12T12:19:49.993423+0000","last_fullsized":"2024-09-12T12:19:49.993423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:11.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-12T12:19:47.761902+0000","last_change":"2024-09-12T12:19:47.759563+0000","last_active":"2024-09-12T12:19:47.761902+0000","last_peered":"2024-09-12T12:19:47.761902+0000","last_clean":"2024-09-12T12:19:47.761902+0000","last_became_active":"2024-09-12T12:19:47.759012+0000","last_became_peered":"2024-09-12T12:19:47.759012+0000","last_unstale":"2024-09-12T12:19:47.761902+0000","last_undegraded":"2024-09-12T12:19:47.761902+0000","last_fullsized":"2024-09-12T12:19:47.761902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-12T12:19:47.735760+0000","last_change":"2024-09-12T12:19:47.735760+0000","last_active":"2024-09-12T12:19:47.718118+0000","last_peered":"2024-09-12T12:19:47.718118+0000","last_clean":"2024-09-12T12:19:47.718118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:19:47.735760+0000","last_undegraded":"2024-09-12T12:19:47.735760+0000","last_fullsized":"2024-09-12T12:19:47.735760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993656+0000","last_change":"2024-09-12T12:19:47.755575+0000","last_active":"2024-09-12T12:19:49.993656+0000","last_peered":"2024-09-12T12:19:49.993656+0000","last_clean":"2024-09-12T12:19:49.993656+0000","last_became_active":"2024-09-12T12:19:47.755279+0000","last_became_peered":"2024-09-12T12:19:47.755279+0000","last_unstale":"2024-09-12T12:19:49.993656+0000","last_undegraded":"2024-09-12T12:19:49.993656+0000","last_fullsized":"2024-09-12T12:19:49.993656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:17:17.262245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":60808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":664833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":73973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":73883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":819200,"data_stored":664889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:19:57.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:19:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667e4ef640 1 Processor -- start 2024-09-12T12:19:57.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667e4ef640 1 -- start start 2024-09-12T12:19:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667e4ef640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f667808c410 0x7f66780c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667e4ef640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66780ce9b0 con 0x7f66780d2920 2024-09-12T12:19:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667e4ef640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6678163020 con 0x7f6678161ed0 2024-09-12T12:19:57.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667e4ef640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f66780d3000 con 0x7f667808c410 2024-09-12T12:19:57.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f66777fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f667808c410 0x7f66780c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667ca8c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66780d2920 0x7f66780cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45318/0 (socket says 172.21.15.3:45318) 2024-09-12T12:19:57.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f667ca8c640 1 -- 172.21.15.3:0/3543305783 learned_addr learned my addr 172.21.15.3:0/3543305783 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:57.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f66777fe640 1 -- 172.21.15.3:0/3543305783 >> v1:172.21.15.142:6789/0 conn(0x7f6678161ed0 legacy=0x7f6678162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:57.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f66777fe640 1 -- 172.21.15.3:0/3543305783 >> v1:172.21.15.3:6789/0 conn(0x7f66780d2920 legacy=0x7f66780cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.504+0000 7f66777fe640 1 -- 172.21.15.3:0/3543305783 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66780d32e0 con 0x7f667808c410 2024-09-12T12:19:57.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f66777fe640 1 --2- 172.21.15.3:0/3543305783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780c80a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f66600097c0 tx=0x7f666001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36151674ec971798 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:57.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f6676ffd640 1 -- 172.21.15.3:0/3543305783 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600b2070 con 0x7f667808c410 2024-09-12T12:19:57.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f6676ffd640 1 -- 172.21.15.3:0/3543305783 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66600a9070 con 0x7f667808c410 2024-09-12T12:19:57.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f6676ffd640 1 -- 172.21.15.3:0/3543305783 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66600ad880 con 0x7f667808c410 2024-09-12T12:19:57.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 -- 172.21.15.3:0/3543305783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 msgr2=0x7f66780c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3543305783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780c80a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f66600097c0 tx=0x7f666001c380 comp rx=0 tx=0).stop 2024-09-12T12:19:57.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 -- 172.21.15.3:0/3543305783 shutdown_connections 2024-09-12T12:19:57.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3543305783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780c80a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 -- 172.21.15.3:0/3543305783 >> 172.21.15.3:0/3543305783 conn(0x7f667808c000 msgr2=0x7f66780d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:57.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 -- 172.21.15.3:0/3543305783 shutdown_connections 2024-09-12T12:19:57.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 -- 172.21.15.3:0/3543305783 wait complete. 2024-09-12T12:19:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 Processor -- start 2024-09-12T12:19:57.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.508+0000 7f667e4ef640 1 -- start start 2024-09-12T12:19:57.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66780d2920 0x7f6678154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6678161ed0 0x7f6678154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6678162a60 con 0x7f667808c410 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66780b6d80 con 0x7f6678161ed0 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6678162be0 con 0x7f66780d2920 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780b8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32774/0 (socket says 172.21.15.3:32774) 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 -- 172.21.15.3:0/3815688041 learned_addr learned my addr 172.21.15.3:0/3815688041 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667ca8c640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6678161ed0 0x7f6678154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f66777fe640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66780d2920 0x7f6678154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 -- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66780d2920 msgr2=0x7f6678154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66780d2920 0x7f6678154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 -- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6678161ed0 msgr2=0x7f6678154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6678161ed0 0x7f6678154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 -- 172.21.15.3:0/3815688041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66780cc630 con 0x7f667808c410 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6677fff640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780b8620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f666800b410 tx=0x7f666800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6674ff9640 1 -- 172.21.15.3:0/3815688041 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668015030 con 0x7f667808c410 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6674ff9640 1 -- 172.21.15.3:0/3815688041 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6668004020 con 0x7f667808c410 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66780cb870 con 0x7f667808c410 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6674ff9640 1 -- 172.21.15.3:0/3815688041 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668004450 con 0x7f667808c410 2024-09-12T12:19:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66780cbe30 con 0x7f667808c410 2024-09-12T12:19:57.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66780c80a0 con 0x7f667808c410 2024-09-12T12:19:57.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6674ff9640 1 -- 172.21.15.3:0/3815688041 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6668018070 con 0x7f667808c410 2024-09-12T12:19:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6674ff9640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66580678c0 0x7f6658069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f6674ff9640 1 -- 172.21.15.3:0/3815688041 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6668085800 con 0x7f667808c410 2024-09-12T12:19:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.512+0000 7f66777fe640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66580678c0 0x7f6658069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.516+0000 7f66777fe640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66580678c0 0x7f6658069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66600af000 tx=0x7f66600a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:57.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.520+0000 7f6674ff9640 1 -- 172.21.15.3:0/3815688041 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6668056340 con 0x7f667808c410 2024-09-12T12:19:57.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.684+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66780cb620 con 0x7f66580678c0 2024-09-12T12:19:57.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.704+0000 7f6674ff9640 1 -- 172.21.15.3:0/3815688041 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f66780cb620 con 0x7f66580678c0 2024-09-12T12:19:57.708 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:57.710 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66580678c0 msgr2=0x7f6658069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66580678c0 0x7f6658069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66600af000 tx=0x7f66600a7000 comp rx=0 tx=0).stop 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 msgr2=0x7f66780b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780b8620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f666800b410 tx=0x7f666800b8d0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 shutdown_connections 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66580678c0 0x7f6658069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6678161ed0 0x7f6678154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66780d2920 0x7f6678154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 --2- 172.21.15.3:0/3815688041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f667808c410 0x7f66780b8620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 >> 172.21.15.3:0/3815688041 conn(0x7f667808c000 msgr2=0x7f667806d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:57.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 shutdown_connections 2024-09-12T12:19:57.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.708+0000 7f667e4ef640 1 -- 172.21.15.3:0/3815688041 wait complete. 2024-09-12T12:19:57.728 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-12T12:19:57.658550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":2957312,"data_stored":1776620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.918135"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995369+0000","last_change":"2024-09-12T12:19:48.781313+0000","last_active":"2024-09-12T12:19:49.995369+0000","last_peered":"2024-09-12T12:19:49.995369+0000","last_clean":"2024-09-12T12:19:49.995369+0000","last_became_active":"2024-09-12T12:19:48.780836+0000","last_became_peered":"2024-09-12T12:19:48.780836+0000","last_unstale":"2024-09-12T12:19:49.995369+0000","last_undegraded":"2024-09-12T12:19:49.995369+0000","last_fullsized":"2024-09-12T12:19:49.995369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754520+0000","last_change":"2024-09-12T12:19:48.759206+0000","last_active":"2024-09-12T12:19:49.754520+0000","last_peered":"2024-09-12T12:19:49.754520+0000","last_clean":"2024-09-12T12:19:49.754520+0000","last_became_active":"2024-09-12T12:19:48.758450+0000","last_became_peered":"2024-09-12T12:19:48.758450+0000","last_unstale":"2024-09-12T12:19:49.754520+0000","last_undegraded":"2024-09-12T12:19:49.754520+0000","last_fullsized":"2024-09-12T12:19:49.754520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992848+0000","last_change":"2024-09-12T12:19:48.769260+0000","last_active":"2024-09-12T12:19:49.992848+0000","last_peered":"2024-09-12T12:19:49.992848+0000","last_clean":"2024-09-12T12:19:49.992848+0000","last_became_active":"2024-09-12T12:19:48.768869+0000","last_became_peered":"2024-09-12T12:19:48.768869+0000","last_unstale":"2024-09-12T12:19:49.992848+0000","last_undegraded":"2024-09-12T12:19:49.992848+0000","last_fullsized":"2024-09-12T12:19:49.992848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:41:14.637521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994736+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994736+0000","last_peered":"2024-09-12T12:19:49.994736+0000","last_clean":"2024-09-12T12:19:49.994736+0000","last_became_active":"2024-09-12T12:19:48.779065+0000","last_became_peered":"2024-09-12T12:19:48.779065+0000","last_unstale":"2024-09-12T12:19:49.994736+0000","last_undegraded":"2024-09-12T12:19:49.994736+0000","last_fullsized":"2024-09-12T12:19:49.994736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993392+0000","last_change":"2024-09-12T12:19:48.766342+0000","last_active":"2024-09-12T12:19:49.993392+0000","last_peered":"2024-09-12T12:19:49.993392+0000","last_clean":"2024-09-12T12:19:49.993392+0000","last_became_active":"2024-09-12T12:19:48.766063+0000","last_became_peered":"2024-09-12T12:19:48.766063+0000","last_unstale":"2024-09-12T12:19:49.993392+0000","last_undegraded":"2024-09-12T12:19:49.993392+0000","last_fullsized":"2024-09-12T12:19:49.993392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993183+0000","last_change":"2024-09-12T12:19:48.769354+0000","last_active":"2024-09-12T12:19:49.993183+0000","last_peered":"2024-09-12T12:19:49.993183+0000","last_clean":"2024-09-12T12:19:49.993183+0000","last_became_active":"2024-09-12T12:19:48.769084+0000","last_became_peered":"2024-09-12T12:19:48.769084+0000","last_unstale":"2024-09-12T12:19:49.993183+0000","last_undegraded":"2024-09-12T12:19:49.993183+0000","last_fullsized":"2024-09-12T12:19:49.993183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:56.236676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992999+0000","last_change":"2024-09-12T12:19:48.773715+0000","last_active":"2024-09-12T12:19:49.992999+0000","last_peered":"2024-09-12T12:19:49.992999+0000","last_clean":"2024-09-12T12:19:49.992999+0000","last_became_active":"2024-09-12T12:19:48.773419+0000","last_became_peered":"2024-09-12T12:19:48.773419+0000","last_unstale":"2024-09-12T12:19:49.992999+0000","last_undegraded":"2024-09-12T12:19:49.992999+0000","last_fullsized":"2024-09-12T12:19:49.992999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748510+0000","last_change":"2024-09-12T12:19:48.779584+0000","last_active":"2024-09-12T12:19:49.748510+0000","last_peered":"2024-09-12T12:19:49.748510+0000","last_clean":"2024-09-12T12:19:49.748510+0000","last_became_active":"2024-09-12T12:19:48.777119+0000","last_became_peered":"2024-09-12T12:19:48.777119+0000","last_unstale":"2024-09-12T12:19:49.748510+0000","last_undegraded":"2024-09-12T12:19:49.748510+0000","last_fullsized":"2024-09-12T12:19:49.748510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:44.728800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995405+0000","last_change":"2024-09-12T12:19:48.775263+0000","last_active":"2024-09-12T12:19:49.995405+0000","last_peered":"2024-09-12T12:19:49.995405+0000","last_clean":"2024-09-12T12:19:49.995405+0000","last_became_active":"2024-09-12T12:19:48.774506+0000","last_became_peered":"2024-09-12T12:19:48.774506+0000","last_unstale":"2024-09-12T12:19:49.995405+0000","last_undegraded":"2024-09-12T12:19:49.995405+0000","last_fullsized":"2024-09-12T12:19:49.995405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:38:46.549546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759178+0000","last_change":"2024-09-12T12:19:48.778949+0000","last_active":"2024-09-12T12:19:49.759178+0000","last_peered":"2024-09-12T12:19:49.759178+0000","last_clean":"2024-09-12T12:19:49.759178+0000","last_became_active":"2024-09-12T12:19:48.778672+0000","last_became_peered":"2024-09-12T12:19:48.778672+0000","last_unstale":"2024-09-12T12:19:49.759178+0000","last_undegraded":"2024-09-12T12:19:49.759178+0000","last_fullsized":"2024-09-12T12:19:49.759178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758789+0000","last_change":"2024-09-12T12:19:48.779161+0000","last_active":"2024-09-12T12:19:49.758789+0000","last_peered":"2024-09-12T12:19:49.758789+0000","last_clean":"2024-09-12T12:19:49.758789+0000","last_became_active":"2024-09-12T12:19:48.778765+0000","last_became_peered":"2024-09-12T12:19:48.778765+0000","last_unstale":"2024-09-12T12:19:49.758789+0000","last_undegraded":"2024-09-12T12:19:49.758789+0000","last_fullsized":"2024-09-12T12:19:49.758789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749404+0000","last_change":"2024-09-12T12:19:48.762583+0000","last_active":"2024-09-12T12:19:49.749404+0000","last_peered":"2024-09-12T12:19:49.749404+0000","last_clean":"2024-09-12T12:19:49.749404+0000","last_became_active":"2024-09-12T12:19:48.762315+0000","last_became_peered":"2024-09-12T12:19:48.762315+0000","last_unstale":"2024-09-12T12:19:49.749404+0000","last_undegraded":"2024-09-12T12:19:49.749404+0000","last_fullsized":"2024-09-12T12:19:49.749404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749236+0000","last_change":"2024-09-12T12:19:48.784303+0000","last_active":"2024-09-12T12:19:49.749236+0000","last_peered":"2024-09-12T12:19:49.749236+0000","last_clean":"2024-09-12T12:19:49.749236+0000","last_became_active":"2024-09-12T12:19:48.779768+0000","last_became_peered":"2024-09-12T12:19:48.779768+0000","last_unstale":"2024-09-12T12:19:49.749236+0000","last_undegraded":"2024-09-12T12:19:49.749236+0000","last_fullsized":"2024-09-12T12:19:49.749236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:40.107345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993137+0000","last_change":"2024-09-12T12:19:48.763027+0000","last_active":"2024-09-12T12:19:49.993137+0000","last_peered":"2024-09-12T12:19:49.993137+0000","last_clean":"2024-09-12T12:19:49.993137+0000","last_became_active":"2024-09-12T12:19:48.762544+0000","last_became_peered":"2024-09-12T12:19:48.762544+0000","last_unstale":"2024-09-12T12:19:49.993137+0000","last_undegraded":"2024-09-12T12:19:49.993137+0000","last_fullsized":"2024-09-12T12:19:49.993137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:44.728800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753861+0000","last_change":"2024-09-12T12:19:48.770278+0000","last_active":"2024-09-12T12:19:49.753861+0000","last_peered":"2024-09-12T12:19:49.753861+0000","last_clean":"2024-09-12T12:19:49.753861+0000","last_became_active":"2024-09-12T12:19:48.769983+0000","last_became_peered":"2024-09-12T12:19:48.769983+0000","last_unstale":"2024-09-12T12:19:49.753861+0000","last_undegraded":"2024-09-12T12:19:49.753861+0000","last_fullsized":"2024-09-12T12:19:49.753861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992463+0000","last_change":"2024-09-12T12:19:48.769537+0000","last_active":"2024-09-12T12:19:49.992463+0000","last_peered":"2024-09-12T12:19:49.992463+0000","last_clean":"2024-09-12T12:19:49.992463+0000","last_became_active":"2024-09-12T12:19:48.769243+0000","last_became_peered":"2024-09-12T12:19:48.769243+0000","last_unstale":"2024-09-12T12:19:49.992463+0000","last_undegraded":"2024-09-12T12:19:49.992463+0000","last_fullsized":"2024-09-12T12:19:49.992463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995255+0000","last_change":"2024-09-12T12:19:48.766440+0000","last_active":"2024-09-12T12:19:49.995255+0000","last_peered":"2024-09-12T12:19:49.995255+0000","last_clean":"2024-09-12T12:19:49.995255+0000","last_became_active":"2024-09-12T12:19:48.766110+0000","last_became_peered":"2024-09-12T12:19:48.766110+0000","last_unstale":"2024-09-12T12:19:49.995255+0000","last_undegraded":"2024-09-12T12:19:49.995255+0000","last_fullsized":"2024-09-12T12:19:49.995255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992782+0000","last_change":"2024-09-12T12:19:48.780118+0000","last_active":"2024-09-12T12:19:49.992782+0000","last_peered":"2024-09-12T12:19:49.992782+0000","last_clean":"2024-09-12T12:19:49.992782+0000","last_became_active":"2024-09-12T12:19:48.779839+0000","last_became_peered":"2024-09-12T12:19:48.779839+0000","last_unstale":"2024-09-12T12:19:49.992782+0000","last_undegraded":"2024-09-12T12:19:49.992782+0000","last_fullsized":"2024-09-12T12:19:49.992782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993776+0000","last_change":"2024-09-12T12:19:48.766221+0000","last_active":"2024-09-12T12:19:49.993776+0000","last_peered":"2024-09-12T12:19:49.993776+0000","last_clean":"2024-09-12T12:19:49.993776+0000","last_became_active":"2024-09-12T12:19:48.765549+0000","last_became_peered":"2024-09-12T12:19:48.765549+0000","last_unstale":"2024-09-12T12:19:49.993776+0000","last_undegraded":"2024-09-12T12:19:49.993776+0000","last_fullsized":"2024-09-12T12:19:49.993776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758800+0000","last_change":"2024-09-12T12:19:48.764597+0000","last_active":"2024-09-12T12:19:49.758800+0000","last_peered":"2024-09-12T12:19:49.758800+0000","last_clean":"2024-09-12T12:19:49.758800+0000","last_became_active":"2024-09-12T12:19:48.764377+0000","last_became_peered":"2024-09-12T12:19:48.764377+0000","last_unstale":"2024-09-12T12:19:49.758800+0000","last_undegraded":"2024-09-12T12:19:49.758800+0000","last_fullsized":"2024-09-12T12:19:49.758800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992940+0000","last_change":"2024-09-12T12:19:48.763523+0000","last_active":"2024-09-12T12:19:49.992940+0000","last_peered":"2024-09-12T12:19:49.992940+0000","last_clean":"2024-09-12T12:19:49.992940+0000","last_became_active":"2024-09-12T12:19:48.762889+0000","last_became_peered":"2024-09-12T12:19:48.762889+0000","last_unstale":"2024-09-12T12:19:49.992940+0000","last_undegraded":"2024-09-12T12:19:49.992940+0000","last_fullsized":"2024-09-12T12:19:49.992940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:04:18.599614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993579+0000","last_change":"2024-09-12T12:19:48.760812+0000","last_active":"2024-09-12T12:19:49.993579+0000","last_peered":"2024-09-12T12:19:49.993579+0000","last_clean":"2024-09-12T12:19:49.993579+0000","last_became_active":"2024-09-12T12:19:48.760551+0000","last_became_peered":"2024-09-12T12:19:48.760551+0000","last_unstale":"2024-09-12T12:19:49.993579+0000","last_undegraded":"2024-09-12T12:19:49.993579+0000","last_fullsized":"2024-09-12T12:19:49.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748762+0000","last_change":"2024-09-12T12:19:48.760474+0000","last_active":"2024-09-12T12:19:49.748762+0000","last_peered":"2024-09-12T12:19:49.748762+0000","last_clean":"2024-09-12T12:19:49.748762+0000","last_became_active":"2024-09-12T12:19:48.760032+0000","last_became_peered":"2024-09-12T12:19:48.760032+0000","last_unstale":"2024-09-12T12:19:49.748762+0000","last_undegraded":"2024-09-12T12:19:49.748762+0000","last_fullsized":"2024-09-12T12:19:49.748762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:40.506967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992589+0000","last_change":"2024-09-12T12:19:48.780825+0000","last_active":"2024-09-12T12:19:49.992589+0000","last_peered":"2024-09-12T12:19:49.992589+0000","last_clean":"2024-09-12T12:19:49.992589+0000","last_became_active":"2024-09-12T12:19:48.780593+0000","last_became_peered":"2024-09-12T12:19:48.780593+0000","last_unstale":"2024-09-12T12:19:49.992589+0000","last_undegraded":"2024-09-12T12:19:49.992589+0000","last_fullsized":"2024-09-12T12:19:49.992589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:50.202989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749885+0000","last_change":"2024-09-12T12:19:48.759772+0000","last_active":"2024-09-12T12:19:49.749885+0000","last_peered":"2024-09-12T12:19:49.749885+0000","last_clean":"2024-09-12T12:19:49.749885+0000","last_became_active":"2024-09-12T12:19:48.759512+0000","last_became_peered":"2024-09-12T12:19:48.759512+0000","last_unstale":"2024-09-12T12:19:49.749885+0000","last_undegraded":"2024-09-12T12:19:49.749885+0000","last_fullsized":"2024-09-12T12:19:49.749885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:02.195687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759372+0000","last_change":"2024-09-12T12:19:48.774953+0000","last_active":"2024-09-12T12:19:49.759372+0000","last_peered":"2024-09-12T12:19:49.759372+0000","last_clean":"2024-09-12T12:19:49.759372+0000","last_became_active":"2024-09-12T12:19:48.774474+0000","last_became_peered":"2024-09-12T12:19:48.774474+0000","last_unstale":"2024-09-12T12:19:49.759372+0000","last_undegraded":"2024-09-12T12:19:49.759372+0000","last_fullsized":"2024-09-12T12:19:49.759372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754066+0000","last_change":"2024-09-12T12:19:48.771914+0000","last_active":"2024-09-12T12:19:49.754066+0000","last_peered":"2024-09-12T12:19:49.754066+0000","last_clean":"2024-09-12T12:19:49.754066+0000","last_became_active":"2024-09-12T12:19:48.771637+0000","last_became_peered":"2024-09-12T12:19:48.771637+0000","last_unstale":"2024-09-12T12:19:49.754066+0000","last_undegraded":"2024-09-12T12:19:49.754066+0000","last_fullsized":"2024-09-12T12:19:49.754066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749153+0000","last_change":"2024-09-12T12:19:48.759783+0000","last_active":"2024-09-12T12:19:49.749153+0000","last_peered":"2024-09-12T12:19:49.749153+0000","last_clean":"2024-09-12T12:19:49.749153+0000","last_became_active":"2024-09-12T12:19:48.759473+0000","last_became_peered":"2024-09-12T12:19:48.759473+0000","last_unstale":"2024-09-12T12:19:49.749153+0000","last_undegraded":"2024-09-12T12:19:49.749153+0000","last_fullsized":"2024-09-12T12:19:49.749153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:11.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993036+0000","last_change":"2024-09-12T12:19:48.759011+0000","last_active":"2024-09-12T12:19:49.993036+0000","last_peered":"2024-09-12T12:19:49.993036+0000","last_clean":"2024-09-12T12:19:49.993036+0000","last_became_active":"2024-09-12T12:19:48.758267+0000","last_became_peered":"2024-09-12T12:19:48.758267+0000","last_unstale":"2024-09-12T12:19:49.993036+0000","last_undegraded":"2024-09-12T12:19:49.993036+0000","last_fullsized":"2024-09-12T12:19:49.993036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:45.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993953+0000","last_change":"2024-09-12T12:19:48.779776+0000","last_active":"2024-09-12T12:19:49.993953+0000","last_peered":"2024-09-12T12:19:49.993953+0000","last_clean":"2024-09-12T12:19:49.993953+0000","last_became_active":"2024-09-12T12:19:48.779500+0000","last_became_peered":"2024-09-12T12:19:48.779500+0000","last_unstale":"2024-09-12T12:19:49.993953+0000","last_undegraded":"2024-09-12T12:19:49.993953+0000","last_fullsized":"2024-09-12T12:19:49.993953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994016+0000","last_change":"2024-09-12T12:19:48.775150+0000","last_active":"2024-09-12T12:19:49.994016+0000","last_peered":"2024-09-12T12:19:49.994016+0000","last_clean":"2024-09-12T12:19:49.994016+0000","last_became_active":"2024-09-12T12:19:48.774841+0000","last_became_peered":"2024-09-12T12:19:48.774841+0000","last_unstale":"2024-09-12T12:19:49.994016+0000","last_undegraded":"2024-09-12T12:19:49.994016+0000","last_fullsized":"2024-09-12T12:19:49.994016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758353+0000","last_change":"2024-09-12T12:19:48.761408+0000","last_active":"2024-09-12T12:19:49.758353+0000","last_peered":"2024-09-12T12:19:49.758353+0000","last_clean":"2024-09-12T12:19:49.758353+0000","last_became_active":"2024-09-12T12:19:48.761091+0000","last_became_peered":"2024-09-12T12:19:48.761091+0000","last_unstale":"2024-09-12T12:19:49.758353+0000","last_undegraded":"2024-09-12T12:19:49.758353+0000","last_fullsized":"2024-09-12T12:19:49.758353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:51.034838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748611+0000","last_change":"2024-09-12T12:19:48.764078+0000","last_active":"2024-09-12T12:19:49.748611+0000","last_peered":"2024-09-12T12:19:49.748611+0000","last_clean":"2024-09-12T12:19:49.748611+0000","last_became_active":"2024-09-12T12:19:48.763831+0000","last_became_peered":"2024-09-12T12:19:48.763831+0000","last_unstale":"2024-09-12T12:19:49.748611+0000","last_undegraded":"2024-09-12T12:19:49.748611+0000","last_fullsized":"2024-09-12T12:19:49.748611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:06.350696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749493+0000","last_change":"2024-09-12T12:19:47.757834+0000","last_active":"2024-09-12T12:19:49.749493+0000","last_peered":"2024-09-12T12:19:49.749493+0000","last_clean":"2024-09-12T12:19:49.749493+0000","last_became_active":"2024-09-12T12:19:47.757145+0000","last_became_peered":"2024-09-12T12:19:47.757145+0000","last_unstale":"2024-09-12T12:19:49.749493+0000","last_undegraded":"2024-09-12T12:19:49.749493+0000","last_fullsized":"2024-09-12T12:19:49.749493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:13:55.749099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993352+0000","last_change":"2024-09-12T12:19:48.761558+0000","last_active":"2024-09-12T12:19:49.993352+0000","last_peered":"2024-09-12T12:19:49.993352+0000","last_clean":"2024-09-12T12:19:49.993352+0000","last_became_active":"2024-09-12T12:19:48.761230+0000","last_became_peered":"2024-09-12T12:19:48.761230+0000","last_unstale":"2024-09-12T12:19:49.993352+0000","last_undegraded":"2024-09-12T12:19:49.993352+0000","last_fullsized":"2024-09-12T12:19:49.993352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:40.506967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758441+0000","last_change":"2024-09-12T12:19:47.758747+0000","last_active":"2024-09-12T12:19:49.758441+0000","last_peered":"2024-09-12T12:19:49.758441+0000","last_clean":"2024-09-12T12:19:49.758441+0000","last_became_active":"2024-09-12T12:19:47.757870+0000","last_became_peered":"2024-09-12T12:19:47.757870+0000","last_unstale":"2024-09-12T12:19:49.758441+0000","last_undegraded":"2024-09-12T12:19:49.758441+0000","last_fullsized":"2024-09-12T12:19:49.758441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748588+0000","last_change":"2024-09-12T12:19:47.758883+0000","last_active":"2024-09-12T12:19:49.748588+0000","last_peered":"2024-09-12T12:19:49.748588+0000","last_clean":"2024-09-12T12:19:49.748588+0000","last_became_active":"2024-09-12T12:19:47.758396+0000","last_became_peered":"2024-09-12T12:19:47.758396+0000","last_unstale":"2024-09-12T12:19:49.748588+0000","last_undegraded":"2024-09-12T12:19:49.748588+0000","last_fullsized":"2024-09-12T12:19:49.748588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:49:34.294310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748361+0000","last_change":"2024-09-12T12:19:48.779910+0000","last_active":"2024-09-12T12:19:49.748361+0000","last_peered":"2024-09-12T12:19:49.748361+0000","last_clean":"2024-09-12T12:19:49.748361+0000","last_became_active":"2024-09-12T12:19:48.779613+0000","last_became_peered":"2024-09-12T12:19:48.779613+0000","last_unstale":"2024-09-12T12:19:49.748361+0000","last_undegraded":"2024-09-12T12:19:49.748361+0000","last_fullsized":"2024-09-12T12:19:49.748361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:04:18.599614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992602+0000","last_change":"2024-09-12T12:19:47.740584+0000","last_active":"2024-09-12T12:19:49.992602+0000","last_peered":"2024-09-12T12:19:49.992602+0000","last_clean":"2024-09-12T12:19:49.992602+0000","last_became_active":"2024-09-12T12:19:47.739985+0000","last_became_peered":"2024-09-12T12:19:47.739985+0000","last_unstale":"2024-09-12T12:19:49.992602+0000","last_undegraded":"2024-09-12T12:19:49.992602+0000","last_fullsized":"2024-09-12T12:19:49.992602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753847+0000","last_change":"2024-09-12T12:19:48.771435+0000","last_active":"2024-09-12T12:19:49.753847+0000","last_peered":"2024-09-12T12:19:49.753847+0000","last_clean":"2024-09-12T12:19:49.753847+0000","last_became_active":"2024-09-12T12:19:48.771108+0000","last_became_peered":"2024-09-12T12:19:48.771108+0000","last_unstale":"2024-09-12T12:19:49.753847+0000","last_undegraded":"2024-09-12T12:19:49.753847+0000","last_fullsized":"2024-09-12T12:19:49.753847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758579+0000","last_change":"2024-09-12T12:19:47.760876+0000","last_active":"2024-09-12T12:19:49.758579+0000","last_peered":"2024-09-12T12:19:49.758579+0000","last_clean":"2024-09-12T12:19:49.758579+0000","last_became_active":"2024-09-12T12:19:47.760036+0000","last_became_peered":"2024-09-12T12:19:47.760036+0000","last_unstale":"2024-09-12T12:19:49.758579+0000","last_undegraded":"2024-09-12T12:19:49.758579+0000","last_fullsized":"2024-09-12T12:19:49.758579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993540+0000","last_change":"2024-09-12T12:19:48.765500+0000","last_active":"2024-09-12T12:19:49.993540+0000","last_peered":"2024-09-12T12:19:49.993540+0000","last_clean":"2024-09-12T12:19:49.993540+0000","last_became_active":"2024-09-12T12:19:48.765191+0000","last_became_peered":"2024-09-12T12:19:48.765191+0000","last_unstale":"2024-09-12T12:19:49.993540+0000","last_undegraded":"2024-09-12T12:19:49.993540+0000","last_fullsized":"2024-09-12T12:19:49.993540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993676+0000","last_change":"2024-09-12T12:19:47.761042+0000","last_active":"2024-09-12T12:19:49.993676+0000","last_peered":"2024-09-12T12:19:49.993676+0000","last_clean":"2024-09-12T12:19:49.993676+0000","last_became_active":"2024-09-12T12:19:47.760261+0000","last_became_peered":"2024-09-12T12:19:47.760261+0000","last_unstale":"2024-09-12T12:19:49.993676+0000","last_undegraded":"2024-09-12T12:19:49.993676+0000","last_fullsized":"2024-09-12T12:19:49.993676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993444+0000","last_change":"2024-09-12T12:19:48.769338+0000","last_active":"2024-09-12T12:19:49.993444+0000","last_peered":"2024-09-12T12:19:49.993444+0000","last_clean":"2024-09-12T12:19:49.993444+0000","last_became_active":"2024-09-12T12:19:48.768955+0000","last_became_peered":"2024-09-12T12:19:48.768955+0000","last_unstale":"2024-09-12T12:19:49.993444+0000","last_undegraded":"2024-09-12T12:19:49.993444+0000","last_fullsized":"2024-09-12T12:19:49.993444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.656049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754127+0000","last_change":"2024-09-12T12:19:47.750999+0000","last_active":"2024-09-12T12:19:49.754127+0000","last_peered":"2024-09-12T12:19:49.754127+0000","last_clean":"2024-09-12T12:19:49.754127+0000","last_became_active":"2024-09-12T12:19:47.750658+0000","last_became_peered":"2024-09-12T12:19:47.750658+0000","last_unstale":"2024-09-12T12:19:49.754127+0000","last_undegraded":"2024-09-12T12:19:49.754127+0000","last_fullsized":"2024-09-12T12:19:49.754127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749115+0000","last_change":"2024-09-12T12:19:48.767953+0000","last_active":"2024-09-12T12:19:49.749115+0000","last_peered":"2024-09-12T12:19:49.749115+0000","last_clean":"2024-09-12T12:19:49.749115+0000","last_became_active":"2024-09-12T12:19:48.767595+0000","last_became_peered":"2024-09-12T12:19:48.767595+0000","last_unstale":"2024-09-12T12:19:49.749115+0000","last_undegraded":"2024-09-12T12:19:49.749115+0000","last_fullsized":"2024-09-12T12:19:49.749115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758590+0000","last_change":"2024-09-12T12:19:47.760548+0000","last_active":"2024-09-12T12:19:49.758590+0000","last_peered":"2024-09-12T12:19:49.758590+0000","last_clean":"2024-09-12T12:19:49.758590+0000","last_became_active":"2024-09-12T12:19:47.760033+0000","last_became_peered":"2024-09-12T12:19:47.760033+0000","last_unstale":"2024-09-12T12:19:49.758590+0000","last_undegraded":"2024-09-12T12:19:49.758590+0000","last_fullsized":"2024-09-12T12:19:49.758590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994749+0000","last_change":"2024-09-12T12:19:48.773829+0000","last_active":"2024-09-12T12:19:49.994749+0000","last_peered":"2024-09-12T12:19:49.994749+0000","last_clean":"2024-09-12T12:19:49.994749+0000","last_became_active":"2024-09-12T12:19:48.773359+0000","last_became_peered":"2024-09-12T12:19:48.773359+0000","last_unstale":"2024-09-12T12:19:49.994749+0000","last_undegraded":"2024-09-12T12:19:49.994749+0000","last_fullsized":"2024-09-12T12:19:49.994749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993173+0000","last_change":"2024-09-12T12:19:47.757426+0000","last_active":"2024-09-12T12:19:49.993173+0000","last_peered":"2024-09-12T12:19:49.993173+0000","last_clean":"2024-09-12T12:19:49.993173+0000","last_became_active":"2024-09-12T12:19:47.756782+0000","last_became_peered":"2024-09-12T12:19:47.756782+0000","last_unstale":"2024-09-12T12:19:49.993173+0000","last_undegraded":"2024-09-12T12:19:49.993173+0000","last_fullsized":"2024-09-12T12:19:49.993173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992472+0000","last_change":"2024-09-12T12:19:48.759150+0000","last_active":"2024-09-12T12:19:49.992472+0000","last_peered":"2024-09-12T12:19:49.992472+0000","last_clean":"2024-09-12T12:19:49.992472+0000","last_became_active":"2024-09-12T12:19:48.758654+0000","last_became_peered":"2024-09-12T12:19:48.758654+0000","last_unstale":"2024-09-12T12:19:49.992472+0000","last_undegraded":"2024-09-12T12:19:49.992472+0000","last_fullsized":"2024-09-12T12:19:49.992472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:37.527639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994079+0000","last_change":"2024-09-12T12:19:47.747437+0000","last_active":"2024-09-12T12:19:49.994079+0000","last_peered":"2024-09-12T12:19:49.994079+0000","last_clean":"2024-09-12T12:19:49.994079+0000","last_became_active":"2024-09-12T12:19:47.746926+0000","last_became_peered":"2024-09-12T12:19:47.746926+0000","last_unstale":"2024-09-12T12:19:49.994079+0000","last_undegraded":"2024-09-12T12:19:49.994079+0000","last_fullsized":"2024-09-12T12:19:49.994079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993790+0000","last_change":"2024-09-12T12:19:48.775580+0000","last_active":"2024-09-12T12:19:49.993790+0000","last_peered":"2024-09-12T12:19:49.993790+0000","last_clean":"2024-09-12T12:19:49.993790+0000","last_became_active":"2024-09-12T12:19:48.767021+0000","last_became_peered":"2024-09-12T12:19:48.767021+0000","last_unstale":"2024-09-12T12:19:49.993790+0000","last_undegraded":"2024-09-12T12:19:49.993790+0000","last_fullsized":"2024-09-12T12:19:49.993790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758162+0000","last_change":"2024-09-12T12:19:47.754802+0000","last_active":"2024-09-12T12:19:49.758162+0000","last_peered":"2024-09-12T12:19:49.758162+0000","last_clean":"2024-09-12T12:19:49.758162+0000","last_became_active":"2024-09-12T12:19:47.754242+0000","last_became_peered":"2024-09-12T12:19:47.754242+0000","last_unstale":"2024-09-12T12:19:49.758162+0000","last_undegraded":"2024-09-12T12:19:49.758162+0000","last_fullsized":"2024-09-12T12:19:49.758162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758450+0000","last_change":"2024-09-12T12:19:48.761340+0000","last_active":"2024-09-12T12:19:49.758450+0000","last_peered":"2024-09-12T12:19:49.758450+0000","last_clean":"2024-09-12T12:19:49.758450+0000","last_became_active":"2024-09-12T12:19:48.761037+0000","last_became_peered":"2024-09-12T12:19:48.761037+0000","last_unstale":"2024-09-12T12:19:49.758450+0000","last_undegraded":"2024-09-12T12:19:49.758450+0000","last_fullsized":"2024-09-12T12:19:49.758450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992730+0000","last_change":"2024-09-12T12:19:47.761479+0000","last_active":"2024-09-12T12:19:49.992730+0000","last_peered":"2024-09-12T12:19:49.992730+0000","last_clean":"2024-09-12T12:19:49.992730+0000","last_became_active":"2024-09-12T12:19:47.760812+0000","last_became_peered":"2024-09-12T12:19:47.760812+0000","last_unstale":"2024-09-12T12:19:49.992730+0000","last_undegraded":"2024-09-12T12:19:49.992730+0000","last_fullsized":"2024-09-12T12:19:49.992730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993407+0000","last_change":"2024-09-12T12:19:48.772189+0000","last_active":"2024-09-12T12:19:49.993407+0000","last_peered":"2024-09-12T12:19:49.993407+0000","last_clean":"2024-09-12T12:19:49.993407+0000","last_became_active":"2024-09-12T12:19:48.765342+0000","last_became_peered":"2024-09-12T12:19:48.765342+0000","last_unstale":"2024-09-12T12:19:49.993407+0000","last_undegraded":"2024-09-12T12:19:49.993407+0000","last_fullsized":"2024-09-12T12:19:49.993407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749324+0000","last_change":"2024-09-12T12:19:47.754107+0000","last_active":"2024-09-12T12:19:49.749324+0000","last_peered":"2024-09-12T12:19:49.749324+0000","last_clean":"2024-09-12T12:19:49.749324+0000","last_became_active":"2024-09-12T12:19:47.753433+0000","last_became_peered":"2024-09-12T12:19:47.753433+0000","last_unstale":"2024-09-12T12:19:49.749324+0000","last_undegraded":"2024-09-12T12:19:49.749324+0000","last_fullsized":"2024-09-12T12:19:49.749324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:28.270374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995095+0000","last_change":"2024-09-12T12:19:48.770240+0000","last_active":"2024-09-12T12:19:49.995095+0000","last_peered":"2024-09-12T12:19:49.995095+0000","last_clean":"2024-09-12T12:19:49.995095+0000","last_became_active":"2024-09-12T12:19:48.766786+0000","last_became_peered":"2024-09-12T12:19:48.766786+0000","last_unstale":"2024-09-12T12:19:49.995095+0000","last_undegraded":"2024-09-12T12:19:49.995095+0000","last_fullsized":"2024-09-12T12:19:49.995095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992870+0000","last_change":"2024-09-12T12:19:47.757967+0000","last_active":"2024-09-12T12:19:49.992870+0000","last_peered":"2024-09-12T12:19:49.992870+0000","last_clean":"2024-09-12T12:19:49.992870+0000","last_became_active":"2024-09-12T12:19:47.757370+0000","last_became_peered":"2024-09-12T12:19:47.757370+0000","last_unstale":"2024-09-12T12:19:49.992870+0000","last_undegraded":"2024-09-12T12:19:49.992870+0000","last_fullsized":"2024-09-12T12:19:49.992870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995730+0000","last_change":"2024-09-12T12:19:48.766283+0000","last_active":"2024-09-12T12:19:49.995730+0000","last_peered":"2024-09-12T12:19:49.995730+0000","last_clean":"2024-09-12T12:19:49.995730+0000","last_became_active":"2024-09-12T12:19:48.765694+0000","last_became_peered":"2024-09-12T12:19:48.765694+0000","last_unstale":"2024-09-12T12:19:49.995730+0000","last_undegraded":"2024-09-12T12:19:49.995730+0000","last_fullsized":"2024-09-12T12:19:49.995730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:42:42.831767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754237+0000","last_change":"2024-09-12T12:19:47.751839+0000","last_active":"2024-09-12T12:19:49.754237+0000","last_peered":"2024-09-12T12:19:49.754237+0000","last_clean":"2024-09-12T12:19:49.754237+0000","last_became_active":"2024-09-12T12:19:47.751482+0000","last_became_peered":"2024-09-12T12:19:47.751482+0000","last_unstale":"2024-09-12T12:19:49.754237+0000","last_undegraded":"2024-09-12T12:19:49.754237+0000","last_fullsized":"2024-09-12T12:19:49.754237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995497+0000","last_change":"2024-09-12T12:19:48.768636+0000","last_active":"2024-09-12T12:19:49.995497+0000","last_peered":"2024-09-12T12:19:49.995497+0000","last_clean":"2024-09-12T12:19:49.995497+0000","last_became_active":"2024-09-12T12:19:48.768342+0000","last_became_peered":"2024-09-12T12:19:48.768342+0000","last_unstale":"2024-09-12T12:19:49.995497+0000","last_undegraded":"2024-09-12T12:19:49.995497+0000","last_fullsized":"2024-09-12T12:19:49.995497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993248+0000","last_change":"2024-09-12T12:19:47.755829+0000","last_active":"2024-09-12T12:19:49.993248+0000","last_peered":"2024-09-12T12:19:49.993248+0000","last_clean":"2024-09-12T12:19:49.993248+0000","last_became_active":"2024-09-12T12:19:47.748443+0000","last_became_peered":"2024-09-12T12:19:47.748443+0000","last_unstale":"2024-09-12T12:19:49.993248+0000","last_undegraded":"2024-09-12T12:19:49.993248+0000","last_fullsized":"2024-09-12T12:19:49.993248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993070+0000","last_change":"2024-09-12T12:19:48.775968+0000","last_active":"2024-09-12T12:19:49.993070+0000","last_peered":"2024-09-12T12:19:49.993070+0000","last_clean":"2024-09-12T12:19:49.993070+0000","last_became_active":"2024-09-12T12:19:48.775731+0000","last_became_peered":"2024-09-12T12:19:48.775731+0000","last_unstale":"2024-09-12T12:19:49.993070+0000","last_undegraded":"2024-09-12T12:19:49.993070+0000","last_fullsized":"2024-09-12T12:19:49.993070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992710+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:19:49.992710+0000","last_peered":"2024-09-12T12:19:49.992710+0000","last_clean":"2024-09-12T12:19:49.992710+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:19:49.992710+0000","last_undegraded":"2024-09-12T12:19:49.992710+0000","last_fullsized":"2024-09-12T12:19:49.992710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993779+0000","last_change":"2024-09-12T12:19:47.757991+0000","last_active":"2024-09-12T12:19:49.993779+0000","last_peered":"2024-09-12T12:19:49.993779+0000","last_clean":"2024-09-12T12:19:49.993779+0000","last_became_active":"2024-09-12T12:19:47.757562+0000","last_became_peered":"2024-09-12T12:19:47.757562+0000","last_unstale":"2024-09-12T12:19:49.993779+0000","last_undegraded":"2024-09-12T12:19:49.993779+0000","last_fullsized":"2024-09-12T12:19:49.993779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749165+0000","last_change":"2024-09-12T12:19:48.779259+0000","last_active":"2024-09-12T12:19:49.749165+0000","last_peered":"2024-09-12T12:19:49.749165+0000","last_clean":"2024-09-12T12:19:49.749165+0000","last_became_active":"2024-09-12T12:19:48.778963+0000","last_became_peered":"2024-09-12T12:19:48.778963+0000","last_unstale":"2024-09-12T12:19:49.749165+0000","last_undegraded":"2024-09-12T12:19:49.749165+0000","last_fullsized":"2024-09-12T12:19:49.749165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.656049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992441+0000","last_change":"2024-09-12T12:19:47.761015+0000","last_active":"2024-09-12T12:19:49.992441+0000","last_peered":"2024-09-12T12:19:49.992441+0000","last_clean":"2024-09-12T12:19:49.992441+0000","last_became_active":"2024-09-12T12:19:47.760494+0000","last_became_peered":"2024-09-12T12:19:47.760494+0000","last_unstale":"2024-09-12T12:19:49.992441+0000","last_undegraded":"2024-09-12T12:19:49.992441+0000","last_fullsized":"2024-09-12T12:19:49.992441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992375+0000","last_change":"2024-09-12T12:19:48.765122+0000","last_active":"2024-09-12T12:19:49.992375+0000","last_peered":"2024-09-12T12:19:49.992375+0000","last_clean":"2024-09-12T12:19:49.992375+0000","last_became_active":"2024-09-12T12:19:48.764670+0000","last_became_peered":"2024-09-12T12:19:48.764670+0000","last_unstale":"2024-09-12T12:19:49.992375+0000","last_undegraded":"2024-09-12T12:19:49.992375+0000","last_fullsized":"2024-09-12T12:19:49.992375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:57:01.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993998+0000","last_change":"2024-09-12T12:19:47.758292+0000","last_active":"2024-09-12T12:19:49.993998+0000","last_peered":"2024-09-12T12:19:49.993998+0000","last_clean":"2024-09-12T12:19:49.993998+0000","last_became_active":"2024-09-12T12:19:47.757643+0000","last_became_peered":"2024-09-12T12:19:47.757643+0000","last_unstale":"2024-09-12T12:19:49.993998+0000","last_undegraded":"2024-09-12T12:19:49.993998+0000","last_fullsized":"2024-09-12T12:19:49.993998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753888+0000","last_change":"2024-09-12T12:19:48.763042+0000","last_active":"2024-09-12T12:19:49.753888+0000","last_peered":"2024-09-12T12:19:49.753888+0000","last_clean":"2024-09-12T12:19:49.753888+0000","last_became_active":"2024-09-12T12:19:48.762214+0000","last_became_peered":"2024-09-12T12:19:48.762214+0000","last_unstale":"2024-09-12T12:19:49.753888+0000","last_undegraded":"2024-09-12T12:19:49.753888+0000","last_fullsized":"2024-09-12T12:19:49.753888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994326+0000","last_change":"2024-09-12T12:19:47.752251+0000","last_active":"2024-09-12T12:19:49.994326+0000","last_peered":"2024-09-12T12:19:49.994326+0000","last_clean":"2024-09-12T12:19:49.994326+0000","last_became_active":"2024-09-12T12:19:47.751530+0000","last_became_peered":"2024-09-12T12:19:47.751530+0000","last_unstale":"2024-09-12T12:19:49.994326+0000","last_undegraded":"2024-09-12T12:19:49.994326+0000","last_fullsized":"2024-09-12T12:19:49.994326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749074+0000","last_change":"2024-09-12T12:19:48.762555+0000","last_active":"2024-09-12T12:19:49.749074+0000","last_peered":"2024-09-12T12:19:49.749074+0000","last_clean":"2024-09-12T12:19:49.749074+0000","last_became_active":"2024-09-12T12:19:48.760885+0000","last_became_peered":"2024-09-12T12:19:48.760885+0000","last_unstale":"2024-09-12T12:19:49.749074+0000","last_undegraded":"2024-09-12T12:19:49.749074+0000","last_fullsized":"2024-09-12T12:19:49.749074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:16.559958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753838+0000","last_change":"2024-09-12T12:19:47.758162+0000","last_active":"2024-09-12T12:19:49.753838+0000","last_peered":"2024-09-12T12:19:49.753838+0000","last_clean":"2024-09-12T12:19:49.753838+0000","last_became_active":"2024-09-12T12:19:47.757816+0000","last_became_peered":"2024-09-12T12:19:47.757816+0000","last_unstale":"2024-09-12T12:19:49.753838+0000","last_undegraded":"2024-09-12T12:19:49.753838+0000","last_fullsized":"2024-09-12T12:19:49.753838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994340+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994340+0000","last_peered":"2024-09-12T12:19:49.994340+0000","last_clean":"2024-09-12T12:19:49.994340+0000","last_became_active":"2024-09-12T12:19:48.779234+0000","last_became_peered":"2024-09-12T12:19:48.779234+0000","last_unstale":"2024-09-12T12:19:49.994340+0000","last_undegraded":"2024-09-12T12:19:49.994340+0000","last_fullsized":"2024-09-12T12:19:49.994340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993920+0000","last_change":"2024-09-12T12:19:47.765924+0000","last_active":"2024-09-12T12:19:49.993920+0000","last_peered":"2024-09-12T12:19:49.993920+0000","last_clean":"2024-09-12T12:19:49.993920+0000","last_became_active":"2024-09-12T12:19:47.765244+0000","last_became_peered":"2024-09-12T12:19:47.765244+0000","last_unstale":"2024-09-12T12:19:49.993920+0000","last_undegraded":"2024-09-12T12:19:49.993920+0000","last_fullsized":"2024-09-12T12:19:49.993920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754093+0000","last_change":"2024-09-12T12:19:48.762307+0000","last_active":"2024-09-12T12:19:49.754093+0000","last_peered":"2024-09-12T12:19:49.754093+0000","last_clean":"2024-09-12T12:19:49.754093+0000","last_became_active":"2024-09-12T12:19:48.761728+0000","last_became_peered":"2024-09-12T12:19:48.761728+0000","last_unstale":"2024-09-12T12:19:49.754093+0000","last_undegraded":"2024-09-12T12:19:49.754093+0000","last_fullsized":"2024-09-12T12:19:49.754093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754345+0000","last_change":"2024-09-12T12:19:47.760726+0000","last_active":"2024-09-12T12:19:49.754345+0000","last_peered":"2024-09-12T12:19:49.754345+0000","last_clean":"2024-09-12T12:19:49.754345+0000","last_became_active":"2024-09-12T12:19:47.760059+0000","last_became_peered":"2024-09-12T12:19:47.760059+0000","last_unstale":"2024-09-12T12:19:49.754345+0000","last_undegraded":"2024-09-12T12:19:49.754345+0000","last_fullsized":"2024-09-12T12:19:49.754345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.750068+0000","last_change":"2024-09-12T12:19:48.768057+0000","last_active":"2024-09-12T12:19:49.750068+0000","last_peered":"2024-09-12T12:19:49.750068+0000","last_clean":"2024-09-12T12:19:49.750068+0000","last_became_active":"2024-09-12T12:19:48.766995+0000","last_became_peered":"2024-09-12T12:19:48.766995+0000","last_unstale":"2024-09-12T12:19:49.750068+0000","last_undegraded":"2024-09-12T12:19:49.750068+0000","last_fullsized":"2024-09-12T12:19:49.750068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:05.941564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758743+0000","last_change":"2024-09-12T12:19:47.765527+0000","last_active":"2024-09-12T12:19:49.758743+0000","last_peered":"2024-09-12T12:19:49.758743+0000","last_clean":"2024-09-12T12:19:49.758743+0000","last_became_active":"2024-09-12T12:19:47.764743+0000","last_became_peered":"2024-09-12T12:19:47.764743+0000","last_unstale":"2024-09-12T12:19:49.758743+0000","last_undegraded":"2024-09-12T12:19:49.758743+0000","last_fullsized":"2024-09-12T12:19:49.758743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992737+0000","last_change":"2024-09-12T12:19:48.761030+0000","last_active":"2024-09-12T12:19:49.992737+0000","last_peered":"2024-09-12T12:19:49.992737+0000","last_clean":"2024-09-12T12:19:49.992737+0000","last_became_active":"2024-09-12T12:19:48.760740+0000","last_became_peered":"2024-09-12T12:19:48.760740+0000","last_unstale":"2024-09-12T12:19:49.992737+0000","last_undegraded":"2024-09-12T12:19:49.992737+0000","last_fullsized":"2024-09-12T12:19:49.992737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:12.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992155+0000","last_change":"2024-09-12T12:19:47.764445+0000","last_active":"2024-09-12T12:19:49.992155+0000","last_peered":"2024-09-12T12:19:49.992155+0000","last_clean":"2024-09-12T12:19:49.992155+0000","last_became_active":"2024-09-12T12:19:47.763777+0000","last_became_peered":"2024-09-12T12:19:47.763777+0000","last_unstale":"2024-09-12T12:19:49.992155+0000","last_undegraded":"2024-09-12T12:19:49.992155+0000","last_fullsized":"2024-09-12T12:19:49.992155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748601+0000","last_change":"2024-09-12T12:19:48.755118+0000","last_active":"2024-09-12T12:19:49.748601+0000","last_peered":"2024-09-12T12:19:49.748601+0000","last_clean":"2024-09-12T12:19:49.748601+0000","last_became_active":"2024-09-12T12:19:48.754819+0000","last_became_peered":"2024-09-12T12:19:48.754819+0000","last_unstale":"2024-09-12T12:19:49.748601+0000","last_undegraded":"2024-09-12T12:19:49.748601+0000","last_fullsized":"2024-09-12T12:19:49.748601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:33.261675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753992+0000","last_change":"2024-09-12T12:19:47.755131+0000","last_active":"2024-09-12T12:19:49.753992+0000","last_peered":"2024-09-12T12:19:49.753992+0000","last_clean":"2024-09-12T12:19:49.753992+0000","last_became_active":"2024-09-12T12:19:47.754580+0000","last_became_peered":"2024-09-12T12:19:47.754580+0000","last_unstale":"2024-09-12T12:19:49.753992+0000","last_undegraded":"2024-09-12T12:19:49.753992+0000","last_fullsized":"2024-09-12T12:19:49.753992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748534+0000","last_change":"2024-09-12T12:19:48.778824+0000","last_active":"2024-09-12T12:19:49.748534+0000","last_peered":"2024-09-12T12:19:49.748534+0000","last_clean":"2024-09-12T12:19:49.748534+0000","last_became_active":"2024-09-12T12:19:48.766383+0000","last_became_peered":"2024-09-12T12:19:48.766383+0000","last_unstale":"2024-09-12T12:19:49.748534+0000","last_undegraded":"2024-09-12T12:19:49.748534+0000","last_fullsized":"2024-09-12T12:19:49.748534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:45.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753795+0000","last_change":"2024-09-12T12:19:47.751410+0000","last_active":"2024-09-12T12:19:49.753795+0000","last_peered":"2024-09-12T12:19:49.753795+0000","last_clean":"2024-09-12T12:19:49.753795+0000","last_became_active":"2024-09-12T12:19:47.750752+0000","last_became_peered":"2024-09-12T12:19:47.750752+0000","last_unstale":"2024-09-12T12:19:49.753795+0000","last_undegraded":"2024-09-12T12:19:49.753795+0000","last_fullsized":"2024-09-12T12:19:49.753795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749406+0000","last_change":"2024-09-12T12:19:48.779436+0000","last_active":"2024-09-12T12:19:49.749406+0000","last_peered":"2024-09-12T12:19:49.749406+0000","last_clean":"2024-09-12T12:19:49.749406+0000","last_became_active":"2024-09-12T12:19:48.779114+0000","last_became_peered":"2024-09-12T12:19:48.779114+0000","last_unstale":"2024-09-12T12:19:49.749406+0000","last_undegraded":"2024-09-12T12:19:49.749406+0000","last_fullsized":"2024-09-12T12:19:49.749406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:21.169633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993986+0000","last_change":"2024-09-12T12:19:47.768458+0000","last_active":"2024-09-12T12:19:49.993986+0000","last_peered":"2024-09-12T12:19:49.993986+0000","last_clean":"2024-09-12T12:19:49.993986+0000","last_became_active":"2024-09-12T12:19:47.767666+0000","last_became_peered":"2024-09-12T12:19:47.767666+0000","last_unstale":"2024-09-12T12:19:49.993986+0000","last_undegraded":"2024-09-12T12:19:49.993986+0000","last_fullsized":"2024-09-12T12:19:49.993986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758358+0000","last_change":"2024-09-12T12:19:48.760279+0000","last_active":"2024-09-12T12:19:49.758358+0000","last_peered":"2024-09-12T12:19:49.758358+0000","last_clean":"2024-09-12T12:19:49.758358+0000","last_became_active":"2024-09-12T12:19:48.759972+0000","last_became_peered":"2024-09-12T12:19:48.759972+0000","last_unstale":"2024-09-12T12:19:49.758358+0000","last_undegraded":"2024-09-12T12:19:49.758358+0000","last_fullsized":"2024-09-12T12:19:49.758358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993934+0000","last_change":"2024-09-12T12:19:48.765639+0000","last_active":"2024-09-12T12:19:49.993934+0000","last_peered":"2024-09-12T12:19:49.993934+0000","last_clean":"2024-09-12T12:19:49.993934+0000","last_became_active":"2024-09-12T12:19:48.765358+0000","last_became_peered":"2024-09-12T12:19:48.765358+0000","last_unstale":"2024-09-12T12:19:49.993934+0000","last_undegraded":"2024-09-12T12:19:49.993934+0000","last_fullsized":"2024-09-12T12:19:49.993934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994200+0000","last_change":"2024-09-12T12:19:47.763682+0000","last_active":"2024-09-12T12:19:49.994200+0000","last_peered":"2024-09-12T12:19:49.994200+0000","last_clean":"2024-09-12T12:19:49.994200+0000","last_became_active":"2024-09-12T12:19:47.762792+0000","last_became_peered":"2024-09-12T12:19:47.762792+0000","last_unstale":"2024-09-12T12:19:49.994200+0000","last_undegraded":"2024-09-12T12:19:49.994200+0000","last_fullsized":"2024-09-12T12:19:49.994200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995111+0000","last_change":"2024-09-12T12:19:48.765714+0000","last_active":"2024-09-12T12:19:49.995111+0000","last_peered":"2024-09-12T12:19:49.995111+0000","last_clean":"2024-09-12T12:19:49.995111+0000","last_became_active":"2024-09-12T12:19:48.765396+0000","last_became_peered":"2024-09-12T12:19:48.765396+0000","last_unstale":"2024-09-12T12:19:49.995111+0000","last_undegraded":"2024-09-12T12:19:49.995111+0000","last_fullsized":"2024-09-12T12:19:49.995111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993218+0000","last_change":"2024-09-12T12:19:47.770906+0000","last_active":"2024-09-12T12:19:49.993218+0000","last_peered":"2024-09-12T12:19:49.993218+0000","last_clean":"2024-09-12T12:19:49.993218+0000","last_became_active":"2024-09-12T12:19:47.770451+0000","last_became_peered":"2024-09-12T12:19:47.770451+0000","last_unstale":"2024-09-12T12:19:49.993218+0000","last_undegraded":"2024-09-12T12:19:49.993218+0000","last_fullsized":"2024-09-12T12:19:49.993218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993423+0000","last_change":"2024-09-12T12:19:48.774830+0000","last_active":"2024-09-12T12:19:49.993423+0000","last_peered":"2024-09-12T12:19:49.993423+0000","last_clean":"2024-09-12T12:19:49.993423+0000","last_became_active":"2024-09-12T12:19:48.774146+0000","last_became_peered":"2024-09-12T12:19:48.774146+0000","last_unstale":"2024-09-12T12:19:49.993423+0000","last_undegraded":"2024-09-12T12:19:49.993423+0000","last_fullsized":"2024-09-12T12:19:49.993423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:11.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749805+0000","last_change":"2024-09-12T12:19:47.759563+0000","last_active":"2024-09-12T12:19:49.749805+0000","last_peered":"2024-09-12T12:19:49.749805+0000","last_clean":"2024-09-12T12:19:49.749805+0000","last_became_active":"2024-09-12T12:19:47.759012+0000","last_became_peered":"2024-09-12T12:19:47.759012+0000","last_unstale":"2024-09-12T12:19:49.749805+0000","last_undegraded":"2024-09-12T12:19:49.749805+0000","last_fullsized":"2024-09-12T12:19:49.749805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:17:17.262245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749697+0000","last_change":"2024-09-12T12:19:48.784674+0000","last_active":"2024-09-12T12:19:49.749697+0000","last_peered":"2024-09-12T12:19:49.749697+0000","last_clean":"2024-09-12T12:19:49.749697+0000","last_became_active":"2024-09-12T12:19:48.784362+0000","last_became_peered":"2024-09-12T12:19:48.784362+0000","last_unstale":"2024-09-12T12:19:49.749697+0000","last_undegraded":"2024-09-12T12:19:49.749697+0000","last_fullsized":"2024-09-12T12:19:49.749697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:08:49.515897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993656+0000","last_change":"2024-09-12T12:19:47.755575+0000","last_active":"2024-09-12T12:19:49.993656+0000","last_peered":"2024-09-12T12:19:49.993656+0000","last_clean":"2024-09-12T12:19:49.993656+0000","last_became_active":"2024-09-12T12:19:47.755279+0000","last_became_peered":"2024-09-12T12:19:47.755279+0000","last_unstale":"2024-09-12T12:19:49.993656+0000","last_undegraded":"2024-09-12T12:19:49.993656+0000","last_fullsized":"2024-09-12T12:19:49.993656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:17:17.262245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":664833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":73973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":73883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":819200,"data_stored":664889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:19:57.732 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:19:57.732 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T12:19:57.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-12T12:19:57.733 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-12T12:19:57.733 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-12T12:19:57.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e9c74640 1 Processor -- start 2024-09-12T12:19:57.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e9c74640 1 -- start start 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e9c74640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7e415b2d0 0x7fa7e40b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e9c74640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7e4073640 0x7fa7e40aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e9c74640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e415f150 con 0x7fa7e406f6e0 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e9c74640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e40ae920 con 0x7fa7e415b2d0 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e9c74640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e415b6b0 con 0x7fa7e4073640 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e37fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7e415b2d0 0x7fa7e40b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e37fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa7e415b2d0 0x7fa7e40b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32788/0 (socket says 172.21.15.3:32788) 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e37fe640 1 -- 172.21.15.3:0/778238117 learned_addr learned my addr 172.21.15.3:0/778238117 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:57.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e3fff640 1 --2- 172.21.15.3:0/778238117 >> v2:172.21.15.142:3300/0 conn(0x7fa7e4073640 0x7fa7e40aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e37fe640 1 -- 172.21.15.3:0/778238117 >> v2:172.21.15.142:3300/0 conn(0x7fa7e4073640 msgr2=0x7fa7e40aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:57.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e37fe640 1 --2- 172.21.15.3:0/778238117 >> v2:172.21.15.142:3300/0 conn(0x7fa7e4073640 0x7fa7e40aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e37fe640 1 -- 172.21.15.3:0/778238117 >> v1:172.21.15.142:6789/0 conn(0x7fa7e406f6e0 legacy=0x7fa7e4072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:57.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.900+0000 7fa7e37fe640 1 -- 172.21.15.3:0/778238117 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e415ee70 con 0x7fa7e415b2d0 2024-09-12T12:19:57.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 --2- 172.21.15.3:0/778238117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e415b2d0 0x7fa7e40b3080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40cc970 tx=0x7fa7d401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64172a32494b775d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:57.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e27fc640 1 -- 172.21.15.3:0/778238117 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40aa610 con 0x7fa7e415b2d0 2024-09-12T12:19:57.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e27fc640 1 -- 172.21.15.3:0/778238117 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40af070 con 0x7fa7e415b2d0 2024-09-12T12:19:57.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e27fc640 1 -- 172.21.15.3:0/778238117 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40aa9d0 con 0x7fa7e415b2d0 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- 172.21.15.3:0/778238117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e415b2d0 msgr2=0x7fa7e40b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/778238117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e415b2d0 0x7fa7e40b3080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40cc970 tx=0x7fa7d401c4b0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- 172.21.15.3:0/778238117 shutdown_connections 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/778238117 >> v2:172.21.15.142:3300/0 conn(0x7fa7e4073640 0x7fa7e40aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/778238117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e415b2d0 0x7fa7e40b3080 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- 172.21.15.3:0/778238117 >> 172.21.15.3:0/778238117 conn(0x7fa7e40b1d80 msgr2=0x7fa7e40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- 172.21.15.3:0/778238117 shutdown_connections 2024-09-12T12:19:57.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- 172.21.15.3:0/778238117 wait complete. 2024-09-12T12:19:57.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 Processor -- start 2024-09-12T12:19:57.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- start start 2024-09-12T12:19:57.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e406f6e0 0x7fa7e414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7e4073640 0x7fa7e40ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7e415b2d0 0x7fa7e414e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40af700 con 0x7fa7e406f6e0 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e406d8d0 con 0x7fa7e4073640 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e40af880 con 0x7fa7e415b2d0 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e406f6e0 0x7fa7e414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e406f6e0 0x7fa7e414b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32790/0 (socket says 172.21.15.3:32790) 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 -- 172.21.15.3:0/3402017288 learned_addr learned my addr 172.21.15.3:0/3402017288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e2ffd640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7e4073640 0x7fa7e40ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e3fff640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7e415b2d0 0x7fa7e414e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 -- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7e415b2d0 msgr2=0x7fa7e414e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7e415b2d0 0x7fa7e414e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 -- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7e4073640 msgr2=0x7fa7e40ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7e4073640 0x7fa7e40ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 -- 172.21.15.3:0/3402017288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e40cf570 con 0x7fa7e406f6e0 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e2ffd640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7e4073640 0x7fa7e40ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e37fe640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e406f6e0 0x7fa7e414b160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa7d401a0e0 tx=0x7fa7d401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7cbfff640 1 -- 172.21.15.3:0/3402017288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40aae60 con 0x7fa7e406f6e0 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7cbfff640 1 -- 172.21.15.3:0/3402017288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40b3800 con 0x7fa7e406f6e0 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e40cf850 con 0x7fa7e406f6e0 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7cbfff640 1 -- 172.21.15.3:0/3402017288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40b3bf0 con 0x7fa7e406f6e0 2024-09-12T12:19:57.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.904+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e40cfe10 con 0x7fa7e406f6e0 2024-09-12T12:19:57.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.908+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e40d0050 con 0x7fa7e406f6e0 2024-09-12T12:19:57.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.908+0000 7fa7cbfff640 1 -- 172.21.15.3:0/3402017288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa7d40c7420 con 0x7fa7e406f6e0 2024-09-12T12:19:57.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.908+0000 7fa7cbfff640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7d00678c0 0x7fa7d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:57.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.908+0000 7fa7cbfff640 1 -- 172.21.15.3:0/3402017288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa7d41205f0 con 0x7fa7e406f6e0 2024-09-12T12:19:57.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.908+0000 7fa7e2ffd640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7d00678c0 0x7fa7d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:57.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.908+0000 7fa7e2ffd640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7d00678c0 0x7fa7d0069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa7e406fb00 tx=0x7fa7d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:57.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:57.912+0000 7fa7cbfff640 1 -- 172.21.15.3:0/3402017288 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa7e40d0050 con 0x7fa7e406f6e0 2024-09-12T12:19:58.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.116+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa7e40aedb0 con 0x7fa7e406f6e0 2024-09-12T12:19:58.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.124+0000 7fa7cbfff640 1 -- 172.21.15.3:0/3402017288 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166893 (secure 0 0 0) 0x7fa7d40c3060 con 0x7fa7e406f6e0 2024-09-12T12:19:58.127 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7d00678c0 msgr2=0x7fa7d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7d00678c0 0x7fa7d0069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa7e406fb00 tx=0x7fa7d8008040 comp rx=0 tx=0).stop 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e406f6e0 msgr2=0x7fa7e414b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e406f6e0 0x7fa7e414b160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa7d401a0e0 tx=0x7fa7d401acf0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 shutdown_connections 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7d00678c0 0x7fa7d0069d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7e415b2d0 0x7fa7e414e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7e4073640 0x7fa7e40ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 --2- 172.21.15.3:0/3402017288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7e406f6e0 0x7fa7e414b160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 >> 172.21.15.3:0/3402017288 conn(0x7fa7e40b1d80 msgr2=0x7fa7e406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 shutdown_connections 2024-09-12T12:19:58.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.128+0000 7fa7e9c74640 1 -- 172.21.15.3:0/3402017288 wait complete. 2024-09-12T12:19:58.144 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":4257875619},{"type":"v1","addr":"172.21.15.142:6833","nonce":4257875619}]},"active_addr":"172.21.15.142:6833/4257875619","active_change":"2024-09-12T12:19:29.641719+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":1628563468}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":2942015123}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":565848737}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":4236703881}]}]} 2024-09-12T12:19:58.148 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-12T12:19:58.148 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-12T12:19:58.149 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a911c640 1 Processor -- start 2024-09-12T12:19:58.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a911c640 1 -- start start 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a911c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f20a409aa70 0x7f20a415e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a911c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a40b5940 con 0x7f20a415b5c0 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a911c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a415d6a0 con 0x7f20a415e6c0 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a911c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f20a40b5c20 con 0x7f20a409aa70 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a2d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20a415b5c0 0x7f20a40cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36034/0 (socket says 172.21.15.3:36034) 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a2d9d640 1 -- 172.21.15.3:0/1132599766 learned_addr learned my addr 172.21.15.3:0/1132599766 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a259c640 1 --2- 172.21.15.3:0/1132599766 >> v2:172.21.15.142:3300/0 conn(0x7f20a409aa70 0x7f20a415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132697033 0 0) 0x7f20a40b5940 con 0x7f20a415b5c0 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f208c003530 con 0x7f20a415b5c0 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2530194552 0 0) 0x7f208c003530 con 0x7f20a415b5c0 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 >> v2:172.21.15.142:3300/0 conn(0x7f20a409aa70 msgr2=0x7f20a415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 --2- 172.21.15.3:0/1132599766 >> v2:172.21.15.142:3300/0 conn(0x7f20a409aa70 0x7f20a415e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 >> v1:172.21.15.142:6789/0 conn(0x7f20a415e6c0 legacy=0x7f20a40b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a415d980 con 0x7f20a415b5c0 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f2094001680 con 0x7f20a415b5c0 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 >> v1:172.21.15.3:6789/0 conn(0x7f20a415b5c0 legacy=0x7f20a40cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f208c0039c0 0x7f208c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f208c004810 0x7f208c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f208c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20a40b5c20 con 0x7f208c0051d0 2024-09-12T12:19:58.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f208c00e380 con 0x7f208c0039c0 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f208c00e580 con 0x7f208c004810 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a359e640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f208c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a2d9d640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f208c0039c0 0x7f208c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a259c640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f208c004810 0x7f208c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a359e640 1 -- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f208c004810 msgr2=0x7f208c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a359e640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f208c004810 0x7f208c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a359e640 1 -- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f208c0039c0 msgr2=0x7f208c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a359e640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f208c0039c0 0x7f208c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a359e640 1 -- 172.21.15.3:0/1132599766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c003530 con 0x7f208c0051d0 2024-09-12T12:19:58.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.264+0000 7f20a2d9d640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f208c0039c0 0x7f208c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a359e640 1 -- 172.21.15.3:0/1132599766 reap_dead start 2024-09-12T12:19:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a359e640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f208c00dbe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f209c001780 tx=0x7f209c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec61b9125628f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c0013a0 con 0x7f208c0051d0 2024-09-12T12:19:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209c0a9070 con 0x7f208c0051d0 2024-09-12T12:19:58.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a1d9b640 1 -- 172.21.15.3:0/1132599766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c003600 con 0x7f208c0051d0 2024-09-12T12:19:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a911c640 1 -- 172.21.15.3:0/1132599766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 msgr2=0x7f208c00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a911c640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f208c00dbe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f209c001780 tx=0x7f209c01c420 comp rx=0 tx=0).stop 2024-09-12T12:19:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a911c640 1 -- 172.21.15.3:0/1132599766 shutdown_connections 2024-09-12T12:19:58.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a911c640 1 --2- 172.21.15.3:0/1132599766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f208c00dbe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a911c640 1 -- 172.21.15.3:0/1132599766 >> 172.21.15.3:0/1132599766 conn(0x7f20a40734b0 msgr2=0x7f20a40b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a911c640 1 -- 172.21.15.3:0/1132599766 shutdown_connections 2024-09-12T12:19:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.268+0000 7f20a911c640 1 -- 172.21.15.3:0/1132599766 wait complete. 2024-09-12T12:19:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 Processor -- start 2024-09-12T12:19:58.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 -- start start 2024-09-12T12:19:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f20a4149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20a41efe80 0x7f20a4148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20a413cb90 0x7f20a4152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20a40b46c0 con 0x7f208c0051d0 2024-09-12T12:19:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20a409aeb0 con 0x7f20a41efe80 2024-09-12T12:19:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20a415eb50 con 0x7f20a413cb90 2024-09-12T12:19:58.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f20a4149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f20a4149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40782/0 (socket says 172.21.15.3:40782) 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 -- 172.21.15.3:0/491528699 learned_addr learned my addr 172.21.15.3:0/491528699 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a359e640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20a413cb90 0x7f20a4152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 -- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20a413cb90 msgr2=0x7f20a4152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a259c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20a41efe80 0x7f20a4148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20a413cb90 0x7f20a4152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 -- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20a41efe80 msgr2=0x7f20a4148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20a41efe80 0x7f20a4148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 -- 172.21.15.3:0/491528699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41f0300 con 0x7f208c0051d0 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a2d9d640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f20a4149e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f20940116a0 tx=0x7f2094011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f208b7fe640 1 -- 172.21.15.3:0/491528699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094010040 con 0x7f208c0051d0 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f208b7fe640 1 -- 172.21.15.3:0/491528699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209401a070 con 0x7f208c0051d0 2024-09-12T12:19:58.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f208b7fe640 1 -- 172.21.15.3:0/491528699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209400fb50 con 0x7f208c0051d0 2024-09-12T12:19:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a4148890 con 0x7f208c0051d0 2024-09-12T12:19:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a414fcf0 con 0x7f208c0051d0 2024-09-12T12:19:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.272+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a415e180 con 0x7f208c0051d0 2024-09-12T12:19:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.276+0000 7f208b7fe640 1 -- 172.21.15.3:0/491528699 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f209400fcf0 con 0x7f208c0051d0 2024-09-12T12:19:58.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.276+0000 7f208b7fe640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20900678c0 0x7f2090069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.276+0000 7f208b7fe640 1 -- 172.21.15.3:0/491528699 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2094088d20 con 0x7f208c0051d0 2024-09-12T12:19:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.276+0000 7f20a259c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20900678c0 0x7f2090069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.276+0000 7f20a259c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20900678c0 0x7f2090069d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f20a415c5f0 tx=0x7f209800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.280+0000 7f208b7fe640 1 -- 172.21.15.3:0/491528699 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f20a4148890 con 0x7f208c0051d0 2024-09-12T12:19:58.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.443+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20a415c1b0 con 0x7f208c0051d0 2024-09-12T12:19:58.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f208b7fe640 1 -- 172.21.15.3:0/491528699 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f20a41f0300 con 0x7f208c0051d0 2024-09-12T12:19:58.449 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:58.450 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:49.738546+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:58.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20900678c0 msgr2=0x7f2090069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20900678c0 0x7f2090069d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f20a415c5f0 tx=0x7f209800b040 comp rx=0 tx=0).stop 2024-09-12T12:19:58.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 msgr2=0x7f20a4149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f20a4149e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f20940116a0 tx=0x7f2094011b60 comp rx=0 tx=0).stop 2024-09-12T12:19:58.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 shutdown_connections 2024-09-12T12:19:58.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20900678c0 0x7f2090069d80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20a413cb90 0x7f20a4152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20a41efe80 0x7f20a4148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 --2- 172.21.15.3:0/491528699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f208c0051d0 0x7f20a4149e20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 >> 172.21.15.3:0/491528699 conn(0x7f20a40734b0 msgr2=0x7f20a4159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 shutdown_connections 2024-09-12T12:19:58.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.447+0000 7f20a911c640 1 -- 172.21.15.3:0/491528699 wait complete. 2024-09-12T12:19:58.469 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-12T12:19:58.469 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:19:58.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.583+0000 7fad9e881640 1 Processor -- start 2024-09-12T12:19:58.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.583+0000 7fad9e881640 1 -- start start 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad9e881640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad9815b2d0 0x7fad980b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad9e881640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad9806f6e0 0x7fad98072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad9e881640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad9e881640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fad980ae920 con 0x7fad9806f6e0 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad9e881640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fad9815f150 con 0x7fad9815b2d0 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad9e881640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fad9815b6b0 con 0x7fad980cc300 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad9806f6e0 0x7fad98072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fad9806f6e0 0x7fad98072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40798/0 (socket says 172.21.15.3:40798) 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 -- 172.21.15.3:0/1138437945 learned_addr learned my addr 172.21.15.3:0/1138437945 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad97fff640 1 --2- 172.21.15.3:0/1138437945 >> v2:172.21.15.142:3300/0 conn(0x7fad9815b2d0 0x7fad980b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad9ce1e640 1 --2- 172.21.15.3:0/1138437945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 -- 172.21.15.3:0/1138437945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 msgr2=0x7fad98073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 --2- 172.21.15.3:0/1138437945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 -- 172.21.15.3:0/1138437945 >> v2:172.21.15.142:3300/0 conn(0x7fad9815b2d0 msgr2=0x7fad980b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 --2- 172.21.15.3:0/1138437945 >> v2:172.21.15.142:3300/0 conn(0x7fad9815b2d0 0x7fad980b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 -- 172.21.15.3:0/1138437945 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9815ee70 con 0x7fad9806f6e0 2024-09-12T12:19:58.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad977fe640 1 --2- 172.21.15.3:0/1138437945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9806f6e0 0x7fad98072f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad9806fe30 tx=0x7fad8c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3fb01941896648 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad96ffd640 1 -- 172.21.15.3:0/1138437945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a5840 con 0x7fad9806f6e0 2024-09-12T12:19:58.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad96ffd640 1 -- 172.21.15.3:0/1138437945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c0a59e0 con 0x7fad9806f6e0 2024-09-12T12:19:58.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.587+0000 7fad96ffd640 1 -- 172.21.15.3:0/1138437945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a5db0 con 0x7fad9806f6e0 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- 172.21.15.3:0/1138437945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9806f6e0 msgr2=0x7fad98072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 --2- 172.21.15.3:0/1138437945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9806f6e0 0x7fad98072f30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad9806fe30 tx=0x7fad8c01c9c0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- 172.21.15.3:0/1138437945 shutdown_connections 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 --2- 172.21.15.3:0/1138437945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 --2- 172.21.15.3:0/1138437945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9806f6e0 0x7fad98072f30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 --2- 172.21.15.3:0/1138437945 >> v2:172.21.15.142:3300/0 conn(0x7fad9815b2d0 0x7fad980b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- 172.21.15.3:0/1138437945 >> 172.21.15.3:0/1138437945 conn(0x7fad980b1d80 msgr2=0x7fad980b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- 172.21.15.3:0/1138437945 shutdown_connections 2024-09-12T12:19:58.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- 172.21.15.3:0/1138437945 wait complete. 2024-09-12T12:19:58.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 Processor -- start 2024-09-12T12:19:58.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- start start 2024-09-12T12:19:58.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad9806f6e0 0x7fad98152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9815b2d0 0x7fad98164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad980b35c0 con 0x7fad9815b2d0 2024-09-12T12:19:58.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fad9806d8b0 con 0x7fad9806f6e0 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fad980b3740 con 0x7fad980cc300 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9815b2d0 0x7fad98164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9815b2d0 0x7fad98164b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40810/0 (socket says 172.21.15.3:40810) 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 -- 172.21.15.3:0/3119756679 learned_addr learned my addr 172.21.15.3:0/3119756679 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad977fe640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad97fff640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad9806f6e0 0x7fad98152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 -- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 msgr2=0x7fad98162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 -- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad9806f6e0 msgr2=0x7fad98152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad9806f6e0 0x7fad98152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 -- 172.21.15.3:0/3119756679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad98166120 con 0x7fad9815b2d0 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9ce1e640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9815b2d0 0x7fad98164b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d5d0 tx=0x7fad9000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad94ff9640 1 -- 172.21.15.3:0/3119756679 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90018020 con 0x7fad9815b2d0 2024-09-12T12:19:58.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad94ff9640 1 -- 172.21.15.3:0/3119756679 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad90014620 con 0x7fad9815b2d0 2024-09-12T12:19:58.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad94ff9640 1 -- 172.21.15.3:0/3119756679 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad900149f0 con 0x7fad9815b2d0 2024-09-12T12:19:58.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad98166400 con 0x7fad9815b2d0 2024-09-12T12:19:58.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.591+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad981669c0 con 0x7fad9815b2d0 2024-09-12T12:19:58.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.595+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad98153aa0 con 0x7fad9815b2d0 2024-09-12T12:19:58.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.595+0000 7fad94ff9640 1 -- 172.21.15.3:0/3119756679 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fad9001b020 con 0x7fad9815b2d0 2024-09-12T12:19:58.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.595+0000 7fad94ff9640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fad840678c0 0x7fad84069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.595+0000 7fad94ff9640 1 -- 172.21.15.3:0/3119756679 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fad900860a0 con 0x7fad9815b2d0 2024-09-12T12:19:58.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.595+0000 7fad97fff640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fad840678c0 0x7fad84069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.595+0000 7fad97fff640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fad840678c0 0x7fad84069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad88009790 tx=0x7fad88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.599+0000 7fad94ff9640 1 -- 172.21.15.3:0/3119756679 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fad90022020 con 0x7fad9815b2d0 2024-09-12T12:19:58.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.759+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad9815e740 con 0x7fad9815b2d0 2024-09-12T12:19:58.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.759+0000 7fad94ff9640 1 -- 172.21.15.3:0/3119756679 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fad90056c70 con 0x7fad9815b2d0 2024-09-12T12:19:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:19:58.763 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:49.738546+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:19:58.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fad840678c0 msgr2=0x7fad84069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fad840678c0 0x7fad84069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad88009790 tx=0x7fad88008040 comp rx=0 tx=0).stop 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9815b2d0 msgr2=0x7fad98164b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9815b2d0 0x7fad98164b00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fad9000d5d0 tx=0x7fad9000da90 comp rx=0 tx=0).stop 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 shutdown_connections 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fad840678c0 0x7fad84069d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad9815b2d0 0x7fad98164b00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad980cc300 0x7fad98162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 --2- 172.21.15.3:0/3119756679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad9806f6e0 0x7fad98152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 >> 172.21.15.3:0/3119756679 conn(0x7fad980b1d80 msgr2=0x7fad9806e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 shutdown_connections 2024-09-12T12:19:58.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.763+0000 7fad9e881640 1 -- 172.21.15.3:0/3119756679 wait complete. 2024-09-12T12:19:58.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-12T12:19:58.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-12T12:19:58.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-12T12:19:58.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-12T12:19:58.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-12T12:19:58.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-12T12:19:58.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-12T12:19:58.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-12T12:19:58.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.959+0000 7fe262863640 1 Processor -- start 2024-09-12T12:19:58.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.959+0000 7fe262863640 1 -- start start 2024-09-12T12:19:58.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f07704e6640 1 Processor -- start 2024-09-12T12:19:58.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe262863640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe25c15a9e0 0x7fe25c072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe262863640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe25c0b35c0 con 0x7fe25c0b5190 2024-09-12T12:19:58.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe262863640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe25c0b38a0 con 0x7fe25c158040 2024-09-12T12:19:58.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe25c0b5190 0x7fe25c071c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36044/0 (socket says 172.21.15.3:36044) 2024-09-12T12:19:58.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe262863640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe25c0b3b80 con 0x7fe25c15a9e0 2024-09-12T12:19:58.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25b7fe640 1 -- 172.21.15.3:0/2160635514 learned_addr learned my addr 172.21.15.3:0/2160635514 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:58.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f07704e6640 1 -- start start 2024-09-12T12:19:58.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f3477640 1 Processor -- start 2024-09-12T12:19:58.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f3477640 1 -- start start 2024-09-12T12:19:58.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331304809 0 0) 0x7fe25c0b35c0 con 0x7fe25c0b5190 2024-09-12T12:19:58.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe244003530 con 0x7fe25c0b5190 2024-09-12T12:19:58.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe260e00640 1 --2- 172.21.15.3:0/2160635514 >> v2:172.21.15.142:3300/0 conn(0x7fe25c15a9e0 0x7fe25c072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009660413 0 0) 0x7fe25c0b38a0 con 0x7fe25c158040 2024-09-12T12:19:58.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe25c0b35c0 con 0x7fe25c158040 2024-09-12T12:19:58.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1920485395 0 0) 0x7fe244003530 con 0x7fe25c0b5190 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 >> v2:172.21.15.142:3300/0 conn(0x7fe25c15a9e0 msgr2=0x7fe25c072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 --2- 172.21.15.3:0/2160635514 >> v2:172.21.15.142:3300/0 conn(0x7fe25c15a9e0 0x7fe25c072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 >> v1:172.21.15.142:6789/0 conn(0x7fe25c158040 legacy=0x7fe25c0b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c0b3e60 con 0x7fe25c0b5190 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fe254001a00 con 0x7fe25c0b5190 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 >> v1:172.21.15.3:6789/0 conn(0x7fe25c0b5190 legacy=0x7fe25c071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2440037b0 0x7fe244003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe244004120 0x7fe24400b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24400b5d0 0x7fe24400da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c0b3b80 con 0x7fe2440037b0 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe24400e370 con 0x7fe24400b5d0 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe24400e570 con 0x7fe244004120 2024-09-12T12:19:58.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25bfff640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2440037b0 0x7fe244003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f07704e6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07600eb3c0 0x7f076004eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f07704e6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0760047580 0x7f076004f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f07704e6640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07600eb910 con 0x7f0760033980 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f07704e6640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f076001c5d0 con 0x7f0760047580 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f07704e6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f076001c8b0 con 0x7f07600eb3c0 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076e282640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0760033980 0x7f0760009b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36052/0 (socket says 172.21.15.3:36052) 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076e282640 1 -- 172.21.15.3:0/2738660839 learned_addr learned my addr 172.21.15.3:0/2738660839 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:58.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076ea83640 1 --2- 172.21.15.3:0/2738660839 >> v2:172.21.15.3:3300/0 conn(0x7f0760047580 0x7f076004f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076da81640 1 --2- 172.21.15.3:0/2738660839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07600eb3c0 0x7f076004eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076ea83640 1 -- 172.21.15.3:0/2738660839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07600eb3c0 msgr2=0x7f076004eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076ea83640 1 --2- 172.21.15.3:0/2738660839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07600eb3c0 0x7f076004eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076ea83640 1 -- 172.21.15.3:0/2738660839 >> v1:172.21.15.3:6789/0 conn(0x7f0760033980 legacy=0x7f0760009b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f3477640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0adf40 0x7f67ec0af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f3477640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67ec0af5a0 0x7f67ec0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25b7fe640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe244004120 0x7fe24400b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f3477640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67ec0ae4e0 con 0x7f67ec147cd0 2024-09-12T12:19:58.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f3477640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec0afa30 con 0x7f67ec0af5a0 2024-09-12T12:19:58.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f3477640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec0ae7c0 con 0x7f67ec0adf40 2024-09-12T12:19:58.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f2c76640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67ec0af5a0 0x7f67ec0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f076ea83640 1 -- 172.21.15.3:0/2738660839 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076001cb90 con 0x7f0760047580 2024-09-12T12:19:58.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f2c76640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67ec0af5a0 0x7f67ec0b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56472/0 (socket says 172.21.15.3:56472) 2024-09-12T12:19:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f2c76640 1 -- 172.21.15.3:0/1001294802 learned_addr learned my addr 172.21.15.3:0/1001294802 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f1c74640 1 --2- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0adf40 0x7f67ec0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f2c76640 1 -- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0adf40 msgr2=0x7f67ec0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f2c76640 1 --2- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0adf40 0x7f67ec0af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f2c76640 1 -- 172.21.15.3:0/1001294802 >> v1:172.21.15.3:6789/0 conn(0x7f67ec147cd0 legacy=0x7f67ec072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7f67f2c76640 1 -- 172.21.15.3:0/1001294802 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec0aeaa0 con 0x7f67ec0af5a0 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe260e00640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24400b5d0 0x7fe24400da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25bfff640 1 -- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe244004120 msgr2=0x7fe24400b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25bfff640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe244004120 0x7fe24400b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25bfff640 1 -- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24400b5d0 msgr2=0x7fe24400da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25bfff640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24400b5d0 0x7fe24400da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25bfff640 1 -- 172.21.15.3:0/2160635514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c0b3e60 con 0x7fe2440037b0 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe25b7fe640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe244004120 0x7fe24400b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:58.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.963+0000 7fe260e00640 1 -- 172.21.15.3:0/2160635514 reap_dead start 2024-09-12T12:19:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce2697640 1 Processor -- start 2024-09-12T12:19:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b1926a640 1 Processor -- start 2024-09-12T12:19:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce2697640 1 -- start start 2024-09-12T12:19:58.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b1926a640 1 -- start start 2024-09-12T12:19:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb759e640 1 Processor -- start 2024-09-12T12:19:58.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb759e640 1 -- start start 2024-09-12T12:19:58.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f2fe4165640 1 Processor -- start 2024-09-12T12:19:58.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f2fe4165640 1 -- start start 2024-09-12T12:19:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7fe25bfff640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2440037b0 0x7fe244003b90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe24c007f50 tx=0x7fe24c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed141545ceb4bd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b1926a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b140b3c70 0x7f5b1406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b1926a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5b140b66d0 0x7f5b140cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b1926a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b140d30c0 con 0x7f5b140b62f0 2024-09-12T12:19:58.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b1926a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5b140ced70 con 0x7f5b140b3c70 2024-09-12T12:19:58.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b1926a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5b140d5510 con 0x7f5b140b66d0 2024-09-12T12:19:58.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce2697640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8cd4056d40 0x7f8cd40571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f076ea83640 1 --2- 172.21.15.3:0/2738660839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760047580 0x7f076004f0b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07680512d0 tx=0x7f07680982b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba845c1ba58ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b0a7fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5b140b3c70 0x7f5b1406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b12ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5b140b62f0 0x7f5b1406cde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36072/0 (socket says 172.21.15.3:36072) 2024-09-12T12:19:58.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b12ffd640 1 -- 172.21.15.3:0/396048051 learned_addr learned my addr 172.21.15.3:0/396048051 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:58.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b137fe640 1 --2- 172.21.15.3:0/396048051 >> v2:172.21.15.142:3300/0 conn(0x7f5b140b66d0 0x7f5b140cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f67f2c76640 1 --2- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec0b3760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009e50 tx=0x7f67e401c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1712b322bd130a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce2697640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd403c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce2697640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd403bb60 con 0x7f8cd40e5f90 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 0x7f4eb80b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb759e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4eb80d0650 0x7f4eb80cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb759e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb8073280 con 0x7f4eb8072e40 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb759e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb80b3a00 con 0x7f4eb80d0650 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb759e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb80b3ce0 con 0x7f4eb8162590 2024-09-12T12:19:58.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb5d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4eb8072e40 0x7f4eb80cff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36080/0 (socket says 172.21.15.3:36080) 2024-09-12T12:19:58.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce1695640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8cd40e5f90 0x7f8cd4053a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:37864/0 (socket says 172.21.15.3:37864) 2024-09-12T12:19:58.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb5d9b640 1 -- 172.21.15.3:0/2695782751 learned_addr learned my addr 172.21.15.3:0/2695782751 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:58.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce0e94640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8cd4056d40 0x7f8cd40571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c007b20 con 0x7fe2440037b0 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24c0a9070 con 0x7fe2440037b0 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7fe25affd640 1 -- 172.21.15.3:0/2160635514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c003600 con 0x7fe2440037b0 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7fe262863640 1 -- 172.21.15.3:0/2160635514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2440037b0 msgr2=0x7fe244003b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7fe262863640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2440037b0 0x7fe244003b90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe24c007f50 tx=0x7fe24c01c9c0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f076d280640 1 -- 172.21.15.3:0/2738660839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768127640 con 0x7f0760047580 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f076d280640 1 -- 172.21.15.3:0/2738660839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076806de50 con 0x7f0760047580 2024-09-12T12:19:58.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f076d280640 1 -- 172.21.15.3:0/2738660839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768127a80 con 0x7f0760047580 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b0a7fc640 1 -- 172.21.15.3:0/396048051 >> v2:172.21.15.142:3300/0 conn(0x7f5b140b66d0 msgr2=0x7f5b140cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b0a7fc640 1 --2- 172.21.15.3:0/396048051 >> v2:172.21.15.142:3300/0 conn(0x7f5b140b66d0 0x7f5b140cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b0a7fc640 1 -- 172.21.15.3:0/396048051 >> v1:172.21.15.3:6789/0 conn(0x7f5b140b62f0 legacy=0x7f5b1406cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f5b0a7fc640 1 -- 172.21.15.3:0/396048051 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b140d4ec0 con 0x7f5b140b3c70 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f2fe4165640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc09b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe1f01640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc09b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe4165640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2fdc09bb70 0x7f2fdc0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe1f01640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc09b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42062/0 (socket says 172.21.15.3:42062) 2024-09-12T12:19:58.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe1f01640 1 -- 172.21.15.3:0/859301992 learned_addr learned my addr 172.21.15.3:0/859301992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:58.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f67f1473640 1 -- 172.21.15.3:0/1001294802 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40a5860 con 0x7f67ec0af5a0 2024-09-12T12:19:58.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f67f1473640 1 -- 172.21.15.3:0/1001294802 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e40a5a00 con 0x7f67ec0af5a0 2024-09-12T12:19:58.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f67f1473640 1 -- 172.21.15.3:0/1001294802 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e40a5dd0 con 0x7f67ec0af5a0 2024-09-12T12:19:58.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce0e94640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8cd4056d40 0x7f8cd40571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40860/0 (socket says 172.21.15.3:40860) 2024-09-12T12:19:58.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce1e96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd403c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce1e96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd403c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42054/0 (socket says 172.21.15.3:42054) 2024-09-12T12:19:58.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce1e96640 1 -- 172.21.15.3:0/2411033696 learned_addr learned my addr 172.21.15.3:0/2411033696 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f8ce2697640 1 -- 172.21.15.3:0/2411033696 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd4057ce0 con 0x7f8cd4056d40 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f8ce2697640 1 -- 172.21.15.3:0/2411033696 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd4057fc0 con 0x7f8cd40576f0 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f8ce0e94640 1 -- 172.21.15.3:0/2411033696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 msgr2=0x7f8cd403c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f8ce0e94640 1 --2- 172.21.15.3:0/2411033696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd403c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb6d9d640 1 --2- 172.21.15.3:0/2695782751 >> v2:172.21.15.3:3300/0 conn(0x7f4eb80d0650 0x7f4eb80cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.967+0000 7f4eb659c640 1 --2- 172.21.15.3:0/2695782751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 0x7f4eb80b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fdc0b32c0 con 0x7f2fdc0d0bf0 2024-09-12T12:19:58.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2fdc0ceac0 con 0x7f2fdc09bb70 2024-09-12T12:19:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc0d3020 con 0x7f2fdc0d0810 2024-09-12T12:19:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f8ce0e94640 1 -- 172.21.15.3:0/2411033696 >> v1:172.21.15.142:6789/0 conn(0x7f8cd40e5f90 legacy=0x7f8cd4053a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f4eb6d9d640 1 -- 172.21.15.3:0/2695782751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 msgr2=0x7f4eb80b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f4eb6d9d640 1 --2- 172.21.15.3:0/2695782751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 0x7f4eb80b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f4eb6d9d640 1 -- 172.21.15.3:0/2695782751 >> v1:172.21.15.3:6789/0 conn(0x7f4eb8072e40 legacy=0x7f4eb80cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f4eb6d9d640 1 -- 172.21.15.3:0/2695782751 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb80ccb50 con 0x7f4eb80d0650 2024-09-12T12:19:58.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f8ce0e94640 1 -- 172.21.15.3:0/2411033696 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd403be00 con 0x7f8cd4056d40 2024-09-12T12:19:58.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cf59e640 1 Processor -- start 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 -- 172.21.15.3:0/2738660839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760047580 msgr2=0x7f076004f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 --2- 172.21.15.3:0/2738660839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760047580 0x7f076004f0b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07680512d0 tx=0x7f07680982b0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 -- 172.21.15.3:0/2738660839 shutdown_connections 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 --2- 172.21.15.3:0/2738660839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760047580 0x7f076004f0b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 --2- 172.21.15.3:0/2738660839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07600eb3c0 0x7f076004eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 -- 172.21.15.3:0/2738660839 >> 172.21.15.3:0/2738660839 conn(0x7f07600360a0 msgr2=0x7f0760036480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 -- 172.21.15.3:0/2738660839 shutdown_connections 2024-09-12T12:19:58.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7fe262863640 1 -- 172.21.15.3:0/2160635514 shutdown_connections 2024-09-12T12:19:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7fe262863640 1 --2- 172.21.15.3:0/2160635514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe2440037b0 0x7fe244003b90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7fe262863640 1 -- 172.21.15.3:0/2160635514 >> 172.21.15.3:0/2160635514 conn(0x7fe25c06dcb0 msgr2=0x7fe25c06e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 -- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 msgr2=0x7f67ec0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 --2- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec0b3760 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67e4009e50 tx=0x7f67e401c9e0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7fe262863640 1 -- 172.21.15.3:0/2160635514 shutdown_connections 2024-09-12T12:19:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f5b0a7fc640 1 --2- 172.21.15.3:0/396048051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b3c70 0x7f5b1406d4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b1406e050 tx=0x7f5afc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14bb10f7b7ee008 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 -- 172.21.15.3:0/1001294802 shutdown_connections 2024-09-12T12:19:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 --2- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec0b3760 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 --2- 172.21.15.3:0/1001294802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0adf40 0x7f67ec0af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 -- 172.21.15.3:0/1001294802 >> 172.21.15.3:0/1001294802 conn(0x7f67ec0b1d40 msgr2=0x7f67ec0b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:58.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 -- 172.21.15.3:0/1001294802 shutdown_connections 2024-09-12T12:19:58.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cf59e640 1 -- start start 2024-09-12T12:19:58.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cf59e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f21d0073140 0x7f21d00d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:58.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cf59e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21d013e5b0 con 0x7f21d0140070 2024-09-12T12:19:58.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cf59e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21d01407c0 con 0x7f21d0072cd0 2024-09-12T12:19:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cf59e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f21d015f9a0 con 0x7f21d0073140 2024-09-12T12:19:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cdd9b640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f21d0072cd0 0x7f21d00cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:37866/0 (socket says 172.21.15.3:37866) 2024-09-12T12:19:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 -- 172.21.15.3:0/2738660839 wait complete. 2024-09-12T12:19:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cdd9b640 1 -- 172.21.15.3:0/3319171269 learned_addr learned my addr 172.21.15.3:0/3319171269 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:58.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f07704e6640 1 Processor -- start 2024-09-12T12:19:58.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158026845 0 0) 0x7f21d01407c0 con 0x7f21d0072cd0 2024-09-12T12:19:58.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f67f3477640 1 -- 172.21.15.3:0/1001294802 wait complete. 2024-09-12T12:19:58.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21b8005740 con 0x7f21d0072cd0 2024-09-12T12:19:58.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 390857896 0 0) 0x7f21b8005740 con 0x7f21d0072cd0 2024-09-12T12:19:58.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b127fc640 1 -- 172.21.15.3:0/396048051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0b5070 con 0x7f5b140b3c70 2024-09-12T12:19:58.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce0e94640 1 --2- 172.21.15.3:0/2411033696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd4056d40 0x7f8cd40571b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80097f0 tx=0x7f8cd801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34955f4fa2f7cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:58.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b127fc640 1 -- 172.21.15.3:0/396048051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc0a8950 con 0x7f5b140b3c70 2024-09-12T12:19:58.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b127fc640 1 -- 172.21.15.3:0/396048051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0a8d20 con 0x7f5b140b3c70 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe1f01640 1 -- 172.21.15.3:0/859301992 >> v2:172.21.15.3:3300/0 conn(0x7f2fdc09bb70 msgr2=0x7f2fdc0b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe1f01640 1 --2- 172.21.15.3:0/859301992 >> v2:172.21.15.3:3300/0 conn(0x7f2fdc09bb70 0x7f2fdc0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe1f01640 1 -- 172.21.15.3:0/859301992 >> v1:172.21.15.3:6789/0 conn(0x7f2fdc0d0bf0 legacy=0x7f2fdc1605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f2fe1f01640 1 -- 172.21.15.3:0/859301992 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc0ce750 con 0x7f2fdc0d0810 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 >> v2:172.21.15.3:3300/0 conn(0x7f21d0073140 msgr2=0x7f21d00d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 --2- 172.21.15.3:0/3319171269 >> v2:172.21.15.3:3300/0 conn(0x7f21d0073140 0x7f21d00d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 >> v1:172.21.15.3:6789/0 conn(0x7f21d0140070 legacy=0x7f21d015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:58.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d0140aa0 con 0x7f21d0072cd0 2024-09-12T12:19:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f21c4001680 con 0x7f21d0072cd0 2024-09-12T12:19:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.971+0000 7fe262863640 1 -- 172.21.15.3:0/2160635514 wait complete. 2024-09-12T12:19:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 >> v1:172.21.15.142:6789/0 conn(0x7f21d0072cd0 legacy=0x7f21d00cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b8005bd0 0x7f21b8005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b8006a20 0x7f21b8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b80073e0 0x7f21b80147d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21d015f9a0 con 0x7f21b80073e0 2024-09-12T12:19:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21b800bde0 con 0x7f21b8005bd0 2024-09-12T12:19:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21b8014d10 con 0x7f21b8006a20 2024-09-12T12:19:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 Processor -- start 2024-09-12T12:19:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb6d9d640 1 --2- 172.21.15.3:0/2695782751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb80cd350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009d90 tx=0x7f4eb001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d2f5d222c6d2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cdd9b640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b8006a20 0x7f21b8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8cd3fff640 1 -- 172.21.15.3:0/2411033696 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80b0070 con 0x7f8cd4056d40 2024-09-12T12:19:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21ced9d640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b80073e0 0x7f21b80147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8cd3fff640 1 -- 172.21.15.3:0/2411033696 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd80ab600 con 0x7f8cd4056d40 2024-09-12T12:19:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe1f01640 1 --2- 172.21.15.3:0/859301992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc09b630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd801c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1858c49c7645455d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21ce59c640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b8005bd0 0x7f21b8005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cdd9b640 1 -- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b8005bd0 msgr2=0x7f21b8005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cdd9b640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b8005bd0 0x7f21b8005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cdd9b640 1 -- 172.21.15.3:0/3319171269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b80073e0 msgr2=0x7f21b80147d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 Processor -- start 2024-09-12T12:19:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cdd9b640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21b80073e0 0x7f21b80147d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cdd9b640 1 -- 172.21.15.3:0/3319171269 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b8005740 con 0x7f21b8006a20 2024-09-12T12:19:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21ce59c640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b8005bd0 0x7f21b8005fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb559a640 1 -- 172.21.15.3:0/2695782751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00b2070 con 0x7f4eb80d0650 2024-09-12T12:19:59.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb559a640 1 -- 172.21.15.3:0/2695782751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb00a58b0 con 0x7f4eb80d0650 2024-09-12T12:19:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb559a640 1 -- 172.21.15.3:0/2695782751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00a5c80 con 0x7f4eb80d0650 2024-09-12T12:19:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 -- start start 2024-09-12T12:19:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe0eff640 1 -- 172.21.15.3:0/859301992 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80b0070 con 0x7f2fdc0d0810 2024-09-12T12:19:59.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 -- 172.21.15.3:0/396048051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b3c70 msgr2=0x7f5b1406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 --2- 172.21.15.3:0/396048051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b3c70 0x7f5b1406d4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b1406e050 tx=0x7f5afc01ec70 comp rx=0 tx=0).stop 2024-09-12T12:19:59.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe0eff640 1 -- 172.21.15.3:0/859301992 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd80ab600 con 0x7f2fdc0d0810 2024-09-12T12:19:59.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe0eff640 1 -- 172.21.15.3:0/859301992 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80ab9b0 con 0x7f2fdc0d0810 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8cd3fff640 1 -- 172.21.15.3:0/2411033696 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80b0070 con 0x7f8cd4056d40 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 -- 172.21.15.3:0/2411033696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd4056d40 msgr2=0x7f8cd40571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 --2- 172.21.15.3:0/2411033696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd4056d40 0x7f8cd40571b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80097f0 tx=0x7f8cd801c490 comp rx=0 tx=0).stop 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 -- 172.21.15.3:0/2411033696 shutdown_connections 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 --2- 172.21.15.3:0/2411033696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd403c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 -- 172.21.15.3:0/2695782751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 msgr2=0x7f4eb80cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 --2- 172.21.15.3:0/2695782751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb80cd350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009d90 tx=0x7f4eb001c8e0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f07704e6640 1 -- start start 2024-09-12T12:19:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 --2- 172.21.15.3:0/2411033696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd4056d40 0x7f8cd40571b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec0adf40 0x7f67ec15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec1fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 -- 172.21.15.3:0/2411033696 >> 172.21.15.3:0/2411033696 conn(0x7f8cd403a4b0 msgr2=0x7f8cd403a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec147cd0 0x7f67ec1fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 msgr2=0x7f2fdc09b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 -- 172.21.15.3:0/2411033696 shutdown_connections 2024-09-12T12:19:59.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 --2- 172.21.15.3:0/859301992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc09b630 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd801c360 comp rx=0 tx=0).stop 2024-09-12T12:19:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f8ce2697640 1 -- 172.21.15.3:0/2411033696 wait complete. 2024-09-12T12:19:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 -- start start 2024-09-12T12:19:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2440037b0 0x7fe25c0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f2475640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec0adf40 0x7f67ec15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f1c74640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec1fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f2475640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec0adf40 0x7f67ec15dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40908/0 (socket says 172.21.15.3:40908) 2024-09-12T12:19:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f1c74640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec1fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56498/0 (socket says 172.21.15.3:56498) 2024-09-12T12:19:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 -- 172.21.15.3:0/2695782751 shutdown_connections 2024-09-12T12:19:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 --2- 172.21.15.3:0/2695782751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb80cd350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 --2- 172.21.15.3:0/2695782751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 0x7f4eb80b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 -- 172.21.15.3:0/2695782751 >> 172.21.15.3:0/2695782751 conn(0x7f4eb806f600 msgr2=0x7f4eb806bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 shutdown_connections 2024-09-12T12:19:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 --2- 172.21.15.3:0/859301992 >> v2:172.21.15.3:3300/0 conn(0x7f2fdc09bb70 0x7f2fdc0b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 --2- 172.21.15.3:0/859301992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc09b630 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 -- 172.21.15.3:0/2695782751 shutdown_connections 2024-09-12T12:19:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 >> 172.21.15.3:0/859301992 conn(0x7f2fdc0d2380 msgr2=0x7f2fdc0d2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 -- 172.21.15.3:0/396048051 shutdown_connections 2024-09-12T12:19:59.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 --2- 172.21.15.3:0/396048051 >> v2:172.21.15.142:3300/0 conn(0x7f5b140b66d0 0x7f5b140cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 --2- 172.21.15.3:0/396048051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b3c70 0x7f5b1406d4f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 -- 172.21.15.3:0/396048051 >> 172.21.15.3:0/396048051 conn(0x7f5b1408c000 msgr2=0x7f5b140d4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe25c0a9de0 0x7fe25c203ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f2c76640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec147cd0 0x7f67ec1fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f2c76640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec147cd0 0x7f67ec1fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42074/0 (socket says 172.21.15.3:42074) 2024-09-12T12:19:59.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21ced9d640 1 -- 172.21.15.3:0/3319171269 reap_dead start 2024-09-12T12:19:59.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 shutdown_connections 2024-09-12T12:19:59.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe25c204400 0x7fe25c204880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f2475640 1 -- 172.21.15.3:0/3601274474 learned_addr learned my addr 172.21.15.3:0/3601274474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0b4840 con 0x7f67ec0adf40 2024-09-12T12:19:59.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0b46c0 con 0x7f67ec0af5a0 2024-09-12T12:19:59.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f67f3477640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec0b49c0 con 0x7f67ec147cd0 2024-09-12T12:19:59.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f2fe4165640 1 -- 172.21.15.3:0/859301992 wait complete. 2024-09-12T12:19:59.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 -- 172.21.15.3:0/396048051 shutdown_connections 2024-09-12T12:19:59.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c0b47e0 con 0x7fe25c0a9de0 2024-09-12T12:19:59.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c0b4660 con 0x7fe2440037b0 2024-09-12T12:19:59.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cdd9b640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b8006a20 0x7f21b8006ea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f21c400c270 tx=0x7f21c401f350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1e482afdaf3f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f5b1926a640 1 -- 172.21.15.3:0/396048051 wait complete. 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe25b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe25c0a9de0 0x7fe25c203ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe25b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe25c0a9de0 0x7fe25c203ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40912/0 (socket says 172.21.15.3:40912) 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe25b7fe640 1 -- 172.21.15.3:0/3244591566 learned_addr learned my addr 172.21.15.3:0/3244591566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe25bfff640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2440037b0 0x7fe25c0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe260e00640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe25c204400 0x7fe25c204880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7fe262863640 1 -- 172.21.15.3:0/3244591566 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe25c06e7d0 con 0x7fe25c204400 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4007d20 con 0x7f21b8006a20 2024-09-12T12:19:59.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f07704e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760033980 0x7f07600e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f07704e6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0760047580 0x7f07600d5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f07704e6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07600eb3c0 0x7f07600317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f07704e6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f076001dd20 con 0x7f0760033980 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f07704e6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f076001dba0 con 0x7f07600eb3c0 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f07704e6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f076001dea0 con 0x7f0760047580 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f076da81640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0760047580 0x7f07600d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f076da81640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0760047580 0x7f07600d5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42092/0 (socket says 172.21.15.3:42092) 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f076da81640 1 -- 172.21.15.3:0/20866459 learned_addr learned my addr 172.21.15.3:0/20866459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076ea83640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07600eb3c0 0x7f07600317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f1c74640 1 -- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec147cd0 msgr2=0x7f67ec1fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f1c74640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec147cd0 0x7f67ec1fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f1c74640 1 -- 172.21.15.3:0/3601274474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec0adf40 msgr2=0x7f67ec15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f1c74640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec0adf40 0x7f67ec15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f1c74640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec1fc320 con 0x7f67ec0af5a0 2024-09-12T12:19:59.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.975+0000 7f4eb759e640 1 -- 172.21.15.3:0/2695782751 wait complete. 2024-09-12T12:19:59.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 Processor -- start 2024-09-12T12:19:59.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c4005ae0 con 0x7f21b8006a20 2024-09-12T12:19:59.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cd59a640 1 -- 172.21.15.3:0/3319171269 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4007070 con 0x7f21b8006a20 2024-09-12T12:19:59.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 -- start start 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076e282640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760033980 0x7f07600e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076da81640 1 -- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07600eb3c0 msgr2=0x7f07600317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076da81640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07600eb3c0 0x7f07600317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076da81640 1 -- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760033980 msgr2=0x7f07600e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076da81640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760033980 0x7f07600e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076da81640 1 -- 172.21.15.3:0/20866459 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07600d5880 con 0x7f0760047580 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f1c74640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec1fb360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e8a0 tx=0x7f67e800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076e282640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0760033980 0x7f07600e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:59.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe25b7fe640 1 -- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe25c204400 msgr2=0x7fe25c204880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe25b7fe640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe25c204400 0x7fe25c204880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe25b7fe640 1 -- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2440037b0 msgr2=0x7fe25c0af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe25b7fe640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2440037b0 0x7fe25c0af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe25b7fe640 1 -- 172.21.15.3:0/3244591566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c0b0800 con 0x7fe25c0a9de0 2024-09-12T12:19:59.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 -- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b8006a20 msgr2=0x7f21b8006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b8006a20 0x7f21b8006ea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f21c400c270 tx=0x7f21c401f350 comp rx=0 tx=0).stop 2024-09-12T12:19:59.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8017070 con 0x7f67ec0af5a0 2024-09-12T12:19:59.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 -- 172.21.15.3:0/3319171269 shutdown_connections 2024-09-12T12:19:59.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 --2- 172.21.15.3:0/3319171269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b8006a20 0x7f21b8006ea0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 -- 172.21.15.3:0/3319171269 >> 172.21.15.3:0/3319171269 conn(0x7f21d00b3020 msgr2=0x7f21d00b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 Processor -- start 2024-09-12T12:19:59.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e800c940 con 0x7f67ec0af5a0 2024-09-12T12:19:59.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800ccf0 con 0x7f67ec0af5a0 2024-09-12T12:19:59.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f3477640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec143c00 con 0x7f67ec0af5a0 2024-09-12T12:19:59.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f5b1926a640 1 Processor -- start 2024-09-12T12:19:59.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f3477640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec1446e0 con 0x7f67ec0af5a0 2024-09-12T12:19:59.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f076da81640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0760047580 0x7f07600d5340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f075800e9e0 tx=0x7f075800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe25b7fe640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe25c0a9de0 0x7fe25c203ec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe254007b70 tx=0x7fe25400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f4eb759e640 1 Processor -- start 2024-09-12T12:19:59.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd4056d40 0x7f8cd4052b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f5b1926a640 1 -- start start 2024-09-12T12:19:59.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f4eb759e640 1 -- start start 2024-09-12T12:19:59.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd40d8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 -- 172.21.15.3:0/3319171269 shutdown_connections 2024-09-12T12:19:59.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd40e5f90 0x7f8cd404d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce1695640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd4056d40 0x7f8cd4052b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce1695640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd4056d40 0x7f8cd4052b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56512/0 (socket says 172.21.15.3:56512) 2024-09-12T12:19:59.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce0e94640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd40d8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce0e94640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd40d8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42106/0 (socket says 172.21.15.3:42106) 2024-09-12T12:19:59.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 -- start start 2024-09-12T12:19:59.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce1695640 1 -- 172.21.15.3:0/3006315071 learned_addr learned my addr 172.21.15.3:0/3006315071 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fdc09bb70 0x7f2fdc0b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc0d0bf0 0x7f2fdc0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0ce180 con 0x7f2fdc09bb70 2024-09-12T12:19:59.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0b3870 con 0x7f2fdc0d0bf0 2024-09-12T12:19:59.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe4165640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc0b23a0 con 0x7f2fdc0d0810 2024-09-12T12:19:59.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd403d4d0 con 0x7f8cd40e5f90 2024-09-12T12:19:59.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd403d350 con 0x7f8cd4056d40 2024-09-12T12:19:59.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd403d650 con 0x7f8cd40576f0 2024-09-12T12:19:59.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f67f3477640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67ec15e500 con 0x7f67ec0af5a0 2024-09-12T12:19:59.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fdc09bb70 0x7f2fdc0b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fdc09bb70 0x7f2fdc0b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40942/0 (socket says 172.21.15.3:40942) 2024-09-12T12:19:59.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 -- 172.21.15.3:0/2824111645 learned_addr learned my addr 172.21.15.3:0/2824111645 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce1e96640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd40e5f90 0x7f8cd404d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1700640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe2702640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc0d0bf0 0x7f2fdc0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 -- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 msgr2=0x7f2fdc0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2fdc0d0810 0x7f2fdc0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 -- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc0d0bf0 msgr2=0x7f2fdc0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fdc0d0bf0 0x7f2fdc0b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 -- 172.21.15.3:0/2824111645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc0b5780 con 0x7f2fdc09bb70 2024-09-12T12:19:59.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 -- 172.21.15.3:0/3319171269 wait complete. 2024-09-12T12:19:59.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 Processor -- start 2024-09-12T12:19:59.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f2fe1f01640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fdc09bb70 0x7f2fdc0b5c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd8019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0758017070 con 0x7f0760047580 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f075800c9d0 con 0x7f0760047580 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075800cda0 con 0x7f0760047580 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254019070 con 0x7fe25c0a9de0 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe254014480 con 0x7fe25c0a9de0 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254014830 con 0x7fe25c0a9de0 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe262863640 1 -- 172.21.15.3:0/3244591566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c0b0ae0 con 0x7fe25c0a9de0 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe262863640 1 -- 172.21.15.3:0/3244591566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c0aa260 con 0x7fe25c0a9de0 2024-09-12T12:19:59.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7fe262863640 1 -- 172.21.15.3:0/3244591566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe25c0b38a0 con 0x7fe25c0a9de0 2024-09-12T12:19:59.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce0e94640 1 -- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd4056d40 msgr2=0x7f8cd4052b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce0e94640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cd4056d40 0x7f8cd4052b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f8ce0e94640 1 -- 172.21.15.3:0/3006315071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd40e5f90 msgr2=0x7f8cd404d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8ce0e94640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8cd40e5f90 0x7f8cd404d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8ce0e94640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd40e0470 con 0x7f8cd40576f0 2024-09-12T12:19:59.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb759e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb8072e40 0x7f4eb8150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.979+0000 7f21cf59e640 1 -- start start 2024-09-12T12:19:59.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb8163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb759e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 0x7f4eb815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb759e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80ce430 con 0x7f4eb80d0650 2024-09-12T12:19:59.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cf59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d01587f0 0x7f21d0141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cf59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d013b7c0 0x7f21d0159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cf59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d00b1b70 0x7f21d00b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cf59e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0073570 con 0x7f21d013b7c0 2024-09-12T12:19:59.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cf59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0141060 con 0x7f21d00b1b70 2024-09-12T12:19:59.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cf59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21d0140450 con 0x7f21d01587f0 2024-09-12T12:19:59.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb759e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb80ce2b0 con 0x7f4eb8072e40 2024-09-12T12:19:59.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8ce0e94640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd40d8d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80ab000 tx=0x7f8cd8019230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07600d6700 con 0x7f0760047580 2024-09-12T12:19:59.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f07600d6cc0 con 0x7f0760047580 2024-09-12T12:19:59.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67e801a070 con 0x7f67ec0af5a0 2024-09-12T12:19:59.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d013b7c0 0x7f21d0159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d013b7c0 0x7f21d0159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40954/0 (socket says 172.21.15.3:40954) 2024-09-12T12:19:59.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 -- 172.21.15.3:0/1716188348 learned_addr learned my addr 172.21.15.3:0/1716188348 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21ce59c640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d01587f0 0x7f21d0141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b1926a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b3c70 0x7f5b1415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67e2ffd640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67d0067910 0x7f67d0069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67e80862f0 con 0x7f67ec0af5a0 2024-09-12T12:19:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b1926a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b62f0 0x7f5b141f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b1926a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b140b66d0 0x7f5b140b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b1926a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1406e5f0 con 0x7f5b140b62f0 2024-09-12T12:19:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67e2ffd640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7f67d006fb90 0x7f67d0071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67e800ce90 con 0x7f67d006fb90 2024-09-12T12:19:59.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f67e80867d0 con 0x7f67ec0af5a0 2024-09-12T12:19:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b1926a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b140d33a0 con 0x7f5b140b3c70 2024-09-12T12:19:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b3c70 0x7f5b1415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b0a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b62f0 0x7f5b141f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b12ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b3c70 0x7f5b1415f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56546/0 (socket says 172.21.15.3:56546) 2024-09-12T12:19:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b12ffd640 1 -- 172.21.15.3:0/2195233683 learned_addr learned my addr 172.21.15.3:0/2195233683 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1406e770 con 0x7f5b140b66d0 2024-09-12T12:19:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80b0040 con 0x7f8cd40576f0 2024-09-12T12:19:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67f2c76640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7f67d006fb90 0x7f67d0071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67f2475640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67d0067910 0x7f67d0069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b0a7fc640 1 -- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b140b66d0 msgr2=0x7f5b140b84b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:19:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b0a7fc640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b140b66d0 0x7f5b140b84b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b0a7fc640 1 -- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b3c70 msgr2=0x7f5b1415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b0a7fc640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b3c70 0x7f5b1415f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b0a7fc640 1 -- 172.21.15.3:0/2195233683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b140b8ce0 con 0x7f5b140b62f0 2024-09-12T12:19:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f67f2c76640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7f67d006fb90 0x7f67d0071fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd40e0750 con 0x7f8cd40576f0 2024-09-12T12:19:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd40e0d10 con 0x7f8cd40576f0 2024-09-12T12:19:59.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cd400de30 con 0x7f8cd40576f0 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd80bb070 con 0x7f8cd40576f0 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80acdb0 con 0x7f8cd40576f0 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 -- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d01587f0 msgr2=0x7f21d0141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21d01587f0 0x7f21d0141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 -- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d00b1b70 msgr2=0x7f21d00b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21d00b1b70 0x7f21d00b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f21cdd9b640 1 -- 172.21.15.3:0/1716188348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d006ff50 con 0x7f21d013b7c0 2024-09-12T12:19:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0750001040 con 0x7f0760047580 2024-09-12T12:19:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe25401c070 con 0x7fe25c0a9de0 2024-09-12T12:19:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7fe258ff9640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe248067990 0x7fe248069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe254085c80 con 0x7fe25c0a9de0 2024-09-12T12:19:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7fe258ff9640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7fe24806fb30 0x7fe248071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe248072600 con 0x7fe24806fb30 2024-09-12T12:19:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7fe25bfff640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe248067990 0x7fe248069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7fe260e00640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7fe24806fb30 0x7fe248071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f075801a070 con 0x7f0760047580 2024-09-12T12:19:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f075effd640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f074c067990 0x7f074c069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0758086090 con 0x7f0760047580 2024-09-12T12:19:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f075effd640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f074c06fc10 0x7f074c072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 --> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f074c0726e0 con 0x7f074c06fc10 2024-09-12T12:19:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0758086480 con 0x7f0760047580 2024-09-12T12:19:59.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f5b0a7fc640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b62f0 0x7f5b141f7ca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0af000 tx=0x7f5afc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80b0040 con 0x7f2fdc09bb70 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd80ba070 con 0x7f2fdc09bb70 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80ace30 con 0x7f2fdc09bb70 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc0b61c0 con 0x7f2fdc09bb70 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc0ca7a0 con 0x7f2fdc09bb70 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fdc06fc70 con 0x7f2fdc09bb70 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb659c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb8072e40 0x7f4eb8150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb8163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb8163750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40948/0 (socket says 172.21.15.3:40948) 2024-09-12T12:19:59.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb5d9b640 1 -- 172.21.15.3:0/976903932 learned_addr learned my addr 172.21.15.3:0/976903932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb6d9d640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 0x7f4eb815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f4eb759e640 1 -- 172.21.15.3:0/976903932 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb80ce5b0 con 0x7f4eb8162590 2024-09-12T12:19:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7fe260e00640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7fe24806fb30 0x7fe248071f50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7fe25bfff640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe248067990 0x7fe248069e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0ab000 tx=0x7fe24c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== osd.0 v2:172.21.15.3:6816/3370158904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe248072600 con 0x7fe24806fb30 2024-09-12T12:19:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f076ea83640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f074c06fc10 0x7f074c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.983+0000 7f076e282640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f074c067990 0x7f074c069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f076e282640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f074c067990 0x7f074c069e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07600e4b10 tx=0x7f0764009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f076ea83640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f074c06fc10 0x7f074c072030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0b5040 con 0x7f5b140b62f0 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc0bf070 con 0x7f5b140b62f0 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0b1a50 con 0x7f5b140b62f0 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2fd8003620 con 0x7f2fdc09bb70 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fd27fc640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2fc00678c0 0x7f2fc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2fd811f730 con 0x7f2fdc09bb70 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fd27fc640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f2fc006fa60 0x7f2fc0071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 --> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fd80ba070 con 0x7f2fc006fa60 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fe1700640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2fc00678c0 0x7f2fc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fe2702640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f2fc006fa60 0x7f2fc0071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2fd8126050 con 0x7f2fdc09bb70 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fe2702640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f2fc006fa60 0x7f2fc0071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f67f2475640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67d0067910 0x7f67d0069dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0098b0 tx=0x7f67dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== osd.4 v2:172.21.15.142:6800/4142884196 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67e800ce90 con 0x7f67d006fb90 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8cd8003620 con 0x7f8cd40576f0 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb5d9b640 1 -- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 msgr2=0x7f4eb815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb5d9b640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4eb8162590 0x7f4eb815c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb5d9b640 1 -- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb8072e40 msgr2=0x7f4eb8150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb5d9b640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4eb8072e40 0x7f4eb8150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb5d9b640 1 -- 172.21.15.3:0/976903932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81569f0 con 0x7f4eb80d0650 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb5d9b640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb8163750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00c4d0 tx=0x7f4eac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac017070 con 0x7f4eb80d0650 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eac005cf0 con 0x7f4eb80d0650 2024-09-12T12:19:59.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac007270 con 0x7f4eb80d0650 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== osd.1 v2:172.21.15.3:6800/4123050499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f074c0726e0 con 0x7f074c06fc10 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8cd1ffb640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cc00678c0 0x7f8cc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8cd81201d0 con 0x7f8cd40576f0 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8ce1695640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cc00678c0 0x7f8cc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8cd1ffb640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f8cc006fa60 0x7f8cc0071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cc0072530 con 0x7f8cc006fa60 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8cd8120660 con 0x7f8cd40576f0 2024-09-12T12:19:59.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8ce1e96640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f8cc006fa60 0x7f8cc0071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f21cdd9b640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d013b7c0 0x7f21d0159a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21c401f180 tx=0x7f21b8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe254052480 con 0x7fe25c0a9de0 2024-09-12T12:19:59.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b140b8fc0 con 0x7f5b140b62f0 2024-09-12T12:19:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b140b9580 con 0x7f5b140b62f0 2024-09-12T12:19:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb759e640 1 -- 172.21.15.3:0/976903932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb8156cd0 con 0x7f4eb80d0650 2024-09-12T12:19:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c40047f0 con 0x7f21d013b7c0 2024-09-12T12:19:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c4004990 con 0x7f21d013b7c0 2024-09-12T12:19:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4004d80 con 0x7f21d013b7c0 2024-09-12T12:19:59.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fe1700640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2fc00678c0 0x7f2fc0069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc006fd0 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4eb759e640 1 -- 172.21.15.3:0/976903932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb8157290 con 0x7f4eb80d0650 2024-09-12T12:19:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== osd.3 v2:172.21.15.3:6808/63313814 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fd80ba070 con 0x7f2fc006fa60 2024-09-12T12:19:59.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8ce1695640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cc00678c0 0x7f8cc0069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc000f70 tx=0x7f8ccc007f10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f8ce1e96640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f8cc006fa60 0x7f8cc0071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b1406db50 con 0x7f5b140b62f0 2024-09-12T12:19:59.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.987+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e90001040 con 0x7f4eb80d0650 2024-09-12T12:19:59.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d0141fd0 con 0x7f21d013b7c0 2024-09-12T12:19:59.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d0159280 con 0x7f21d013b7c0 2024-09-12T12:19:59.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21d006e530 con 0x7f21d013b7c0 2024-09-12T12:19:59.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5afc0b1bf0 con 0x7f5b140b62f0 2024-09-12T12:19:59.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b0bfff640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5af40678c0 0x7f5af4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5afc1239a0 con 0x7f5b140b62f0 2024-09-12T12:19:59.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b0bfff640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f5af406fa60 0x7f5af4071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 --> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5afc0b12a0 con 0x7f5af406fa60 2024-09-12T12:19:59.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5afc123d50 con 0x7f5b140b62f0 2024-09-12T12:19:59.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b137fe640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f5af406fa60 0x7f5af4071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== osd.5 v2:172.21.15.142:6816/4110985109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cc0072530 con 0x7f8cc006fa60 2024-09-12T12:19:59.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b12ffd640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5af40678c0 0x7f5af4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f5b137fe640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f5af406fa60 0x7f5af4071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f21c40070e0 con 0x7f21d013b7c0 2024-09-12T12:19:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4eac01a070 con 0x7f4eb80d0650 2024-09-12T12:19:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f4ea6ffd640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e98067990 0x7f4e98069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4eac084f30 con 0x7f4eb80d0650 2024-09-12T12:19:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f4ea6ffd640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f4e9806fc10 0x7f4e98072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 --> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4eac016ec0 con 0x7f4e9806fc10 2024-09-12T12:19:59.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4eac087590 con 0x7f4eb80d0650 2024-09-12T12:19:59.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.991+0000 7f4eb659c640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e98067990 0x7f4e98069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f4eb6d9d640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f4e9806fc10 0x7f4e98072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f21beffd640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21b00678f0 0x7f21b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f5b12ffd640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5af40678c0 0x7f5af4069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b04001730 tx=0x7f5b04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== osd.2 v2:172.21.15.3:6824/3021596546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5afc0b12a0 con 0x7f5af406fa60 2024-09-12T12:19:59.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f21c4123260 con 0x7f21d013b7c0 2024-09-12T12:19:59.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f21beffd640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7f21b006fb10 0x7f21b0071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 --> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21b00725e0 con 0x7f21b006fb10 2024-09-12T12:19:59.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f21c4123650 con 0x7f21d013b7c0 2024-09-12T12:19:59.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f4eb659c640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e98067990 0x7f4e98069e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8163150 tx=0x7f4ea8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f21ce59c640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21b00678f0 0x7f21b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f4eb6d9d640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f4e9806fc10 0x7f4e98072030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:58.995+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== osd.6 v2:172.21.15.142:6808/3052505942 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eac016ec0 con 0x7f4e9806fc10 2024-09-12T12:19:59.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.003+0000 7f21ced9d640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7f21b006fb10 0x7f21b0071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.003+0000 7f21ced9d640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7f21b006fb10 0x7f21b0071f30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.003+0000 7f21ce59c640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21b00678f0 0x7f21b0069db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21c8007fd0 tx=0x7f21c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.003+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== osd.7 v2:172.21.15.142:6824/177123471 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21b00725e0 con 0x7f21b006fb10 2024-09-12T12:19:59.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe262863640 1 -- 172.21.15.3:0/3244591566 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe25c15ca20 con 0x7fe24806fb30 2024-09-12T12:19:59.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe258ff9640 1 -- 172.21.15.3:0/3244591566 <== osd.0 v2:172.21.15.3:6816/3370158904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe25c15ca20 con 0x7fe24806fb30 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 -- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7fe24806fb30 msgr2=0x7fe248071f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7fe24806fb30 0x7fe248071f50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 -- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe248067990 msgr2=0x7fe248069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe248067990 0x7fe248069e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0ab000 tx=0x7fe24c0a7070 comp rx=0 tx=0).stop 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 -- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe25c0a9de0 msgr2=0x7fe25c203ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 --2- 172.21.15.3:0/3244591566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe25c0a9de0 0x7fe25c203ec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe254007b70 tx=0x7fe25400e530 comp rx=0 tx=0).stop 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe260e00640 1 -- 172.21.15.3:0/3244591566 reap_dead start 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 -- 172.21.15.3:0/3244591566 shutdown_connections 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 -- 172.21.15.3:0/3244591566 >> 172.21.15.3:0/3244591566 conn(0x7fe25c06dcb0 msgr2=0x7fe25c15d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 -- 172.21.15.3:0/3244591566 shutdown_connections 2024-09-12T12:19:59.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.011+0000 7fe2427fc640 1 -- 172.21.15.3:0/3244591566 wait complete. 2024-09-12T12:19:59.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67f3477640 1 -- 172.21.15.3:0/3601274474 --> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f67ec069230 con 0x7f67d006fb90 2024-09-12T12:19:59.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e2ffd640 1 -- 172.21.15.3:0/3601274474 <== osd.4 v2:172.21.15.142:6800/4142884196 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f67e4009170 con 0x7f67d006fb90 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 -- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7f67d006fb90 msgr2=0x7f67d0071fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7f67d006fb90 0x7f67d0071fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 -- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67d0067910 msgr2=0x7f67d0069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67d0067910 0x7f67d0069dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0098b0 tx=0x7f67dc008040 comp rx=0 tx=0).stop 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 -- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 msgr2=0x7f67ec1fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 --2- 172.21.15.3:0/3601274474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec0af5a0 0x7f67ec1fb360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67e800e8a0 tx=0x7f67e800ed60 comp rx=0 tx=0).stop 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67f2c76640 1 -- 172.21.15.3:0/3601274474 reap_dead start 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 -- 172.21.15.3:0/3601274474 shutdown_connections 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 -- 172.21.15.3:0/3601274474 >> 172.21.15.3:0/3601274474 conn(0x7f67ec0b1d40 msgr2=0x7f67ec071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 --> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0750002d60 con 0x7f074c06fc10 2024-09-12T12:19:59.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 -- 172.21.15.3:0/3601274474 shutdown_connections 2024-09-12T12:19:59.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f67e0ff9640 1 -- 172.21.15.3:0/3601274474 wait complete. 2024-09-12T12:19:59.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.015+0000 7f075effd640 1 -- 172.21.15.3:0/20866459 <== osd.1 v2:172.21.15.3:6800/4123050499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0750002d60 con 0x7f074c06fc10 2024-09-12T12:19:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 --> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fdc0b5a60 con 0x7f2fc006fa60 2024-09-12T12:19:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fd27fc640 1 -- 172.21.15.3:0/2824111645 <== osd.3 v2:172.21.15.3:6808/63313814 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2fdc0b5a60 con 0x7f2fc006fa60 2024-09-12T12:19:59.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 --> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8cd40585b0 con 0x7f8cc006fa60 2024-09-12T12:19:59.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f2fc006fa60 msgr2=0x7f2fc0071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f2fc006fa60 0x7f2fc0071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f074c06fc10 msgr2=0x7f074c072030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f074c06fc10 0x7f074c072030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2fc00678c0 msgr2=0x7f2fc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2fc00678c0 0x7f2fc0069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc006fd0 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fdc09bb70 msgr2=0x7f2fdc0b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 --2- 172.21.15.3:0/2824111645 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fdc09bb70 0x7f2fdc0b5c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009870 tx=0x7f2fd8019100 comp rx=0 tx=0).stop 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe2702640 1 -- 172.21.15.3:0/2824111645 reap_dead start 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 shutdown_connections 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 >> 172.21.15.3:0/2824111645 conn(0x7f2fdc0d2380 msgr2=0x7f2fdc0750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 shutdown_connections 2024-09-12T12:19:59.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f074c067990 msgr2=0x7f074c069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f074c067990 0x7f074c069e50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07600e4b10 tx=0x7f0764009040 comp rx=0 tx=0).stop 2024-09-12T12:19:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0760047580 msgr2=0x7f07600d5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 --2- 172.21.15.3:0/20866459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0760047580 0x7f07600d5340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f075800e9e0 tx=0x7f075800eea0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f076ea83640 1 -- 172.21.15.3:0/20866459 reap_dead start 2024-09-12T12:19:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f2fe4165640 1 -- 172.21.15.3:0/2824111645 wait complete. 2024-09-12T12:19:59.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f8cd1ffb640 1 -- 172.21.15.3:0/3006315071 <== osd.5 v2:172.21.15.142:6816/4110985109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8cd40585b0 con 0x7f8cc006fa60 2024-09-12T12:19:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 shutdown_connections 2024-09-12T12:19:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 >> 172.21.15.3:0/20866459 conn(0x7f07600360a0 msgr2=0x7f0760005ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 shutdown_connections 2024-09-12T12:19:59.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.019+0000 7f07704e6640 1 -- 172.21.15.3:0/20866459 wait complete. 2024-09-12T12:19:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f8cc006fa60 msgr2=0x7f8cc0071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f8cc006fa60 0x7f8cc0071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cc00678c0 msgr2=0x7f8cc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cc00678c0 0x7f8cc0069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc000f70 tx=0x7f8ccc007f10 comp rx=0 tx=0).stop 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 msgr2=0x7f8cd40d8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 --2- 172.21.15.3:0/3006315071 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8cd40576f0 0x7f8cd40d8d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80ab000 tx=0x7f8cd8019230 comp rx=0 tx=0).stop 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce1e96640 1 -- 172.21.15.3:0/3006315071 reap_dead start 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 shutdown_connections 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.023+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 >> 172.21.15.3:0/3006315071 conn(0x7f8cd403a4b0 msgr2=0x7f8cd40e28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 --> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b140d1bc0 con 0x7f5af406fa60 2024-09-12T12:19:59.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 shutdown_connections 2024-09-12T12:19:59.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 --> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e90002de0 con 0x7f4e9806fc10 2024-09-12T12:19:59.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b0bfff640 1 -- 172.21.15.3:0/2195233683 <== osd.2 v2:172.21.15.3:6824/3021596546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5b140d1bc0 con 0x7f5af406fa60 2024-09-12T12:19:59.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea6ffd640 1 -- 172.21.15.3:0/976903932 <== osd.6 v2:172.21.15.142:6808/3052505942 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4e90002de0 con 0x7f4e9806fc10 2024-09-12T12:19:59.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 --> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21d006e530 con 0x7f21b006fb10 2024-09-12T12:19:59.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f4e9806fc10 msgr2=0x7f4e98072030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f4e9806fc10 0x7f4e98072030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f5af406fa60 msgr2=0x7f5af4071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f5af406fa60 0x7f5af4071e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e98067990 msgr2=0x7f4e98069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e98067990 0x7f4e98069e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8163150 tx=0x7f4ea8009190 comp rx=0 tx=0).stop 2024-09-12T12:19:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 msgr2=0x7f4eb8163750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 --2- 172.21.15.3:0/976903932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4eb80d0650 0x7f4eb8163750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00c4d0 tx=0x7f4eac00c990 comp rx=0 tx=0).stop 2024-09-12T12:19:59.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4eb6d9d640 1 -- 172.21.15.3:0/976903932 reap_dead start 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5af40678c0 msgr2=0x7f5af4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5af40678c0 0x7f5af4069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5b04001730 tx=0x7f5b04009040 comp rx=0 tx=0).stop 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b62f0 msgr2=0x7f5b141f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 --2- 172.21.15.3:0/2195233683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b140b62f0 0x7f5b141f7ca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0af000 tx=0x7f5afc01ba10 comp rx=0 tx=0).stop 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b137fe640 1 -- 172.21.15.3:0/2195233683 reap_dead start 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 shutdown_connections 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 >> 172.21.15.3:0/2195233683 conn(0x7f5b1408c000 msgr2=0x7f5b1415f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 shutdown_connections 2024-09-12T12:19:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 >> 172.21.15.3:0/976903932 conn(0x7f4eb806f600 msgr2=0x7f4eb80cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 shutdown_connections 2024-09-12T12:19:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 shutdown_connections 2024-09-12T12:19:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f4ea4ff9640 1 -- 172.21.15.3:0/976903932 wait complete. 2024-09-12T12:19:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f5b1926a640 1 -- 172.21.15.3:0/2195233683 wait complete. 2024-09-12T12:19:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21beffd640 1 -- 172.21.15.3:0/1716188348 <== osd.7 v2:172.21.15.142:6824/177123471 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21d006e530 con 0x7f21b006fb10 2024-09-12T12:19:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f8ce2697640 1 -- 172.21.15.3:0/3006315071 wait complete. 2024-09-12T12:19:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7f21b006fb10 msgr2=0x7f21b0071f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7f21b006fb10 0x7f21b0071f30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21b00678f0 msgr2=0x7f21b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21b00678f0 0x7f21b0069db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21c8007fd0 tx=0x7f21c8009040 comp rx=0 tx=0).stop 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d013b7c0 msgr2=0x7f21d0159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 --2- 172.21.15.3:0/1716188348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21d013b7c0 0x7f21d0159a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f21c401f180 tx=0x7f21b8007a50 comp rx=0 tx=0).stop 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21ced9d640 1 -- 172.21.15.3:0/1716188348 reap_dead start 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 shutdown_connections 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 >> 172.21.15.3:0/1716188348 conn(0x7f21d00b3020 msgr2=0x7f21d015b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 shutdown_connections 2024-09-12T12:19:59.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.027+0000 7f21cf59e640 1 -- 172.21.15.3:0/1716188348 wait complete. 2024-09-12T12:19:59.085 INFO:teuthology.orchestra.run.smithi003.stdout:60129542150 2024-09-12T12:19:59.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T12:19:59.086 INFO:teuthology.orchestra.run.smithi003.stdout:60129542151 2024-09-12T12:19:59.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T12:19:59.087 INFO:teuthology.orchestra.run.smithi003.stdout:64424509448 2024-09-12T12:19:59.088 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T12:19:59.088 INFO:teuthology.orchestra.run.smithi003.stdout:60129542151 2024-09-12T12:19:59.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T12:19:59.089 INFO:teuthology.orchestra.run.smithi003.stdout:60129542150 2024-09-12T12:19:59.090 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T12:19:59.090 INFO:teuthology.orchestra.run.smithi003.stdout:60129542152 2024-09-12T12:19:59.091 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T12:19:59.092 INFO:teuthology.orchestra.run.smithi003.stdout:60129542152 2024-09-12T12:19:59.092 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T12:19:59.093 INFO:teuthology.orchestra.run.smithi003.stdout:60129542151 2024-09-12T12:19:59.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T12:19:59.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55b5332640 1 Processor -- start 2024-09-12T12:19:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55b5332640 1 -- start start 2024-09-12T12:19:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3e6ce640 1 Processor -- start 2024-09-12T12:19:59.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3e6ce640 1 -- start start 2024-09-12T12:19:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f36c9b21640 1 Processor -- start 2024-09-12T12:19:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f36c9b21640 1 -- start start 2024-09-12T12:19:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f04e08f1640 1 Processor -- start 2024-09-12T12:19:59.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f04e08f1640 1 -- start start 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3e6ce640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a301009c0 0x7f3a30009ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3e6ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a30006fe0 0x7f3a30036030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3d6cc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a301009c0 0x7f3a30009ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3d6cc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a301009c0 0x7f3a30009ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40984/0 (socket says 172.21.15.3:40984) 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3d6cc640 1 -- 172.21.15.3:0/2118673943 learned_addr learned my addr 172.21.15.3:0/2118673943 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a30054d60 con 0x7f3a3000b700 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3a30055040 con 0x7f3a301009c0 2024-09-12T12:19:59.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3a30055320 con 0x7f3a30006fe0 2024-09-12T12:19:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f36c9b21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c4070390 0x7f36c4072a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55b5332640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55b015e250 0x7f55b00d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55b5332640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b00b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55b5332640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55b00d47c0 con 0x7f55b00cf270 2024-09-12T12:19:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55b5332640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f55b00cb290 con 0x7f55b015e250 2024-09-12T12:19:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55b5332640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f55b00d4aa0 con 0x7f55b00d19c0 2024-09-12T12:19:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55aeffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55b015e250 0x7f55b00d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55aeffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f55b015e250 0x7f55b00d40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40970/0 (socket says 172.21.15.3:40970) 2024-09-12T12:19:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f55aeffd640 1 -- 172.21.15.3:0/2769395363 learned_addr learned my addr 172.21.15.3:0/2769395363 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f55af7fe640 1 --2- 172.21.15.3:0/2769395363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b00b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f36c9b21640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c40b4020 con 0x7f36c40b31e0 2024-09-12T12:19:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.299+0000 7f36c37fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f36c40b31e0 0x7f36c40b35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36116/0 (socket says 172.21.15.3:36116) 2024-09-12T12:19:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f36c37fe640 1 -- 172.21.15.3:0/791138973 learned_addr learned my addr 172.21.15.3:0/791138973 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:19:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f36c2ffd640 1 --2- 172.21.15.3:0/791138973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c4070390 0x7f36c4072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f55aeffd640 1 -- 172.21.15.3:0/2769395363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 msgr2=0x7f55b00b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f55aeffd640 1 --2- 172.21.15.3:0/2769395363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b00b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f55aeffd640 1 -- 172.21.15.3:0/2769395363 >> v1:172.21.15.3:6789/0 conn(0x7f55b00cf270 legacy=0x7f55b00d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f36c9b21640 1 -- 172.21.15.3:0/791138973 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c40b4d30 con 0x7f36c408b620 2024-09-12T12:19:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f36c9b21640 1 -- 172.21.15.3:0/791138973 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f36c40b52f0 con 0x7f36c4070390 2024-09-12T12:19:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f55aeffd640 1 -- 172.21.15.3:0/2769395363 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b00d4d80 con 0x7f55b015e250 2024-09-12T12:19:59.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04e08f1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04d40df7f0 0x7f04d400c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04e08f1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04e08f1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d40dfc40 con 0x7f04d4008c70 2024-09-12T12:19:59.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfddd19640 1 Processor -- start 2024-09-12T12:19:59.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04e08f1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f04d400cd40 con 0x7f04d40df7f0 2024-09-12T12:19:59.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04e08f1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f04d4038010 con 0x7f04d4008150 2024-09-12T12:19:59.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04db59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04db59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42162/0 (socket says 172.21.15.3:42162) 2024-09-12T12:19:59.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04db59e640 1 -- 172.21.15.3:0/1339019932 learned_addr learned my addr 172.21.15.3:0/1339019932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6d34640 1 Processor -- start 2024-09-12T12:19:59.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 Processor -- start 2024-09-12T12:19:59.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 -- start start 2024-09-12T12:19:59.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfddd19640 1 -- start start 2024-09-12T12:19:59.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6d34640 1 -- start start 2024-09-12T12:19:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f36c2ffd640 1 -- 172.21.15.3:0/791138973 >> v1:172.21.15.142:6789/0 conn(0x7f36c408b620 legacy=0x7f36c4072fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f36c2ffd640 1 -- 172.21.15.3:0/791138973 >> v1:172.21.15.3:6789/0 conn(0x7f36c40b31e0 legacy=0x7f36c40b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f36c2ffd640 1 -- 172.21.15.3:0/791138973 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c40b5010 con 0x7f36c4070390 2024-09-12T12:19:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfddd19640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80b4950 0x7fbfd80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfddd19640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbfd80ad060 0x7fbfd80ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfddd19640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd80b38b0 con 0x7fbfd8162800 2024-09-12T12:19:59.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfddd19640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbfd80b44c0 con 0x7fbfd80ad060 2024-09-12T12:19:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfddd19640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd80b3b90 con 0x7fbfd80b4950 2024-09-12T12:19:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80b4950 0x7fbfd80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6d34640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc7c8036b90 0x7fc7c804efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd6ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80b4950 0x7fbfd80b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42174/0 (socket says 172.21.15.3:42174) 2024-09-12T12:19:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd6ffd640 1 -- 172.21.15.3:0/3740079446 learned_addr learned my addr 172.21.15.3:0/3740079446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd7fff640 1 --2- 172.21.15.3:0/3740079446 >> v2:172.21.15.142:3300/0 conn(0x7fbfd80ad060 0x7fbfd80ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf1c036290 0x7fcf1c036670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf1c036bb0 0x7fcf1c039c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcf1c047b80 con 0x7fcf1c036290 2024-09-12T12:19:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcf1c047830 con 0x7fcf1c036bb0 2024-09-12T12:19:59.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf2a673640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c0337c0 con 0x7fcf1c03a1b0 2024-09-12T12:19:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6d34640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c800a6b0 con 0x7fc7c8004a40 2024-09-12T12:19:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6d34640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c8007860 con 0x7fc7c8007410 2024-09-12T12:19:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f3a3d6cc640 1 -- 172.21.15.3:0/2118673943 >> v2:172.21.15.142:3300/0 conn(0x7f3a30006fe0 msgr2=0x7f3a30036030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T12:19:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f3a3d6cc640 1 --2- 172.21.15.3:0/2118673943 >> v2:172.21.15.142:3300/0 conn(0x7f3a30006fe0 0x7f3a30036030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f3a3d6cc640 1 -- 172.21.15.3:0/2118673943 >> v1:172.21.15.142:6789/0 conn(0x7f3a3000b700 legacy=0x7f3a30039260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f3a3d6cc640 1 -- 172.21.15.3:0/2118673943 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a30006470 con 0x7f3a301009c0 2024-09-12T12:19:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04db59e640 1 -- 172.21.15.3:0/1339019932 >> v2:172.21.15.3:3300/0 conn(0x7f04d40df7f0 msgr2=0x7f04d400c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04db59e640 1 --2- 172.21.15.3:0/1339019932 >> v2:172.21.15.3:3300/0 conn(0x7f04d40df7f0 0x7f04d400c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04db59e640 1 -- 172.21.15.3:0/1339019932 >> v1:172.21.15.3:6789/0 conn(0x7f04d4008c70 legacy=0x7f04d4038d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7f04db59e640 1 -- 172.21.15.3:0/1339019932 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d400af10 con 0x7f04d4008150 2024-09-12T12:19:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6d34640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c80558c0 con 0x7fc7c8036b90 2024-09-12T12:19:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6533640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc7c8036b90 0x7fc7c804efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6533640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc7c8036b90 0x7fc7c804efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56578/0 (socket says 172.21.15.3:56578) 2024-09-12T12:19:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fc7d6533640 1 -- 172.21.15.3:0/2080127634 learned_addr learned my addr 172.21.15.3:0/2080127634 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf29e72640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf29e72640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42190/0 (socket says 172.21.15.3:42190) 2024-09-12T12:19:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf29e72640 1 -- 172.21.15.3:0/1071853233 learned_addr learned my addr 172.21.15.3:0/1071853233 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd6ffd640 1 -- 172.21.15.3:0/3740079446 >> v2:172.21.15.142:3300/0 conn(0x7fbfd80ad060 msgr2=0x7fbfd80ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd6ffd640 1 --2- 172.21.15.3:0/3740079446 >> v2:172.21.15.142:3300/0 conn(0x7fbfd80ad060 0x7fbfd80ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd6ffd640 1 -- 172.21.15.3:0/3740079446 >> v1:172.21.15.3:6789/0 conn(0x7fbfd8162800 legacy=0x7fbfd80cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd6ffd640 1 -- 172.21.15.3:0/3740079446 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80b3e70 con 0x7fbfd80b4950 2024-09-12T12:19:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fbfd7fff640 1 --2- 172.21.15.3:0/3740079446 >> v2:172.21.15.142:3300/0 conn(0x7fbfd80ad060 0x7fbfd80ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.303+0000 7fcf28e70640 1 --2- 172.21.15.3:0/1071853233 >> v2:172.21.15.142:3300/0 conn(0x7fcf1c036bb0 0x7fcf1c039c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3d6cc640 1 --2- 172.21.15.3:0/2118673943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301009c0 0x7f3a30009ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000e8e0 tx=0x7f3a2801e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5745a28bdc8764b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 Processor -- start 2024-09-12T12:19:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 -- start start 2024-09-12T12:19:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a2ffff640 1 -- 172.21.15.3:0/2118673943 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a280aa620 con 0x7f3a301009c0 2024-09-12T12:19:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a2ffff640 1 -- 172.21.15.3:0/2118673943 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a280aa7c0 con 0x7f3a301009c0 2024-09-12T12:19:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf29e72640 1 -- 172.21.15.3:0/1071853233 >> v2:172.21.15.142:3300/0 conn(0x7fcf1c036bb0 msgr2=0x7fcf1c039c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf29e72640 1 --2- 172.21.15.3:0/1071853233 >> v2:172.21.15.142:3300/0 conn(0x7fcf1c036bb0 0x7fcf1c039c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf29e72640 1 -- 172.21.15.3:0/1071853233 >> v2:172.21.15.3:3300/0 conn(0x7fcf1c036290 msgr2=0x7fcf1c036670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf29e72640 1 --2- 172.21.15.3:0/1071853233 >> v2:172.21.15.3:3300/0 conn(0x7fcf1c036290 0x7fcf1c036670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf29e72640 1 -- 172.21.15.3:0/1071853233 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c0334e0 con 0x7fcf1c03a1b0 2024-09-12T12:19:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a2ffff640 1 -- 172.21.15.3:0/2118673943 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a280aab90 con 0x7f3a301009c0 2024-09-12T12:19:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc7d4d30640 1 -- 172.21.15.3:0/2080127634 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836309508 0 0) 0x7fc7c800a6b0 con 0x7fc7c8004a40 2024-09-12T12:19:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc7d4d30640 1 -- 172.21.15.3:0/2080127634 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7bc003530 con 0x7fc7c8004a40 2024-09-12T12:19:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55aeffd640 1 --2- 172.21.15.3:0/2769395363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b015e250 0x7f55b00d40c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55a0013150 tx=0x7f55a0027470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01988c33860b750 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c2ffd640 1 --2- 172.21.15.3:0/791138973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c4070390 0x7f36c4072a90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009cb0 tx=0x7f36bc01d5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79745c0426baaba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04db59e640 1 --2- 172.21.15.3:0/1339019932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40085e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04dc076440 tx=0x7f04dc09ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4026b48a5c4980 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc7d6533640 1 -- 172.21.15.3:0/2080127634 >> v1:172.21.15.142:6789/0 conn(0x7fc7c8007410 legacy=0x7fc7c8036480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc7d6533640 1 -- 172.21.15.3:0/2080127634 >> v1:172.21.15.3:6789/0 conn(0x7fc7c8004a40 legacy=0x7fc7c8006ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc7d6533640 1 -- 172.21.15.3:0/2080127634 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c800a3d0 con 0x7fc7c8036b90 2024-09-12T12:19:59.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc19c15b240 0x7fc19c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301009c0 msgr2=0x7f3a30009ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 --2- 172.21.15.3:0/2118673943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301009c0 0x7f3a30009ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000e8e0 tx=0x7f3a2801e6e0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc19c0b5a70 0x7fc19c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55ae7fc640 1 -- 172.21.15.3:0/2769395363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a00bc070 con 0x7f55b015e250 2024-09-12T12:19:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55ae7fc640 1 -- 172.21.15.3:0/2769395363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55a00112d0 con 0x7f55b015e250 2024-09-12T12:19:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55ae7fc640 1 -- 172.21.15.3:0/2769395363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a00116a0 con 0x7f55b015e250 2024-09-12T12:19:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 -- 172.21.15.3:0/2769395363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b015e250 msgr2=0x7f55b00d40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 --2- 172.21.15.3:0/2769395363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b015e250 0x7f55b00d40c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55a0013150 tx=0x7f55a0027470 comp rx=0 tx=0).stop 2024-09-12T12:19:59.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 -- 172.21.15.3:0/2769395363 shutdown_connections 2024-09-12T12:19:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 --2- 172.21.15.3:0/2769395363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b00b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 --2- 172.21.15.3:0/2769395363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b015e250 0x7f55b00d40c0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55a0013150 tx=0x7f55a0027470 comp rx=0 tx=0).stop 2024-09-12T12:19:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 -- 172.21.15.3:0/2769395363 >> 172.21.15.3:0/2769395363 conn(0x7f55b0067e00 msgr2=0x7f55b0068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c27fc640 1 -- 172.21.15.3:0/791138973 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0b2070 con 0x7f36c4070390 2024-09-12T12:19:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c27fc640 1 -- 172.21.15.3:0/791138973 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc0ad440 con 0x7f36c4070390 2024-09-12T12:19:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c27fc640 1 -- 172.21.15.3:0/791138973 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0a5e60 con 0x7f36c4070390 2024-09-12T12:19:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c9b21640 1 -- 172.21.15.3:0/791138973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c4070390 msgr2=0x7f36c4072a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c9b21640 1 --2- 172.21.15.3:0/791138973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c4070390 0x7f36c4072a90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f36bc009cb0 tx=0x7f36bc01d5a0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc19c15b240 0x7fc19c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc19c15b240 0x7fc19c0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41008/0 (socket says 172.21.15.3:41008) 2024-09-12T12:19:59.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 -- 172.21.15.3:0/954196624 learned_addr learned my addr 172.21.15.3:0/954196624 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 --2- 172.21.15.3:0/954196624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 0x7fc19c06bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04d9d9b640 1 -- 172.21.15.3:0/1339019932 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc1249f0 con 0x7f04d4008150 2024-09-12T12:19:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04d9d9b640 1 -- 172.21.15.3:0/1339019932 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc06de50 con 0x7f04d4008150 2024-09-12T12:19:59.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04d9d9b640 1 -- 172.21.15.3:0/1339019932 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc124e30 con 0x7f04d4008150 2024-09-12T12:19:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc19c0b5ee0 con 0x7fc19c15b240 2024-09-12T12:19:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fbfd6ffd640 1 --2- 172.21.15.3:0/3740079446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80b4950 0x7fbfd80b4db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009870 tx=0x7fbfc801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ecac7deaafc177 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c9b21640 1 -- 172.21.15.3:0/791138973 shutdown_connections 2024-09-12T12:19:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c9b21640 1 --2- 172.21.15.3:0/791138973 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c4070390 0x7f36c4072a90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c9b21640 1 -- 172.21.15.3:0/791138973 >> 172.21.15.3:0/791138973 conn(0x7f36c406dcb0 msgr2=0x7f36c406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc19c0b4af0 con 0x7fc19c0b5a70 2024-09-12T12:19:59.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc19c070600 con 0x7fc19c0b46c0 2024-09-12T12:19:59.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c9b21640 1 -- 172.21.15.3:0/791138973 shutdown_connections 2024-09-12T12:19:59.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f36c9b21640 1 -- 172.21.15.3:0/791138973 wait complete. 2024-09-12T12:19:59.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 shutdown_connections 2024-09-12T12:19:59.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 --2- 172.21.15.3:0/2118673943 >> v2:172.21.15.142:3300/0 conn(0x7f3a30006fe0 0x7f3a30036030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 --2- 172.21.15.3:0/2118673943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a301009c0 0x7f3a30009ba0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 >> 172.21.15.3:0/2118673943 conn(0x7f3a3001d940 msgr2=0x7f3a300545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 -- 172.21.15.3:0/2769395363 shutdown_connections 2024-09-12T12:19:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19ad9d640 1 --2- 172.21.15.3:0/954196624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 0x7fc19c06bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fbfd67fc640 1 -- 172.21.15.3:0/3740079446 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80b0070 con 0x7fbfd80b4950 2024-09-12T12:19:59.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf29e72640 1 --2- 172.21.15.3:0/1071853233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0520f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c053dc0 tx=0x7fcf1401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2d6d899a30ac04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 shutdown_connections 2024-09-12T12:19:59.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 -- 172.21.15.3:0/1339019932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 msgr2=0x7f04d40085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 --2- 172.21.15.3:0/1339019932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40085e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f04dc076440 tx=0x7f04dc09ad10 comp rx=0 tx=0).stop 2024-09-12T12:19:59.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/2118673943 wait complete. 2024-09-12T12:19:59.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 -- 172.21.15.3:0/954196624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 msgr2=0x7fc19c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:59.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 --2- 172.21.15.3:0/954196624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 0x7fc19c06bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 -- 172.21.15.3:0/954196624 >> v2:172.21.15.142:3300/0 conn(0x7fc19c0b5a70 msgr2=0x7fc19c0b4180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T12:19:59.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 -- 172.21.15.3:0/1339019932 shutdown_connections 2024-09-12T12:19:59.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 --2- 172.21.15.3:0/1339019932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40085e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 --2- 172.21.15.3:0/1339019932 >> v2:172.21.15.3:3300/0 conn(0x7f04d40df7f0 0x7f04d400c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 -- 172.21.15.3:0/1339019932 >> 172.21.15.3:0/1339019932 conn(0x7f04d401ea60 msgr2=0x7f04d4038660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f55b5332640 1 -- 172.21.15.3:0/2769395363 wait complete. 2024-09-12T12:19:59.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf1bfff640 1 -- 172.21.15.3:0/1071853233 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf140b1070 con 0x7fcf1c03a1b0 2024-09-12T12:19:59.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf1bfff640 1 -- 172.21.15.3:0/1071853233 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf140ac640 con 0x7fcf1c03a1b0 2024-09-12T12:19:59.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fcf1bfff640 1 -- 172.21.15.3:0/1071853233 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf140aca10 con 0x7fcf1c03a1b0 2024-09-12T12:19:59.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fbfd67fc640 1 -- 172.21.15.3:0/3740079446 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc80ab600 con 0x7fbfd80b4950 2024-09-12T12:19:59.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fbfd67fc640 1 -- 172.21.15.3:0/3740079446 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80ab9d0 con 0x7fbfd80b4950 2024-09-12T12:19:59.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 -- 172.21.15.3:0/1339019932 shutdown_connections 2024-09-12T12:19:59.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7f04e08f1640 1 -- 172.21.15.3:0/1339019932 wait complete. 2024-09-12T12:19:59.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fbfddd19640 1 -- 172.21.15.3:0/3740079446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80b4950 msgr2=0x7fbfd80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3740079446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80b4950 0x7fbfd80b4db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009870 tx=0x7fbfc801c520 comp rx=0 tx=0).stop 2024-09-12T12:19:59.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- 172.21.15.3:0/3740079446 shutdown_connections 2024-09-12T12:19:59.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3740079446 >> v2:172.21.15.142:3300/0 conn(0x7fbfd80ad060 0x7fbfd80ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3740079446 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80b4950 0x7fbfd80b4db0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009870 tx=0x7fbfc801c520 comp rx=0 tx=0).stop 2024-09-12T12:19:59.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- 172.21.15.3:0/3740079446 >> 172.21.15.3:0/3740079446 conn(0x7fbfd806be90 msgr2=0x7fbfd806c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 --2- 172.21.15.3:0/954196624 >> v2:172.21.15.142:3300/0 conn(0x7fc19c0b5a70 0x7fc19c0b4180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.307+0000 7fc19a59c640 1 -- 172.21.15.3:0/954196624 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c074300 con 0x7fc19c15b240 2024-09-12T12:19:59.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 Processor -- start 2024-09-12T12:19:59.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 Processor -- start 2024-09-12T12:19:59.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 -- start start 2024-09-12T12:19:59.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 Processor -- start 2024-09-12T12:19:59.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 -- 172.21.15.3:0/1071853233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 msgr2=0x7fcf1c0520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 --2- 172.21.15.3:0/1071853233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0520f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c053dc0 tx=0x7fcf1401c540 comp rx=0 tx=0).stop 2024-09-12T12:19:59.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 -- 172.21.15.3:0/1071853233 shutdown_connections 2024-09-12T12:19:59.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 --2- 172.21.15.3:0/1071853233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0520f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 --2- 172.21.15.3:0/1071853233 >> v2:172.21.15.142:3300/0 conn(0x7fcf1c036bb0 0x7fcf1c039c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 --2- 172.21.15.3:0/1071853233 >> v2:172.21.15.3:3300/0 conn(0x7fcf1c036290 0x7fcf1c036670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 -- 172.21.15.3:0/1071853233 >> 172.21.15.3:0/1071853233 conn(0x7fcf1c01d940 msgr2=0x7fcf1c0545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 -- 172.21.15.3:0/1071853233 shutdown_connections 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 Processor -- start 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 -- start start 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36c4070390 0x7f36c416da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36c408b620 0x7f36c40cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c40b31e0 0x7f36c416bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4179740 con 0x7f36c4070390 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41795c0 con 0x7f36c408b620 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c9b21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f36c41798c0 con 0x7f36c40b31e0 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fc7d6533640 1 --2- 172.21.15.3:0/2080127634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804efc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0076440 tx=0x7fc7d0098550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba5f901d46618d64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fc7d4d30640 1 -- 172.21.15.3:0/2080127634 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d012c070 con 0x7fc7c8036b90 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fc7d4d30640 1 -- 172.21.15.3:0/2080127634 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d006de50 con 0x7fc7c8036b90 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fc7d4d30640 1 -- 172.21.15.3:0/2080127634 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d01278c0 con 0x7fc7c8036b90 2024-09-12T12:19:59.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- 172.21.15.3:0/3740079446 shutdown_connections 2024-09-12T12:19:59.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- 172.21.15.3:0/3740079446 wait complete. 2024-09-12T12:19:59.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 Processor -- start 2024-09-12T12:19:59.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36c4070390 0x7f36c416da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36c4070390 0x7f36c416da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41014/0 (socket says 172.21.15.3:41014) 2024-09-12T12:19:59.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 -- 172.21.15.3:0/656003517 learned_addr learned my addr 172.21.15.3:0/656003517 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 -- start start 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b00cf270 0x7f55b0157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b0159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b01615e0 0x7f55b00ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f55b00b7360 con 0x7f55b01615e0 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f55b015e630 con 0x7f55b00cf270 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55b5332640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f55b00b74e0 con 0x7f55b00d19c0 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b0159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b0159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42222/0 (socket says 172.21.15.3:42222) 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55a7fff640 1 -- 172.21.15.3:0/1122328030 learned_addr learned my addr 172.21.15.3:0/1122328030 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55af7fe640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b01615e0 0x7f55b00ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f55aeffd640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b00cf270 0x7f55b0157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a30006fe0 0x7f3a300f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3000b700 0x7f3a300ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a301009c0 0x7f3a30035030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3d6cc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a30006fe0 0x7f3a300f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010d390 con 0x7f3a3000b700 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3010d510 con 0x7f3a301009c0 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3e6ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3010d690 con 0x7f3a30006fe0 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3d6cc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a30006fe0 0x7f3a300f0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42228/0 (socket says 172.21.15.3:42228) 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3d6cc640 1 -- 172.21.15.3:0/1124687031 learned_addr learned my addr 172.21.15.3:0/1124687031 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3cecb640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3000b700 0x7f3a300ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 -- start start 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04d4008c70 0x7f04d404bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d40df7f0 0x7f04d4049e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4034830 con 0x7f04d4008c70 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04d400acc0 con 0x7f04d40df7f0 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04e08f1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f04d4034b10 con 0x7f04d4008150 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04d4008c70 0x7f04d404bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04d4008c70 0x7f04d404bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41020/0 (socket says 172.21.15.3:41020) 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 -- 172.21.15.3:0/1359831404 learned_addr learned my addr 172.21.15.3:0/1359831404 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04dad9d640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 -- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 msgr2=0x7f04d40cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 -- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d40df7f0 msgr2=0x7f04d4049e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d40df7f0 0x7f04d4049e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 -- 172.21.15.3:0/1359831404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d40cdf10 con 0x7f04d4008c70 2024-09-12T12:19:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04da59c640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04d4008c70 0x7f04d404bd00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b410 tx=0x7f04cc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fcf2a673640 1 -- 172.21.15.3:0/1071853233 wait complete. 2024-09-12T12:19:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 -- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c40b31e0 msgr2=0x7f36c416bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c40b31e0 0x7f36c416bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 -- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36c408b620 msgr2=0x7f36c40cdb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:19:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36c408b620 0x7f36c40cdb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 -- 172.21.15.3:0/656003517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c416dfd0 con 0x7f36c4070390 2024-09-12T12:19:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36c37fe640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36c4070390 0x7f36c416da90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d910 tx=0x7f36b800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36abfff640 1 -- 172.21.15.3:0/656003517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8014070 con 0x7f36c4070390 2024-09-12T12:19:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36abfff640 1 -- 172.21.15.3:0/656003517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b8001a80 con 0x7f36c4070390 2024-09-12T12:19:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f36abfff640 1 -- 172.21.15.3:0/656003517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8007050 con 0x7f36c4070390 2024-09-12T12:19:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fc19a59c640 1 --2- 172.21.15.3:0/954196624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c15b240 0x7fc19c0b5530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc19c071530 tx=0x7fc18c01c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e788ce5df29df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fc7d6d34640 1 -- 172.21.15.3:0/2080127634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 msgr2=0x7fc7c804efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fc7d6d34640 1 --2- 172.21.15.3:0/2080127634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804efc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0076440 tx=0x7fc7d0098550 comp rx=0 tx=0).stop 2024-09-12T12:19:59.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3decd640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a301009c0 0x7f3a30035030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- start start 2024-09-12T12:19:59.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80ad060 0x7fbfd815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd8162800 0x7fbfd809bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd815c990 0x7fbfd80adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd816f3c0 con 0x7fbfd8162800 2024-09-12T12:19:59.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd80b3560 con 0x7fbfd815c990 2024-09-12T12:19:59.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7fbfddd19640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd816f540 con 0x7fbfd80ad060 2024-09-12T12:19:59.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f04c37fe640 1 -- 172.21.15.3:0/1359831404 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc015020 con 0x7f04d4008c70 2024-09-12T12:19:59.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3cecb640 1 -- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a30006fe0 msgr2=0x7f3a300f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3cecb640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a30006fe0 0x7f3a300f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3cecb640 1 -- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a301009c0 msgr2=0x7f3a30035030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3cecb640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a301009c0 0x7f3a30035030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.311+0000 7f3a3cecb640 1 -- 172.21.15.3:0/1124687031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3004b740 con 0x7f3a3000b700 2024-09-12T12:19:59.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd815c990 0x7fbfd80adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd815c990 0x7fbfd80adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56618/0 (socket says 172.21.15.3:56618) 2024-09-12T12:19:59.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 -- 172.21.15.3:0/3212144943 learned_addr learned my addr 172.21.15.3:0/3212144943 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a7fff640 1 -- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b00cf270 msgr2=0x7f55b0157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a7fff640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b00cf270 0x7f55b0157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a7fff640 1 -- 172.21.15.3:0/1122328030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b01615e0 msgr2=0x7f55b00ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a7fff640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b01615e0 0x7f55b00ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19959a640 1 -- 172.21.15.3:0/954196624 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0b0070 con 0x7fc19c15b240 2024-09-12T12:19:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19959a640 1 -- 172.21.15.3:0/954196624 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c0ab600 con 0x7fc19c15b240 2024-09-12T12:19:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a7fff640 1 -- 172.21.15.3:0/1122328030 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b00c9650 con 0x7f55b00d19c0 2024-09-12T12:19:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19959a640 1 -- 172.21.15.3:0/954196624 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0ab9d0 con 0x7fc19c15b240 2024-09-12T12:19:59.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f3a3cecb640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3000b700 0x7f3a300ff4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400b3f0 tx=0x7f3a3400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f04e08f1640 1 -- 172.21.15.3:0/1359831404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d40358e0 con 0x7f04d4008c70 2024-09-12T12:19:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f04e08f1640 1 -- 172.21.15.3:0/1359831404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d4035ea0 con 0x7f04d4008c70 2024-09-12T12:19:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd6ffd640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd8162800 0x7fbfd809bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd77fe640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80ad060 0x7fbfd815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc7d6d34640 1 -- 172.21.15.3:0/2080127634 shutdown_connections 2024-09-12T12:19:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc7d6d34640 1 --2- 172.21.15.3:0/2080127634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804efc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc7d6d34640 1 -- 172.21.15.3:0/2080127634 >> 172.21.15.3:0/2080127634 conn(0x7fc7c801dd90 msgr2=0x7fc7c8054a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f3a2dffb640 1 -- 172.21.15.3:0/1124687031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34015020 con 0x7f3a3000b700 2024-09-12T12:19:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/1124687031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a300f14a0 con 0x7f3a3000b700 2024-09-12T12:19:59.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a7fff640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b0159750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f559800c450 tx=0x7f559800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 Processor -- start 2024-09-12T12:19:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 -- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80ad060 msgr2=0x7fbfd815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80ad060 0x7fbfd815c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 -- 172.21.15.3:0/3212144943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd8162800 msgr2=0x7fbfd809bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd8162800 0x7fbfd809bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 -- 172.21.15.3:0/3212144943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd809c8b0 con 0x7fbfd815c990 2024-09-12T12:19:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc7d6d34640 1 -- 172.21.15.3:0/2080127634 shutdown_connections 2024-09-12T12:19:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55aeffd640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b00cf270 0x7f55b0157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c15b240 msgr2=0x7fc19c0b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 --2- 172.21.15.3:0/954196624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c15b240 0x7fc19c0b5530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc19c071530 tx=0x7fc18c01c1f0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfd7fff640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd815c990 0x7fbfd80adb90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00e950 tx=0x7fbfcc00ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f04e08f1640 1 -- 172.21.15.3:0/1359831404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d40085e0 con 0x7f04d4008c70 2024-09-12T12:19:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfbffff640 1 -- 172.21.15.3:0/3212144943 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc017070 con 0x7fbfd815c990 2024-09-12T12:19:59.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfbffff640 1 -- 172.21.15.3:0/3212144943 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfcc00ca00 con 0x7fbfd815c990 2024-09-12T12:19:59.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c40ce420 con 0x7f36c4070390 2024-09-12T12:19:59.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/1124687031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a300f1a60 con 0x7f3a3000b700 2024-09-12T12:19:59.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/1124687031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a30047df0 con 0x7f3a3000b700 2024-09-12T12:19:59.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c40ce9e0 con 0x7f36c4070390 2024-09-12T12:19:59.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f04c37fe640 1 -- 172.21.15.3:0/1359831404 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04cc011830 con 0x7f04d4008c70 2024-09-12T12:19:59.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a77fe640 1 -- 172.21.15.3:0/1122328030 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980052d0 con 0x7f55b00d19c0 2024-09-12T12:19:59.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a77fe640 1 -- 172.21.15.3:0/1122328030 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5598005470 con 0x7f55b00d19c0 2024-09-12T12:19:59.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55a77fe640 1 -- 172.21.15.3:0/1122328030 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598012850 con 0x7f55b00d19c0 2024-09-12T12:19:59.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfbffff640 1 -- 172.21.15.3:0/3212144943 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00cdd0 con 0x7fbfd815c990 2024-09-12T12:19:59.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd809a430 con 0x7fbfd815c990 2024-09-12T12:19:59.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd815c650 con 0x7fbfd815c990 2024-09-12T12:19:59.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b00c9930 con 0x7f55b00d19c0 2024-09-12T12:19:59.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b00c9ef0 con 0x7f55b00d19c0 2024-09-12T12:19:59.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc7d6d34640 1 -- 172.21.15.3:0/2080127634 wait complete. 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc7d6d34640 1 Processor -- start 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 -- start start 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf1c036290 0x7fcf1c04bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c036bb0 0x7fcf1c005370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0ebeb0 con 0x7fcf1c036bb0 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0ebd30 con 0x7fcf1c036290 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf2a673640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c0ec030 con 0x7fcf1c03a1b0 2024-09-12T12:19:59.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c036bb0 0x7fcf1c005370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c036bb0 0x7fcf1c005370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41072/0 (socket says 172.21.15.3:41072) 2024-09-12T12:19:59.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 -- 172.21.15.3:0/2052011630 learned_addr learned my addr 172.21.15.3:0/2052011630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c40b35c0 con 0x7f36c4070390 2024-09-12T12:19:59.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f04c37fe640 1 -- 172.21.15.3:0/1359831404 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc011c00 con 0x7f04d4008c70 2024-09-12T12:19:59.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd80b4db0 con 0x7fbfd815c990 2024-09-12T12:19:59.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc7d6d34640 1 -- start start 2024-09-12T12:19:59.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b00d40c0 con 0x7f55b00d19c0 2024-09-12T12:19:59.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f3a2dffb640 1 -- 172.21.15.3:0/1124687031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a34011850 con 0x7f3a3000b700 2024-09-12T12:19:59.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7f3a2dffb640 1 -- 172.21.15.3:0/1124687031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a34011c40 con 0x7f3a3000b700 2024-09-12T12:19:59.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 -- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 msgr2=0x7fcf1c0e4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 -- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf1c036290 msgr2=0x7fcf1c04bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf1c036290 0x7fcf1c04bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 -- 172.21.15.3:0/2052011630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c0058b0 con 0x7fcf1c036bb0 2024-09-12T12:19:59.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fcf28e70640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c036bb0 0x7fcf1c005370 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf2406bb00 tx=0x7fcf24069820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 shutdown_connections 2024-09-12T12:19:59.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 --2- 172.21.15.3:0/954196624 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 0x7fc19c06bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 --2- 172.21.15.3:0/954196624 >> v2:172.21.15.142:3300/0 conn(0x7fc19c0b5a70 0x7fc19c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 --2- 172.21.15.3:0/954196624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c15b240 0x7fc19c0b5530 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 >> 172.21.15.3:0/954196624 conn(0x7fc19c06ed40 msgr2=0x7fc19c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.315+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 shutdown_connections 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6d34640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c8004a40 0x7fc7c8185510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6d34640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8007410 0x7fc7c818bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6d34640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6d34640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c804fa90 con 0x7fc7c8004a40 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6d34640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c804f910 con 0x7fc7c8036b90 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d5531640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8007410 0x7fc7c818bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6d34640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c80ec7d0 con 0x7fc7c8007410 2024-09-12T12:19:59.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d5531640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8007410 0x7fc7c818bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42260/0 (socket says 172.21.15.3:42260) 2024-09-12T12:19:59.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d5531640 1 -- 172.21.15.3:0/1206306476 learned_addr learned my addr 172.21.15.3:0/1206306476 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 -- 172.21.15.3:0/954196624 wait complete. 2024-09-12T12:19:59.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6533640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fbfbffff640 1 -- 172.21.15.3:0/3212144943 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbfcc004020 con 0x7fbfd815c990 2024-09-12T12:19:59.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fbfbffff640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbfc40678c0 0x7fbfc4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 Processor -- start 2024-09-12T12:19:59.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fbfbffff640 1 -- 172.21.15.3:0/3212144943 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbfcc085f30 con 0x7fbfd815c990 2024-09-12T12:19:59.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6533640 1 -- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8007410 msgr2=0x7fc7c818bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6533640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8007410 0x7fc7c818bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6533640 1 -- 172.21.15.3:0/1206306476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c8004a40 msgr2=0x7fc7c8185510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6533640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c8004a40 0x7fc7c8185510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6533640 1 -- 172.21.15.3:0/1206306476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c8187f70 con 0x7fc7c8036b90 2024-09-12T12:19:59.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fbfd77fe640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbfc40678c0 0x7fbfc4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f36abfff640 1 -- 172.21.15.3:0/656003517 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f36b8017070 con 0x7f36c4070390 2024-09-12T12:19:59.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f36abfff640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f36b0067990 0x7f36b0069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f36abfff640 1 -- 172.21.15.3:0/656003517 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f36b80865b0 con 0x7f36c4070390 2024-09-12T12:19:59.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f04c37fe640 1 -- 172.21.15.3:0/1359831404 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f04cc018020 con 0x7f04d4008c70 2024-09-12T12:19:59.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc7d6533640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804ab20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00746e0 tx=0x7fc7d00952f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 -- start start 2024-09-12T12:19:59.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 0x7fc19c15cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c0b5a70 0x7fc19c1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c15b240 0x7fc19c16b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c0cf190 con 0x7fc19c0b5a70 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c0cec10 con 0x7fc19c15b240 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19b59e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc19c067a00 con 0x7fc19c0b46c0 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f04c37fe640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04c80678c0 0x7f04c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f04c37fe640 1 -- 172.21.15.3:0/1359831404 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f04cc086080 con 0x7f04d4008c70 2024-09-12T12:19:59.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f36c2ffd640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f36b0067990 0x7f36b0069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c0b5a70 0x7fc19c1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc19ad9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c15b240 0x7fc19c16b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c0b5a70 0x7fc19c1494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41096/0 (socket says 172.21.15.3:41096) 2024-09-12T12:19:59.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 -- 172.21.15.3:0/7993658 learned_addr learned my addr 172.21.15.3:0/7993658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:19:59.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fbfd77fe640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbfc40678c0 0x7fbfc4069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfd815d000 tx=0x7fbfd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f55a77fe640 1 -- 172.21.15.3:0/1122328030 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f55980129f0 con 0x7f55b00d19c0 2024-09-12T12:19:59.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 -- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 msgr2=0x7fc19c15cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 0x7fc19c15cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 -- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c15b240 msgr2=0x7fc19c16b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c15b240 0x7fc19c16b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 -- 172.21.15.3:0/7993658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c15bba0 con 0x7fc19c0b5a70 2024-09-12T12:19:59.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f55a77fe640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5590067990 0x7f5590069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f36c2ffd640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f36b0067990 0x7f36b0069e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0af000 tx=0x7f36bc004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f55a77fe640 1 -- 172.21.15.3:0/1122328030 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f55980862e0 con 0x7f55b00d19c0 2024-09-12T12:19:59.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f04dad9d640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04c80678c0 0x7f04c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7fc199d9b640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c0b5a70 0x7fc19c1494f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b4d0 tx=0x7fc19000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f55aeffd640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5590067990 0x7f5590069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.319+0000 7f3a2dffb640 1 -- 172.21.15.3:0/1124687031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3a34018020 con 0x7f3a3000b700 2024-09-12T12:19:59.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f3a2dffb640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a1c067910 0x7f3a1c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f3a2dffb640 1 -- 172.21.15.3:0/1124687031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3a34086060 con 0x7f3a3000b700 2024-09-12T12:19:59.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f3a3d6cc640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a1c067910 0x7f3a1c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f55aeffd640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5590067990 0x7f5590069e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55a00b9000 tx=0x7f55a00b2000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f04dad9d640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04c80678c0 0x7f04c8069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04d40d38a0 tx=0x7f04d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f3a3d6cc640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a1c067910 0x7f3a1c069dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a28000c10 tx=0x7f3a280a9000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.323+0000 7f04c37fe640 1 -- 172.21.15.3:0/1359831404 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f04cc01f020 con 0x7f04d4008c70 2024-09-12T12:19:59.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fc18affd640 1 -- 172.21.15.3:0/7993658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190014070 con 0x7fc19c0b5a70 2024-09-12T12:19:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7f3a2dffb640 1 -- 172.21.15.3:0/1124687031 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3a3401f020 con 0x7f3a3000b700 2024-09-12T12:19:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fcf19ffb640 1 -- 172.21.15.3:0/2052011630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24098070 con 0x7fcf1c036bb0 2024-09-12T12:19:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c04c3a0 con 0x7fcf1c036bb0 2024-09-12T12:19:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c04c960 con 0x7fcf1c036bb0 2024-09-12T12:19:59.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c15be80 con 0x7fc19c0b5a70 2024-09-12T12:19:59.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c15c440 con 0x7fc19c0b5a70 2024-09-12T12:19:59.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c0b4180 con 0x7fc19c0b5a70 2024-09-12T12:19:59.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.327+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c039c70 con 0x7fcf1c036bb0 2024-09-12T12:19:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.331+0000 7fcf19ffb640 1 -- 172.21.15.3:0/2052011630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2406de50 con 0x7fcf1c036bb0 2024-09-12T12:19:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.331+0000 7fcf19ffb640 1 -- 172.21.15.3:0/2052011630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24075a30 con 0x7fcf1c036bb0 2024-09-12T12:19:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.331+0000 7fc18affd640 1 -- 172.21.15.3:0/7993658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc190010870 con 0x7fc19c0b5a70 2024-09-12T12:19:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.331+0000 7fc18affd640 1 -- 172.21.15.3:0/7993658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19001e060 con 0x7fc19c0b5a70 2024-09-12T12:19:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.331+0000 7f36abfff640 1 -- 172.21.15.3:0/656003517 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f36b80570d0 con 0x7f36c4070390 2024-09-12T12:19:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7c67fc640 1 -- 172.21.15.3:0/1206306476 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0135020 con 0x7fc7c8036b90 2024-09-12T12:19:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7c67fc640 1 -- 172.21.15.3:0/1206306476 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d0127d30 con 0x7fc7c8036b90 2024-09-12T12:19:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7c67fc640 1 -- 172.21.15.3:0/1206306476 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d012c040 con 0x7fc7c8036b90 2024-09-12T12:19:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7d6d34640 1 -- 172.21.15.3:0/1206306476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c8188230 con 0x7fc7c8036b90 2024-09-12T12:19:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7d6d34640 1 -- 172.21.15.3:0/1206306476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c81887f0 con 0x7fc7c8036b90 2024-09-12T12:19:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7f55a77fe640 1 -- 172.21.15.3:0/1122328030 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5598056e00 con 0x7f55b00d19c0 2024-09-12T12:19:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fbfbffff640 1 -- 172.21.15.3:0/3212144943 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbfcc056ae0 con 0x7fbfd815c990 2024-09-12T12:19:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fcf19ffb640 1 -- 172.21.15.3:0/2052011630 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcf2409b070 con 0x7fcf1c036bb0 2024-09-12T12:19:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fcf19ffb640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf080678c0 0x7fcf08069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fcf19ffb640 1 -- 172.21.15.3:0/2052011630 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcf241010f0 con 0x7fcf1c036bb0 2024-09-12T12:19:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fcf29671640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf080678c0 0x7fcf08069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc18affd640 1 -- 172.21.15.3:0/7993658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc190017070 con 0x7fc19c0b5a70 2024-09-12T12:19:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc18affd640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc17c0678c0 0x7fc17c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc18affd640 1 -- 172.21.15.3:0/7993658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc190085ee0 con 0x7fc19c0b5a70 2024-09-12T12:19:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fcf29671640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf080678c0 0x7fcf08069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c04e2f0 tx=0x7fcf20006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7d6d34640 1 -- 172.21.15.3:0/1206306476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c804efc0 con 0x7fc7c8036b90 2024-09-12T12:19:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc19a59c640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc17c0678c0 0x7fc17c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7c67fc640 1 -- 172.21.15.3:0/1206306476 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc7d0138020 con 0x7fc7c8036b90 2024-09-12T12:19:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7c67fc640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7b4067910 0x7fc7b4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:59.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.335+0000 7fc7c67fc640 1 -- 172.21.15.3:0/1206306476 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc7d019aff0 con 0x7fc7c8036b90 2024-09-12T12:19:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.339+0000 7fc19a59c640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc17c0678c0 0x7fc17c069d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0ad000 tx=0x7fc18c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.339+0000 7fc7d5d32640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7b4067910 0x7fc7b4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.339+0000 7fc7d5d32640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7b4067910 0x7fc7b4069dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000f00 tx=0x7fc7cc007890 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.339+0000 7fcf19ffb640 1 -- 172.21.15.3:0/2052011630 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcf240a2080 con 0x7fcf1c036bb0 2024-09-12T12:19:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.343+0000 7fc18affd640 1 -- 172.21.15.3:0/7993658 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc190056390 con 0x7fc19c0b5a70 2024-09-12T12:19:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.351+0000 7fc7c67fc640 1 -- 172.21.15.3:0/1206306476 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc7d013f020 con 0x7fc7c8036b90 2024-09-12T12:19:59.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.651+0000 7f04e08f1640 1 -- 172.21.15.3:0/1359831404 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f04d403b5b0 con 0x7f04d4008c70 2024-09-12T12:19:59.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.651+0000 7f04c37fe640 1 -- 172.21.15.3:0/1359831404 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f04cc056ba0 con 0x7f04d4008c70 2024-09-12T12:19:59.656 INFO:teuthology.orchestra.run.smithi003.stdout:64424509446 2024-09-12T12:19:59.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f3a3e6ce640 1 -- 172.21.15.3:0/1124687031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3a300f1c30 con 0x7f3a3000b700 2024-09-12T12:19:59.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 -- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04c80678c0 msgr2=0x7f04c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04c80678c0 0x7f04c8069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f04d40d38a0 tx=0x7f04d0009040 comp rx=0 tx=0).stop 2024-09-12T12:19:59.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 -- 172.21.15.3:0/1359831404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04d4008c70 msgr2=0x7f04d404bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04d4008c70 0x7f04d404bd00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b410 tx=0x7f04cc00b8d0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f3a2dffb640 1 -- 172.21.15.3:0/1124687031 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a34056c80 con 0x7f3a3000b700 2024-09-12T12:19:59.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 -- 172.21.15.3:0/1359831404 shutdown_connections 2024-09-12T12:19:59.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04c80678c0 0x7f04c8069d80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04d40df7f0 0x7f04d4049e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04d4008c70 0x7f04d404bd00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 --2- 172.21.15.3:0/1359831404 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04d4008150 0x7f04d40cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 -- 172.21.15.3:0/1359831404 >> 172.21.15.3:0/1359831404 conn(0x7f04d401ea60 msgr2=0x7f04d4053e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.663 INFO:teuthology.orchestra.run.smithi003.stdout:60129542149 2024-09-12T12:19:59.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 -- 172.21.15.3:0/1359831404 shutdown_connections 2024-09-12T12:19:59.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.659+0000 7f04c17fa640 1 -- 172.21.15.3:0/1359831404 wait complete. 2024-09-12T12:19:59.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f36c416a570 con 0x7f36c4070390 2024-09-12T12:19:59.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36abfff640 1 -- 172.21.15.3:0/656003517 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36b80569d0 con 0x7f36c4070390 2024-09-12T12:19:59.665 INFO:teuthology.orchestra.run.smithi003.stdout:60129542148 2024-09-12T12:19:59.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 -- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a1c067910 msgr2=0x7f3a1c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a1c067910 0x7f3a1c069dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a28000c10 tx=0x7f3a280a9000 comp rx=0 tx=0).stop 2024-09-12T12:19:59.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f36b0067990 msgr2=0x7f36b0069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f36b0067990 0x7f36b0069e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0af000 tx=0x7f36bc004ac0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36c4070390 msgr2=0x7f36c416da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36c4070390 0x7f36c416da90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d910 tx=0x7f36b800ddd0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 -- 172.21.15.3:0/1124687031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3000b700 msgr2=0x7f3a300ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3000b700 0x7f3a300ff4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400b3f0 tx=0x7f3a3400b8b0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 -- 172.21.15.3:0/1124687031 shutdown_connections 2024-09-12T12:19:59.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a1c067910 0x7f3a1c069dd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 shutdown_connections 2024-09-12T12:19:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f36b0067990 0x7f36b0069e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36c40b31e0 0x7f36c416bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36c408b620 0x7f36c40cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 --2- 172.21.15.3:0/656003517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f36c4070390 0x7f36c416da90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 >> 172.21.15.3:0/656003517 conn(0x7f36c406dcb0 msgr2=0x7f36c4168810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a301009c0 0x7f3a30035030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a3000b700 0x7f3a300ff4d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 --2- 172.21.15.3:0/1124687031 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a30006fe0 0x7f3a300f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 -- 172.21.15.3:0/1124687031 >> 172.21.15.3:0/1124687031 conn(0x7f3a3001d940 msgr2=0x7f3a30039260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 shutdown_connections 2024-09-12T12:19:59.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 -- 172.21.15.3:0/1124687031 shutdown_connections 2024-09-12T12:19:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f36c9b21640 1 -- 172.21.15.3:0/656003517 wait complete. 2024-09-12T12:19:59.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.663+0000 7f3a237fe640 1 -- 172.21.15.3:0/1124687031 wait complete. 2024-09-12T12:19:59.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcf1c04b390 con 0x7fcf1c036bb0 2024-09-12T12:19:59.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7fcf19ffb640 1 -- 172.21.15.3:0/2052011630 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf24090070 con 0x7fcf1c036bb0 2024-09-12T12:19:59.674 INFO:teuthology.orchestra.run.smithi003.stdout:60129542149 2024-09-12T12:19:59.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbfd80aedb0 con 0x7fbfd815c990 2024-09-12T12:19:59.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f55b0068430 con 0x7f55b00d19c0 2024-09-12T12:19:59.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc19c0b5700 con 0x7fc19c0b5a70 2024-09-12T12:19:59.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7fc18affd640 1 -- 172.21.15.3:0/7993658 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc19005a690 con 0x7fc19c0b5a70 2024-09-12T12:19:59.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7f55a77fe640 1 -- 172.21.15.3:0/1122328030 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5598056700 con 0x7f55b00d19c0 2024-09-12T12:19:59.676 INFO:teuthology.orchestra.run.smithi003.stdout:60129542150 2024-09-12T12:19:59.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.671+0000 7fbfbffff640 1 -- 172.21.15.3:0/3212144943 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfcc0563e0 con 0x7fbfd815c990 2024-09-12T12:19:59.677 INFO:teuthology.orchestra.run.smithi003.stdout:60129542148 2024-09-12T12:19:59.677 INFO:teuthology.orchestra.run.smithi003.stdout:60129542149 2024-09-12T12:19:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf080678c0 msgr2=0x7fcf08069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf080678c0 0x7fcf08069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c04e2f0 tx=0x7fcf20006bf0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c036bb0 msgr2=0x7fcf1c005370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c036bb0 0x7fcf1c005370 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf2406bb00 tx=0x7fcf24069820 comp rx=0 tx=0).stop 2024-09-12T12:19:59.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 shutdown_connections 2024-09-12T12:19:59.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf080678c0 0x7fcf08069d80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1c03a1b0 0x7fcf1c0e4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf1c036bb0 0x7fcf1c005370 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 --2- 172.21.15.3:0/2052011630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf1c036290 0x7fcf1c04bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc17c0678c0 msgr2=0x7fc17c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc17c0678c0 0x7fc17c069d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0ad000 tx=0x7fc18c004740 comp rx=0 tx=0).stop 2024-09-12T12:19:59.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c0b5a70 msgr2=0x7fc19c1494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c0b5a70 0x7fc19c1494f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b4d0 tx=0x7fc19000b990 comp rx=0 tx=0).stop 2024-09-12T12:19:59.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 >> 172.21.15.3:0/2052011630 conn(0x7fcf1c01d940 msgr2=0x7fcf1c035750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 shutdown_connections 2024-09-12T12:19:59.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc17c0678c0 0x7fc17c069d80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c15b240 0x7fc19c16b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc19c0b5a70 0x7fc19c1494f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 --2- 172.21.15.3:0/7993658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c0b46c0 0x7fc19c15cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5590067990 msgr2=0x7f5590069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 >> 172.21.15.3:0/7993658 conn(0x7fc19c06ed40 msgr2=0x7fc19c0b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5590067990 0x7f5590069e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55a00b9000 tx=0x7f55a00b2000 comp rx=0 tx=0).stop 2024-09-12T12:19:59.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbfc40678c0 msgr2=0x7fbfc4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbfc40678c0 0x7fbfc4069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfd815d000 tx=0x7fbfd0009040 comp rx=0 tx=0).stop 2024-09-12T12:19:59.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd815c990 msgr2=0x7fbfd80adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd815c990 0x7fbfd80adb90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00e950 tx=0x7fbfcc00ee10 comp rx=0 tx=0).stop 2024-09-12T12:19:59.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 msgr2=0x7f55b0159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b0159750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f559800c450 tx=0x7f559800c910 comp rx=0 tx=0).stop 2024-09-12T12:19:59.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 shutdown_connections 2024-09-12T12:19:59.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 shutdown_connections 2024-09-12T12:19:59.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbfc40678c0 0x7fbfc4069d80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfd815c990 0x7fbfd80adb90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfd8162800 0x7fbfd809bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 --2- 172.21.15.3:0/3212144943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfd80ad060 0x7fbfd815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 >> 172.21.15.3:0/3212144943 conn(0x7fbfd806be90 msgr2=0x7fbfd80b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fc19b59e640 1 -- 172.21.15.3:0/7993658 wait complete. 2024-09-12T12:19:59.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 shutdown_connections 2024-09-12T12:19:59.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5590067990 0x7f5590069e50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55b01615e0 0x7f55b00ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f55b00d19c0 0x7f55b0159750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 --2- 172.21.15.3:0/1122328030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55b00cf270 0x7f55b0157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 >> 172.21.15.3:0/1122328030 conn(0x7f55b0067e00 msgr2=0x7f55b00b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 shutdown_connections 2024-09-12T12:19:59.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 shutdown_connections 2024-09-12T12:19:59.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fbfddd19640 1 -- 172.21.15.3:0/3212144943 wait complete. 2024-09-12T12:19:59.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7f55b5332640 1 -- 172.21.15.3:0/1122328030 wait complete. 2024-09-12T12:19:59.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 shutdown_connections 2024-09-12T12:19:59.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.675+0000 7fcf2a673640 1 -- 172.21.15.3:0/2052011630 wait complete. 2024-09-12T12:19:59.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.687+0000 7fc7d6d34640 1 -- 172.21.15.3:0/1206306476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc7c8008f30 con 0x7fc7c8036b90 2024-09-12T12:19:59.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.687+0000 7fc7c67fc640 1 -- 172.21.15.3:0/1206306476 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7c80df090 con 0x7fc7c8036b90 2024-09-12T12:19:59.692 INFO:teuthology.orchestra.run.smithi003.stdout:60129542150 2024-09-12T12:19:59.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 -- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7b4067910 msgr2=0x7fc7b4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7b4067910 0x7fc7b4069dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000f00 tx=0x7fc7cc007890 comp rx=0 tx=0).stop 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 -- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 msgr2=0x7fc7c804ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804ab20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00746e0 tx=0x7fc7d00952f0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 -- 172.21.15.3:0/1206306476 shutdown_connections 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7b4067910 0x7fc7b4069dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7c8036b90 0x7fc7c804ab20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc7c8007410 0x7fc7c818bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 --2- 172.21.15.3:0/1206306476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7c8004a40 0x7fc7c8185510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 -- 172.21.15.3:0/1206306476 >> 172.21.15.3:0/1206306476 conn(0x7fc7c801dd90 msgr2=0x7fc7c804fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 -- 172.21.15.3:0/1206306476 shutdown_connections 2024-09-12T12:19:59.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:19:59.691+0000 7fc7b3fff640 1 -- 172.21.15.3:0/1206306476 wait complete. 2024-09-12T12:19:59.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.3 2024-09-12T12:19:59.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.2 2024-09-12T12:19:59.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.4 2024-09-12T12:19:59.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.0 2024-09-12T12:19:59.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.1 2024-09-12T12:19:59.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.7 2024-09-12T12:19:59.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.5 2024-09-12T12:19:59.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.6 2024-09-12T12:20:00.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-12T12:20:00.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-12T12:20:00.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-12T12:20:00.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-12T12:20:00.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-12T12:20:00.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-12T12:20:00.717 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-12T12:20:00.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-12T12:20:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.879+0000 7fe5bd18c640 1 Processor -- start 2024-09-12T12:20:00.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.879+0000 7fe5bd18c640 1 -- start start 2024-09-12T12:20:00.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5bd18c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5b8070200 0x7fe5b815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5bd18c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5b8157de0 0x7fe5b80b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5bd18c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b8161130 con 0x7fe5b80b4fc0 2024-09-12T12:20:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5bd18c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe5b80b5410 con 0x7fe5b8070200 2024-09-12T12:20:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5bd18c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe5b80b38a0 con 0x7fe5b8157de0 2024-09-12T12:20:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b759e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe5b80b4fc0 0x7fe5b815dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36164/0 (socket says 172.21.15.3:36164) 2024-09-12T12:20:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b759e640 1 -- 172.21.15.3:0/891012006 learned_addr learned my addr 172.21.15.3:0/891012006 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b659c640 1 --2- 172.21.15.3:0/891012006 >> v2:172.21.15.142:3300/0 conn(0x7fe5b8157de0 0x7fe5b80b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b6d9d640 1 --2- 172.21.15.3:0/891012006 >> v2:172.21.15.3:3300/0 conn(0x7fe5b8070200 0x7fe5b815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b6d9d640 1 -- 172.21.15.3:0/891012006 >> v2:172.21.15.142:3300/0 conn(0x7fe5b8157de0 msgr2=0x7fe5b80b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b6d9d640 1 --2- 172.21.15.3:0/891012006 >> v2:172.21.15.142:3300/0 conn(0x7fe5b8157de0 0x7fe5b80b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b6d9d640 1 -- 172.21.15.3:0/891012006 >> v1:172.21.15.3:6789/0 conn(0x7fe5b80b4fc0 legacy=0x7fe5b815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.883+0000 7fe5b6d9d640 1 -- 172.21.15.3:0/891012006 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b80b3b80 con 0x7fe5b8070200 2024-09-12T12:20:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.887+0000 7fe5b6d9d640 1 --2- 172.21.15.3:0/891012006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b815a9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5b806d920 tx=0x7fe5ac01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158c2b3874a2bf96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5b5d9b640 1 -- 172.21.15.3:0/891012006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0b0070 con 0x7fe5b8070200 2024-09-12T12:20:00.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5b5d9b640 1 -- 172.21.15.3:0/891012006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ac0ab600 con 0x7fe5b8070200 2024-09-12T12:20:00.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5b5d9b640 1 -- 172.21.15.3:0/891012006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0ab9d0 con 0x7fe5b8070200 2024-09-12T12:20:00.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 -- 172.21.15.3:0/891012006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 msgr2=0x7fe5b815a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/891012006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b815a9e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5b806d920 tx=0x7fe5ac01c310 comp rx=0 tx=0).stop 2024-09-12T12:20:00.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 -- 172.21.15.3:0/891012006 shutdown_connections 2024-09-12T12:20:00.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/891012006 >> v2:172.21.15.142:3300/0 conn(0x7fe5b8157de0 0x7fe5b80b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/891012006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b815a9e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 -- 172.21.15.3:0/891012006 >> 172.21.15.3:0/891012006 conn(0x7fe5b806dcb0 msgr2=0x7fe5b806e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 -- 172.21.15.3:0/891012006 shutdown_connections 2024-09-12T12:20:00.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 -- 172.21.15.3:0/891012006 wait complete. 2024-09-12T12:20:00.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 Processor -- start 2024-09-12T12:20:00.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.899+0000 7fe5bd18c640 1 -- start start 2024-09-12T12:20:00.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b8149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b80b4fc0 0x7fe5b8144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8157de0 0x7fe5b8145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b80b31e0 con 0x7fe5b8070200 2024-09-12T12:20:00.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b80b4140 con 0x7fe5b80b4fc0 2024-09-12T12:20:00.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b80b3360 con 0x7fe5b8157de0 2024-09-12T12:20:00.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b8149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b8149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41114/0 (socket says 172.21.15.3:41114) 2024-09-12T12:20:00.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 -- 172.21.15.3:0/2352023461 learned_addr learned my addr 172.21.15.3:0/2352023461 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b659c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b80b4fc0 0x7fe5b8144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 -- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8157de0 msgr2=0x7fe5b8145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8157de0 0x7fe5b8145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 -- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b80b4fc0 msgr2=0x7fe5b8144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b80b4fc0 0x7fe5b8144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 -- 172.21.15.3:0/2352023461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b8146e30 con 0x7fe5b8070200 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5b6d9d640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b8149820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac019f40 tx=0x7fe5ac0190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe59f7fe640 1 -- 172.21.15.3:0/2352023461 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0b0040 con 0x7fe5b8070200 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe59f7fe640 1 -- 172.21.15.3:0/2352023461 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ac0b9070 con 0x7fe5b8070200 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe59f7fe640 1 -- 172.21.15.3:0/2352023461 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0aab30 con 0x7fe5b8070200 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b81436f0 con 0x7fe5b8070200 2024-09-12T12:20:00.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.903+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b813dca0 con 0x7fe5b8070200 2024-09-12T12:20:00.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.907+0000 7fe59f7fe640 1 -- 172.21.15.3:0/2352023461 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe5ac0ab4c0 con 0x7fe5b8070200 2024-09-12T12:20:00.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.907+0000 7fe59f7fe640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe5a0067a10 0x7fe5a0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.907+0000 7fe59f7fe640 1 -- 172.21.15.3:0/2352023461 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe5ac11fb50 con 0x7fe5b8070200 2024-09-12T12:20:00.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.907+0000 7fe5b659c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe5a0067a10 0x7fe5a0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.907+0000 7fe5b659c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe5a0067a10 0x7fe5a0069ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5b81442c0 tx=0x7fe5b000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.907+0000 7fe59d7fa640 1 -- 172.21.15.3:0/2352023461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5980052a0 con 0x7fe5b8070200 2024-09-12T12:20:00.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.915+0000 7fe59f7fe640 1 -- 172.21.15.3:0/2352023461 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe5ac015f90 con 0x7fe5b8070200 2024-09-12T12:20:00.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fde1c0640 1 Processor -- start 2024-09-12T12:20:00.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7168dfd640 1 Processor -- start 2024-09-12T12:20:00.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7168dfd640 1 -- start start 2024-09-12T12:20:00.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fde1c0640 1 -- start start 2024-09-12T12:20:00.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7168dfd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716415ce40 0x7f71640b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7168dfd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7164073030 0x7f716406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fde1c0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0fd81627e0 0x7f0fd806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fde1c0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd80c5af0 0x7f0fd80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fde1c0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd809b940 con 0x7f0fd806d320 2024-09-12T12:20:00.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fde1c0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd809bc20 con 0x7f0fd81627e0 2024-09-12T12:20:00.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fde1c0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd80b1900 con 0x7f0fd80c5af0 2024-09-12T12:20:00.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7168dfd640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71640b3e40 con 0x7f71640b3890 2024-09-12T12:20:00.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e1de52640 1 Processor -- start 2024-09-12T12:20:00.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e1de52640 1 -- start start 2024-09-12T12:20:00.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd6ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0fd806d320 0x7f0fd80c53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36168/0 (socket says 172.21.15.3:36168) 2024-09-12T12:20:00.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd6ffd640 1 -- 172.21.15.3:0/2450939262 learned_addr learned my addr 172.21.15.3:0/2450939262 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:00.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd77fe640 1 --2- 172.21.15.3:0/2450939262 >> v2:172.21.15.142:3300/0 conn(0x7f0fd81627e0 0x7f0fd806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f71637fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716415ce40 0x7f71640b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f71637fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716415ce40 0x7f71640b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42302/0 (socket says 172.21.15.3:42302) 2024-09-12T12:20:00.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f71637fe640 1 -- 172.21.15.3:0/2464521778 learned_addr learned my addr 172.21.15.3:0/2464521778 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd7fff640 1 --2- 172.21.15.3:0/2450939262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd80c5af0 0x7f0fd80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd67fc640 1 -- 172.21.15.3:0/2450939262 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1794226951 0 0) 0x7f0fd809b940 con 0x7f0fd806d320 2024-09-12T12:20:00.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd67fc640 1 -- 172.21.15.3:0/2450939262 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fc0003530 con 0x7f0fd806d320 2024-09-12T12:20:00.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7fea1783d640 1 Processor -- start 2024-09-12T12:20:00.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7168dfd640 1 -- 172.21.15.3:0/2464521778 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7164072270 con 0x7f7164073030 2024-09-12T12:20:00.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7168dfd640 1 -- 172.21.15.3:0/2464521778 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7164072550 con 0x7f716415ce40 2024-09-12T12:20:00.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7162ffd640 1 --2- 172.21.15.3:0/2464521778 >> v2:172.21.15.3:3300/0 conn(0x7f7164073030 0x7f716406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7fea1783d640 1 -- start start 2024-09-12T12:20:00.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7162ffd640 1 -- 172.21.15.3:0/2464521778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716415ce40 msgr2=0x7f71640b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:00.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7162ffd640 1 --2- 172.21.15.3:0/2464521778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716415ce40 0x7f71640b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7162ffd640 1 -- 172.21.15.3:0/2464521778 >> v1:172.21.15.3:6789/0 conn(0x7f71640b3890 legacy=0x7f716406f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f71627fc640 1 -- 172.21.15.3:0/2464521778 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224425900 0 0) 0x7f71640b3e40 con 0x7f71640b3890 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f7162ffd640 1 -- 172.21.15.3:0/2464521778 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7164072830 con 0x7f7164073030 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f71637fe640 1 --2- 172.21.15.3:0/2464521778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716415ce40 0x7f71640b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e1de52640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e180d0b80 0x7f0e180b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e1de52640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1806e270 con 0x7f0e1806dd40 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e1de52640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1806e550 con 0x7f0e1815a210 2024-09-12T12:20:00.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e1de52640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1806fc40 con 0x7f0e180d0b80 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e17fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0e1806dd40 0x7f0e1815e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36186/0 (socket says 172.21.15.3:36186) 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e17fff640 1 -- 172.21.15.3:0/99094862 learned_addr learned my addr 172.21.15.3:0/99094862 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0e16ffd640 1 --2- 172.21.15.3:0/99094862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e180d0b80 0x7f0e180b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7fea1783d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea100b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7fea1783d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fea100ad060 0x7fea100ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7fea155d9640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea10162800 0x7fea100cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36192/0 (socket says 172.21.15.3:36192) 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7fea155d9640 1 -- 172.21.15.3:0/1305258498 learned_addr learned my addr 172.21.15.3:0/1305258498 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea100b38b0 con 0x7fea10162800 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fea100b44c0 con 0x7fea100ad060 2024-09-12T12:20:00.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea100b3b90 con 0x7fea100b4950 2024-09-12T12:20:00.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea14dd8640 1 --2- 172.21.15.3:0/1305258498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea100b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea15dda640 1 --2- 172.21.15.3:0/1305258498 >> v2:172.21.15.142:3300/0 conn(0x7fea100ad060 0x7fea100ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0e167fc640 1 -- 172.21.15.3:0/99094862 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147679193 0 0) 0x7f0e1806e550 con 0x7f0e1815a210 2024-09-12T12:20:00.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0e167fc640 1 -- 172.21.15.3:0/99094862 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e00004640 con 0x7f0e1815a210 2024-09-12T12:20:00.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea07fff640 1 -- 172.21.15.3:0/1305258498 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2643156620 0 0) 0x7fea100b38b0 con 0x7fea10162800 2024-09-12T12:20:00.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea07fff640 1 -- 172.21.15.3:0/1305258498 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9fc003530 con 0x7fea10162800 2024-09-12T12:20:00.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0e167fc640 1 -- 172.21.15.3:0/99094862 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3584880266 0 0) 0x7f0e1806e270 con 0x7f0e1806dd40 2024-09-12T12:20:00.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0e167fc640 1 -- 172.21.15.3:0/99094862 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e1806e550 con 0x7f0e1806dd40 2024-09-12T12:20:00.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea14dd8640 1 -- 172.21.15.3:0/1305258498 >> v2:172.21.15.142:3300/0 conn(0x7fea100ad060 msgr2=0x7fea100ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:00.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea14dd8640 1 --2- 172.21.15.3:0/1305258498 >> v2:172.21.15.142:3300/0 conn(0x7fea100ad060 0x7fea100ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea14dd8640 1 -- 172.21.15.3:0/1305258498 >> v1:172.21.15.3:6789/0 conn(0x7fea10162800 legacy=0x7fea100cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea14dd8640 1 -- 172.21.15.3:0/1305258498 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea100b3e70 con 0x7fea100b4950 2024-09-12T12:20:00.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd7fff640 1 -- 172.21.15.3:0/2450939262 >> v2:172.21.15.142:3300/0 conn(0x7f0fd81627e0 msgr2=0x7f0fd806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:00.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd7fff640 1 --2- 172.21.15.3:0/2450939262 >> v2:172.21.15.142:3300/0 conn(0x7f0fd81627e0 0x7f0fd806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd7fff640 1 -- 172.21.15.3:0/2450939262 >> v1:172.21.15.3:6789/0 conn(0x7f0fd806d320 legacy=0x7f0fd80c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.927+0000 7f0fd7fff640 1 -- 172.21.15.3:0/2450939262 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd80b1be0 con 0x7f0fd80c5af0 2024-09-12T12:20:00.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7b58e47640 1 Processor -- start 2024-09-12T12:20:00.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fd7fff640 1 --2- 172.21.15.3:0/2450939262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd80c5af0 0x7f0fd80b23e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009d90 tx=0x7f0fd001c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0eb1e875499a0de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7b58e47640 1 -- start start 2024-09-12T12:20:00.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7162ffd640 1 --2- 172.21.15.3:0/2464521778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7164073030 0x7f716406da30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7158009a20 tx=0x7f715801c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00e859f9296a482 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0e16ffd640 1 -- 172.21.15.3:0/99094862 >> v1:172.21.15.142:6789/0 conn(0x7f0e1815a210 legacy=0x7f0e180d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0e16ffd640 1 -- 172.21.15.3:0/99094862 >> v1:172.21.15.3:6789/0 conn(0x7f0e1806dd40 legacy=0x7f0e1815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0e16ffd640 1 -- 172.21.15.3:0/99094862 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1806e830 con 0x7f0e180d0b80 2024-09-12T12:20:00.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea14dd8640 1 --2- 172.21.15.3:0/1305258498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea100b4db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea000097c0 tx=0x7fea0001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812278dd77d256e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fd67fc640 1 -- 172.21.15.3:0/2450939262 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00b2070 con 0x7f0fd80c5af0 2024-09-12T12:20:00.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fd67fc640 1 -- 172.21.15.3:0/2450939262 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd00a5a40 con 0x7f0fd80c5af0 2024-09-12T12:20:00.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fd67fc640 1 -- 172.21.15.3:0/2450939262 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00a5e50 con 0x7f0fd80c5af0 2024-09-12T12:20:00.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea07fff640 1 -- 172.21.15.3:0/1305258498 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000b0070 con 0x7fea100b4950 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea07fff640 1 -- 172.21.15.3:0/1305258498 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea000ab600 con 0x7fea100b4950 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7fea07fff640 1 -- 172.21.15.3:0/1305258498 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000aba10 con 0x7fea100b4950 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 -- 172.21.15.3:0/2450939262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd80c5af0 msgr2=0x7f0fd80b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/2450939262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd80c5af0 0x7f0fd80b23e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009d90 tx=0x7f0fd001c960 comp rx=0 tx=0).stop 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 -- 172.21.15.3:0/2450939262 shutdown_connections 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/2450939262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd80c5af0 0x7f0fd80b23e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/2450939262 >> v2:172.21.15.142:3300/0 conn(0x7f0fd81627e0 0x7f0fd806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 -- 172.21.15.3:0/2450939262 >> 172.21.15.3:0/2450939262 conn(0x7f0fd80b3f00 msgr2=0x7f0fd80b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7b58e47640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7b540b4070 0x7f7b540b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7b58e47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b540ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7b58e47640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b5415c6a0 con 0x7f7b54073cb0 2024-09-12T12:20:00.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7b58e47640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7b540cc8d0 con 0x7f7b540b4070 2024-09-12T12:20:00.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f7b58e47640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7b540b3220 con 0x7f7b5415b890 2024-09-12T12:20:00.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 -- 172.21.15.3:0/2450939262 shutdown_connections 2024-09-12T12:20:00.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f0fde1c0640 1 -- 172.21.15.3:0/2450939262 wait complete. 2024-09-12T12:20:00.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f71627fc640 1 -- 172.21.15.3:0/2464521778 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580b0600 con 0x7f7164073030 2024-09-12T12:20:00.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f71627fc640 1 -- 172.21.15.3:0/2464521778 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71580b07a0 con 0x7f7164073030 2024-09-12T12:20:00.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.931+0000 7f71627fc640 1 -- 172.21.15.3:0/2464521778 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580b0b70 con 0x7f7164073030 2024-09-12T12:20:00.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b52d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b540ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b51d9b640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f7b54073cb0 0x7f7b5415d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:37920/0 (socket says 172.21.15.3:37920) 2024-09-12T12:20:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b51d9b640 1 -- 172.21.15.3:0/4153588783 learned_addr learned my addr 172.21.15.3:0/4153588783 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b5259c640 1 --2- 172.21.15.3:0/4153588783 >> v2:172.21.15.142:3300/0 conn(0x7f7b540b4070 0x7f7b540b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 msgr2=0x7fea100b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 --2- 172.21.15.3:0/1305258498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea100b4db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea000097c0 tx=0x7fea0001c480 comp rx=0 tx=0).stop 2024-09-12T12:20:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 shutdown_connections 2024-09-12T12:20:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 --2- 172.21.15.3:0/1305258498 >> v2:172.21.15.142:3300/0 conn(0x7fea100ad060 0x7fea100ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 --2- 172.21.15.3:0/1305258498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea100b4db0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 >> 172.21.15.3:0/1305258498 conn(0x7fea1006be90 msgr2=0x7fea1006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 shutdown_connections 2024-09-12T12:20:00.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b5159a640 1 -- 172.21.15.3:0/4153588783 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2073185474 0 0) 0x7f7b5415c6a0 con 0x7f7b54073cb0 2024-09-12T12:20:00.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b5159a640 1 -- 172.21.15.3:0/4153588783 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b3c003530 con 0x7f7b54073cb0 2024-09-12T12:20:00.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b52d9d640 1 -- 172.21.15.3:0/4153588783 >> v2:172.21.15.142:3300/0 conn(0x7f7b540b4070 msgr2=0x7f7b540b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b52d9d640 1 --2- 172.21.15.3:0/4153588783 >> v2:172.21.15.142:3300/0 conn(0x7f7b540b4070 0x7f7b540b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b52d9d640 1 -- 172.21.15.3:0/4153588783 >> v1:172.21.15.142:6789/0 conn(0x7f7b54073cb0 legacy=0x7f7b5415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b52d9d640 1 -- 172.21.15.3:0/4153588783 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b540b3500 con 0x7f7b5415b890 2024-09-12T12:20:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 -- 172.21.15.3:0/2464521778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7164073030 msgr2=0x7f716406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 --2- 172.21.15.3:0/2464521778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7164073030 0x7f716406da30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7158009a20 tx=0x7f715801c6d0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 Processor -- start 2024-09-12T12:20:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- 172.21.15.3:0/1305258498 wait complete. 2024-09-12T12:20:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 Processor -- start 2024-09-12T12:20:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 -- 172.21.15.3:0/2464521778 shutdown_connections 2024-09-12T12:20:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 --2- 172.21.15.3:0/2464521778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7164073030 0x7f716406da30 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7158009a20 tx=0x7f715801c6d0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 --2- 172.21.15.3:0/2464521778 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f716415ce40 0x7f71640b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 -- 172.21.15.3:0/2464521778 >> 172.21.15.3:0/2464521778 conn(0x7f71640b1d40 msgr2=0x7f71640b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e16ffd640 1 --2- 172.21.15.3:0/99094862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e180d0b80 0x7f0e180b4c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009770 tx=0x7f0e0c01c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1639c5327c9c308c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e167fc640 1 -- 172.21.15.3:0/99094862 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0b0070 con 0x7f0e180d0b80 2024-09-12T12:20:00.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e167fc640 1 -- 172.21.15.3:0/99094862 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0ab400 con 0x7f0e180d0b80 2024-09-12T12:20:00.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e167fc640 1 -- 172.21.15.3:0/99094862 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10004fe0 con 0x7f0e180d0b80 2024-09-12T12:20:00.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 -- start start 2024-09-12T12:20:00.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd806d320 0x7f0fd815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd80c5af0 0x7f0fd80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd81627e0 0x7f0fd815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd80b34c0 con 0x7f0fd806d320 2024-09-12T12:20:00.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd80b3340 con 0x7f0fd80c5af0 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd80b3640 con 0x7f0fd81627e0 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd806d320 0x7f0fd815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd806d320 0x7f0fd815aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41132/0 (socket says 172.21.15.3:41132) 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 -- 172.21.15.3:0/499840391 learned_addr learned my addr 172.21.15.3:0/499840391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd6ffd640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd80c5af0 0x7f0fd80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd7fff640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd81627e0 0x7f0fd815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 -- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd81627e0 msgr2=0x7f0fd815ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd81627e0 0x7f0fd815ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 -- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd80c5af0 msgr2=0x7f0fd80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd80c5af0 0x7f0fd80ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 -- 172.21.15.3:0/499840391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd80caeb0 con 0x7f0fd806d320 2024-09-12T12:20:00.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd6ffd640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd80c5af0 0x7f0fd80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- start start 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea100ad060 0x7fea1009c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea1015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10162800 0x7fea101594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea1016f3c0 con 0x7fea10162800 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea100b3560 con 0x7fea100ad060 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea1783d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea1016f540 con 0x7fea100b4950 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10162800 0x7fea101594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10162800 0x7fea101594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41146/0 (socket says 172.21.15.3:41146) 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 -- 172.21.15.3:0/752952059 learned_addr learned my addr 172.21.15.3:0/752952059 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea14dd8640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea1015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea155d9640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea100ad060 0x7fea1009c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e1de52640 1 -- 172.21.15.3:0/99094862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e180d0b80 msgr2=0x7f0e180b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e1de52640 1 --2- 172.21.15.3:0/99094862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e180d0b80 0x7f0e180b4c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009770 tx=0x7f0e0c01c2b0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fd77fe640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd806d320 0x7f0fd815aa20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800e870 tx=0x7f0fc800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e1de52640 1 -- 172.21.15.3:0/99094862 shutdown_connections 2024-09-12T12:20:00.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e1de52640 1 --2- 172.21.15.3:0/99094862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e180d0b80 0x7f0e180b4c40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e1de52640 1 -- 172.21.15.3:0/99094862 >> 172.21.15.3:0/99094862 conn(0x7f0e1806be90 msgr2=0x7f0e1806c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e1de52640 1 -- 172.21.15.3:0/99094862 shutdown_connections 2024-09-12T12:20:00.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0e1de52640 1 -- 172.21.15.3:0/99094862 wait complete. 2024-09-12T12:20:00.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 -- 172.21.15.3:0/2464521778 shutdown_connections 2024-09-12T12:20:00.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b52d9d640 1 --2- 172.21.15.3:0/4153588783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b540ad1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009d90 tx=0x7f7b4c01c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b80d62c60d70c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7168dfd640 1 -- 172.21.15.3:0/2464521778 wait complete. 2024-09-12T12:20:00.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fbffff640 1 -- 172.21.15.3:0/499840391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8017070 con 0x7f0fd806d320 2024-09-12T12:20:00.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fbffff640 1 -- 172.21.15.3:0/499840391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc800c940 con 0x7f0fd806d320 2024-09-12T12:20:00.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd80cb170 con 0x7f0fd806d320 2024-09-12T12:20:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fbffff640 1 -- 172.21.15.3:0/499840391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800ccf0 con 0x7f0fd806d320 2024-09-12T12:20:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f7b5159a640 1 -- 172.21.15.3:0/4153588783 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0b2070 con 0x7f7b5415b890 2024-09-12T12:20:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd80cb730 con 0x7f0fd806d320 2024-09-12T12:20:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 -- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 msgr2=0x7fea1015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea1015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 -- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea100ad060 msgr2=0x7fea1009c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea100ad060 0x7fea1009c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.935+0000 7fea15dda640 1 -- 172.21.15.3:0/752952059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1015efc0 con 0x7fea10162800 2024-09-12T12:20:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b5159a640 1 -- 172.21.15.3:0/4153588783 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4c0a5950 con 0x7f7b5415b890 2024-09-12T12:20:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b5159a640 1 -- 172.21.15.3:0/4153588783 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0a5d40 con 0x7f7b5415b890 2024-09-12T12:20:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 -- 172.21.15.3:0/4153588783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 msgr2=0x7f7b540ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 --2- 172.21.15.3:0/4153588783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b540ad1d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009d90 tx=0x7f7b4c01c9b0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 Processor -- start 2024-09-12T12:20:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 -- start start 2024-09-12T12:20:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 Processor -- start 2024-09-12T12:20:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 -- start start 2024-09-12T12:20:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7fea15dda640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10162800 0x7fea101594d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fea08009dd0 tx=0x7fea0800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7fea05ffb640 1 -- 172.21.15.3:0/752952059 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08018020 con 0x7fea10162800 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7fea05ffb640 1 -- 172.21.15.3:0/752952059 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea08014480 con 0x7fea10162800 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7fea05ffb640 1 -- 172.21.15.3:0/752952059 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08014870 con 0x7fea10162800 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1009a310 con 0x7fea10162800 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea1009a8d0 con 0x7fea10162800 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 -- 172.21.15.3:0/4153588783 shutdown_connections 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 --2- 172.21.15.3:0/4153588783 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b540ad1d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 --2- 172.21.15.3:0/4153588783 >> v2:172.21.15.142:3300/0 conn(0x7f7b540b4070 0x7f7b540b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 -- 172.21.15.3:0/4153588783 >> 172.21.15.3:0/4153588783 conn(0x7f7b54073190 msgr2=0x7f7b540735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0fbffff640 1 -- 172.21.15.3:0/499840391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0fc801a070 con 0x7f0fd806d320 2024-09-12T12:20:00.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 -- 172.21.15.3:0/4153588783 shutdown_connections 2024-09-12T12:20:00.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 -- 172.21.15.3:0/4153588783 wait complete. 2024-09-12T12:20:00.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71640b3890 0x7f716415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716415ce40 0x7f716414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7164160780 0x7f7164159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f716416f570 con 0x7f71640b3890 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f716416f3f0 con 0x7f716415ce40 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f716416f6f0 con 0x7f7164160780 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716415ce40 0x7f716414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716415ce40 0x7f716414ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56712/0 (socket says 172.21.15.3:56712) 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 -- 172.21.15.3:0/3411780758 learned_addr learned my addr 172.21.15.3:0/3411780758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f71637fe640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71640b3890 0x7f716415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7163fff640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7164160780 0x7f7164159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1806dd40 0x7f0e18153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e180d0b80 0x7f0e18159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e1815a210 0x7f0e180cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181680b0 con 0x7f0e180d0b80 2024-09-12T12:20:00.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1815b090 con 0x7f0e1806dd40 2024-09-12T12:20:00.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e1de52640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e18168230 con 0x7f0e1815a210 2024-09-12T12:20:00.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0fbffff640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fc4067990 0x7f0fc4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e1815a210 0x7f0e180cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e1815a210 0x7f0e180cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42364/0 (socket says 172.21.15.3:42364) 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 -- 172.21.15.3:0/2853865024 learned_addr learned my addr 172.21.15.3:0/2853865024 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 -- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7164160780 msgr2=0x7f7164159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7164160780 0x7f7164159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 -- 172.21.15.3:0/3411780758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71640b3890 msgr2=0x7f716415ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71640b3890 0x7f716415ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 -- 172.21.15.3:0/3411780758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716415f370 con 0x7f716415ce40 2024-09-12T12:20:00.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7163fff640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7164160780 0x7f7164159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:00.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0fbffff640 1 -- 172.21.15.3:0/499840391 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0fc8085f30 con 0x7f0fd806d320 2024-09-12T12:20:00.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 -- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1806dd40 msgr2=0x7f0e18153710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:20:00.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1806dd40 0x7f0e18153710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 -- 172.21.15.3:0/2853865024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e180d0b80 msgr2=0x7f0e18159640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:20:00.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e180d0b80 0x7f0e18159640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0e17fff640 1 -- 172.21.15.3:0/2853865024 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e181585a0 con 0x7f0e1815a210 2024-09-12T12:20:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7b58e47640 1 Processor -- start 2024-09-12T12:20:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b58e47640 1 -- start start 2024-09-12T12:20:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7162ffd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716415ce40 0x7f716414ff10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71580192e0 tx=0x7f7158019470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f714bfff640 1 -- 172.21.15.3:0/3411780758 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580b0cc0 con 0x7f716415ce40 2024-09-12T12:20:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716414e9a0 con 0x7f716415ce40 2024-09-12T12:20:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716414e3e0 con 0x7f716415ce40 2024-09-12T12:20:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0fd6ffd640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fc4067990 0x7f0fc4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0e17fff640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e1815a210 0x7f0e180cd7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000bdb0 tx=0x7f0e1000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0dfffff640 1 -- 172.21.15.3:0/2853865024 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10019070 con 0x7f0e1815a210 2024-09-12T12:20:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7fea05ffb640 1 -- 172.21.15.3:0/752952059 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fea0801b020 con 0x7fea10162800 2024-09-12T12:20:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1009aad0 con 0x7fea10162800 2024-09-12T12:20:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7fea05ffb640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9f4067990 0x7fe9f4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e18158880 con 0x7f0e1815a210 2024-09-12T12:20:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0dfffff640 1 -- 172.21.15.3:0/2853865024 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e10014440 con 0x7f0e1815a210 2024-09-12T12:20:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0dfffff640 1 -- 172.21.15.3:0/2853865024 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10014850 con 0x7f0e1815a210 2024-09-12T12:20:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e18159120 con 0x7f0e1815a210 2024-09-12T12:20:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b58e47640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b54073cb0 0x7f7b54145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b58e47640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b540b4070 0x7f7b5414a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b58e47640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b54150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b58e47640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b540b3bb0 con 0x7f7b540b4070 2024-09-12T12:20:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b58e47640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b540b3a30 con 0x7f7b54073cb0 2024-09-12T12:20:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b58e47640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7b540b3d30 con 0x7f7b5415b890 2024-09-12T12:20:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.939+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd80b23e0 con 0x7f0fd806d320 2024-09-12T12:20:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b54150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b54150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42366/0 (socket says 172.21.15.3:42366) 2024-09-12T12:20:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 -- 172.21.15.3:0/762686755 learned_addr learned my addr 172.21.15.3:0/762686755 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b5259c640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b54073cb0 0x7f7b54145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e18158c70 con 0x7f0e1815a210 2024-09-12T12:20:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 -- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b54073cb0 msgr2=0x7f7b54145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b54073cb0 0x7f7b54145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 -- 172.21.15.3:0/762686755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b540b4070 msgr2=0x7f7b5414a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b540b4070 0x7f7b5414a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 -- 172.21.15.3:0/762686755 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b541547b0 con 0x7f7b5415b890 2024-09-12T12:20:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7b52d9d640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b54150d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0b0040 tx=0x7f7b4c019750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0fd6ffd640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fc4067990 0x7f0fc4069e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80c5600 tx=0x7f0fcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0dfffff640 1 -- 172.21.15.3:0/2853865024 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0e1001c070 con 0x7f0e1815a210 2024-09-12T12:20:00.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0dfffff640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e040678c0 0x7f0e04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f0dfffff640 1 -- 172.21.15.3:0/2853865024 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0e10085e20 con 0x7f0e1815a210 2024-09-12T12:20:00.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f2987640 1 Processor -- start 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f714bfff640 1 -- 172.21.15.3:0/3411780758 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7158004020 con 0x7f716415ce40 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f714bfff640 1 -- 172.21.15.3:0/3411780758 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71580b0e60 con 0x7f716415ce40 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716406da30 con 0x7f716415ce40 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f714bfff640 1 -- 172.21.15.3:0/3411780758 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f71580c0060 con 0x7f716415ce40 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f714bfff640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71500678c0 0x7f7150069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f714bfff640 1 -- 172.21.15.3:0/3411780758 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f715811fb40 con 0x7f716415ce40 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.943+0000 7f71637fe640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71500678c0 0x7f7150069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f0e177fe640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e040678c0 0x7f0e04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f2987640 1 -- start start 2024-09-12T12:20:00.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97517b640 1 Processor -- start 2024-09-12T12:20:00.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97517b640 1 -- start start 2024-09-12T12:20:00.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f0e177fe640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e040678c0 0x7f0e04069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007be0 tx=0x7f0e080010e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f71637fe640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71500678c0 0x7f7150069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f715c0072a0 tx=0x7f715c007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f2987640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16ec15c660 0x7f16ec0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f2987640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16ec0b1ad0 con 0x7f16ec0cde80 2024-09-12T12:20:00.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f2987640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16ec0cffd0 con 0x7f16ec09bab0 2024-09-12T12:20:00.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f2987640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f16ec0d02b0 con 0x7f16ec15c660 2024-09-12T12:20:00.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f0f24640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f16ec0cde80 0x7f16ec0b26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36208/0 (socket says 172.21.15.3:36208) 2024-09-12T12:20:00.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16f0f24640 1 -- 172.21.15.3:0/1705822741 learned_addr learned my addr 172.21.15.3:0/1705822741 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:00.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fea05ffb640 1 -- 172.21.15.3:0/752952059 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fea08086630 con 0x7fea10162800 2024-09-12T12:20:00.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97517b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd9680df7f0 0x7fd96800c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97517b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96800af10 con 0x7fd968008150 2024-09-12T12:20:00.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97517b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96800b1f0 con 0x7fd968008c70 2024-09-12T12:20:00.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97517b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd96800cd40 con 0x7fd9680df7f0 2024-09-12T12:20:00.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16ebfff640 1 --2- 172.21.15.3:0/1705822741 >> v2:172.21.15.142:3300/0 conn(0x7f16ec15c660 0x7f16ec0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97497a640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd968008150 0x7fd968037190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36218/0 (socket says 172.21.15.3:36218) 2024-09-12T12:20:00.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd97497a640 1 -- 172.21.15.3:0/2897291763 learned_addr learned my addr 172.21.15.3:0/2897291763 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:00.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd96ffff640 1 --2- 172.21.15.3:0/2897291763 >> v2:172.21.15.142:3300/0 conn(0x7fd9680df7f0 0x7fd96800c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fea05ffb640 1 -- 172.21.15.3:0/752952059 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fea1009aad0 con 0x7fea10162800 2024-09-12T12:20:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fea155d9640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9f4067990 0x7fe9f4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16ebfff640 1 -- 172.21.15.3:0/1705822741 >> v1:172.21.15.142:6789/0 conn(0x7f16ec09bab0 legacy=0x7f16ec0b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16ebfff640 1 -- 172.21.15.3:0/1705822741 >> v1:172.21.15.3:6789/0 conn(0x7f16ec0cde80 legacy=0x7f16ec0b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7f16ebfff640 1 -- 172.21.15.3:0/1705822741 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec09b7b0 con 0x7f16ec15c660 2024-09-12T12:20:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd96ffff640 1 -- 172.21.15.3:0/2897291763 >> v1:172.21.15.142:6789/0 conn(0x7fd968008c70 legacy=0x7fd968038d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd96ffff640 1 -- 172.21.15.3:0/2897291763 >> v1:172.21.15.3:6789/0 conn(0x7fd968008150 legacy=0x7fd968037190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fd96ffff640 1 -- 172.21.15.3:0/2897291763 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd968034830 con 0x7fd9680df7f0 2024-09-12T12:20:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.947+0000 7fea155d9640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9f4067990 0x7fe9f4069e50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f70 tx=0x7fea0c007dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.951+0000 7f16ebfff640 1 --2- 172.21.15.3:0/1705822741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec15c660 0x7f16ec0d0940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16e00097c0 tx=0x7f16e001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d70d9e88221b907 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.951+0000 7fd96ffff640 1 --2- 172.21.15.3:0/2897291763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9680df7f0 0x7fd96800c800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd9640097a0 tx=0x7fd96401ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f86cb3616f3a8fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.955+0000 7f0fbffff640 1 -- 172.21.15.3:0/499840391 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0fc8056a50 con 0x7f0fd806d320 2024-09-12T12:20:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16eaffd640 1 -- 172.21.15.3:0/1705822741 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00b5070 con 0x7f16ec15c660 2024-09-12T12:20:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16eaffd640 1 -- 172.21.15.3:0/1705822741 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e00a8990 con 0x7f16ec15c660 2024-09-12T12:20:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16eaffd640 1 -- 172.21.15.3:0/1705822741 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00a8d60 con 0x7f16ec15c660 2024-09-12T12:20:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f7b42ffd640 1 -- 172.21.15.3:0/762686755 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0b2050 con 0x7f7b5415b890 2024-09-12T12:20:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7fd96effd640 1 -- 172.21.15.3:0/2897291763 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b5070 con 0x7fd9680df7f0 2024-09-12T12:20:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7fd96effd640 1 -- 172.21.15.3:0/2897291763 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9640a8930 con 0x7fd9680df7f0 2024-09-12T12:20:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f0dfffff640 1 -- 172.21.15.3:0/2853865024 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0e10056940 con 0x7f0e1815a210 2024-09-12T12:20:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b54154a90 con 0x7f7b5415b890 2024-09-12T12:20:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b54155050 con 0x7f7b5415b890 2024-09-12T12:20:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16f2987640 1 -- 172.21.15.3:0/1705822741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec15c660 msgr2=0x7f16ec0d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16f2987640 1 --2- 172.21.15.3:0/1705822741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec15c660 0x7f16ec0d0940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16e00097c0 tx=0x7f16e001ec90 comp rx=0 tx=0).stop 2024-09-12T12:20:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16f2987640 1 -- 172.21.15.3:0/1705822741 shutdown_connections 2024-09-12T12:20:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16f2987640 1 --2- 172.21.15.3:0/1705822741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec15c660 0x7f16ec0d0940 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16e00097c0 tx=0x7f16e001ec90 comp rx=0 tx=0).stop 2024-09-12T12:20:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16f2987640 1 -- 172.21.15.3:0/1705822741 >> 172.21.15.3:0/1705822741 conn(0x7f16ec0b3e50 msgr2=0x7f16ec0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f7b42ffd640 1 -- 172.21.15.3:0/762686755 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4c0be070 con 0x7f7b5415b890 2024-09-12T12:20:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f7b42ffd640 1 -- 172.21.15.3:0/762686755 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c004040 con 0x7f7b5415b890 2024-09-12T12:20:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7fd96effd640 1 -- 172.21.15.3:0/2897291763 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b5070 con 0x7fd9680df7f0 2024-09-12T12:20:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7fd97517b640 1 -- 172.21.15.3:0/2897291763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9680df7f0 msgr2=0x7fd96800c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7fd97517b640 1 --2- 172.21.15.3:0/2897291763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9680df7f0 0x7fd96800c800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd9640097a0 tx=0x7fd96401ec30 comp rx=0 tx=0).stop 2024-09-12T12:20:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f714bfff640 1 -- 172.21.15.3:0/3411780758 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f71580ac030 con 0x7f716415ce40 2024-09-12T12:20:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b380052a0 con 0x7f7b5415b890 2024-09-12T12:20:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16f2987640 1 -- 172.21.15.3:0/1705822741 shutdown_connections 2024-09-12T12:20:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.959+0000 7f16f2987640 1 -- 172.21.15.3:0/1705822741 wait complete. 2024-09-12T12:20:00.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 Processor -- start 2024-09-12T12:20:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f7b42ffd640 1 -- 172.21.15.3:0/762686755 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7b4c0041e0 con 0x7f7b5415b890 2024-09-12T12:20:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f7b42ffd640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b340678f0 0x7f7b34069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f7b42ffd640 1 -- 172.21.15.3:0/762686755 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7b4c1213d0 con 0x7f7b5415b890 2024-09-12T12:20:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 -- start start 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec09bab0 0x7f16ec1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16ec0cde80 0x7f16ec0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec0c5e90 0x7f16ec156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec0d0e80 con 0x7f16ec09bab0 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec09bab0 0x7f16ec1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec09bab0 0x7f16ec1556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41182/0 (socket says 172.21.15.3:41182) 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 -- 172.21.15.3:0/3192964346 learned_addr learned my addr 172.21.15.3:0/3192964346 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec09b480 con 0x7f16ec0c5e90 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f16ec0d1000 con 0x7f16ec0cde80 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16eb7fe640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16ec0cde80 0x7f16ec0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f0f24640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec0c5e90 0x7f16ec156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 -- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16ec0cde80 msgr2=0x7f16ec0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16ec0cde80 0x7f16ec0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 -- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec0c5e90 msgr2=0x7f16ec156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec0c5e90 0x7f16ec156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 -- 172.21.15.3:0/3192964346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec0b6760 con 0x7f16ec09bab0 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f0f24640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec0c5e90 0x7f16ec156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16ebfff640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec09bab0 0x7f16ec1556f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16e001c8c0 tx=0x7f16e001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16e8ff9640 1 -- 172.21.15.3:0/3192964346 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00b5040 con 0x7f16ec09bab0 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ec0b6a40 con 0x7f16ec09bab0 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16e8ff9640 1 -- 172.21.15.3:0/3192964346 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e00c0070 con 0x7f16ec09bab0 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16e8ff9640 1 -- 172.21.15.3:0/3192964346 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00b0aa0 con 0x7f16ec09bab0 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.963+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ec0b7000 con 0x7f16ec09bab0 2024-09-12T12:20:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7fd97517b640 1 -- 172.21.15.3:0/2897291763 shutdown_connections 2024-09-12T12:20:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7fd97517b640 1 --2- 172.21.15.3:0/2897291763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9680df7f0 0x7fd96800c800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7fd97517b640 1 -- 172.21.15.3:0/2897291763 >> 172.21.15.3:0/2897291763 conn(0x7fd96801ea60 msgr2=0x7fd968038660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ec0b71d0 con 0x7f16ec09bab0 2024-09-12T12:20:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7fd97517b640 1 -- 172.21.15.3:0/2897291763 shutdown_connections 2024-09-12T12:20:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7fd97517b640 1 -- 172.21.15.3:0/2897291763 wait complete. 2024-09-12T12:20:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7f16e8ff9640 1 -- 172.21.15.3:0/3192964346 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f16e00b0c40 con 0x7f16ec09bab0 2024-09-12T12:20:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7fd97517b640 1 Processor -- start 2024-09-12T12:20:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7fd97517b640 1 -- start start 2024-09-12T12:20:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7f16e8ff9640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f16d80678c0 0x7f16d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7f16e8ff9640 1 -- 172.21.15.3:0/3192964346 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f16e0123d00 con 0x7f16ec09bab0 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.967+0000 7f16eb7fe640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f16d80678c0 0x7f16d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97517b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd968008150 0x7fd9680d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97517b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd968008c70 0x7fd968049da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97517b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9680df7f0 0x7fd9680dd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97517b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd96800acc0 con 0x7fd968008150 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97517b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd968039480 con 0x7fd968008c70 2024-09-12T12:20:00.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97517b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd9680dfbd0 con 0x7fd9680df7f0 2024-09-12T12:20:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7f7b5259c640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b340678f0 0x7f7b34069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd968008c70 0x7fd968049da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96f7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd968008c70 0x7fd968049da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56756/0 (socket says 172.21.15.3:56756) 2024-09-12T12:20:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96f7fe640 1 -- 172.21.15.3:0/3882252912 learned_addr learned my addr 172.21.15.3:0/3882252912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:00.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96ffff640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd968008150 0x7fd9680d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97497a640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9680df7f0 0x7fd9680dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7f7b5259c640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b340678f0 0x7f7b34069db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009b60 tx=0x7f7b44009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96ffff640 1 -- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9680df7f0 msgr2=0x7fd9680dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96ffff640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9680df7f0 0x7fd9680dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96ffff640 1 -- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd968008c70 msgr2=0x7fd968049da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96ffff640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd968008c70 0x7fd968049da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96ffff640 1 -- 172.21.15.3:0/3882252912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9680d21a0 con 0x7fd968008150 2024-09-12T12:20:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd97497a640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9680df7f0 0x7fd9680dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7f16eb7fe640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f16d80678c0 0x7f16d8069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16ec15b640 tx=0x7f16dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.971+0000 7fd96ffff640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd968008150 0x7fd9680d1c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd9640af000 tx=0x7fd96401b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.975+0000 7f7b42ffd640 1 -- 172.21.15.3:0/762686755 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7b4c0165a0 con 0x7f7b5415b890 2024-09-12T12:20:00.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.979+0000 7fd96cff9640 1 -- 172.21.15.3:0/3882252912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640bd070 con 0x7fd968008150 2024-09-12T12:20:00.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.979+0000 7fd96cff9640 1 -- 172.21.15.3:0/3882252912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd964005c70 con 0x7fd968008150 2024-09-12T12:20:00.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.979+0000 7fd96cff9640 1 -- 172.21.15.3:0/3882252912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b5070 con 0x7fd968008150 2024-09-12T12:20:00.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.979+0000 7f16e8ff9640 1 -- 172.21.15.3:0/3192964346 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f16e00187f0 con 0x7f16ec09bab0 2024-09-12T12:20:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.979+0000 7fd97517b640 1 -- 172.21.15.3:0/3882252912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd968035cc0 con 0x7fd968008150 2024-09-12T12:20:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.979+0000 7fd97517b640 1 -- 172.21.15.3:0/3882252912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd968036280 con 0x7fd968008150 2024-09-12T12:20:00.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.979+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9480052a0 con 0x7fd968008150 2024-09-12T12:20:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.983+0000 7fd96cff9640 1 -- 172.21.15.3:0/3882252912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9640c0070 con 0x7fd968008150 2024-09-12T12:20:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.983+0000 7fd96cff9640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd9600679b0 0x7fd960069e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.983+0000 7fd96cff9640 1 -- 172.21.15.3:0/3882252912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd964123960 con 0x7fd968008150 2024-09-12T12:20:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.983+0000 7fd96f7fe640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd9600679b0 0x7fd960069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.983+0000 7fd96f7fe640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd9600679b0 0x7fd960069e70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9680e7e80 tx=0x7fd95c003700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:00.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:00.991+0000 7fd96cff9640 1 -- 172.21.15.3:0/3882252912 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd964018790 con 0x7fd968008150 2024-09-12T12:20:01.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.243+0000 7fe59d7fa640 1 -- 172.21.15.3:0/2352023461 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe598005590 con 0x7fe5b8070200 2024-09-12T12:20:01.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.243+0000 7fe59f7fe640 1 -- 172.21.15.3:0/2352023461 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5ac0a9300 con 0x7fe5b8070200 2024-09-12T12:20:01.248 INFO:teuthology.orchestra.run.smithi003.stdout:64424509448 2024-09-12T12:20:01.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe5a0067a10 msgr2=0x7fe5a0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe5a0067a10 0x7fe5a0069ed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5b81442c0 tx=0x7fe5b000b040 comp rx=0 tx=0).stop 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 msgr2=0x7fe5b8149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b8149820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac019f40 tx=0x7fe5ac0190b0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 shutdown_connections 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe5a0067a10 0x7fe5a0069ed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5b8157de0 0x7fe5b8145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b80b4fc0 0x7fe5b8144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 --2- 172.21.15.3:0/2352023461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5b8070200 0x7fe5b8149820 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 >> 172.21.15.3:0/2352023461 conn(0x7fe5b806dcb0 msgr2=0x7fe5b814a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 shutdown_connections 2024-09-12T12:20:01.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.247+0000 7fe5bd18c640 1 -- 172.21.15.3:0/2352023461 wait complete. 2024-09-12T12:20:01.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2024-09-12T12:20:01.272 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.271+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0e180b4c40 con 0x7f0e1815a210 2024-09-12T12:20:01.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.275+0000 7f0dfffff640 1 -- 172.21.15.3:0/2853865024 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e10056240 con 0x7f0e1815a210 2024-09-12T12:20:01.277 INFO:teuthology.orchestra.run.smithi003.stdout:60129542152 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.275+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e040678c0 msgr2=0x7f0e04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.275+0000 7f0e1de52640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e040678c0 0x7f0e04069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007be0 tx=0x7f0e080010e0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.275+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e1815a210 msgr2=0x7f0e180cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.275+0000 7f0e1de52640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e1815a210 0x7f0e180cd7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000bdb0 tx=0x7f0e1000bea0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 shutdown_connections 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e040678c0 0x7f0e04069d80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e1815a210 0x7f0e180cd7a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e180d0b80 0x7f0e18159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 --2- 172.21.15.3:0/2853865024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e1806dd40 0x7f0e18153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 >> 172.21.15.3:0/2853865024 conn(0x7f0e1806be90 msgr2=0x7f0e1815d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 shutdown_connections 2024-09-12T12:20:01.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.279+0000 7f0e1de52640 1 -- 172.21.15.3:0/2853865024 wait complete. 2024-09-12T12:20:01.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7164073e90 con 0x7f716415ce40 2024-09-12T12:20:01.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f714bfff640 1 -- 172.21.15.3:0/3411780758 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71580162f0 con 0x7f716415ce40 2024-09-12T12:20:01.286 INFO:teuthology.orchestra.run.smithi003.stdout:60129542151 2024-09-12T12:20:01.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0fd80b8390 con 0x7f0fd806d320 2024-09-12T12:20:01.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f0fbffff640 1 -- 172.21.15.3:0/499840391 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0fc8056350 con 0x7f0fd806d320 2024-09-12T12:20:01.287 INFO:teuthology.orchestra.run.smithi003.stdout:60129542151 2024-09-12T12:20:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71500678c0 msgr2=0x7f7150069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71500678c0 0x7f7150069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f715c0072a0 tx=0x7f715c007a40 comp rx=0 tx=0).stop 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716415ce40 msgr2=0x7f716414ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716415ce40 0x7f716414ff10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71580192e0 tx=0x7f7158019470 comp rx=0 tx=0).stop 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 shutdown_connections 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71500678c0 0x7f7150069d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7164160780 0x7f7164159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.283+0000 7f7168dfd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f716415ce40 0x7f716414ff10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f7168dfd640 1 --2- 172.21.15.3:0/3411780758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71640b3890 0x7f716415ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 >> 172.21.15.3:0/3411780758 conn(0x7f71640b1d40 msgr2=0x7f71640b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 shutdown_connections 2024-09-12T12:20:01.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f7168dfd640 1 -- 172.21.15.3:0/3411780758 wait complete. 2024-09-12T12:20:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fc4067990 msgr2=0x7f0fc4069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fc4067990 0x7f0fc4069e50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0fd80c5600 tx=0x7f0fcc009040 comp rx=0 tx=0).stop 2024-09-12T12:20:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd806d320 msgr2=0x7f0fd815aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd806d320 0x7f0fd815aa20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800e870 tx=0x7f0fc800ec70 comp rx=0 tx=0).stop 2024-09-12T12:20:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 shutdown_connections 2024-09-12T12:20:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fc4067990 0x7f0fc4069e50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fd81627e0 0x7f0fd815ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd80c5af0 0x7f0fd80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 --2- 172.21.15.3:0/499840391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fd806d320 0x7f0fd815aa20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 >> 172.21.15.3:0/499840391 conn(0x7f0fd80b3f00 msgr2=0x7f0fd80afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 shutdown_connections 2024-09-12T12:20:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7f0fde1c0640 1 -- 172.21.15.3:0/499840391 wait complete. 2024-09-12T12:20:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fea100ad510 con 0x7fea10162800 2024-09-12T12:20:01.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.287+0000 7fea05ffb640 1 -- 172.21.15.3:0/752952059 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fea08022070 con 0x7fea10162800 2024-09-12T12:20:01.292 INFO:teuthology.orchestra.run.smithi003.stdout:60129542152 2024-09-12T12:20:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9f4067990 msgr2=0x7fe9f4069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9f4067990 0x7fe9f4069e50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea0c000f70 tx=0x7fea0c007dd0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10162800 msgr2=0x7fea101594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10162800 0x7fea101594d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fea08009dd0 tx=0x7fea0800c490 comp rx=0 tx=0).stop 2024-09-12T12:20:01.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 shutdown_connections 2024-09-12T12:20:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9f4067990 0x7fe9f4069e50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea10162800 0x7fea101594d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea100b4950 0x7fea1015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 --2- 172.21.15.3:0/752952059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea100ad060 0x7fea1009c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 >> 172.21.15.3:0/752952059 conn(0x7fea1006be90 msgr2=0x7fea1016c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 shutdown_connections 2024-09-12T12:20:01.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.291+0000 7fea1783d640 1 -- 172.21.15.3:0/752952059 wait complete. 2024-09-12T12:20:01.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.7 2024-09-12T12:20:01.310 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2024-09-12T12:20:01.311 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7b38005860 con 0x7f7b5415b890 2024-09-12T12:20:01.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b42ffd640 1 -- 172.21.15.3:0/762686755 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b4c0f1ff0 con 0x7f7b5415b890 2024-09-12T12:20:01.313 INFO:teuthology.orchestra.run.smithi003.stdout:60129542151 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b340678f0 msgr2=0x7f7b34069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b340678f0 0x7f7b34069db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009b60 tx=0x7f7b44009190 comp rx=0 tx=0).stop 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 msgr2=0x7f7b54150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b54150d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0b0040 tx=0x7f7b4c019750 comp rx=0 tx=0).stop 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 shutdown_connections 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b340678f0 0x7f7b34069db0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b5415b890 0x7f7b54150d60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b540b4070 0x7f7b5414a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 --2- 172.21.15.3:0/762686755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b54073cb0 0x7f7b54145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 >> 172.21.15.3:0/762686755 conn(0x7f7b54073190 msgr2=0x7f7b540ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 shutdown_connections 2024-09-12T12:20:01.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.311+0000 7f7b58e47640 1 -- 172.21.15.3:0/762686755 wait complete. 2024-09-12T12:20:01.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2024-09-12T12:20:01.322 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.319+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f16ec0b2e00 con 0x7f16ec09bab0 2024-09-12T12:20:01.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.319+0000 7f16e8ff9640 1 -- 172.21.15.3:0/3192964346 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16e00f4960 con 0x7f16ec09bab0 2024-09-12T12:20:01.323 INFO:teuthology.orchestra.run.smithi003.stdout:60129542150 2024-09-12T12:20:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.319+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd948005590 con 0x7fd968008150 2024-09-12T12:20:01.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.319+0000 7fd96cff9640 1 -- 172.21.15.3:0/3882252912 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd964005e20 con 0x7fd968008150 2024-09-12T12:20:01.324 INFO:teuthology.orchestra.run.smithi003.stdout:60129542150 2024-09-12T12:20:01.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.6 2024-09-12T12:20:01.326 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f16d80678c0 msgr2=0x7f16d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f16d80678c0 0x7f16d8069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f16ec15b640 tx=0x7f16dc009040 comp rx=0 tx=0).stop 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec09bab0 msgr2=0x7f16ec1556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec09bab0 0x7f16ec1556f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f16e001c8c0 tx=0x7f16e001ba30 comp rx=0 tx=0).stop 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 shutdown_connections 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f16d80678c0 0x7f16d8069d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16ec0c5e90 0x7f16ec156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16ec0cde80 0x7f16ec0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 --2- 172.21.15.3:0/3192964346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16ec09bab0 0x7f16ec1556f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 >> 172.21.15.3:0/3192964346 conn(0x7f16ec0b3e50 msgr2=0x7f16ec0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 shutdown_connections 2024-09-12T12:20:01.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7f16f2987640 1 -- 172.21.15.3:0/3192964346 wait complete. 2024-09-12T12:20:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd9600679b0 msgr2=0x7fd960069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd9600679b0 0x7fd960069e70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9680e7e80 tx=0x7fd95c003700 comp rx=0 tx=0).stop 2024-09-12T12:20:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd968008150 msgr2=0x7fd9680d1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd968008150 0x7fd9680d1c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd9640af000 tx=0x7fd96401b9d0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 shutdown_connections 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd9600679b0 0x7fd960069e70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9680df7f0 0x7fd9680dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd968008c70 0x7fd968049da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 --2- 172.21.15.3:0/3882252912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd968008150 0x7fd9680d1c60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 >> 172.21.15.3:0/3882252912 conn(0x7fd96801ea60 msgr2=0x7fd968038cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 shutdown_connections 2024-09-12T12:20:01.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.323+0000 7fd95a7fc640 1 -- 172.21.15.3:0/3882252912 wait complete. 2024-09-12T12:20:01.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2024-09-12T12:20:01.335 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-09-12T12:20:01.350 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-09-12T12:20:01.350 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:01.350 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:20:01.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:20:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa7f60640 1 Processor -- start 2024-09-12T12:20:01.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa7f60640 1 -- start start 2024-09-12T12:20:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa7f60640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0aa015e5d0 0x7f0aa00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa7f60640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa00d31a0 con 0x7f0aa00cfbe0 2024-09-12T12:20:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa7f60640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aa00d3480 con 0x7f0aa00d5050 2024-09-12T12:20:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa7f60640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0aa00b1060 con 0x7f0aa015e5d0 2024-09-12T12:20:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0aa00cfbe0 0x7f0aa00b14f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36228/0 (socket says 172.21.15.3:36228) 2024-09-12T12:20:01.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 -- 172.21.15.3:0/2052194715 learned_addr learned my addr 172.21.15.3:0/2052194715 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa5cfc640 1 --2- 172.21.15.3:0/2052194715 >> v2:172.21.15.142:3300/0 conn(0x7f0aa015e5d0 0x7f0aa00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516373323 0 0) 0x7f0aa00d31a0 con 0x7f0aa00cfbe0 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a8c003530 con 0x7f0aa00cfbe0 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4164288756 0 0) 0x7f0a8c003530 con 0x7f0aa00cfbe0 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 >> v2:172.21.15.142:3300/0 conn(0x7f0aa015e5d0 msgr2=0x7f0aa00ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 --2- 172.21.15.3:0/2052194715 >> v2:172.21.15.142:3300/0 conn(0x7f0aa015e5d0 0x7f0aa00ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 >> v1:172.21.15.142:6789/0 conn(0x7f0aa00d5050 legacy=0x7f0aa00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00d3760 con 0x7f0aa00cfbe0 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f0a98001a00 con 0x7f0aa00cfbe0 2024-09-12T12:20:01.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 >> v1:172.21.15.3:6789/0 conn(0x7f0aa00cfbe0 legacy=0x7f0aa00b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a8c0039c0 0x7f0a8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c004810 0x7f0a8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a8c0051d0 0x7f0a8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00b1060 con 0x7f0a8c0051d0 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c00c170 con 0x7f0a8c0039c0 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c00c370 con 0x7f0a8c004810 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a8c0051d0 0x7f0a8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa5cfc640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a8c0039c0 0x7f0a8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 -- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c004810 msgr2=0x7f0a8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c004810 0x7f0a8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 -- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a8c0039c0 msgr2=0x7f0a8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a8c0039c0 0x7f0a8c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa64fd640 1 -- 172.21.15.3:0/2052194715 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c003530 con 0x7f0a8c0051d0 2024-09-12T12:20:01.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.499+0000 7f0aa5cfc640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a8c0039c0 0x7f0a8c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa64fd640 1 -- 172.21.15.3:0/2052194715 reap_dead start 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa64fd640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a8c0051d0 0x7f0a8c00b9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800a860 tx=0x7f0a9801fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a6088eb8620b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800a480 con 0x7f0a8c0051d0 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a980aa710 con 0x7f0a8c0051d0 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa4cfa640 1 -- 172.21.15.3:0/2052194715 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a980aaae0 con 0x7f0a8c0051d0 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 -- 172.21.15.3:0/2052194715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a8c0051d0 msgr2=0x7f0a8c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a8c0051d0 0x7f0a8c00b9d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800a860 tx=0x7f0a9801fae0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 -- 172.21.15.3:0/2052194715 shutdown_connections 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/2052194715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a8c0051d0 0x7f0a8c00b9d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 -- 172.21.15.3:0/2052194715 >> 172.21.15.3:0/2052194715 conn(0x7f0aa008c000 msgr2=0x7f0aa00d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 -- 172.21.15.3:0/2052194715 shutdown_connections 2024-09-12T12:20:01.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 -- 172.21.15.3:0/2052194715 wait complete. 2024-09-12T12:20:01.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 Processor -- start 2024-09-12T12:20:01.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.503+0000 7f0aa7f60640 1 -- start start 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c0051d0 0x7f0aa0161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa00b8c30 0x7f0aa00b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0aa00cbe60 0x7f0aa00cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa00cf4d0 con 0x7f0aa00cbe60 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa015ebc0 con 0x7f0aa00b8c30 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa00cf650 con 0x7f0a8c0051d0 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0aa00cbe60 0x7f0aa00cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0aa00cbe60 0x7f0aa00cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41206/0 (socket says 172.21.15.3:41206) 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 -- 172.21.15.3:0/3366617464 learned_addr learned my addr 172.21.15.3:0/3366617464 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa54fb640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa00b8c30 0x7f0aa00b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa5cfc640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c0051d0 0x7f0aa0161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 -- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c0051d0 msgr2=0x7f0aa0161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c0051d0 0x7f0aa0161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 -- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa00b8c30 msgr2=0x7f0aa00b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa00b8c30 0x7f0aa00b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 -- 172.21.15.3:0/3366617464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa00cc820 con 0x7f0aa00cbe60 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa64fd640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0aa00cbe60 0x7f0aa00cc2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a9801f910 tx=0x7f0a8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0a967fc640 1 -- 172.21.15.3:0/3366617464 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a980bd070 con 0x7f0aa00cbe60 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0a967fc640 1 -- 172.21.15.3:0/3366617464 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a98001c20 con 0x7f0aa00cbe60 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0a967fc640 1 -- 172.21.15.3:0/3366617464 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800a480 con 0x7f0aa00cbe60 2024-09-12T12:20:01.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa00ccab0 con 0x7f0aa00cbe60 2024-09-12T12:20:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa01f4210 con 0x7f0aa00cbe60 2024-09-12T12:20:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa00ceb60 con 0x7f0aa00cbe60 2024-09-12T12:20:01.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0a967fc640 1 -- 172.21.15.3:0/3366617464 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a980c0070 con 0x7f0aa00cbe60 2024-09-12T12:20:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0a967fc640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a840678f0 0x7f0a84069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0a967fc640 1 -- 172.21.15.3:0/3366617464 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a98122b60 con 0x7f0aa00cbe60 2024-09-12T12:20:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.507+0000 7f0aa5cfc640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a840678f0 0x7f0a84069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.511+0000 7f0aa5cfc640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a840678f0 0x7f0a84069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0099e0 tx=0x7f0a9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:01.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.515+0000 7f0a967fc640 1 -- 172.21.15.3:0/3366617464 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0a980197f0 con 0x7f0aa00cbe60 2024-09-12T12:20:01.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.667+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0aa00ba490 con 0x7f0a840678f0 2024-09-12T12:20:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.687+0000 7f0a967fc640 1 -- 172.21.15.3:0/3366617464 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f0aa00ba490 con 0x7f0a840678f0 2024-09-12T12:20:01.691 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:01.692 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:20:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.687+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a840678f0 msgr2=0x7f0a84069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.687+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a840678f0 0x7f0a84069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0099e0 tx=0x7f0a9c008040 comp rx=0 tx=0).stop 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0aa00cbe60 msgr2=0x7f0aa00cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0aa00cbe60 0x7f0aa00cc2e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a9801f910 tx=0x7f0a8c005840 comp rx=0 tx=0).stop 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 shutdown_connections 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a840678f0 0x7f0a84069db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0aa00cbe60 0x7f0aa00cc2e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0aa00b8c30 0x7f0aa00b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 --2- 172.21.15.3:0/3366617464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a8c0051d0 0x7f0aa0161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 >> 172.21.15.3:0/3366617464 conn(0x7f0aa008c000 msgr2=0x7f0aa006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 shutdown_connections 2024-09-12T12:20:01.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.691+0000 7f0aa7f60640 1 -- 172.21.15.3:0/3366617464 wait complete. 2024-09-12T12:20:01.708 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-12T12:20:01.660130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":2957312,"data_stored":1776620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.915340"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995369+0000","last_change":"2024-09-12T12:19:48.781313+0000","last_active":"2024-09-12T12:19:49.995369+0000","last_peered":"2024-09-12T12:19:49.995369+0000","last_clean":"2024-09-12T12:19:49.995369+0000","last_became_active":"2024-09-12T12:19:48.780836+0000","last_became_peered":"2024-09-12T12:19:48.780836+0000","last_unstale":"2024-09-12T12:19:49.995369+0000","last_undegraded":"2024-09-12T12:19:49.995369+0000","last_fullsized":"2024-09-12T12:19:49.995369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754520+0000","last_change":"2024-09-12T12:19:48.759206+0000","last_active":"2024-09-12T12:19:49.754520+0000","last_peered":"2024-09-12T12:19:49.754520+0000","last_clean":"2024-09-12T12:19:49.754520+0000","last_became_active":"2024-09-12T12:19:48.758450+0000","last_became_peered":"2024-09-12T12:19:48.758450+0000","last_unstale":"2024-09-12T12:19:49.754520+0000","last_undegraded":"2024-09-12T12:19:49.754520+0000","last_fullsized":"2024-09-12T12:19:49.754520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992848+0000","last_change":"2024-09-12T12:19:48.769260+0000","last_active":"2024-09-12T12:19:49.992848+0000","last_peered":"2024-09-12T12:19:49.992848+0000","last_clean":"2024-09-12T12:19:49.992848+0000","last_became_active":"2024-09-12T12:19:48.768869+0000","last_became_peered":"2024-09-12T12:19:48.768869+0000","last_unstale":"2024-09-12T12:19:49.992848+0000","last_undegraded":"2024-09-12T12:19:49.992848+0000","last_fullsized":"2024-09-12T12:19:49.992848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:41:14.637521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994736+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994736+0000","last_peered":"2024-09-12T12:19:49.994736+0000","last_clean":"2024-09-12T12:19:49.994736+0000","last_became_active":"2024-09-12T12:19:48.779065+0000","last_became_peered":"2024-09-12T12:19:48.779065+0000","last_unstale":"2024-09-12T12:19:49.994736+0000","last_undegraded":"2024-09-12T12:19:49.994736+0000","last_fullsized":"2024-09-12T12:19:49.994736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993392+0000","last_change":"2024-09-12T12:19:48.766342+0000","last_active":"2024-09-12T12:19:49.993392+0000","last_peered":"2024-09-12T12:19:49.993392+0000","last_clean":"2024-09-12T12:19:49.993392+0000","last_became_active":"2024-09-12T12:19:48.766063+0000","last_became_peered":"2024-09-12T12:19:48.766063+0000","last_unstale":"2024-09-12T12:19:49.993392+0000","last_undegraded":"2024-09-12T12:19:49.993392+0000","last_fullsized":"2024-09-12T12:19:49.993392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993183+0000","last_change":"2024-09-12T12:19:48.769354+0000","last_active":"2024-09-12T12:19:49.993183+0000","last_peered":"2024-09-12T12:19:49.993183+0000","last_clean":"2024-09-12T12:19:49.993183+0000","last_became_active":"2024-09-12T12:19:48.769084+0000","last_became_peered":"2024-09-12T12:19:48.769084+0000","last_unstale":"2024-09-12T12:19:49.993183+0000","last_undegraded":"2024-09-12T12:19:49.993183+0000","last_fullsized":"2024-09-12T12:19:49.993183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:56.236676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992999+0000","last_change":"2024-09-12T12:19:48.773715+0000","last_active":"2024-09-12T12:19:49.992999+0000","last_peered":"2024-09-12T12:19:49.992999+0000","last_clean":"2024-09-12T12:19:49.992999+0000","last_became_active":"2024-09-12T12:19:48.773419+0000","last_became_peered":"2024-09-12T12:19:48.773419+0000","last_unstale":"2024-09-12T12:19:49.992999+0000","last_undegraded":"2024-09-12T12:19:49.992999+0000","last_fullsized":"2024-09-12T12:19:49.992999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748510+0000","last_change":"2024-09-12T12:19:48.779584+0000","last_active":"2024-09-12T12:19:49.748510+0000","last_peered":"2024-09-12T12:19:49.748510+0000","last_clean":"2024-09-12T12:19:49.748510+0000","last_became_active":"2024-09-12T12:19:48.777119+0000","last_became_peered":"2024-09-12T12:19:48.777119+0000","last_unstale":"2024-09-12T12:19:49.748510+0000","last_undegraded":"2024-09-12T12:19:49.748510+0000","last_fullsized":"2024-09-12T12:19:49.748510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:44.728800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995405+0000","last_change":"2024-09-12T12:19:48.775263+0000","last_active":"2024-09-12T12:19:49.995405+0000","last_peered":"2024-09-12T12:19:49.995405+0000","last_clean":"2024-09-12T12:19:49.995405+0000","last_became_active":"2024-09-12T12:19:48.774506+0000","last_became_peered":"2024-09-12T12:19:48.774506+0000","last_unstale":"2024-09-12T12:19:49.995405+0000","last_undegraded":"2024-09-12T12:19:49.995405+0000","last_fullsized":"2024-09-12T12:19:49.995405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:38:46.549546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759178+0000","last_change":"2024-09-12T12:19:48.778949+0000","last_active":"2024-09-12T12:19:49.759178+0000","last_peered":"2024-09-12T12:19:49.759178+0000","last_clean":"2024-09-12T12:19:49.759178+0000","last_became_active":"2024-09-12T12:19:48.778672+0000","last_became_peered":"2024-09-12T12:19:48.778672+0000","last_unstale":"2024-09-12T12:19:49.759178+0000","last_undegraded":"2024-09-12T12:19:49.759178+0000","last_fullsized":"2024-09-12T12:19:49.759178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758789+0000","last_change":"2024-09-12T12:19:48.779161+0000","last_active":"2024-09-12T12:19:49.758789+0000","last_peered":"2024-09-12T12:19:49.758789+0000","last_clean":"2024-09-12T12:19:49.758789+0000","last_became_active":"2024-09-12T12:19:48.778765+0000","last_became_peered":"2024-09-12T12:19:48.778765+0000","last_unstale":"2024-09-12T12:19:49.758789+0000","last_undegraded":"2024-09-12T12:19:49.758789+0000","last_fullsized":"2024-09-12T12:19:49.758789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749404+0000","last_change":"2024-09-12T12:19:48.762583+0000","last_active":"2024-09-12T12:19:49.749404+0000","last_peered":"2024-09-12T12:19:49.749404+0000","last_clean":"2024-09-12T12:19:49.749404+0000","last_became_active":"2024-09-12T12:19:48.762315+0000","last_became_peered":"2024-09-12T12:19:48.762315+0000","last_unstale":"2024-09-12T12:19:49.749404+0000","last_undegraded":"2024-09-12T12:19:49.749404+0000","last_fullsized":"2024-09-12T12:19:49.749404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749236+0000","last_change":"2024-09-12T12:19:48.784303+0000","last_active":"2024-09-12T12:19:49.749236+0000","last_peered":"2024-09-12T12:19:49.749236+0000","last_clean":"2024-09-12T12:19:49.749236+0000","last_became_active":"2024-09-12T12:19:48.779768+0000","last_became_peered":"2024-09-12T12:19:48.779768+0000","last_unstale":"2024-09-12T12:19:49.749236+0000","last_undegraded":"2024-09-12T12:19:49.749236+0000","last_fullsized":"2024-09-12T12:19:49.749236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:40.107345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993137+0000","last_change":"2024-09-12T12:19:48.763027+0000","last_active":"2024-09-12T12:19:49.993137+0000","last_peered":"2024-09-12T12:19:49.993137+0000","last_clean":"2024-09-12T12:19:49.993137+0000","last_became_active":"2024-09-12T12:19:48.762544+0000","last_became_peered":"2024-09-12T12:19:48.762544+0000","last_unstale":"2024-09-12T12:19:49.993137+0000","last_undegraded":"2024-09-12T12:19:49.993137+0000","last_fullsized":"2024-09-12T12:19:49.993137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:44.728800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753861+0000","last_change":"2024-09-12T12:19:48.770278+0000","last_active":"2024-09-12T12:19:49.753861+0000","last_peered":"2024-09-12T12:19:49.753861+0000","last_clean":"2024-09-12T12:19:49.753861+0000","last_became_active":"2024-09-12T12:19:48.769983+0000","last_became_peered":"2024-09-12T12:19:48.769983+0000","last_unstale":"2024-09-12T12:19:49.753861+0000","last_undegraded":"2024-09-12T12:19:49.753861+0000","last_fullsized":"2024-09-12T12:19:49.753861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992463+0000","last_change":"2024-09-12T12:19:48.769537+0000","last_active":"2024-09-12T12:19:49.992463+0000","last_peered":"2024-09-12T12:19:49.992463+0000","last_clean":"2024-09-12T12:19:49.992463+0000","last_became_active":"2024-09-12T12:19:48.769243+0000","last_became_peered":"2024-09-12T12:19:48.769243+0000","last_unstale":"2024-09-12T12:19:49.992463+0000","last_undegraded":"2024-09-12T12:19:49.992463+0000","last_fullsized":"2024-09-12T12:19:49.992463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995255+0000","last_change":"2024-09-12T12:19:48.766440+0000","last_active":"2024-09-12T12:19:49.995255+0000","last_peered":"2024-09-12T12:19:49.995255+0000","last_clean":"2024-09-12T12:19:49.995255+0000","last_became_active":"2024-09-12T12:19:48.766110+0000","last_became_peered":"2024-09-12T12:19:48.766110+0000","last_unstale":"2024-09-12T12:19:49.995255+0000","last_undegraded":"2024-09-12T12:19:49.995255+0000","last_fullsized":"2024-09-12T12:19:49.995255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992782+0000","last_change":"2024-09-12T12:19:48.780118+0000","last_active":"2024-09-12T12:19:49.992782+0000","last_peered":"2024-09-12T12:19:49.992782+0000","last_clean":"2024-09-12T12:19:49.992782+0000","last_became_active":"2024-09-12T12:19:48.779839+0000","last_became_peered":"2024-09-12T12:19:48.779839+0000","last_unstale":"2024-09-12T12:19:49.992782+0000","last_undegraded":"2024-09-12T12:19:49.992782+0000","last_fullsized":"2024-09-12T12:19:49.992782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993776+0000","last_change":"2024-09-12T12:19:48.766221+0000","last_active":"2024-09-12T12:19:49.993776+0000","last_peered":"2024-09-12T12:19:49.993776+0000","last_clean":"2024-09-12T12:19:49.993776+0000","last_became_active":"2024-09-12T12:19:48.765549+0000","last_became_peered":"2024-09-12T12:19:48.765549+0000","last_unstale":"2024-09-12T12:19:49.993776+0000","last_undegraded":"2024-09-12T12:19:49.993776+0000","last_fullsized":"2024-09-12T12:19:49.993776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758800+0000","last_change":"2024-09-12T12:19:48.764597+0000","last_active":"2024-09-12T12:19:49.758800+0000","last_peered":"2024-09-12T12:19:49.758800+0000","last_clean":"2024-09-12T12:19:49.758800+0000","last_became_active":"2024-09-12T12:19:48.764377+0000","last_became_peered":"2024-09-12T12:19:48.764377+0000","last_unstale":"2024-09-12T12:19:49.758800+0000","last_undegraded":"2024-09-12T12:19:49.758800+0000","last_fullsized":"2024-09-12T12:19:49.758800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992940+0000","last_change":"2024-09-12T12:19:48.763523+0000","last_active":"2024-09-12T12:19:49.992940+0000","last_peered":"2024-09-12T12:19:49.992940+0000","last_clean":"2024-09-12T12:19:49.992940+0000","last_became_active":"2024-09-12T12:19:48.762889+0000","last_became_peered":"2024-09-12T12:19:48.762889+0000","last_unstale":"2024-09-12T12:19:49.992940+0000","last_undegraded":"2024-09-12T12:19:49.992940+0000","last_fullsized":"2024-09-12T12:19:49.992940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:04:18.599614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993579+0000","last_change":"2024-09-12T12:19:48.760812+0000","last_active":"2024-09-12T12:19:49.993579+0000","last_peered":"2024-09-12T12:19:49.993579+0000","last_clean":"2024-09-12T12:19:49.993579+0000","last_became_active":"2024-09-12T12:19:48.760551+0000","last_became_peered":"2024-09-12T12:19:48.760551+0000","last_unstale":"2024-09-12T12:19:49.993579+0000","last_undegraded":"2024-09-12T12:19:49.993579+0000","last_fullsized":"2024-09-12T12:19:49.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748762+0000","last_change":"2024-09-12T12:19:48.760474+0000","last_active":"2024-09-12T12:19:49.748762+0000","last_peered":"2024-09-12T12:19:49.748762+0000","last_clean":"2024-09-12T12:19:49.748762+0000","last_became_active":"2024-09-12T12:19:48.760032+0000","last_became_peered":"2024-09-12T12:19:48.760032+0000","last_unstale":"2024-09-12T12:19:49.748762+0000","last_undegraded":"2024-09-12T12:19:49.748762+0000","last_fullsized":"2024-09-12T12:19:49.748762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:40.506967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992589+0000","last_change":"2024-09-12T12:19:48.780825+0000","last_active":"2024-09-12T12:19:49.992589+0000","last_peered":"2024-09-12T12:19:49.992589+0000","last_clean":"2024-09-12T12:19:49.992589+0000","last_became_active":"2024-09-12T12:19:48.780593+0000","last_became_peered":"2024-09-12T12:19:48.780593+0000","last_unstale":"2024-09-12T12:19:49.992589+0000","last_undegraded":"2024-09-12T12:19:49.992589+0000","last_fullsized":"2024-09-12T12:19:49.992589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:50.202989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749885+0000","last_change":"2024-09-12T12:19:48.759772+0000","last_active":"2024-09-12T12:19:49.749885+0000","last_peered":"2024-09-12T12:19:49.749885+0000","last_clean":"2024-09-12T12:19:49.749885+0000","last_became_active":"2024-09-12T12:19:48.759512+0000","last_became_peered":"2024-09-12T12:19:48.759512+0000","last_unstale":"2024-09-12T12:19:49.749885+0000","last_undegraded":"2024-09-12T12:19:49.749885+0000","last_fullsized":"2024-09-12T12:19:49.749885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:02.195687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759372+0000","last_change":"2024-09-12T12:19:48.774953+0000","last_active":"2024-09-12T12:19:49.759372+0000","last_peered":"2024-09-12T12:19:49.759372+0000","last_clean":"2024-09-12T12:19:49.759372+0000","last_became_active":"2024-09-12T12:19:48.774474+0000","last_became_peered":"2024-09-12T12:19:48.774474+0000","last_unstale":"2024-09-12T12:19:49.759372+0000","last_undegraded":"2024-09-12T12:19:49.759372+0000","last_fullsized":"2024-09-12T12:19:49.759372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754066+0000","last_change":"2024-09-12T12:19:48.771914+0000","last_active":"2024-09-12T12:19:49.754066+0000","last_peered":"2024-09-12T12:19:49.754066+0000","last_clean":"2024-09-12T12:19:49.754066+0000","last_became_active":"2024-09-12T12:19:48.771637+0000","last_became_peered":"2024-09-12T12:19:48.771637+0000","last_unstale":"2024-09-12T12:19:49.754066+0000","last_undegraded":"2024-09-12T12:19:49.754066+0000","last_fullsized":"2024-09-12T12:19:49.754066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749153+0000","last_change":"2024-09-12T12:19:48.759783+0000","last_active":"2024-09-12T12:19:49.749153+0000","last_peered":"2024-09-12T12:19:49.749153+0000","last_clean":"2024-09-12T12:19:49.749153+0000","last_became_active":"2024-09-12T12:19:48.759473+0000","last_became_peered":"2024-09-12T12:19:48.759473+0000","last_unstale":"2024-09-12T12:19:49.749153+0000","last_undegraded":"2024-09-12T12:19:49.749153+0000","last_fullsized":"2024-09-12T12:19:49.749153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:11.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993036+0000","last_change":"2024-09-12T12:19:48.759011+0000","last_active":"2024-09-12T12:19:49.993036+0000","last_peered":"2024-09-12T12:19:49.993036+0000","last_clean":"2024-09-12T12:19:49.993036+0000","last_became_active":"2024-09-12T12:19:48.758267+0000","last_became_peered":"2024-09-12T12:19:48.758267+0000","last_unstale":"2024-09-12T12:19:49.993036+0000","last_undegraded":"2024-09-12T12:19:49.993036+0000","last_fullsized":"2024-09-12T12:19:49.993036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:45.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993953+0000","last_change":"2024-09-12T12:19:48.779776+0000","last_active":"2024-09-12T12:19:49.993953+0000","last_peered":"2024-09-12T12:19:49.993953+0000","last_clean":"2024-09-12T12:19:49.993953+0000","last_became_active":"2024-09-12T12:19:48.779500+0000","last_became_peered":"2024-09-12T12:19:48.779500+0000","last_unstale":"2024-09-12T12:19:49.993953+0000","last_undegraded":"2024-09-12T12:19:49.993953+0000","last_fullsized":"2024-09-12T12:19:49.993953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994016+0000","last_change":"2024-09-12T12:19:48.775150+0000","last_active":"2024-09-12T12:19:49.994016+0000","last_peered":"2024-09-12T12:19:49.994016+0000","last_clean":"2024-09-12T12:19:49.994016+0000","last_became_active":"2024-09-12T12:19:48.774841+0000","last_became_peered":"2024-09-12T12:19:48.774841+0000","last_unstale":"2024-09-12T12:19:49.994016+0000","last_undegraded":"2024-09-12T12:19:49.994016+0000","last_fullsized":"2024-09-12T12:19:49.994016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758353+0000","last_change":"2024-09-12T12:19:48.761408+0000","last_active":"2024-09-12T12:19:49.758353+0000","last_peered":"2024-09-12T12:19:49.758353+0000","last_clean":"2024-09-12T12:19:49.758353+0000","last_became_active":"2024-09-12T12:19:48.761091+0000","last_became_peered":"2024-09-12T12:19:48.761091+0000","last_unstale":"2024-09-12T12:19:49.758353+0000","last_undegraded":"2024-09-12T12:19:49.758353+0000","last_fullsized":"2024-09-12T12:19:49.758353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:51.034838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748611+0000","last_change":"2024-09-12T12:19:48.764078+0000","last_active":"2024-09-12T12:19:49.748611+0000","last_peered":"2024-09-12T12:19:49.748611+0000","last_clean":"2024-09-12T12:19:49.748611+0000","last_became_active":"2024-09-12T12:19:48.763831+0000","last_became_peered":"2024-09-12T12:19:48.763831+0000","last_unstale":"2024-09-12T12:19:49.748611+0000","last_undegraded":"2024-09-12T12:19:49.748611+0000","last_fullsized":"2024-09-12T12:19:49.748611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:06.350696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749493+0000","last_change":"2024-09-12T12:19:47.757834+0000","last_active":"2024-09-12T12:19:49.749493+0000","last_peered":"2024-09-12T12:19:49.749493+0000","last_clean":"2024-09-12T12:19:49.749493+0000","last_became_active":"2024-09-12T12:19:47.757145+0000","last_became_peered":"2024-09-12T12:19:47.757145+0000","last_unstale":"2024-09-12T12:19:49.749493+0000","last_undegraded":"2024-09-12T12:19:49.749493+0000","last_fullsized":"2024-09-12T12:19:49.749493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:13:55.749099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993352+0000","last_change":"2024-09-12T12:19:48.761558+0000","last_active":"2024-09-12T12:19:49.993352+0000","last_peered":"2024-09-12T12:19:49.993352+0000","last_clean":"2024-09-12T12:19:49.993352+0000","last_became_active":"2024-09-12T12:19:48.761230+0000","last_became_peered":"2024-09-12T12:19:48.761230+0000","last_unstale":"2024-09-12T12:19:49.993352+0000","last_undegraded":"2024-09-12T12:19:49.993352+0000","last_fullsized":"2024-09-12T12:19:49.993352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:40.506967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758441+0000","last_change":"2024-09-12T12:19:47.758747+0000","last_active":"2024-09-12T12:19:49.758441+0000","last_peered":"2024-09-12T12:19:49.758441+0000","last_clean":"2024-09-12T12:19:49.758441+0000","last_became_active":"2024-09-12T12:19:47.757870+0000","last_became_peered":"2024-09-12T12:19:47.757870+0000","last_unstale":"2024-09-12T12:19:49.758441+0000","last_undegraded":"2024-09-12T12:19:49.758441+0000","last_fullsized":"2024-09-12T12:19:49.758441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748588+0000","last_change":"2024-09-12T12:19:47.758883+0000","last_active":"2024-09-12T12:19:49.748588+0000","last_peered":"2024-09-12T12:19:49.748588+0000","last_clean":"2024-09-12T12:19:49.748588+0000","last_became_active":"2024-09-12T12:19:47.758396+0000","last_became_peered":"2024-09-12T12:19:47.758396+0000","last_unstale":"2024-09-12T12:19:49.748588+0000","last_undegraded":"2024-09-12T12:19:49.748588+0000","last_fullsized":"2024-09-12T12:19:49.748588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:49:34.294310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748361+0000","last_change":"2024-09-12T12:19:48.779910+0000","last_active":"2024-09-12T12:19:49.748361+0000","last_peered":"2024-09-12T12:19:49.748361+0000","last_clean":"2024-09-12T12:19:49.748361+0000","last_became_active":"2024-09-12T12:19:48.779613+0000","last_became_peered":"2024-09-12T12:19:48.779613+0000","last_unstale":"2024-09-12T12:19:49.748361+0000","last_undegraded":"2024-09-12T12:19:49.748361+0000","last_fullsized":"2024-09-12T12:19:49.748361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:04:18.599614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992602+0000","last_change":"2024-09-12T12:19:47.740584+0000","last_active":"2024-09-12T12:19:49.992602+0000","last_peered":"2024-09-12T12:19:49.992602+0000","last_clean":"2024-09-12T12:19:49.992602+0000","last_became_active":"2024-09-12T12:19:47.739985+0000","last_became_peered":"2024-09-12T12:19:47.739985+0000","last_unstale":"2024-09-12T12:19:49.992602+0000","last_undegraded":"2024-09-12T12:19:49.992602+0000","last_fullsized":"2024-09-12T12:19:49.992602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753847+0000","last_change":"2024-09-12T12:19:48.771435+0000","last_active":"2024-09-12T12:19:49.753847+0000","last_peered":"2024-09-12T12:19:49.753847+0000","last_clean":"2024-09-12T12:19:49.753847+0000","last_became_active":"2024-09-12T12:19:48.771108+0000","last_became_peered":"2024-09-12T12:19:48.771108+0000","last_unstale":"2024-09-12T12:19:49.753847+0000","last_undegraded":"2024-09-12T12:19:49.753847+0000","last_fullsized":"2024-09-12T12:19:49.753847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758579+0000","last_change":"2024-09-12T12:19:47.760876+0000","last_active":"2024-09-12T12:19:49.758579+0000","last_peered":"2024-09-12T12:19:49.758579+0000","last_clean":"2024-09-12T12:19:49.758579+0000","last_became_active":"2024-09-12T12:19:47.760036+0000","last_became_peered":"2024-09-12T12:19:47.760036+0000","last_unstale":"2024-09-12T12:19:49.758579+0000","last_undegraded":"2024-09-12T12:19:49.758579+0000","last_fullsized":"2024-09-12T12:19:49.758579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993540+0000","last_change":"2024-09-12T12:19:48.765500+0000","last_active":"2024-09-12T12:19:49.993540+0000","last_peered":"2024-09-12T12:19:49.993540+0000","last_clean":"2024-09-12T12:19:49.993540+0000","last_became_active":"2024-09-12T12:19:48.765191+0000","last_became_peered":"2024-09-12T12:19:48.765191+0000","last_unstale":"2024-09-12T12:19:49.993540+0000","last_undegraded":"2024-09-12T12:19:49.993540+0000","last_fullsized":"2024-09-12T12:19:49.993540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993676+0000","last_change":"2024-09-12T12:19:47.761042+0000","last_active":"2024-09-12T12:19:49.993676+0000","last_peered":"2024-09-12T12:19:49.993676+0000","last_clean":"2024-09-12T12:19:49.993676+0000","last_became_active":"2024-09-12T12:19:47.760261+0000","last_became_peered":"2024-09-12T12:19:47.760261+0000","last_unstale":"2024-09-12T12:19:49.993676+0000","last_undegraded":"2024-09-12T12:19:49.993676+0000","last_fullsized":"2024-09-12T12:19:49.993676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993444+0000","last_change":"2024-09-12T12:19:48.769338+0000","last_active":"2024-09-12T12:19:49.993444+0000","last_peered":"2024-09-12T12:19:49.993444+0000","last_clean":"2024-09-12T12:19:49.993444+0000","last_became_active":"2024-09-12T12:19:48.768955+0000","last_became_peered":"2024-09-12T12:19:48.768955+0000","last_unstale":"2024-09-12T12:19:49.993444+0000","last_undegraded":"2024-09-12T12:19:49.993444+0000","last_fullsized":"2024-09-12T12:19:49.993444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.656049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754127+0000","last_change":"2024-09-12T12:19:47.750999+0000","last_active":"2024-09-12T12:19:49.754127+0000","last_peered":"2024-09-12T12:19:49.754127+0000","last_clean":"2024-09-12T12:19:49.754127+0000","last_became_active":"2024-09-12T12:19:47.750658+0000","last_became_peered":"2024-09-12T12:19:47.750658+0000","last_unstale":"2024-09-12T12:19:49.754127+0000","last_undegraded":"2024-09-12T12:19:49.754127+0000","last_fullsized":"2024-09-12T12:19:49.754127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749115+0000","last_change":"2024-09-12T12:19:48.767953+0000","last_active":"2024-09-12T12:19:49.749115+0000","last_peered":"2024-09-12T12:19:49.749115+0000","last_clean":"2024-09-12T12:19:49.749115+0000","last_became_active":"2024-09-12T12:19:48.767595+0000","last_became_peered":"2024-09-12T12:19:48.767595+0000","last_unstale":"2024-09-12T12:19:49.749115+0000","last_undegraded":"2024-09-12T12:19:49.749115+0000","last_fullsized":"2024-09-12T12:19:49.749115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758590+0000","last_change":"2024-09-12T12:19:47.760548+0000","last_active":"2024-09-12T12:19:49.758590+0000","last_peered":"2024-09-12T12:19:49.758590+0000","last_clean":"2024-09-12T12:19:49.758590+0000","last_became_active":"2024-09-12T12:19:47.760033+0000","last_became_peered":"2024-09-12T12:19:47.760033+0000","last_unstale":"2024-09-12T12:19:49.758590+0000","last_undegraded":"2024-09-12T12:19:49.758590+0000","last_fullsized":"2024-09-12T12:19:49.758590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994749+0000","last_change":"2024-09-12T12:19:48.773829+0000","last_active":"2024-09-12T12:19:49.994749+0000","last_peered":"2024-09-12T12:19:49.994749+0000","last_clean":"2024-09-12T12:19:49.994749+0000","last_became_active":"2024-09-12T12:19:48.773359+0000","last_became_peered":"2024-09-12T12:19:48.773359+0000","last_unstale":"2024-09-12T12:19:49.994749+0000","last_undegraded":"2024-09-12T12:19:49.994749+0000","last_fullsized":"2024-09-12T12:19:49.994749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993173+0000","last_change":"2024-09-12T12:19:47.757426+0000","last_active":"2024-09-12T12:19:49.993173+0000","last_peered":"2024-09-12T12:19:49.993173+0000","last_clean":"2024-09-12T12:19:49.993173+0000","last_became_active":"2024-09-12T12:19:47.756782+0000","last_became_peered":"2024-09-12T12:19:47.756782+0000","last_unstale":"2024-09-12T12:19:49.993173+0000","last_undegraded":"2024-09-12T12:19:49.993173+0000","last_fullsized":"2024-09-12T12:19:49.993173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992472+0000","last_change":"2024-09-12T12:19:48.759150+0000","last_active":"2024-09-12T12:19:49.992472+0000","last_peered":"2024-09-12T12:19:49.992472+0000","last_clean":"2024-09-12T12:19:49.992472+0000","last_became_active":"2024-09-12T12:19:48.758654+0000","last_became_peered":"2024-09-12T12:19:48.758654+0000","last_unstale":"2024-09-12T12:19:49.992472+0000","last_undegraded":"2024-09-12T12:19:49.992472+0000","last_fullsized":"2024-09-12T12:19:49.992472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:37.527639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994079+0000","last_change":"2024-09-12T12:19:47.747437+0000","last_active":"2024-09-12T12:19:49.994079+0000","last_peered":"2024-09-12T12:19:49.994079+0000","last_clean":"2024-09-12T12:19:49.994079+0000","last_became_active":"2024-09-12T12:19:47.746926+0000","last_became_peered":"2024-09-12T12:19:47.746926+0000","last_unstale":"2024-09-12T12:19:49.994079+0000","last_undegraded":"2024-09-12T12:19:49.994079+0000","last_fullsized":"2024-09-12T12:19:49.994079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993790+0000","last_change":"2024-09-12T12:19:48.775580+0000","last_active":"2024-09-12T12:19:49.993790+0000","last_peered":"2024-09-12T12:19:49.993790+0000","last_clean":"2024-09-12T12:19:49.993790+0000","last_became_active":"2024-09-12T12:19:48.767021+0000","last_became_peered":"2024-09-12T12:19:48.767021+0000","last_unstale":"2024-09-12T12:19:49.993790+0000","last_undegraded":"2024-09-12T12:19:49.993790+0000","last_fullsized":"2024-09-12T12:19:49.993790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758162+0000","last_change":"2024-09-12T12:19:47.754802+0000","last_active":"2024-09-12T12:19:49.758162+0000","last_peered":"2024-09-12T12:19:49.758162+0000","last_clean":"2024-09-12T12:19:49.758162+0000","last_became_active":"2024-09-12T12:19:47.754242+0000","last_became_peered":"2024-09-12T12:19:47.754242+0000","last_unstale":"2024-09-12T12:19:49.758162+0000","last_undegraded":"2024-09-12T12:19:49.758162+0000","last_fullsized":"2024-09-12T12:19:49.758162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758450+0000","last_change":"2024-09-12T12:19:48.761340+0000","last_active":"2024-09-12T12:19:49.758450+0000","last_peered":"2024-09-12T12:19:49.758450+0000","last_clean":"2024-09-12T12:19:49.758450+0000","last_became_active":"2024-09-12T12:19:48.761037+0000","last_became_peered":"2024-09-12T12:19:48.761037+0000","last_unstale":"2024-09-12T12:19:49.758450+0000","last_undegraded":"2024-09-12T12:19:49.758450+0000","last_fullsized":"2024-09-12T12:19:49.758450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992730+0000","last_change":"2024-09-12T12:19:47.761479+0000","last_active":"2024-09-12T12:19:49.992730+0000","last_peered":"2024-09-12T12:19:49.992730+0000","last_clean":"2024-09-12T12:19:49.992730+0000","last_became_active":"2024-09-12T12:19:47.760812+0000","last_became_peered":"2024-09-12T12:19:47.760812+0000","last_unstale":"2024-09-12T12:19:49.992730+0000","last_undegraded":"2024-09-12T12:19:49.992730+0000","last_fullsized":"2024-09-12T12:19:49.992730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993407+0000","last_change":"2024-09-12T12:19:48.772189+0000","last_active":"2024-09-12T12:19:49.993407+0000","last_peered":"2024-09-12T12:19:49.993407+0000","last_clean":"2024-09-12T12:19:49.993407+0000","last_became_active":"2024-09-12T12:19:48.765342+0000","last_became_peered":"2024-09-12T12:19:48.765342+0000","last_unstale":"2024-09-12T12:19:49.993407+0000","last_undegraded":"2024-09-12T12:19:49.993407+0000","last_fullsized":"2024-09-12T12:19:49.993407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749324+0000","last_change":"2024-09-12T12:19:47.754107+0000","last_active":"2024-09-12T12:19:49.749324+0000","last_peered":"2024-09-12T12:19:49.749324+0000","last_clean":"2024-09-12T12:19:49.749324+0000","last_became_active":"2024-09-12T12:19:47.753433+0000","last_became_peered":"2024-09-12T12:19:47.753433+0000","last_unstale":"2024-09-12T12:19:49.749324+0000","last_undegraded":"2024-09-12T12:19:49.749324+0000","last_fullsized":"2024-09-12T12:19:49.749324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:28.270374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995095+0000","last_change":"2024-09-12T12:19:48.770240+0000","last_active":"2024-09-12T12:19:49.995095+0000","last_peered":"2024-09-12T12:19:49.995095+0000","last_clean":"2024-09-12T12:19:49.995095+0000","last_became_active":"2024-09-12T12:19:48.766786+0000","last_became_peered":"2024-09-12T12:19:48.766786+0000","last_unstale":"2024-09-12T12:19:49.995095+0000","last_undegraded":"2024-09-12T12:19:49.995095+0000","last_fullsized":"2024-09-12T12:19:49.995095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992870+0000","last_change":"2024-09-12T12:19:47.757967+0000","last_active":"2024-09-12T12:19:49.992870+0000","last_peered":"2024-09-12T12:19:49.992870+0000","last_clean":"2024-09-12T12:19:49.992870+0000","last_became_active":"2024-09-12T12:19:47.757370+0000","last_became_peered":"2024-09-12T12:19:47.757370+0000","last_unstale":"2024-09-12T12:19:49.992870+0000","last_undegraded":"2024-09-12T12:19:49.992870+0000","last_fullsized":"2024-09-12T12:19:49.992870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995730+0000","last_change":"2024-09-12T12:19:48.766283+0000","last_active":"2024-09-12T12:19:49.995730+0000","last_peered":"2024-09-12T12:19:49.995730+0000","last_clean":"2024-09-12T12:19:49.995730+0000","last_became_active":"2024-09-12T12:19:48.765694+0000","last_became_peered":"2024-09-12T12:19:48.765694+0000","last_unstale":"2024-09-12T12:19:49.995730+0000","last_undegraded":"2024-09-12T12:19:49.995730+0000","last_fullsized":"2024-09-12T12:19:49.995730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:42:42.831767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754237+0000","last_change":"2024-09-12T12:19:47.751839+0000","last_active":"2024-09-12T12:19:49.754237+0000","last_peered":"2024-09-12T12:19:49.754237+0000","last_clean":"2024-09-12T12:19:49.754237+0000","last_became_active":"2024-09-12T12:19:47.751482+0000","last_became_peered":"2024-09-12T12:19:47.751482+0000","last_unstale":"2024-09-12T12:19:49.754237+0000","last_undegraded":"2024-09-12T12:19:49.754237+0000","last_fullsized":"2024-09-12T12:19:49.754237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995497+0000","last_change":"2024-09-12T12:19:48.768636+0000","last_active":"2024-09-12T12:19:49.995497+0000","last_peered":"2024-09-12T12:19:49.995497+0000","last_clean":"2024-09-12T12:19:49.995497+0000","last_became_active":"2024-09-12T12:19:48.768342+0000","last_became_peered":"2024-09-12T12:19:48.768342+0000","last_unstale":"2024-09-12T12:19:49.995497+0000","last_undegraded":"2024-09-12T12:19:49.995497+0000","last_fullsized":"2024-09-12T12:19:49.995497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993248+0000","last_change":"2024-09-12T12:19:47.755829+0000","last_active":"2024-09-12T12:19:49.993248+0000","last_peered":"2024-09-12T12:19:49.993248+0000","last_clean":"2024-09-12T12:19:49.993248+0000","last_became_active":"2024-09-12T12:19:47.748443+0000","last_became_peered":"2024-09-12T12:19:47.748443+0000","last_unstale":"2024-09-12T12:19:49.993248+0000","last_undegraded":"2024-09-12T12:19:49.993248+0000","last_fullsized":"2024-09-12T12:19:49.993248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993070+0000","last_change":"2024-09-12T12:19:48.775968+0000","last_active":"2024-09-12T12:19:49.993070+0000","last_peered":"2024-09-12T12:19:49.993070+0000","last_clean":"2024-09-12T12:19:49.993070+0000","last_became_active":"2024-09-12T12:19:48.775731+0000","last_became_peered":"2024-09-12T12:19:48.775731+0000","last_unstale":"2024-09-12T12:19:49.993070+0000","last_undegraded":"2024-09-12T12:19:49.993070+0000","last_fullsized":"2024-09-12T12:19:49.993070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992710+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:19:49.992710+0000","last_peered":"2024-09-12T12:19:49.992710+0000","last_clean":"2024-09-12T12:19:49.992710+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:19:49.992710+0000","last_undegraded":"2024-09-12T12:19:49.992710+0000","last_fullsized":"2024-09-12T12:19:49.992710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993779+0000","last_change":"2024-09-12T12:19:47.757991+0000","last_active":"2024-09-12T12:19:49.993779+0000","last_peered":"2024-09-12T12:19:49.993779+0000","last_clean":"2024-09-12T12:19:49.993779+0000","last_became_active":"2024-09-12T12:19:47.757562+0000","last_became_peered":"2024-09-12T12:19:47.757562+0000","last_unstale":"2024-09-12T12:19:49.993779+0000","last_undegraded":"2024-09-12T12:19:49.993779+0000","last_fullsized":"2024-09-12T12:19:49.993779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749165+0000","last_change":"2024-09-12T12:19:48.779259+0000","last_active":"2024-09-12T12:19:49.749165+0000","last_peered":"2024-09-12T12:19:49.749165+0000","last_clean":"2024-09-12T12:19:49.749165+0000","last_became_active":"2024-09-12T12:19:48.778963+0000","last_became_peered":"2024-09-12T12:19:48.778963+0000","last_unstale":"2024-09-12T12:19:49.749165+0000","last_undegraded":"2024-09-12T12:19:49.749165+0000","last_fullsized":"2024-09-12T12:19:49.749165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.656049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992441+0000","last_change":"2024-09-12T12:19:47.761015+0000","last_active":"2024-09-12T12:19:49.992441+0000","last_peered":"2024-09-12T12:19:49.992441+0000","last_clean":"2024-09-12T12:19:49.992441+0000","last_became_active":"2024-09-12T12:19:47.760494+0000","last_became_peered":"2024-09-12T12:19:47.760494+0000","last_unstale":"2024-09-12T12:19:49.992441+0000","last_undegraded":"2024-09-12T12:19:49.992441+0000","last_fullsized":"2024-09-12T12:19:49.992441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992375+0000","last_change":"2024-09-12T12:19:48.765122+0000","last_active":"2024-09-12T12:19:49.992375+0000","last_peered":"2024-09-12T12:19:49.992375+0000","last_clean":"2024-09-12T12:19:49.992375+0000","last_became_active":"2024-09-12T12:19:48.764670+0000","last_became_peered":"2024-09-12T12:19:48.764670+0000","last_unstale":"2024-09-12T12:19:49.992375+0000","last_undegraded":"2024-09-12T12:19:49.992375+0000","last_fullsized":"2024-09-12T12:19:49.992375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:57:01.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993998+0000","last_change":"2024-09-12T12:19:47.758292+0000","last_active":"2024-09-12T12:19:49.993998+0000","last_peered":"2024-09-12T12:19:49.993998+0000","last_clean":"2024-09-12T12:19:49.993998+0000","last_became_active":"2024-09-12T12:19:47.757643+0000","last_became_peered":"2024-09-12T12:19:47.757643+0000","last_unstale":"2024-09-12T12:19:49.993998+0000","last_undegraded":"2024-09-12T12:19:49.993998+0000","last_fullsized":"2024-09-12T12:19:49.993998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753888+0000","last_change":"2024-09-12T12:19:48.763042+0000","last_active":"2024-09-12T12:19:49.753888+0000","last_peered":"2024-09-12T12:19:49.753888+0000","last_clean":"2024-09-12T12:19:49.753888+0000","last_became_active":"2024-09-12T12:19:48.762214+0000","last_became_peered":"2024-09-12T12:19:48.762214+0000","last_unstale":"2024-09-12T12:19:49.753888+0000","last_undegraded":"2024-09-12T12:19:49.753888+0000","last_fullsized":"2024-09-12T12:19:49.753888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994326+0000","last_change":"2024-09-12T12:19:47.752251+0000","last_active":"2024-09-12T12:19:49.994326+0000","last_peered":"2024-09-12T12:19:49.994326+0000","last_clean":"2024-09-12T12:19:49.994326+0000","last_became_active":"2024-09-12T12:19:47.751530+0000","last_became_peered":"2024-09-12T12:19:47.751530+0000","last_unstale":"2024-09-12T12:19:49.994326+0000","last_undegraded":"2024-09-12T12:19:49.994326+0000","last_fullsized":"2024-09-12T12:19:49.994326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749074+0000","last_change":"2024-09-12T12:19:48.762555+0000","last_active":"2024-09-12T12:19:49.749074+0000","last_peered":"2024-09-12T12:19:49.749074+0000","last_clean":"2024-09-12T12:19:49.749074+0000","last_became_active":"2024-09-12T12:19:48.760885+0000","last_became_peered":"2024-09-12T12:19:48.760885+0000","last_unstale":"2024-09-12T12:19:49.749074+0000","last_undegraded":"2024-09-12T12:19:49.749074+0000","last_fullsized":"2024-09-12T12:19:49.749074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:16.559958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753838+0000","last_change":"2024-09-12T12:19:47.758162+0000","last_active":"2024-09-12T12:19:49.753838+0000","last_peered":"2024-09-12T12:19:49.753838+0000","last_clean":"2024-09-12T12:19:49.753838+0000","last_became_active":"2024-09-12T12:19:47.757816+0000","last_became_peered":"2024-09-12T12:19:47.757816+0000","last_unstale":"2024-09-12T12:19:49.753838+0000","last_undegraded":"2024-09-12T12:19:49.753838+0000","last_fullsized":"2024-09-12T12:19:49.753838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994340+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994340+0000","last_peered":"2024-09-12T12:19:49.994340+0000","last_clean":"2024-09-12T12:19:49.994340+0000","last_became_active":"2024-09-12T12:19:48.779234+0000","last_became_peered":"2024-09-12T12:19:48.779234+0000","last_unstale":"2024-09-12T12:19:49.994340+0000","last_undegraded":"2024-09-12T12:19:49.994340+0000","last_fullsized":"2024-09-12T12:19:49.994340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993920+0000","last_change":"2024-09-12T12:19:47.765924+0000","last_active":"2024-09-12T12:19:49.993920+0000","last_peered":"2024-09-12T12:19:49.993920+0000","last_clean":"2024-09-12T12:19:49.993920+0000","last_became_active":"2024-09-12T12:19:47.765244+0000","last_became_peered":"2024-09-12T12:19:47.765244+0000","last_unstale":"2024-09-12T12:19:49.993920+0000","last_undegraded":"2024-09-12T12:19:49.993920+0000","last_fullsized":"2024-09-12T12:19:49.993920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754093+0000","last_change":"2024-09-12T12:19:48.762307+0000","last_active":"2024-09-12T12:19:49.754093+0000","last_peered":"2024-09-12T12:19:49.754093+0000","last_clean":"2024-09-12T12:19:49.754093+0000","last_became_active":"2024-09-12T12:19:48.761728+0000","last_became_peered":"2024-09-12T12:19:48.761728+0000","last_unstale":"2024-09-12T12:19:49.754093+0000","last_undegraded":"2024-09-12T12:19:49.754093+0000","last_fullsized":"2024-09-12T12:19:49.754093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754345+0000","last_change":"2024-09-12T12:19:47.760726+0000","last_active":"2024-09-12T12:19:49.754345+0000","last_peered":"2024-09-12T12:19:49.754345+0000","last_clean":"2024-09-12T12:19:49.754345+0000","last_became_active":"2024-09-12T12:19:47.760059+0000","last_became_peered":"2024-09-12T12:19:47.760059+0000","last_unstale":"2024-09-12T12:19:49.754345+0000","last_undegraded":"2024-09-12T12:19:49.754345+0000","last_fullsized":"2024-09-12T12:19:49.754345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.750068+0000","last_change":"2024-09-12T12:19:48.768057+0000","last_active":"2024-09-12T12:19:49.750068+0000","last_peered":"2024-09-12T12:19:49.750068+0000","last_clean":"2024-09-12T12:19:49.750068+0000","last_became_active":"2024-09-12T12:19:48.766995+0000","last_became_peered":"2024-09-12T12:19:48.766995+0000","last_unstale":"2024-09-12T12:19:49.750068+0000","last_undegraded":"2024-09-12T12:19:49.750068+0000","last_fullsized":"2024-09-12T12:19:49.750068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:05.941564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758743+0000","last_change":"2024-09-12T12:19:47.765527+0000","last_active":"2024-09-12T12:19:49.758743+0000","last_peered":"2024-09-12T12:19:49.758743+0000","last_clean":"2024-09-12T12:19:49.758743+0000","last_became_active":"2024-09-12T12:19:47.764743+0000","last_became_peered":"2024-09-12T12:19:47.764743+0000","last_unstale":"2024-09-12T12:19:49.758743+0000","last_undegraded":"2024-09-12T12:19:49.758743+0000","last_fullsized":"2024-09-12T12:19:49.758743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992737+0000","last_change":"2024-09-12T12:19:48.761030+0000","last_active":"2024-09-12T12:19:49.992737+0000","last_peered":"2024-09-12T12:19:49.992737+0000","last_clean":"2024-09-12T12:19:49.992737+0000","last_became_active":"2024-09-12T12:19:48.760740+0000","last_became_peered":"2024-09-12T12:19:48.760740+0000","last_unstale":"2024-09-12T12:19:49.992737+0000","last_undegraded":"2024-09-12T12:19:49.992737+0000","last_fullsized":"2024-09-12T12:19:49.992737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:12.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992155+0000","last_change":"2024-09-12T12:19:47.764445+0000","last_active":"2024-09-12T12:19:49.992155+0000","last_peered":"2024-09-12T12:19:49.992155+0000","last_clean":"2024-09-12T12:19:49.992155+0000","last_became_active":"2024-09-12T12:19:47.763777+0000","last_became_peered":"2024-09-12T12:19:47.763777+0000","last_unstale":"2024-09-12T12:19:49.992155+0000","last_undegraded":"2024-09-12T12:19:49.992155+0000","last_fullsized":"2024-09-12T12:19:49.992155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748601+0000","last_change":"2024-09-12T12:19:48.755118+0000","last_active":"2024-09-12T12:19:49.748601+0000","last_peered":"2024-09-12T12:19:49.748601+0000","last_clean":"2024-09-12T12:19:49.748601+0000","last_became_active":"2024-09-12T12:19:48.754819+0000","last_became_peered":"2024-09-12T12:19:48.754819+0000","last_unstale":"2024-09-12T12:19:49.748601+0000","last_undegraded":"2024-09-12T12:19:49.748601+0000","last_fullsized":"2024-09-12T12:19:49.748601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:33.261675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753992+0000","last_change":"2024-09-12T12:19:47.755131+0000","last_active":"2024-09-12T12:19:49.753992+0000","last_peered":"2024-09-12T12:19:49.753992+0000","last_clean":"2024-09-12T12:19:49.753992+0000","last_became_active":"2024-09-12T12:19:47.754580+0000","last_became_peered":"2024-09-12T12:19:47.754580+0000","last_unstale":"2024-09-12T12:19:49.753992+0000","last_undegraded":"2024-09-12T12:19:49.753992+0000","last_fullsized":"2024-09-12T12:19:49.753992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748534+0000","last_change":"2024-09-12T12:19:48.778824+0000","last_active":"2024-09-12T12:19:49.748534+0000","last_peered":"2024-09-12T12:19:49.748534+0000","last_clean":"2024-09-12T12:19:49.748534+0000","last_became_active":"2024-09-12T12:19:48.766383+0000","last_became_peered":"2024-09-12T12:19:48.766383+0000","last_unstale":"2024-09-12T12:19:49.748534+0000","last_undegraded":"2024-09-12T12:19:49.748534+0000","last_fullsized":"2024-09-12T12:19:49.748534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:45.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753795+0000","last_change":"2024-09-12T12:19:47.751410+0000","last_active":"2024-09-12T12:19:49.753795+0000","last_peered":"2024-09-12T12:19:49.753795+0000","last_clean":"2024-09-12T12:19:49.753795+0000","last_became_active":"2024-09-12T12:19:47.750752+0000","last_became_peered":"2024-09-12T12:19:47.750752+0000","last_unstale":"2024-09-12T12:19:49.753795+0000","last_undegraded":"2024-09-12T12:19:49.753795+0000","last_fullsized":"2024-09-12T12:19:49.753795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749406+0000","last_change":"2024-09-12T12:19:48.779436+0000","last_active":"2024-09-12T12:19:49.749406+0000","last_peered":"2024-09-12T12:19:49.749406+0000","last_clean":"2024-09-12T12:19:49.749406+0000","last_became_active":"2024-09-12T12:19:48.779114+0000","last_became_peered":"2024-09-12T12:19:48.779114+0000","last_unstale":"2024-09-12T12:19:49.749406+0000","last_undegraded":"2024-09-12T12:19:49.749406+0000","last_fullsized":"2024-09-12T12:19:49.749406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:21.169633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993986+0000","last_change":"2024-09-12T12:19:47.768458+0000","last_active":"2024-09-12T12:19:49.993986+0000","last_peered":"2024-09-12T12:19:49.993986+0000","last_clean":"2024-09-12T12:19:49.993986+0000","last_became_active":"2024-09-12T12:19:47.767666+0000","last_became_peered":"2024-09-12T12:19:47.767666+0000","last_unstale":"2024-09-12T12:19:49.993986+0000","last_undegraded":"2024-09-12T12:19:49.993986+0000","last_fullsized":"2024-09-12T12:19:49.993986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758358+0000","last_change":"2024-09-12T12:19:48.760279+0000","last_active":"2024-09-12T12:19:49.758358+0000","last_peered":"2024-09-12T12:19:49.758358+0000","last_clean":"2024-09-12T12:19:49.758358+0000","last_became_active":"2024-09-12T12:19:48.759972+0000","last_became_peered":"2024-09-12T12:19:48.759972+0000","last_unstale":"2024-09-12T12:19:49.758358+0000","last_undegraded":"2024-09-12T12:19:49.758358+0000","last_fullsized":"2024-09-12T12:19:49.758358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993934+0000","last_change":"2024-09-12T12:19:48.765639+0000","last_active":"2024-09-12T12:19:49.993934+0000","last_peered":"2024-09-12T12:19:49.993934+0000","last_clean":"2024-09-12T12:19:49.993934+0000","last_became_active":"2024-09-12T12:19:48.765358+0000","last_became_peered":"2024-09-12T12:19:48.765358+0000","last_unstale":"2024-09-12T12:19:49.993934+0000","last_undegraded":"2024-09-12T12:19:49.993934+0000","last_fullsized":"2024-09-12T12:19:49.993934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994200+0000","last_change":"2024-09-12T12:19:47.763682+0000","last_active":"2024-09-12T12:19:49.994200+0000","last_peered":"2024-09-12T12:19:49.994200+0000","last_clean":"2024-09-12T12:19:49.994200+0000","last_became_active":"2024-09-12T12:19:47.762792+0000","last_became_peered":"2024-09-12T12:19:47.762792+0000","last_unstale":"2024-09-12T12:19:49.994200+0000","last_undegraded":"2024-09-12T12:19:49.994200+0000","last_fullsized":"2024-09-12T12:19:49.994200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995111+0000","last_change":"2024-09-12T12:19:48.765714+0000","last_active":"2024-09-12T12:19:49.995111+0000","last_peered":"2024-09-12T12:19:49.995111+0000","last_clean":"2024-09-12T12:19:49.995111+0000","last_became_active":"2024-09-12T12:19:48.765396+0000","last_became_peered":"2024-09-12T12:19:48.765396+0000","last_unstale":"2024-09-12T12:19:49.995111+0000","last_undegraded":"2024-09-12T12:19:49.995111+0000","last_fullsized":"2024-09-12T12:19:49.995111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993218+0000","last_change":"2024-09-12T12:19:47.770906+0000","last_active":"2024-09-12T12:19:49.993218+0000","last_peered":"2024-09-12T12:19:49.993218+0000","last_clean":"2024-09-12T12:19:49.993218+0000","last_became_active":"2024-09-12T12:19:47.770451+0000","last_became_peered":"2024-09-12T12:19:47.770451+0000","last_unstale":"2024-09-12T12:19:49.993218+0000","last_undegraded":"2024-09-12T12:19:49.993218+0000","last_fullsized":"2024-09-12T12:19:49.993218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993423+0000","last_change":"2024-09-12T12:19:48.774830+0000","last_active":"2024-09-12T12:19:49.993423+0000","last_peered":"2024-09-12T12:19:49.993423+0000","last_clean":"2024-09-12T12:19:49.993423+0000","last_became_active":"2024-09-12T12:19:48.774146+0000","last_became_peered":"2024-09-12T12:19:48.774146+0000","last_unstale":"2024-09-12T12:19:49.993423+0000","last_undegraded":"2024-09-12T12:19:49.993423+0000","last_fullsized":"2024-09-12T12:19:49.993423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:11.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749805+0000","last_change":"2024-09-12T12:19:47.759563+0000","last_active":"2024-09-12T12:19:49.749805+0000","last_peered":"2024-09-12T12:19:49.749805+0000","last_clean":"2024-09-12T12:19:49.749805+0000","last_became_active":"2024-09-12T12:19:47.759012+0000","last_became_peered":"2024-09-12T12:19:47.759012+0000","last_unstale":"2024-09-12T12:19:49.749805+0000","last_undegraded":"2024-09-12T12:19:49.749805+0000","last_fullsized":"2024-09-12T12:19:49.749805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:17:17.262245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749697+0000","last_change":"2024-09-12T12:19:48.784674+0000","last_active":"2024-09-12T12:19:49.749697+0000","last_peered":"2024-09-12T12:19:49.749697+0000","last_clean":"2024-09-12T12:19:49.749697+0000","last_became_active":"2024-09-12T12:19:48.784362+0000","last_became_peered":"2024-09-12T12:19:48.784362+0000","last_unstale":"2024-09-12T12:19:49.749697+0000","last_undegraded":"2024-09-12T12:19:49.749697+0000","last_fullsized":"2024-09-12T12:19:49.749697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:08:49.515897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993656+0000","last_change":"2024-09-12T12:19:47.755575+0000","last_active":"2024-09-12T12:19:49.993656+0000","last_peered":"2024-09-12T12:19:49.993656+0000","last_clean":"2024-09-12T12:19:49.993656+0000","last_became_active":"2024-09-12T12:19:47.755279+0000","last_became_peered":"2024-09-12T12:19:47.755279+0000","last_unstale":"2024-09-12T12:19:49.993656+0000","last_undegraded":"2024-09-12T12:19:49.993656+0000","last_fullsized":"2024-09-12T12:19:49.993656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:17:17.262245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":664833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":73973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":73883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":819200,"data_stored":664889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:20:01.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:20:01.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f936be61640 1 Processor -- start 2024-09-12T12:20:01.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f936be61640 1 -- start start 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f936be61640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f93640d5050 0x7f93640cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f936be61640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 0x7f93640b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f936be61640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93640cfe20 con 0x7f936415e5d0 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f936be61640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f93640b1060 con 0x7f93640d5050 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f936be61640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f93640d31a0 con 0x7f93640cfa10 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f93693fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f93640d5050 0x7f93640cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.863+0000 7f93693fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f93640d5050 0x7f93640cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41210/0 (socket says 172.21.15.3:41210) 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f93693fc640 1 -- 172.21.15.3:0/4061001023 learned_addr learned my addr 172.21.15.3:0/4061001023 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f93693fc640 1 -- 172.21.15.3:0/4061001023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 msgr2=0x7f93640b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f93693fc640 1 --2- 172.21.15.3:0/4061001023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 0x7f93640b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f93693fc640 1 -- 172.21.15.3:0/4061001023 >> v1:172.21.15.142:6789/0 conn(0x7f936415e5d0 legacy=0x7f93640ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f93693fc640 1 -- 172.21.15.3:0/4061001023 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93640d3480 con 0x7f93640d5050 2024-09-12T12:20:01.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f93693fc640 1 --2- 172.21.15.3:0/4061001023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93640d5050 0x7f93640cf4d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9354009870 tx=0x7f935401c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbad2a5e7cbffc3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:01.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f9368bfb640 1 -- 172.21.15.3:0/4061001023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93540b0070 con 0x7f93640d5050 2024-09-12T12:20:01.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f9368bfb640 1 -- 172.21.15.3:0/4061001023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93540ab600 con 0x7f93640d5050 2024-09-12T12:20:01.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f9368bfb640 1 -- 172.21.15.3:0/4061001023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93540ab9d0 con 0x7f93640d5050 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 -- 172.21.15.3:0/4061001023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93640d5050 msgr2=0x7f93640cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 --2- 172.21.15.3:0/4061001023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93640d5050 0x7f93640cf4d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9354009870 tx=0x7f935401c2b0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 -- 172.21.15.3:0/4061001023 shutdown_connections 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 --2- 172.21.15.3:0/4061001023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 0x7f93640b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 --2- 172.21.15.3:0/4061001023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93640d5050 0x7f93640cf4d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 -- 172.21.15.3:0/4061001023 >> 172.21.15.3:0/4061001023 conn(0x7f936408c000 msgr2=0x7f93640d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 -- 172.21.15.3:0/4061001023 shutdown_connections 2024-09-12T12:20:01.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.867+0000 7f936be61640 1 -- 172.21.15.3:0/4061001023 wait complete. 2024-09-12T12:20:01.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 Processor -- start 2024-09-12T12:20:01.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 -- start start 2024-09-12T12:20:01.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 0x7f93640b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93640d5050 0x7f93640c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f936415e5d0 0x7f9364156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f93640b2610 con 0x7f936415e5d0 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93640b2490 con 0x7f93640d5050 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f93640b2790 con 0x7f93640cfa10 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f936415e5d0 0x7f9364156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f936415e5d0 0x7f9364156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41220/0 (socket says 172.21.15.3:41220) 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 -- 172.21.15.3:0/2424589111 learned_addr learned my addr 172.21.15.3:0/2424589111 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f93693fc640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93640d5050 0x7f93640c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 -- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 msgr2=0x7f93640b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 0x7f93640b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 -- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93640d5050 msgr2=0x7f93640c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:01.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93640d5050 0x7f93640c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 -- 172.21.15.3:0/2424589111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936415a0a0 con 0x7f936415e5d0 2024-09-12T12:20:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936a3fe640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f936415e5d0 0x7f9364156c00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b9e0 tx=0x7f935c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f935a7fc640 1 -- 172.21.15.3:0/2424589111 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c019070 con 0x7f936415e5d0 2024-09-12T12:20:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f935a7fc640 1 -- 172.21.15.3:0/2424589111 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c00cac0 con 0x7f936415e5d0 2024-09-12T12:20:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936415a380 con 0x7f936415e5d0 2024-09-12T12:20:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f935a7fc640 1 -- 172.21.15.3:0/2424589111 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c014440 con 0x7f936415e5d0 2024-09-12T12:20:01.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936415a940 con 0x7f936415e5d0 2024-09-12T12:20:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93640b14f0 con 0x7f936415e5d0 2024-09-12T12:20:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f935a7fc640 1 -- 172.21.15.3:0/2424589111 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f935c01c070 con 0x7f936415e5d0 2024-09-12T12:20:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f935a7fc640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93480678c0 0x7f9348069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.871+0000 7f935a7fc640 1 -- 172.21.15.3:0/2424589111 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f935c085980 con 0x7f936415e5d0 2024-09-12T12:20:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.875+0000 7f9369bfd640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93480678c0 0x7f9348069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:01.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.875+0000 7f9369bfd640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93480678c0 0x7f9348069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9360009650 tx=0x7f9360006bb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:01.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:01.879+0000 7f935a7fc640 1 -- 172.21.15.3:0/2424589111 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f935c0564b0 con 0x7f936415e5d0 2024-09-12T12:20:02.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.031+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93640c9470 con 0x7f93480678c0 2024-09-12T12:20:02.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.047+0000 7f935a7fc640 1 -- 172.21.15.3:0/2424589111 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f9360009280 con 0x7f93480678c0 2024-09-12T12:20:02.054 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:02.056 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93480678c0 msgr2=0x7f9348069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93480678c0 0x7f9348069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9360009650 tx=0x7f9360006bb0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f936415e5d0 msgr2=0x7f9364156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f936415e5d0 0x7f9364156c00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b9e0 tx=0x7f935c00bea0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 shutdown_connections 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93480678c0 0x7f9348069d80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f936415e5d0 0x7f9364156c00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93640d5050 0x7f93640c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 --2- 172.21.15.3:0/2424589111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93640cfa10 0x7f93640b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 >> 172.21.15.3:0/2424589111 conn(0x7f936408c000 msgr2=0x7f93640d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 shutdown_connections 2024-09-12T12:20:02.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.051+0000 7f936be61640 1 -- 172.21.15.3:0/2424589111 wait complete. 2024-09-12T12:20:02.071 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-12T12:20:01.660130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":2957312,"data_stored":1776620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.915340"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995369+0000","last_change":"2024-09-12T12:19:48.781313+0000","last_active":"2024-09-12T12:19:49.995369+0000","last_peered":"2024-09-12T12:19:49.995369+0000","last_clean":"2024-09-12T12:19:49.995369+0000","last_became_active":"2024-09-12T12:19:48.780836+0000","last_became_peered":"2024-09-12T12:19:48.780836+0000","last_unstale":"2024-09-12T12:19:49.995369+0000","last_undegraded":"2024-09-12T12:19:49.995369+0000","last_fullsized":"2024-09-12T12:19:49.995369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754520+0000","last_change":"2024-09-12T12:19:48.759206+0000","last_active":"2024-09-12T12:19:49.754520+0000","last_peered":"2024-09-12T12:19:49.754520+0000","last_clean":"2024-09-12T12:19:49.754520+0000","last_became_active":"2024-09-12T12:19:48.758450+0000","last_became_peered":"2024-09-12T12:19:48.758450+0000","last_unstale":"2024-09-12T12:19:49.754520+0000","last_undegraded":"2024-09-12T12:19:49.754520+0000","last_fullsized":"2024-09-12T12:19:49.754520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992848+0000","last_change":"2024-09-12T12:19:48.769260+0000","last_active":"2024-09-12T12:19:49.992848+0000","last_peered":"2024-09-12T12:19:49.992848+0000","last_clean":"2024-09-12T12:19:49.992848+0000","last_became_active":"2024-09-12T12:19:48.768869+0000","last_became_peered":"2024-09-12T12:19:48.768869+0000","last_unstale":"2024-09-12T12:19:49.992848+0000","last_undegraded":"2024-09-12T12:19:49.992848+0000","last_fullsized":"2024-09-12T12:19:49.992848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:41:14.637521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994736+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994736+0000","last_peered":"2024-09-12T12:19:49.994736+0000","last_clean":"2024-09-12T12:19:49.994736+0000","last_became_active":"2024-09-12T12:19:48.779065+0000","last_became_peered":"2024-09-12T12:19:48.779065+0000","last_unstale":"2024-09-12T12:19:49.994736+0000","last_undegraded":"2024-09-12T12:19:49.994736+0000","last_fullsized":"2024-09-12T12:19:49.994736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993392+0000","last_change":"2024-09-12T12:19:48.766342+0000","last_active":"2024-09-12T12:19:49.993392+0000","last_peered":"2024-09-12T12:19:49.993392+0000","last_clean":"2024-09-12T12:19:49.993392+0000","last_became_active":"2024-09-12T12:19:48.766063+0000","last_became_peered":"2024-09-12T12:19:48.766063+0000","last_unstale":"2024-09-12T12:19:49.993392+0000","last_undegraded":"2024-09-12T12:19:49.993392+0000","last_fullsized":"2024-09-12T12:19:49.993392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993183+0000","last_change":"2024-09-12T12:19:48.769354+0000","last_active":"2024-09-12T12:19:49.993183+0000","last_peered":"2024-09-12T12:19:49.993183+0000","last_clean":"2024-09-12T12:19:49.993183+0000","last_became_active":"2024-09-12T12:19:48.769084+0000","last_became_peered":"2024-09-12T12:19:48.769084+0000","last_unstale":"2024-09-12T12:19:49.993183+0000","last_undegraded":"2024-09-12T12:19:49.993183+0000","last_fullsized":"2024-09-12T12:19:49.993183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:56.236676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992999+0000","last_change":"2024-09-12T12:19:48.773715+0000","last_active":"2024-09-12T12:19:49.992999+0000","last_peered":"2024-09-12T12:19:49.992999+0000","last_clean":"2024-09-12T12:19:49.992999+0000","last_became_active":"2024-09-12T12:19:48.773419+0000","last_became_peered":"2024-09-12T12:19:48.773419+0000","last_unstale":"2024-09-12T12:19:49.992999+0000","last_undegraded":"2024-09-12T12:19:49.992999+0000","last_fullsized":"2024-09-12T12:19:49.992999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748510+0000","last_change":"2024-09-12T12:19:48.779584+0000","last_active":"2024-09-12T12:19:49.748510+0000","last_peered":"2024-09-12T12:19:49.748510+0000","last_clean":"2024-09-12T12:19:49.748510+0000","last_became_active":"2024-09-12T12:19:48.777119+0000","last_became_peered":"2024-09-12T12:19:48.777119+0000","last_unstale":"2024-09-12T12:19:49.748510+0000","last_undegraded":"2024-09-12T12:19:49.748510+0000","last_fullsized":"2024-09-12T12:19:49.748510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:44.728800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995405+0000","last_change":"2024-09-12T12:19:48.775263+0000","last_active":"2024-09-12T12:19:49.995405+0000","last_peered":"2024-09-12T12:19:49.995405+0000","last_clean":"2024-09-12T12:19:49.995405+0000","last_became_active":"2024-09-12T12:19:48.774506+0000","last_became_peered":"2024-09-12T12:19:48.774506+0000","last_unstale":"2024-09-12T12:19:49.995405+0000","last_undegraded":"2024-09-12T12:19:49.995405+0000","last_fullsized":"2024-09-12T12:19:49.995405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:38:46.549546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759178+0000","last_change":"2024-09-12T12:19:48.778949+0000","last_active":"2024-09-12T12:19:49.759178+0000","last_peered":"2024-09-12T12:19:49.759178+0000","last_clean":"2024-09-12T12:19:49.759178+0000","last_became_active":"2024-09-12T12:19:48.778672+0000","last_became_peered":"2024-09-12T12:19:48.778672+0000","last_unstale":"2024-09-12T12:19:49.759178+0000","last_undegraded":"2024-09-12T12:19:49.759178+0000","last_fullsized":"2024-09-12T12:19:49.759178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758789+0000","last_change":"2024-09-12T12:19:48.779161+0000","last_active":"2024-09-12T12:19:49.758789+0000","last_peered":"2024-09-12T12:19:49.758789+0000","last_clean":"2024-09-12T12:19:49.758789+0000","last_became_active":"2024-09-12T12:19:48.778765+0000","last_became_peered":"2024-09-12T12:19:48.778765+0000","last_unstale":"2024-09-12T12:19:49.758789+0000","last_undegraded":"2024-09-12T12:19:49.758789+0000","last_fullsized":"2024-09-12T12:19:49.758789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749404+0000","last_change":"2024-09-12T12:19:48.762583+0000","last_active":"2024-09-12T12:19:49.749404+0000","last_peered":"2024-09-12T12:19:49.749404+0000","last_clean":"2024-09-12T12:19:49.749404+0000","last_became_active":"2024-09-12T12:19:48.762315+0000","last_became_peered":"2024-09-12T12:19:48.762315+0000","last_unstale":"2024-09-12T12:19:49.749404+0000","last_undegraded":"2024-09-12T12:19:49.749404+0000","last_fullsized":"2024-09-12T12:19:49.749404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749236+0000","last_change":"2024-09-12T12:19:48.784303+0000","last_active":"2024-09-12T12:19:49.749236+0000","last_peered":"2024-09-12T12:19:49.749236+0000","last_clean":"2024-09-12T12:19:49.749236+0000","last_became_active":"2024-09-12T12:19:48.779768+0000","last_became_peered":"2024-09-12T12:19:48.779768+0000","last_unstale":"2024-09-12T12:19:49.749236+0000","last_undegraded":"2024-09-12T12:19:49.749236+0000","last_fullsized":"2024-09-12T12:19:49.749236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:40.107345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993137+0000","last_change":"2024-09-12T12:19:48.763027+0000","last_active":"2024-09-12T12:19:49.993137+0000","last_peered":"2024-09-12T12:19:49.993137+0000","last_clean":"2024-09-12T12:19:49.993137+0000","last_became_active":"2024-09-12T12:19:48.762544+0000","last_became_peered":"2024-09-12T12:19:48.762544+0000","last_unstale":"2024-09-12T12:19:49.993137+0000","last_undegraded":"2024-09-12T12:19:49.993137+0000","last_fullsized":"2024-09-12T12:19:49.993137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:00:44.728800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753861+0000","last_change":"2024-09-12T12:19:48.770278+0000","last_active":"2024-09-12T12:19:49.753861+0000","last_peered":"2024-09-12T12:19:49.753861+0000","last_clean":"2024-09-12T12:19:49.753861+0000","last_became_active":"2024-09-12T12:19:48.769983+0000","last_became_peered":"2024-09-12T12:19:48.769983+0000","last_unstale":"2024-09-12T12:19:49.753861+0000","last_undegraded":"2024-09-12T12:19:49.753861+0000","last_fullsized":"2024-09-12T12:19:49.753861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992463+0000","last_change":"2024-09-12T12:19:48.769537+0000","last_active":"2024-09-12T12:19:49.992463+0000","last_peered":"2024-09-12T12:19:49.992463+0000","last_clean":"2024-09-12T12:19:49.992463+0000","last_became_active":"2024-09-12T12:19:48.769243+0000","last_became_peered":"2024-09-12T12:19:48.769243+0000","last_unstale":"2024-09-12T12:19:49.992463+0000","last_undegraded":"2024-09-12T12:19:49.992463+0000","last_fullsized":"2024-09-12T12:19:49.992463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995255+0000","last_change":"2024-09-12T12:19:48.766440+0000","last_active":"2024-09-12T12:19:49.995255+0000","last_peered":"2024-09-12T12:19:49.995255+0000","last_clean":"2024-09-12T12:19:49.995255+0000","last_became_active":"2024-09-12T12:19:48.766110+0000","last_became_peered":"2024-09-12T12:19:48.766110+0000","last_unstale":"2024-09-12T12:19:49.995255+0000","last_undegraded":"2024-09-12T12:19:49.995255+0000","last_fullsized":"2024-09-12T12:19:49.995255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992782+0000","last_change":"2024-09-12T12:19:48.780118+0000","last_active":"2024-09-12T12:19:49.992782+0000","last_peered":"2024-09-12T12:19:49.992782+0000","last_clean":"2024-09-12T12:19:49.992782+0000","last_became_active":"2024-09-12T12:19:48.779839+0000","last_became_peered":"2024-09-12T12:19:48.779839+0000","last_unstale":"2024-09-12T12:19:49.992782+0000","last_undegraded":"2024-09-12T12:19:49.992782+0000","last_fullsized":"2024-09-12T12:19:49.992782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993776+0000","last_change":"2024-09-12T12:19:48.766221+0000","last_active":"2024-09-12T12:19:49.993776+0000","last_peered":"2024-09-12T12:19:49.993776+0000","last_clean":"2024-09-12T12:19:49.993776+0000","last_became_active":"2024-09-12T12:19:48.765549+0000","last_became_peered":"2024-09-12T12:19:48.765549+0000","last_unstale":"2024-09-12T12:19:49.993776+0000","last_undegraded":"2024-09-12T12:19:49.993776+0000","last_fullsized":"2024-09-12T12:19:49.993776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758800+0000","last_change":"2024-09-12T12:19:48.764597+0000","last_active":"2024-09-12T12:19:49.758800+0000","last_peered":"2024-09-12T12:19:49.758800+0000","last_clean":"2024-09-12T12:19:49.758800+0000","last_became_active":"2024-09-12T12:19:48.764377+0000","last_became_peered":"2024-09-12T12:19:48.764377+0000","last_unstale":"2024-09-12T12:19:49.758800+0000","last_undegraded":"2024-09-12T12:19:49.758800+0000","last_fullsized":"2024-09-12T12:19:49.758800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992940+0000","last_change":"2024-09-12T12:19:48.763523+0000","last_active":"2024-09-12T12:19:49.992940+0000","last_peered":"2024-09-12T12:19:49.992940+0000","last_clean":"2024-09-12T12:19:49.992940+0000","last_became_active":"2024-09-12T12:19:48.762889+0000","last_became_peered":"2024-09-12T12:19:48.762889+0000","last_unstale":"2024-09-12T12:19:49.992940+0000","last_undegraded":"2024-09-12T12:19:49.992940+0000","last_fullsized":"2024-09-12T12:19:49.992940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:04:18.599614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993579+0000","last_change":"2024-09-12T12:19:48.760812+0000","last_active":"2024-09-12T12:19:49.993579+0000","last_peered":"2024-09-12T12:19:49.993579+0000","last_clean":"2024-09-12T12:19:49.993579+0000","last_became_active":"2024-09-12T12:19:48.760551+0000","last_became_peered":"2024-09-12T12:19:48.760551+0000","last_unstale":"2024-09-12T12:19:49.993579+0000","last_undegraded":"2024-09-12T12:19:49.993579+0000","last_fullsized":"2024-09-12T12:19:49.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748762+0000","last_change":"2024-09-12T12:19:48.760474+0000","last_active":"2024-09-12T12:19:49.748762+0000","last_peered":"2024-09-12T12:19:49.748762+0000","last_clean":"2024-09-12T12:19:49.748762+0000","last_became_active":"2024-09-12T12:19:48.760032+0000","last_became_peered":"2024-09-12T12:19:48.760032+0000","last_unstale":"2024-09-12T12:19:49.748762+0000","last_undegraded":"2024-09-12T12:19:49.748762+0000","last_fullsized":"2024-09-12T12:19:49.748762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:40.506967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992589+0000","last_change":"2024-09-12T12:19:48.780825+0000","last_active":"2024-09-12T12:19:49.992589+0000","last_peered":"2024-09-12T12:19:49.992589+0000","last_clean":"2024-09-12T12:19:49.992589+0000","last_became_active":"2024-09-12T12:19:48.780593+0000","last_became_peered":"2024-09-12T12:19:48.780593+0000","last_unstale":"2024-09-12T12:19:49.992589+0000","last_undegraded":"2024-09-12T12:19:49.992589+0000","last_fullsized":"2024-09-12T12:19:49.992589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:50.202989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749885+0000","last_change":"2024-09-12T12:19:48.759772+0000","last_active":"2024-09-12T12:19:49.749885+0000","last_peered":"2024-09-12T12:19:49.749885+0000","last_clean":"2024-09-12T12:19:49.749885+0000","last_became_active":"2024-09-12T12:19:48.759512+0000","last_became_peered":"2024-09-12T12:19:48.759512+0000","last_unstale":"2024-09-12T12:19:49.749885+0000","last_undegraded":"2024-09-12T12:19:49.749885+0000","last_fullsized":"2024-09-12T12:19:49.749885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:27:02.195687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.759372+0000","last_change":"2024-09-12T12:19:48.774953+0000","last_active":"2024-09-12T12:19:49.759372+0000","last_peered":"2024-09-12T12:19:49.759372+0000","last_clean":"2024-09-12T12:19:49.759372+0000","last_became_active":"2024-09-12T12:19:48.774474+0000","last_became_peered":"2024-09-12T12:19:48.774474+0000","last_unstale":"2024-09-12T12:19:49.759372+0000","last_undegraded":"2024-09-12T12:19:49.759372+0000","last_fullsized":"2024-09-12T12:19:49.759372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:20.695425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754066+0000","last_change":"2024-09-12T12:19:48.771914+0000","last_active":"2024-09-12T12:19:49.754066+0000","last_peered":"2024-09-12T12:19:49.754066+0000","last_clean":"2024-09-12T12:19:49.754066+0000","last_became_active":"2024-09-12T12:19:48.771637+0000","last_became_peered":"2024-09-12T12:19:48.771637+0000","last_unstale":"2024-09-12T12:19:49.754066+0000","last_undegraded":"2024-09-12T12:19:49.754066+0000","last_fullsized":"2024-09-12T12:19:49.754066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:51.038915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749153+0000","last_change":"2024-09-12T12:19:48.759783+0000","last_active":"2024-09-12T12:19:49.749153+0000","last_peered":"2024-09-12T12:19:49.749153+0000","last_clean":"2024-09-12T12:19:49.749153+0000","last_became_active":"2024-09-12T12:19:48.759473+0000","last_became_peered":"2024-09-12T12:19:48.759473+0000","last_unstale":"2024-09-12T12:19:49.749153+0000","last_undegraded":"2024-09-12T12:19:49.749153+0000","last_fullsized":"2024-09-12T12:19:49.749153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:11.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993036+0000","last_change":"2024-09-12T12:19:48.759011+0000","last_active":"2024-09-12T12:19:49.993036+0000","last_peered":"2024-09-12T12:19:49.993036+0000","last_clean":"2024-09-12T12:19:49.993036+0000","last_became_active":"2024-09-12T12:19:48.758267+0000","last_became_peered":"2024-09-12T12:19:48.758267+0000","last_unstale":"2024-09-12T12:19:49.993036+0000","last_undegraded":"2024-09-12T12:19:49.993036+0000","last_fullsized":"2024-09-12T12:19:49.993036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:45.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993953+0000","last_change":"2024-09-12T12:19:48.779776+0000","last_active":"2024-09-12T12:19:49.993953+0000","last_peered":"2024-09-12T12:19:49.993953+0000","last_clean":"2024-09-12T12:19:49.993953+0000","last_became_active":"2024-09-12T12:19:48.779500+0000","last_became_peered":"2024-09-12T12:19:48.779500+0000","last_unstale":"2024-09-12T12:19:49.993953+0000","last_undegraded":"2024-09-12T12:19:49.993953+0000","last_fullsized":"2024-09-12T12:19:49.993953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994016+0000","last_change":"2024-09-12T12:19:48.775150+0000","last_active":"2024-09-12T12:19:49.994016+0000","last_peered":"2024-09-12T12:19:49.994016+0000","last_clean":"2024-09-12T12:19:49.994016+0000","last_became_active":"2024-09-12T12:19:48.774841+0000","last_became_peered":"2024-09-12T12:19:48.774841+0000","last_unstale":"2024-09-12T12:19:49.994016+0000","last_undegraded":"2024-09-12T12:19:49.994016+0000","last_fullsized":"2024-09-12T12:19:49.994016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:38.998936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758353+0000","last_change":"2024-09-12T12:19:48.761408+0000","last_active":"2024-09-12T12:19:49.758353+0000","last_peered":"2024-09-12T12:19:49.758353+0000","last_clean":"2024-09-12T12:19:49.758353+0000","last_became_active":"2024-09-12T12:19:48.761091+0000","last_became_peered":"2024-09-12T12:19:48.761091+0000","last_unstale":"2024-09-12T12:19:49.758353+0000","last_undegraded":"2024-09-12T12:19:49.758353+0000","last_fullsized":"2024-09-12T12:19:49.758353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:51.034838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748611+0000","last_change":"2024-09-12T12:19:48.764078+0000","last_active":"2024-09-12T12:19:49.748611+0000","last_peered":"2024-09-12T12:19:49.748611+0000","last_clean":"2024-09-12T12:19:49.748611+0000","last_became_active":"2024-09-12T12:19:48.763831+0000","last_became_peered":"2024-09-12T12:19:48.763831+0000","last_unstale":"2024-09-12T12:19:49.748611+0000","last_undegraded":"2024-09-12T12:19:49.748611+0000","last_fullsized":"2024-09-12T12:19:49.748611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:06.350696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749493+0000","last_change":"2024-09-12T12:19:47.757834+0000","last_active":"2024-09-12T12:19:49.749493+0000","last_peered":"2024-09-12T12:19:49.749493+0000","last_clean":"2024-09-12T12:19:49.749493+0000","last_became_active":"2024-09-12T12:19:47.757145+0000","last_became_peered":"2024-09-12T12:19:47.757145+0000","last_unstale":"2024-09-12T12:19:49.749493+0000","last_undegraded":"2024-09-12T12:19:49.749493+0000","last_fullsized":"2024-09-12T12:19:49.749493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:13:55.749099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993352+0000","last_change":"2024-09-12T12:19:48.761558+0000","last_active":"2024-09-12T12:19:49.993352+0000","last_peered":"2024-09-12T12:19:49.993352+0000","last_clean":"2024-09-12T12:19:49.993352+0000","last_became_active":"2024-09-12T12:19:48.761230+0000","last_became_peered":"2024-09-12T12:19:48.761230+0000","last_unstale":"2024-09-12T12:19:49.993352+0000","last_undegraded":"2024-09-12T12:19:49.993352+0000","last_fullsized":"2024-09-12T12:19:49.993352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:40.506967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758441+0000","last_change":"2024-09-12T12:19:47.758747+0000","last_active":"2024-09-12T12:19:49.758441+0000","last_peered":"2024-09-12T12:19:49.758441+0000","last_clean":"2024-09-12T12:19:49.758441+0000","last_became_active":"2024-09-12T12:19:47.757870+0000","last_became_peered":"2024-09-12T12:19:47.757870+0000","last_unstale":"2024-09-12T12:19:49.758441+0000","last_undegraded":"2024-09-12T12:19:49.758441+0000","last_fullsized":"2024-09-12T12:19:49.758441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748588+0000","last_change":"2024-09-12T12:19:47.758883+0000","last_active":"2024-09-12T12:19:49.748588+0000","last_peered":"2024-09-12T12:19:49.748588+0000","last_clean":"2024-09-12T12:19:49.748588+0000","last_became_active":"2024-09-12T12:19:47.758396+0000","last_became_peered":"2024-09-12T12:19:47.758396+0000","last_unstale":"2024-09-12T12:19:49.748588+0000","last_undegraded":"2024-09-12T12:19:49.748588+0000","last_fullsized":"2024-09-12T12:19:49.748588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:49:34.294310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748361+0000","last_change":"2024-09-12T12:19:48.779910+0000","last_active":"2024-09-12T12:19:49.748361+0000","last_peered":"2024-09-12T12:19:49.748361+0000","last_clean":"2024-09-12T12:19:49.748361+0000","last_became_active":"2024-09-12T12:19:48.779613+0000","last_became_peered":"2024-09-12T12:19:48.779613+0000","last_unstale":"2024-09-12T12:19:49.748361+0000","last_undegraded":"2024-09-12T12:19:49.748361+0000","last_fullsized":"2024-09-12T12:19:49.748361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:04:18.599614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992602+0000","last_change":"2024-09-12T12:19:47.740584+0000","last_active":"2024-09-12T12:19:49.992602+0000","last_peered":"2024-09-12T12:19:49.992602+0000","last_clean":"2024-09-12T12:19:49.992602+0000","last_became_active":"2024-09-12T12:19:47.739985+0000","last_became_peered":"2024-09-12T12:19:47.739985+0000","last_unstale":"2024-09-12T12:19:49.992602+0000","last_undegraded":"2024-09-12T12:19:49.992602+0000","last_fullsized":"2024-09-12T12:19:49.992602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753847+0000","last_change":"2024-09-12T12:19:48.771435+0000","last_active":"2024-09-12T12:19:49.753847+0000","last_peered":"2024-09-12T12:19:49.753847+0000","last_clean":"2024-09-12T12:19:49.753847+0000","last_became_active":"2024-09-12T12:19:48.771108+0000","last_became_peered":"2024-09-12T12:19:48.771108+0000","last_unstale":"2024-09-12T12:19:49.753847+0000","last_undegraded":"2024-09-12T12:19:49.753847+0000","last_fullsized":"2024-09-12T12:19:49.753847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758579+0000","last_change":"2024-09-12T12:19:47.760876+0000","last_active":"2024-09-12T12:19:49.758579+0000","last_peered":"2024-09-12T12:19:49.758579+0000","last_clean":"2024-09-12T12:19:49.758579+0000","last_became_active":"2024-09-12T12:19:47.760036+0000","last_became_peered":"2024-09-12T12:19:47.760036+0000","last_unstale":"2024-09-12T12:19:49.758579+0000","last_undegraded":"2024-09-12T12:19:49.758579+0000","last_fullsized":"2024-09-12T12:19:49.758579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993540+0000","last_change":"2024-09-12T12:19:48.765500+0000","last_active":"2024-09-12T12:19:49.993540+0000","last_peered":"2024-09-12T12:19:49.993540+0000","last_clean":"2024-09-12T12:19:49.993540+0000","last_became_active":"2024-09-12T12:19:48.765191+0000","last_became_peered":"2024-09-12T12:19:48.765191+0000","last_unstale":"2024-09-12T12:19:49.993540+0000","last_undegraded":"2024-09-12T12:19:49.993540+0000","last_fullsized":"2024-09-12T12:19:49.993540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:17.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993676+0000","last_change":"2024-09-12T12:19:47.761042+0000","last_active":"2024-09-12T12:19:49.993676+0000","last_peered":"2024-09-12T12:19:49.993676+0000","last_clean":"2024-09-12T12:19:49.993676+0000","last_became_active":"2024-09-12T12:19:47.760261+0000","last_became_peered":"2024-09-12T12:19:47.760261+0000","last_unstale":"2024-09-12T12:19:49.993676+0000","last_undegraded":"2024-09-12T12:19:49.993676+0000","last_fullsized":"2024-09-12T12:19:49.993676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993444+0000","last_change":"2024-09-12T12:19:48.769338+0000","last_active":"2024-09-12T12:19:49.993444+0000","last_peered":"2024-09-12T12:19:49.993444+0000","last_clean":"2024-09-12T12:19:49.993444+0000","last_became_active":"2024-09-12T12:19:48.768955+0000","last_became_peered":"2024-09-12T12:19:48.768955+0000","last_unstale":"2024-09-12T12:19:49.993444+0000","last_undegraded":"2024-09-12T12:19:49.993444+0000","last_fullsized":"2024-09-12T12:19:49.993444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.656049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754127+0000","last_change":"2024-09-12T12:19:47.750999+0000","last_active":"2024-09-12T12:19:49.754127+0000","last_peered":"2024-09-12T12:19:49.754127+0000","last_clean":"2024-09-12T12:19:49.754127+0000","last_became_active":"2024-09-12T12:19:47.750658+0000","last_became_peered":"2024-09-12T12:19:47.750658+0000","last_unstale":"2024-09-12T12:19:49.754127+0000","last_undegraded":"2024-09-12T12:19:49.754127+0000","last_fullsized":"2024-09-12T12:19:49.754127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749115+0000","last_change":"2024-09-12T12:19:48.767953+0000","last_active":"2024-09-12T12:19:49.749115+0000","last_peered":"2024-09-12T12:19:49.749115+0000","last_clean":"2024-09-12T12:19:49.749115+0000","last_became_active":"2024-09-12T12:19:48.767595+0000","last_became_peered":"2024-09-12T12:19:48.767595+0000","last_unstale":"2024-09-12T12:19:49.749115+0000","last_undegraded":"2024-09-12T12:19:49.749115+0000","last_fullsized":"2024-09-12T12:19:49.749115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:36.791935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758590+0000","last_change":"2024-09-12T12:19:47.760548+0000","last_active":"2024-09-12T12:19:49.758590+0000","last_peered":"2024-09-12T12:19:49.758590+0000","last_clean":"2024-09-12T12:19:49.758590+0000","last_became_active":"2024-09-12T12:19:47.760033+0000","last_became_peered":"2024-09-12T12:19:47.760033+0000","last_unstale":"2024-09-12T12:19:49.758590+0000","last_undegraded":"2024-09-12T12:19:49.758590+0000","last_fullsized":"2024-09-12T12:19:49.758590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994749+0000","last_change":"2024-09-12T12:19:48.773829+0000","last_active":"2024-09-12T12:19:49.994749+0000","last_peered":"2024-09-12T12:19:49.994749+0000","last_clean":"2024-09-12T12:19:49.994749+0000","last_became_active":"2024-09-12T12:19:48.773359+0000","last_became_peered":"2024-09-12T12:19:48.773359+0000","last_unstale":"2024-09-12T12:19:49.994749+0000","last_undegraded":"2024-09-12T12:19:49.994749+0000","last_fullsized":"2024-09-12T12:19:49.994749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993173+0000","last_change":"2024-09-12T12:19:47.757426+0000","last_active":"2024-09-12T12:19:49.993173+0000","last_peered":"2024-09-12T12:19:49.993173+0000","last_clean":"2024-09-12T12:19:49.993173+0000","last_became_active":"2024-09-12T12:19:47.756782+0000","last_became_peered":"2024-09-12T12:19:47.756782+0000","last_unstale":"2024-09-12T12:19:49.993173+0000","last_undegraded":"2024-09-12T12:19:49.993173+0000","last_fullsized":"2024-09-12T12:19:49.993173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:00.248250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992472+0000","last_change":"2024-09-12T12:19:48.759150+0000","last_active":"2024-09-12T12:19:49.992472+0000","last_peered":"2024-09-12T12:19:49.992472+0000","last_clean":"2024-09-12T12:19:49.992472+0000","last_became_active":"2024-09-12T12:19:48.758654+0000","last_became_peered":"2024-09-12T12:19:48.758654+0000","last_unstale":"2024-09-12T12:19:49.992472+0000","last_undegraded":"2024-09-12T12:19:49.992472+0000","last_fullsized":"2024-09-12T12:19:49.992472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:24:37.527639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994079+0000","last_change":"2024-09-12T12:19:47.747437+0000","last_active":"2024-09-12T12:19:49.994079+0000","last_peered":"2024-09-12T12:19:49.994079+0000","last_clean":"2024-09-12T12:19:49.994079+0000","last_became_active":"2024-09-12T12:19:47.746926+0000","last_became_peered":"2024-09-12T12:19:47.746926+0000","last_unstale":"2024-09-12T12:19:49.994079+0000","last_undegraded":"2024-09-12T12:19:49.994079+0000","last_fullsized":"2024-09-12T12:19:49.994079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993790+0000","last_change":"2024-09-12T12:19:48.775580+0000","last_active":"2024-09-12T12:19:49.993790+0000","last_peered":"2024-09-12T12:19:49.993790+0000","last_clean":"2024-09-12T12:19:49.993790+0000","last_became_active":"2024-09-12T12:19:48.767021+0000","last_became_peered":"2024-09-12T12:19:48.767021+0000","last_unstale":"2024-09-12T12:19:49.993790+0000","last_undegraded":"2024-09-12T12:19:49.993790+0000","last_fullsized":"2024-09-12T12:19:49.993790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:04.066347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758162+0000","last_change":"2024-09-12T12:19:47.754802+0000","last_active":"2024-09-12T12:19:49.758162+0000","last_peered":"2024-09-12T12:19:49.758162+0000","last_clean":"2024-09-12T12:19:49.758162+0000","last_became_active":"2024-09-12T12:19:47.754242+0000","last_became_peered":"2024-09-12T12:19:47.754242+0000","last_unstale":"2024-09-12T12:19:49.758162+0000","last_undegraded":"2024-09-12T12:19:49.758162+0000","last_fullsized":"2024-09-12T12:19:49.758162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758450+0000","last_change":"2024-09-12T12:19:48.761340+0000","last_active":"2024-09-12T12:19:49.758450+0000","last_peered":"2024-09-12T12:19:49.758450+0000","last_clean":"2024-09-12T12:19:49.758450+0000","last_became_active":"2024-09-12T12:19:48.761037+0000","last_became_peered":"2024-09-12T12:19:48.761037+0000","last_unstale":"2024-09-12T12:19:49.758450+0000","last_undegraded":"2024-09-12T12:19:49.758450+0000","last_fullsized":"2024-09-12T12:19:49.758450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:12.393169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992730+0000","last_change":"2024-09-12T12:19:47.761479+0000","last_active":"2024-09-12T12:19:49.992730+0000","last_peered":"2024-09-12T12:19:49.992730+0000","last_clean":"2024-09-12T12:19:49.992730+0000","last_became_active":"2024-09-12T12:19:47.760812+0000","last_became_peered":"2024-09-12T12:19:47.760812+0000","last_unstale":"2024-09-12T12:19:49.992730+0000","last_undegraded":"2024-09-12T12:19:49.992730+0000","last_fullsized":"2024-09-12T12:19:49.992730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:28:29.835593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993407+0000","last_change":"2024-09-12T12:19:48.772189+0000","last_active":"2024-09-12T12:19:49.993407+0000","last_peered":"2024-09-12T12:19:49.993407+0000","last_clean":"2024-09-12T12:19:49.993407+0000","last_became_active":"2024-09-12T12:19:48.765342+0000","last_became_peered":"2024-09-12T12:19:48.765342+0000","last_unstale":"2024-09-12T12:19:49.993407+0000","last_undegraded":"2024-09-12T12:19:49.993407+0000","last_fullsized":"2024-09-12T12:19:49.993407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:32.450493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749324+0000","last_change":"2024-09-12T12:19:47.754107+0000","last_active":"2024-09-12T12:19:49.749324+0000","last_peered":"2024-09-12T12:19:49.749324+0000","last_clean":"2024-09-12T12:19:49.749324+0000","last_became_active":"2024-09-12T12:19:47.753433+0000","last_became_peered":"2024-09-12T12:19:47.753433+0000","last_unstale":"2024-09-12T12:19:49.749324+0000","last_undegraded":"2024-09-12T12:19:49.749324+0000","last_fullsized":"2024-09-12T12:19:49.749324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:28.270374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995095+0000","last_change":"2024-09-12T12:19:48.770240+0000","last_active":"2024-09-12T12:19:49.995095+0000","last_peered":"2024-09-12T12:19:49.995095+0000","last_clean":"2024-09-12T12:19:49.995095+0000","last_became_active":"2024-09-12T12:19:48.766786+0000","last_became_peered":"2024-09-12T12:19:48.766786+0000","last_unstale":"2024-09-12T12:19:49.995095+0000","last_undegraded":"2024-09-12T12:19:49.995095+0000","last_fullsized":"2024-09-12T12:19:49.995095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:33:59.703967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992870+0000","last_change":"2024-09-12T12:19:47.757967+0000","last_active":"2024-09-12T12:19:49.992870+0000","last_peered":"2024-09-12T12:19:49.992870+0000","last_clean":"2024-09-12T12:19:49.992870+0000","last_became_active":"2024-09-12T12:19:47.757370+0000","last_became_peered":"2024-09-12T12:19:47.757370+0000","last_unstale":"2024-09-12T12:19:49.992870+0000","last_undegraded":"2024-09-12T12:19:49.992870+0000","last_fullsized":"2024-09-12T12:19:49.992870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:02.522150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995730+0000","last_change":"2024-09-12T12:19:48.766283+0000","last_active":"2024-09-12T12:19:49.995730+0000","last_peered":"2024-09-12T12:19:49.995730+0000","last_clean":"2024-09-12T12:19:49.995730+0000","last_became_active":"2024-09-12T12:19:48.765694+0000","last_became_peered":"2024-09-12T12:19:48.765694+0000","last_unstale":"2024-09-12T12:19:49.995730+0000","last_undegraded":"2024-09-12T12:19:49.995730+0000","last_fullsized":"2024-09-12T12:19:49.995730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:42:42.831767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754237+0000","last_change":"2024-09-12T12:19:47.751839+0000","last_active":"2024-09-12T12:19:49.754237+0000","last_peered":"2024-09-12T12:19:49.754237+0000","last_clean":"2024-09-12T12:19:49.754237+0000","last_became_active":"2024-09-12T12:19:47.751482+0000","last_became_peered":"2024-09-12T12:19:47.751482+0000","last_unstale":"2024-09-12T12:19:49.754237+0000","last_undegraded":"2024-09-12T12:19:49.754237+0000","last_fullsized":"2024-09-12T12:19:49.754237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995497+0000","last_change":"2024-09-12T12:19:48.768636+0000","last_active":"2024-09-12T12:19:49.995497+0000","last_peered":"2024-09-12T12:19:49.995497+0000","last_clean":"2024-09-12T12:19:49.995497+0000","last_became_active":"2024-09-12T12:19:48.768342+0000","last_became_peered":"2024-09-12T12:19:48.768342+0000","last_unstale":"2024-09-12T12:19:49.995497+0000","last_undegraded":"2024-09-12T12:19:49.995497+0000","last_fullsized":"2024-09-12T12:19:49.995497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:43:34.278241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993248+0000","last_change":"2024-09-12T12:19:47.755829+0000","last_active":"2024-09-12T12:19:49.993248+0000","last_peered":"2024-09-12T12:19:49.993248+0000","last_clean":"2024-09-12T12:19:49.993248+0000","last_became_active":"2024-09-12T12:19:47.748443+0000","last_became_peered":"2024-09-12T12:19:47.748443+0000","last_unstale":"2024-09-12T12:19:49.993248+0000","last_undegraded":"2024-09-12T12:19:49.993248+0000","last_fullsized":"2024-09-12T12:19:49.993248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993070+0000","last_change":"2024-09-12T12:19:48.775968+0000","last_active":"2024-09-12T12:19:49.993070+0000","last_peered":"2024-09-12T12:19:49.993070+0000","last_clean":"2024-09-12T12:19:49.993070+0000","last_became_active":"2024-09-12T12:19:48.775731+0000","last_became_peered":"2024-09-12T12:19:48.775731+0000","last_unstale":"2024-09-12T12:19:49.993070+0000","last_undegraded":"2024-09-12T12:19:49.993070+0000","last_fullsized":"2024-09-12T12:19:49.993070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:13.032780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992710+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:19:49.992710+0000","last_peered":"2024-09-12T12:19:49.992710+0000","last_clean":"2024-09-12T12:19:49.992710+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:19:49.992710+0000","last_undegraded":"2024-09-12T12:19:49.992710+0000","last_fullsized":"2024-09-12T12:19:49.992710+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993779+0000","last_change":"2024-09-12T12:19:47.757991+0000","last_active":"2024-09-12T12:19:49.993779+0000","last_peered":"2024-09-12T12:19:49.993779+0000","last_clean":"2024-09-12T12:19:49.993779+0000","last_became_active":"2024-09-12T12:19:47.757562+0000","last_became_peered":"2024-09-12T12:19:47.757562+0000","last_unstale":"2024-09-12T12:19:49.993779+0000","last_undegraded":"2024-09-12T12:19:49.993779+0000","last_fullsized":"2024-09-12T12:19:49.993779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749165+0000","last_change":"2024-09-12T12:19:48.779259+0000","last_active":"2024-09-12T12:19:49.749165+0000","last_peered":"2024-09-12T12:19:49.749165+0000","last_clean":"2024-09-12T12:19:49.749165+0000","last_became_active":"2024-09-12T12:19:48.778963+0000","last_became_peered":"2024-09-12T12:19:48.778963+0000","last_unstale":"2024-09-12T12:19:49.749165+0000","last_undegraded":"2024-09-12T12:19:49.749165+0000","last_fullsized":"2024-09-12T12:19:49.749165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.656049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992441+0000","last_change":"2024-09-12T12:19:47.761015+0000","last_active":"2024-09-12T12:19:49.992441+0000","last_peered":"2024-09-12T12:19:49.992441+0000","last_clean":"2024-09-12T12:19:49.992441+0000","last_became_active":"2024-09-12T12:19:47.760494+0000","last_became_peered":"2024-09-12T12:19:47.760494+0000","last_unstale":"2024-09-12T12:19:49.992441+0000","last_undegraded":"2024-09-12T12:19:49.992441+0000","last_fullsized":"2024-09-12T12:19:49.992441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992375+0000","last_change":"2024-09-12T12:19:48.765122+0000","last_active":"2024-09-12T12:19:49.992375+0000","last_peered":"2024-09-12T12:19:49.992375+0000","last_clean":"2024-09-12T12:19:49.992375+0000","last_became_active":"2024-09-12T12:19:48.764670+0000","last_became_peered":"2024-09-12T12:19:48.764670+0000","last_unstale":"2024-09-12T12:19:49.992375+0000","last_undegraded":"2024-09-12T12:19:49.992375+0000","last_fullsized":"2024-09-12T12:19:49.992375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:57:01.970397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993998+0000","last_change":"2024-09-12T12:19:47.758292+0000","last_active":"2024-09-12T12:19:49.993998+0000","last_peered":"2024-09-12T12:19:49.993998+0000","last_clean":"2024-09-12T12:19:49.993998+0000","last_became_active":"2024-09-12T12:19:47.757643+0000","last_became_peered":"2024-09-12T12:19:47.757643+0000","last_unstale":"2024-09-12T12:19:49.993998+0000","last_undegraded":"2024-09-12T12:19:49.993998+0000","last_fullsized":"2024-09-12T12:19:49.993998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753888+0000","last_change":"2024-09-12T12:19:48.763042+0000","last_active":"2024-09-12T12:19:49.753888+0000","last_peered":"2024-09-12T12:19:49.753888+0000","last_clean":"2024-09-12T12:19:49.753888+0000","last_became_active":"2024-09-12T12:19:48.762214+0000","last_became_peered":"2024-09-12T12:19:48.762214+0000","last_unstale":"2024-09-12T12:19:49.753888+0000","last_undegraded":"2024-09-12T12:19:49.753888+0000","last_fullsized":"2024-09-12T12:19:49.753888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:56.695388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994326+0000","last_change":"2024-09-12T12:19:47.752251+0000","last_active":"2024-09-12T12:19:49.994326+0000","last_peered":"2024-09-12T12:19:49.994326+0000","last_clean":"2024-09-12T12:19:49.994326+0000","last_became_active":"2024-09-12T12:19:47.751530+0000","last_became_peered":"2024-09-12T12:19:47.751530+0000","last_unstale":"2024-09-12T12:19:49.994326+0000","last_undegraded":"2024-09-12T12:19:49.994326+0000","last_fullsized":"2024-09-12T12:19:49.994326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:15.549633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749074+0000","last_change":"2024-09-12T12:19:48.762555+0000","last_active":"2024-09-12T12:19:49.749074+0000","last_peered":"2024-09-12T12:19:49.749074+0000","last_clean":"2024-09-12T12:19:49.749074+0000","last_became_active":"2024-09-12T12:19:48.760885+0000","last_became_peered":"2024-09-12T12:19:48.760885+0000","last_unstale":"2024-09-12T12:19:49.749074+0000","last_undegraded":"2024-09-12T12:19:49.749074+0000","last_fullsized":"2024-09-12T12:19:49.749074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:16.559958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753838+0000","last_change":"2024-09-12T12:19:47.758162+0000","last_active":"2024-09-12T12:19:49.753838+0000","last_peered":"2024-09-12T12:19:49.753838+0000","last_clean":"2024-09-12T12:19:49.753838+0000","last_became_active":"2024-09-12T12:19:47.757816+0000","last_became_peered":"2024-09-12T12:19:47.757816+0000","last_unstale":"2024-09-12T12:19:49.753838+0000","last_undegraded":"2024-09-12T12:19:49.753838+0000","last_fullsized":"2024-09-12T12:19:49.753838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994340+0000","last_change":"2024-09-12T12:19:48.779987+0000","last_active":"2024-09-12T12:19:49.994340+0000","last_peered":"2024-09-12T12:19:49.994340+0000","last_clean":"2024-09-12T12:19:49.994340+0000","last_became_active":"2024-09-12T12:19:48.779234+0000","last_became_peered":"2024-09-12T12:19:48.779234+0000","last_unstale":"2024-09-12T12:19:49.994340+0000","last_undegraded":"2024-09-12T12:19:49.994340+0000","last_fullsized":"2024-09-12T12:19:49.994340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:18.141405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993920+0000","last_change":"2024-09-12T12:19:47.765924+0000","last_active":"2024-09-12T12:19:49.993920+0000","last_peered":"2024-09-12T12:19:49.993920+0000","last_clean":"2024-09-12T12:19:49.993920+0000","last_became_active":"2024-09-12T12:19:47.765244+0000","last_became_peered":"2024-09-12T12:19:47.765244+0000","last_unstale":"2024-09-12T12:19:49.993920+0000","last_undegraded":"2024-09-12T12:19:49.993920+0000","last_fullsized":"2024-09-12T12:19:49.993920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754093+0000","last_change":"2024-09-12T12:19:48.762307+0000","last_active":"2024-09-12T12:19:49.754093+0000","last_peered":"2024-09-12T12:19:49.754093+0000","last_clean":"2024-09-12T12:19:49.754093+0000","last_became_active":"2024-09-12T12:19:48.761728+0000","last_became_peered":"2024-09-12T12:19:48.761728+0000","last_unstale":"2024-09-12T12:19:49.754093+0000","last_undegraded":"2024-09-12T12:19:49.754093+0000","last_fullsized":"2024-09-12T12:19:49.754093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:28:28.797306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.754345+0000","last_change":"2024-09-12T12:19:47.760726+0000","last_active":"2024-09-12T12:19:49.754345+0000","last_peered":"2024-09-12T12:19:49.754345+0000","last_clean":"2024-09-12T12:19:49.754345+0000","last_became_active":"2024-09-12T12:19:47.760059+0000","last_became_peered":"2024-09-12T12:19:47.760059+0000","last_unstale":"2024-09-12T12:19:49.754345+0000","last_undegraded":"2024-09-12T12:19:49.754345+0000","last_fullsized":"2024-09-12T12:19:49.754345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:27.473751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.750068+0000","last_change":"2024-09-12T12:19:48.768057+0000","last_active":"2024-09-12T12:19:49.750068+0000","last_peered":"2024-09-12T12:19:49.750068+0000","last_clean":"2024-09-12T12:19:49.750068+0000","last_became_active":"2024-09-12T12:19:48.766995+0000","last_became_peered":"2024-09-12T12:19:48.766995+0000","last_unstale":"2024-09-12T12:19:49.750068+0000","last_undegraded":"2024-09-12T12:19:49.750068+0000","last_fullsized":"2024-09-12T12:19:49.750068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:05.941564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758743+0000","last_change":"2024-09-12T12:19:47.765527+0000","last_active":"2024-09-12T12:19:49.758743+0000","last_peered":"2024-09-12T12:19:49.758743+0000","last_clean":"2024-09-12T12:19:49.758743+0000","last_became_active":"2024-09-12T12:19:47.764743+0000","last_became_peered":"2024-09-12T12:19:47.764743+0000","last_unstale":"2024-09-12T12:19:49.758743+0000","last_undegraded":"2024-09-12T12:19:49.758743+0000","last_fullsized":"2024-09-12T12:19:49.758743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992737+0000","last_change":"2024-09-12T12:19:48.761030+0000","last_active":"2024-09-12T12:19:49.992737+0000","last_peered":"2024-09-12T12:19:49.992737+0000","last_clean":"2024-09-12T12:19:49.992737+0000","last_became_active":"2024-09-12T12:19:48.760740+0000","last_became_peered":"2024-09-12T12:19:48.760740+0000","last_unstale":"2024-09-12T12:19:49.992737+0000","last_undegraded":"2024-09-12T12:19:49.992737+0000","last_fullsized":"2024-09-12T12:19:49.992737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:03:12.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.992155+0000","last_change":"2024-09-12T12:19:47.764445+0000","last_active":"2024-09-12T12:19:49.992155+0000","last_peered":"2024-09-12T12:19:49.992155+0000","last_clean":"2024-09-12T12:19:49.992155+0000","last_became_active":"2024-09-12T12:19:47.763777+0000","last_became_peered":"2024-09-12T12:19:47.763777+0000","last_unstale":"2024-09-12T12:19:49.992155+0000","last_undegraded":"2024-09-12T12:19:49.992155+0000","last_fullsized":"2024-09-12T12:19:49.992155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:05.340370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748601+0000","last_change":"2024-09-12T12:19:48.755118+0000","last_active":"2024-09-12T12:19:49.748601+0000","last_peered":"2024-09-12T12:19:49.748601+0000","last_clean":"2024-09-12T12:19:49.748601+0000","last_became_active":"2024-09-12T12:19:48.754819+0000","last_became_peered":"2024-09-12T12:19:48.754819+0000","last_unstale":"2024-09-12T12:19:49.748601+0000","last_undegraded":"2024-09-12T12:19:49.748601+0000","last_fullsized":"2024-09-12T12:19:49.748601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:33.261675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753992+0000","last_change":"2024-09-12T12:19:47.755131+0000","last_active":"2024-09-12T12:19:49.753992+0000","last_peered":"2024-09-12T12:19:49.753992+0000","last_clean":"2024-09-12T12:19:49.753992+0000","last_became_active":"2024-09-12T12:19:47.754580+0000","last_became_peered":"2024-09-12T12:19:47.754580+0000","last_unstale":"2024-09-12T12:19:49.753992+0000","last_undegraded":"2024-09-12T12:19:49.753992+0000","last_fullsized":"2024-09-12T12:19:49.753992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.487788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.748534+0000","last_change":"2024-09-12T12:19:48.778824+0000","last_active":"2024-09-12T12:19:49.748534+0000","last_peered":"2024-09-12T12:19:49.748534+0000","last_clean":"2024-09-12T12:19:49.748534+0000","last_became_active":"2024-09-12T12:19:48.766383+0000","last_became_peered":"2024-09-12T12:19:48.766383+0000","last_unstale":"2024-09-12T12:19:49.748534+0000","last_undegraded":"2024-09-12T12:19:49.748534+0000","last_fullsized":"2024-09-12T12:19:49.748534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:45.047614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.753795+0000","last_change":"2024-09-12T12:19:47.751410+0000","last_active":"2024-09-12T12:19:49.753795+0000","last_peered":"2024-09-12T12:19:49.753795+0000","last_clean":"2024-09-12T12:19:49.753795+0000","last_became_active":"2024-09-12T12:19:47.750752+0000","last_became_peered":"2024-09-12T12:19:47.750752+0000","last_unstale":"2024-09-12T12:19:49.753795+0000","last_undegraded":"2024-09-12T12:19:49.753795+0000","last_fullsized":"2024-09-12T12:19:49.753795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:01:32.251350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749406+0000","last_change":"2024-09-12T12:19:48.779436+0000","last_active":"2024-09-12T12:19:49.749406+0000","last_peered":"2024-09-12T12:19:49.749406+0000","last_clean":"2024-09-12T12:19:49.749406+0000","last_became_active":"2024-09-12T12:19:48.779114+0000","last_became_peered":"2024-09-12T12:19:48.779114+0000","last_unstale":"2024-09-12T12:19:49.749406+0000","last_undegraded":"2024-09-12T12:19:49.749406+0000","last_fullsized":"2024-09-12T12:19:49.749406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:21.169633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993986+0000","last_change":"2024-09-12T12:19:47.768458+0000","last_active":"2024-09-12T12:19:49.993986+0000","last_peered":"2024-09-12T12:19:49.993986+0000","last_clean":"2024-09-12T12:19:49.993986+0000","last_became_active":"2024-09-12T12:19:47.767666+0000","last_became_peered":"2024-09-12T12:19:47.767666+0000","last_unstale":"2024-09-12T12:19:49.993986+0000","last_undegraded":"2024-09-12T12:19:49.993986+0000","last_fullsized":"2024-09-12T12:19:49.993986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:48.671457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.758358+0000","last_change":"2024-09-12T12:19:48.760279+0000","last_active":"2024-09-12T12:19:49.758358+0000","last_peered":"2024-09-12T12:19:49.758358+0000","last_clean":"2024-09-12T12:19:49.758358+0000","last_became_active":"2024-09-12T12:19:48.759972+0000","last_became_peered":"2024-09-12T12:19:48.759972+0000","last_unstale":"2024-09-12T12:19:49.758358+0000","last_undegraded":"2024-09-12T12:19:49.758358+0000","last_fullsized":"2024-09-12T12:19:49.758358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:04.435851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993934+0000","last_change":"2024-09-12T12:19:48.765639+0000","last_active":"2024-09-12T12:19:49.993934+0000","last_peered":"2024-09-12T12:19:49.993934+0000","last_clean":"2024-09-12T12:19:49.993934+0000","last_became_active":"2024-09-12T12:19:48.765358+0000","last_became_peered":"2024-09-12T12:19:48.765358+0000","last_unstale":"2024-09-12T12:19:49.993934+0000","last_undegraded":"2024-09-12T12:19:49.993934+0000","last_fullsized":"2024-09-12T12:19:49.993934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:38.269343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.994200+0000","last_change":"2024-09-12T12:19:47.763682+0000","last_active":"2024-09-12T12:19:49.994200+0000","last_peered":"2024-09-12T12:19:49.994200+0000","last_clean":"2024-09-12T12:19:49.994200+0000","last_became_active":"2024-09-12T12:19:47.762792+0000","last_became_peered":"2024-09-12T12:19:47.762792+0000","last_unstale":"2024-09-12T12:19:49.994200+0000","last_undegraded":"2024-09-12T12:19:49.994200+0000","last_fullsized":"2024-09-12T12:19:49.994200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:47:37.988611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.995111+0000","last_change":"2024-09-12T12:19:48.765714+0000","last_active":"2024-09-12T12:19:49.995111+0000","last_peered":"2024-09-12T12:19:49.995111+0000","last_clean":"2024-09-12T12:19:49.995111+0000","last_became_active":"2024-09-12T12:19:48.765396+0000","last_became_peered":"2024-09-12T12:19:48.765396+0000","last_unstale":"2024-09-12T12:19:49.995111+0000","last_undegraded":"2024-09-12T12:19:49.995111+0000","last_fullsized":"2024-09-12T12:19:49.995111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:10.911841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993218+0000","last_change":"2024-09-12T12:19:47.770906+0000","last_active":"2024-09-12T12:19:49.993218+0000","last_peered":"2024-09-12T12:19:49.993218+0000","last_clean":"2024-09-12T12:19:49.993218+0000","last_became_active":"2024-09-12T12:19:47.770451+0000","last_became_peered":"2024-09-12T12:19:47.770451+0000","last_unstale":"2024-09-12T12:19:49.993218+0000","last_undegraded":"2024-09-12T12:19:49.993218+0000","last_fullsized":"2024-09-12T12:19:49.993218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:35.781610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993423+0000","last_change":"2024-09-12T12:19:48.774830+0000","last_active":"2024-09-12T12:19:49.993423+0000","last_peered":"2024-09-12T12:19:49.993423+0000","last_clean":"2024-09-12T12:19:49.993423+0000","last_became_active":"2024-09-12T12:19:48.774146+0000","last_became_peered":"2024-09-12T12:19:48.774146+0000","last_unstale":"2024-09-12T12:19:49.993423+0000","last_undegraded":"2024-09-12T12:19:49.993423+0000","last_fullsized":"2024-09-12T12:19:49.993423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:11.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749805+0000","last_change":"2024-09-12T12:19:47.759563+0000","last_active":"2024-09-12T12:19:49.749805+0000","last_peered":"2024-09-12T12:19:49.749805+0000","last_clean":"2024-09-12T12:19:49.749805+0000","last_became_active":"2024-09-12T12:19:47.759012+0000","last_became_peered":"2024-09-12T12:19:47.759012+0000","last_unstale":"2024-09-12T12:19:49.749805+0000","last_undegraded":"2024-09-12T12:19:49.749805+0000","last_fullsized":"2024-09-12T12:19:49.749805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:17:17.262245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.749697+0000","last_change":"2024-09-12T12:19:48.784674+0000","last_active":"2024-09-12T12:19:49.749697+0000","last_peered":"2024-09-12T12:19:49.749697+0000","last_clean":"2024-09-12T12:19:49.749697+0000","last_became_active":"2024-09-12T12:19:48.784362+0000","last_became_peered":"2024-09-12T12:19:48.784362+0000","last_unstale":"2024-09-12T12:19:49.749697+0000","last_undegraded":"2024-09-12T12:19:49.749697+0000","last_fullsized":"2024-09-12T12:19:49.749697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:47.718118+0000","last_clean_scrub_stamp":"2024-09-12T12:19:47.718118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:08:49.515897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-12T12:19:49.993656+0000","last_change":"2024-09-12T12:19:47.755575+0000","last_active":"2024-09-12T12:19:49.993656+0000","last_peered":"2024-09-12T12:19:49.993656+0000","last_clean":"2024-09-12T12:19:49.993656+0000","last_became_active":"2024-09-12T12:19:47.755279+0000","last_became_peered":"2024-09-12T12:19:47.755279+0000","last_unstale":"2024-09-12T12:19:49.993656+0000","last_undegraded":"2024-09-12T12:19:49.993656+0000","last_fullsized":"2024-09-12T12:19:49.993656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:46.707792+0000","last_clean_scrub_stamp":"2024-09-12T12:19:46.707792+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:17:17.262245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":664833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":73973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":74009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":73883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":819200,"data_stored":664889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:20:02.075 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:20:02.075 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T12:20:02.076 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-12T12:20:02.076 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-12T12:20:02.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.231+0000 7fe6e0556640 1 Processor -- start 2024-09-12T12:20:02.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.231+0000 7fe6e0556640 1 -- start start 2024-09-12T12:20:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6e0556640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 0x7fe6d806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6e0556640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d8071ed0 con 0x7fe6d8144110 2024-09-12T12:20:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6e0556640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d80721b0 con 0x7fe6d80727b0 2024-09-12T12:20:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6e0556640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d8072490 con 0x7fe6d815ac30 2024-09-12T12:20:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6ddaf1640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe6d8144110 0x7fe6d80b4a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36242/0 (socket says 172.21.15.3:36242) 2024-09-12T12:20:02.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6ddaf1640 1 -- 172.21.15.3:0/158959706 learned_addr learned my addr 172.21.15.3:0/158959706 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:02.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2073894639 0 0) 0x7fe6d8071ed0 con 0x7fe6d8144110 2024-09-12T12:20:02.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6de2f2640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 0x7fe6d806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6c4003450 con 0x7fe6d8144110 2024-09-12T12:20:02.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1516753015 0 0) 0x7fe6c4003450 con 0x7fe6d8144110 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 msgr2=0x7fe6d806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 0x7fe6d806f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 >> v1:172.21.15.142:6789/0 conn(0x7fe6d80727b0 legacy=0x7fe6d80ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d80b4230 con 0x7fe6d8144110 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fe6c8001680 con 0x7fe6d8144110 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 >> v1:172.21.15.3:6789/0 conn(0x7fe6d8144110 legacy=0x7fe6d80b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c4003ae0 0x7fe6c4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 0x7fe6c400a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6c400d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8072490 con 0x7fe6c400b0a0 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c400ddb0 con 0x7fe6c4003ae0 2024-09-12T12:20:02.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c400dfb0 con 0x7fe6d815ac30 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6deaf3640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6c400d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6de2f2640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c4003ae0 0x7fe6c4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6ddaf1640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 0x7fe6c400a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6deaf3640 1 -- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 msgr2=0x7fe6c400a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6deaf3640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d815ac30 0x7fe6c400a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6deaf3640 1 -- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c4003ae0 msgr2=0x7fe6c4003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6deaf3640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6c4003ae0 0x7fe6c4003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6deaf3640 1 -- 172.21.15.3:0/158959706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c4003450 con 0x7fe6c400b0a0 2024-09-12T12:20:02.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.235+0000 7fe6deaf3640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6c400d530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0001e70 tx=0x7fe6d001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5870dcdd5475e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00af070 con 0x7fe6c400b0a0 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d0003b80 con 0x7fe6c400b0a0 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6dd2f0640 1 -- 172.21.15.3:0/158959706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0001a90 con 0x7fe6c400b0a0 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- 172.21.15.3:0/158959706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 msgr2=0x7fe6c400d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 --2- 172.21.15.3:0/158959706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6c400d530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0001e70 tx=0x7fe6d001c780 comp rx=0 tx=0).stop 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6deaf3640 1 -- 172.21.15.3:0/158959706 reap_dead start 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- 172.21.15.3:0/158959706 shutdown_connections 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- 172.21.15.3:0/158959706 >> 172.21.15.3:0/158959706 conn(0x7fe6d80b1d80 msgr2=0x7fe6d80b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- 172.21.15.3:0/158959706 shutdown_connections 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- 172.21.15.3:0/158959706 wait complete. 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 Processor -- start 2024-09-12T12:20:02.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- start start 2024-09-12T12:20:02.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6d8153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d814e150 0x7fe6d81fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d81fbd30 0x7fe6d81fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8072fd0 con 0x7fe6c400b0a0 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8072e50 con 0x7fe6d81fbd30 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d8153d20 con 0x7fe6d814e150 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6d8153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6d8153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41250/0 (socket says 172.21.15.3:41250) 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 -- 172.21.15.3:0/145405816 learned_addr learned my addr 172.21.15.3:0/145405816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6deaf3640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d81fbd30 0x7fe6d81fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6ddaf1640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d814e150 0x7fe6d81fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 -- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d814e150 msgr2=0x7fe6d81fb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d814e150 0x7fe6d81fb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 -- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d81fbd30 msgr2=0x7fe6d81fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d81fbd30 0x7fe6d81fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 -- 172.21.15.3:0/145405816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d8154000 con 0x7fe6c400b0a0 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6de2f2640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6d8153660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40115e0 tx=0x7fe6d4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6ceffd640 1 -- 172.21.15.3:0/145405816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400f8f0 con 0x7fe6c400b0a0 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6ceffd640 1 -- 172.21.15.3:0/145405816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d400fa90 con 0x7fe6c400b0a0 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6ceffd640 1 -- 172.21.15.3:0/145405816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400fe60 con 0x7fe6c400b0a0 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.239+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81542e0 con 0x7fe6c400b0a0 2024-09-12T12:20:02.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.243+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81548a0 con 0x7fe6c400b0a0 2024-09-12T12:20:02.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.243+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d8155180 con 0x7fe6c400b0a0 2024-09-12T12:20:02.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.243+0000 7fe6ceffd640 1 -- 172.21.15.3:0/145405816 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe6d4015460 con 0x7fe6c400b0a0 2024-09-12T12:20:02.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.243+0000 7fe6ceffd640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6bc0678f0 0x7fe6bc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.243+0000 7fe6ceffd640 1 -- 172.21.15.3:0/145405816 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d40865f0 con 0x7fe6c400b0a0 2024-09-12T12:20:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.243+0000 7fe6ddaf1640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6bc0678f0 0x7fe6bc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.243+0000 7fe6ddaf1640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6bc0678f0 0x7fe6bc069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80cc250 tx=0x7fe6c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.247+0000 7fe6ceffd640 1 -- 172.21.15.3:0/145405816 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe6d4019030 con 0x7fe6c400b0a0 2024-09-12T12:20:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.439+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe6d8157e80 con 0x7fe6c400b0a0 2024-09-12T12:20:02.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.439+0000 7fe6ceffd640 1 -- 172.21.15.3:0/145405816 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe6d4057130 con 0x7fe6c400b0a0 2024-09-12T12:20:02.443 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:02.443 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T12:20:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6bc0678f0 msgr2=0x7fe6bc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6bc0678f0 0x7fe6bc069db0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80cc250 tx=0x7fe6c8009040 comp rx=0 tx=0).stop 2024-09-12T12:20:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 msgr2=0x7fe6d8153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6d8153660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40115e0 tx=0x7fe6d4011aa0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 shutdown_connections 2024-09-12T12:20:02.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6bc0678f0 0x7fe6bc069db0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6d81fbd30 0x7fe6d81fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6d814e150 0x7fe6d81fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 --2- 172.21.15.3:0/145405816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6c400b0a0 0x7fe6d8153660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 >> 172.21.15.3:0/145405816 conn(0x7fe6d80b1d80 msgr2=0x7fe6d806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 shutdown_connections 2024-09-12T12:20:02.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.443+0000 7fe6e0556640 1 -- 172.21.15.3:0/145405816 wait complete. 2024-09-12T12:20:02.461 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-12T12:20:02.461 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.575+0000 7f59442d7640 1 Processor -- start 2024-09-12T12:20:02.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.575+0000 7f59442d7640 1 -- start start 2024-09-12T12:20:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f59442d7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 0x7f593c0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f59442d7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f593c070d40 0x7f593c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f59442d7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f593c06efc0 con 0x7f593c15c470 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f59442d7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f593c0b3b60 con 0x7f593c070d40 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f59442d7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f593c06f2a0 con 0x7f593c0b0a70 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5942073640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f593c15c470 0x7f593c070630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36246/0 (socket says 172.21.15.3:36246) 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5942073640 1 -- 172.21.15.3:0/966233346 learned_addr learned my addr 172.21.15.3:0/966233346 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5942874640 1 --2- 172.21.15.3:0/966233346 >> v2:172.21.15.142:3300/0 conn(0x7f593c070d40 0x7f593c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941872640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 0x7f593c0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751386092 0 0) 0x7f593c06efc0 con 0x7f593c15c470 2024-09-12T12:20:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5928003530 con 0x7f593c15c470 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165521438 0 0) 0x7f5928003530 con 0x7f593c15c470 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 msgr2=0x7f593c0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 0x7f593c0b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 >> v2:172.21.15.142:3300/0 conn(0x7f593c070d40 msgr2=0x7f593c0ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 --2- 172.21.15.3:0/966233346 >> v2:172.21.15.142:3300/0 conn(0x7f593c070d40 0x7f593c0ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c06f580 con 0x7f593c15c470 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f592c001680 con 0x7f593c15c470 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 >> v1:172.21.15.3:6789/0 conn(0x7f593c15c470 legacy=0x7f593c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 0x7f59280039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f5928004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928004d50 0x7f59280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f593c0b3b60 con 0x7f5928003f40 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f593c06f2a0 con 0x7f5928004d50 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f592800b9d0 con 0x7f593c0b0a70 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941872640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f5928004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5942874640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928004d50 0x7f59280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5942073640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 0x7f59280039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941872640 1 -- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 msgr2=0x7f59280039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941872640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b0a70 0x7f59280039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941872640 1 -- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928004d50 msgr2=0x7f59280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941872640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928004d50 0x7f59280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5941872640 1 -- 172.21.15.3:0/966233346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5928003530 con 0x7f5928003f40 2024-09-12T12:20:02.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.579+0000 7f5942874640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5928004d50 0x7f59280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5941872640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f5928004810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5938009ac0 tx=0x7f593801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccba9388a773305 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380b2070 con 0x7f5928003f40 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59380a5640 con 0x7f5928003f40 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5941071640 1 -- 172.21.15.3:0/966233346 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938009690 con 0x7f5928003f40 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- 172.21.15.3:0/966233346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 msgr2=0x7f5928004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 --2- 172.21.15.3:0/966233346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f5928004810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5938009ac0 tx=0x7f593801c9f0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942874640 1 -- 172.21.15.3:0/966233346 reap_dead start 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- 172.21.15.3:0/966233346 shutdown_connections 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- 172.21.15.3:0/966233346 >> 172.21.15.3:0/966233346 conn(0x7f593c09aa70 msgr2=0x7f593c0b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- 172.21.15.3:0/966233346 shutdown_connections 2024-09-12T12:20:02.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- 172.21.15.3:0/966233346 wait complete. 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 Processor -- start 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- start start 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f593c0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593c0b5f00 0x7f593c14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b6380 0x7f593c14ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f593c06fe20 con 0x7f5928003f40 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f593c06fb40 con 0x7f593c0b5f00 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f593c070340 con 0x7f593c0b6380 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942073640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f593c0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942874640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b6380 0x7f593c14ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5941872640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593c0b5f00 0x7f593c14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5941872640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593c0b5f00 0x7f593c14e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:56856/0 (socket says 172.21.15.3:56856) 2024-09-12T12:20:02.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5941872640 1 -- 172.21.15.3:0/2595672894 learned_addr learned my addr 172.21.15.3:0/2595672894 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942874640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b6380 0x7f593c14ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:42502/0 (socket says 172.21.15.3:42502) 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942073640 1 -- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b6380 msgr2=0x7f593c14ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942073640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b6380 0x7f593c14ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942073640 1 -- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593c0b5f00 msgr2=0x7f593c14e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942073640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593c0b5f00 0x7f593c14e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942073640 1 -- 172.21.15.3:0/2595672894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c067dd0 con 0x7f5928003f40 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5942073640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f593c0b43e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f592c010aa0 tx=0x7f592c010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5932ffd640 1 -- 172.21.15.3:0/2595672894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c01b070 con 0x7f5928003f40 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5932ffd640 1 -- 172.21.15.3:0/2595672894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c011040 con 0x7f5928003f40 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c0b4920 con 0x7f5928003f40 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f5932ffd640 1 -- 172.21.15.3:0/2595672894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c00ee60 con 0x7f5928003f40 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.583+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c0b4ea0 con 0x7f5928003f40 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.587+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593c0b3620 con 0x7f5928003f40 2024-09-12T12:20:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.587+0000 7f5932ffd640 1 -- 172.21.15.3:0/2595672894 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f592c01e070 con 0x7f5928003f40 2024-09-12T12:20:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.587+0000 7f5932ffd640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59200678c0 0x7f5920069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.587+0000 7f5932ffd640 1 -- 172.21.15.3:0/2595672894 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f592c088740 con 0x7f5928003f40 2024-09-12T12:20:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.587+0000 7f5941872640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59200678c0 0x7f5920069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.587+0000 7f5941872640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59200678c0 0x7f5920069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59380b0040 tx=0x7f59380046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.591+0000 7f5932ffd640 1 -- 172.21.15.3:0/2595672894 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f592c058cf0 con 0x7f5928003f40 2024-09-12T12:20:02.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.783+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f593c0b0710 con 0x7f5928003f40 2024-09-12T12:20:02.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.783+0000 7f5932ffd640 1 -- 172.21.15.3:0/2595672894 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f592c05cff0 con 0x7f5928003f40 2024-09-12T12:20:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:02.787 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:20:02.788 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59200678c0 msgr2=0x7f5920069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59200678c0 0x7f5920069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59380b0040 tx=0x7f59380046d0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 msgr2=0x7f593c0b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f593c0b43e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f592c010aa0 tx=0x7f592c010f60 comp rx=0 tx=0).stop 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 shutdown_connections 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59200678c0 0x7f5920069d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f593c0b6380 0x7f593c14ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593c0b5f00 0x7f593c14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 --2- 172.21.15.3:0/2595672894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5928003f40 0x7f593c0b43e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 >> 172.21.15.3:0/2595672894 conn(0x7f593c09aa70 msgr2=0x7f593c070630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 shutdown_connections 2024-09-12T12:20:02.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.787+0000 7f59442d7640 1 -- 172.21.15.3:0/2595672894 wait complete. 2024-09-12T12:20:02.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6870db5640 1 Processor -- start 2024-09-12T12:20:02.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6870db5640 1 -- start start 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6870db5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c159bd0 0x7f686c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6870db5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f686c159fb0 0x7f686c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6870db5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686c06e400 con 0x7f686c06f4a0 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6870db5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f686c06f010 con 0x7f686c159fb0 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6870db5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f686c06e6e0 con 0x7f686c159bd0 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686ad9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f686c06f4a0 0x7f686c0ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36254/0 (socket says 172.21.15.3:36254) 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686ad9d640 1 -- 172.21.15.3:0/181950849 learned_addr learned my addr 172.21.15.3:0/181950849 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6863fff640 1 --2- 172.21.15.3:0/181950849 >> v2:172.21.15.142:3300/0 conn(0x7f686c159fb0 0x7f686c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686a59c640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c159bd0 0x7f686c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 409963157 0 0) 0x7f686c06e400 con 0x7f686c06f4a0 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6858003530 con 0x7f686c06f4a0 2024-09-12T12:20:02.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3792276927 0 0) 0x7f6858003530 con 0x7f686c06f4a0 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c159bd0 msgr2=0x7f686c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c159bd0 0x7f686c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 >> v2:172.21.15.142:3300/0 conn(0x7f686c159fb0 msgr2=0x7f686c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 --2- 172.21.15.3:0/181950849 >> v2:172.21.15.142:3300/0 conn(0x7f686c159fb0 0x7f686c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c06e9c0 con 0x7f686c06f4a0 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f6864001a00 con 0x7f686c06f4a0 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6863fff640 1 --2- 172.21.15.3:0/181950849 >> v2:172.21.15.142:3300/0 conn(0x7f686c159fb0 0x7f686c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 >> v1:172.21.15.3:6789/0 conn(0x7f686c06f4a0 legacy=0x7f686c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68580039c0 0x7f6858003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6858004810 0x7f6858004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c159bd0 0x7f68580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f686c06f010 con 0x7f68580039c0 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f686c06e6e0 con 0x7f6858004810 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f685800b9d0 con 0x7f686c159bd0 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686a59c640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68580039c0 0x7f6858003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f6863fff640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6858004810 0x7f6858004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686a59c640 1 -- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c159bd0 msgr2=0x7f68580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686a59c640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c159bd0 0x7f68580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686a59c640 1 -- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6858004810 msgr2=0x7f6858004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686a59c640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6858004810 0x7f6858004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.915+0000 7f686a59c640 1 -- 172.21.15.3:0/181950849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858003530 con 0x7f68580039c0 2024-09-12T12:20:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f686a59c640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68580039c0 0x7f6858003de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f685c009ac0 tx=0x7f685c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2e65508f4239bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c009690 con 0x7f68580039c0 2024-09-12T12:20:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c0a64a0 con 0x7f68580039c0 2024-09-12T12:20:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6869d9b640 1 -- 172.21.15.3:0/181950849 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0a6870 con 0x7f68580039c0 2024-09-12T12:20:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- 172.21.15.3:0/181950849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68580039c0 msgr2=0x7f6858003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 --2- 172.21.15.3:0/181950849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68580039c0 0x7f6858003de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f685c009ac0 tx=0x7f685c01ca90 comp rx=0 tx=0).stop 2024-09-12T12:20:02.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f686ad9d640 1 -- 172.21.15.3:0/181950849 reap_dead start 2024-09-12T12:20:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- 172.21.15.3:0/181950849 shutdown_connections 2024-09-12T12:20:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- 172.21.15.3:0/181950849 >> 172.21.15.3:0/181950849 conn(0x7f686c0b1d80 msgr2=0x7f686c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- 172.21.15.3:0/181950849 shutdown_connections 2024-09-12T12:20:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- 172.21.15.3:0/181950849 wait complete. 2024-09-12T12:20:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 Processor -- start 2024-09-12T12:20:02.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- start start 2024-09-12T12:20:02.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68580039c0 0x7f686c145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c09a1e0 0x7f686c09afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c09a660 0x7f686c09b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f686c06dec0 con 0x7f686c09a660 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f686c06dd40 con 0x7f68580039c0 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f6870db5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f686c06e040 con 0x7f686c09a1e0 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f686ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c09a660 0x7f686c09b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f686ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c09a660 0x7f686c09b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41272/0 (socket says 172.21.15.3:41272) 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f686ad9d640 1 -- 172.21.15.3:0/796918716 learned_addr learned my addr 172.21.15.3:0/796918716 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.919+0000 7f686a59c640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68580039c0 0x7f686c145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6863fff640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c09a1e0 0x7f686c09afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686ad9d640 1 -- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c09a1e0 msgr2=0x7f686c09afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686ad9d640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c09a1e0 0x7f686c09afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686ad9d640 1 -- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68580039c0 msgr2=0x7f686c145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686ad9d640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68580039c0 0x7f686c145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686ad9d640 1 -- 172.21.15.3:0/796918716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c0cec50 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6863fff640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c09a1e0 0x7f686c09afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686ad9d640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c09a660 0x7f686c09b4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68640013e0 tx=0x7f686400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6862ffd640 1 -- 172.21.15.3:0/796918716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686400cea0 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6862ffd640 1 -- 172.21.15.3:0/796918716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686400a610 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c0cef30 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6862ffd640 1 -- 172.21.15.3:0/796918716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686400aa00 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c1f63d0 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686c15d2d0 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6862ffd640 1 -- 172.21.15.3:0/796918716 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6864011070 con 0x7f686c09a660 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6862ffd640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f684c0678f0 0x7f684c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f6862ffd640 1 -- 172.21.15.3:0/796918716 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f68640852d0 con 0x7f686c09a660 2024-09-12T12:20:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686a59c640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f684c0678f0 0x7f684c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.923+0000 7f686a59c640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f684c0678f0 0x7f684c069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f685c0ad000 tx=0x7f685c0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:02.927+0000 7f6862ffd640 1 -- 172.21.15.3:0/796918716 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6864055d90 con 0x7f686c09a660 2024-09-12T12:20:03.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.115+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f686c145cb0 con 0x7f686c09a660 2024-09-12T12:20:03.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6862ffd640 1 -- 172.21.15.3:0/796918716 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f6864055690 con 0x7f686c09a660 2024-09-12T12:20:03.121 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:03.121 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:20:03.121 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:20:03.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f684c0678f0 msgr2=0x7f684c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f684c0678f0 0x7f684c069db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f685c0ad000 tx=0x7f685c0b2040 comp rx=0 tx=0).stop 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c09a660 msgr2=0x7f686c09b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c09a660 0x7f686c09b4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68640013e0 tx=0x7f686400cc50 comp rx=0 tx=0).stop 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 shutdown_connections 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f684c0678f0 0x7f684c069db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f686c09a660 0x7f686c09b4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f686c09a1e0 0x7f686c09afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 --2- 172.21.15.3:0/796918716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68580039c0 0x7f686c145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 >> 172.21.15.3:0/796918716 conn(0x7f686c0b1d80 msgr2=0x7f686c15e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 shutdown_connections 2024-09-12T12:20:03.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.119+0000 7f6870db5640 1 -- 172.21.15.3:0/796918716 wait complete. 2024-09-12T12:20:03.139 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:20:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.255+0000 7fa2051e6640 1 Processor -- start 2024-09-12T12:20:03.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.255+0000 7fa2051e6640 1 -- start start 2024-09-12T12:20:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa2051e6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa20015b2d0 0x7fa2000b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa2051e6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa20006f6e0 0x7fa200072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa2051e6640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa20015f150 con 0x7fa2000cc300 2024-09-12T12:20:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa2051e6640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa2000ae920 con 0x7fa20006f6e0 2024-09-12T12:20:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa2051e6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa20015b6b0 con 0x7fa20015b2d0 2024-09-12T12:20:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fe59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa20006f6e0 0x7fa200072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1ff59e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa2000cc300 0x7fa2000aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36262/0 (socket says 172.21.15.3:36262) 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1ff59e640 1 -- 172.21.15.3:0/423624520 learned_addr learned my addr 172.21.15.3:0/423624520 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fe59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa20006f6e0 0x7fa200072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41284/0 (socket says 172.21.15.3:41284) 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fed9d640 1 --2- 172.21.15.3:0/423624520 >> v2:172.21.15.142:3300/0 conn(0x7fa20015b2d0 0x7fa2000b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fe59c640 1 -- 172.21.15.3:0/423624520 >> v2:172.21.15.142:3300/0 conn(0x7fa20015b2d0 msgr2=0x7fa2000b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fe59c640 1 --2- 172.21.15.3:0/423624520 >> v2:172.21.15.142:3300/0 conn(0x7fa20015b2d0 0x7fa2000b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fe59c640 1 -- 172.21.15.3:0/423624520 >> v1:172.21.15.3:6789/0 conn(0x7fa2000cc300 legacy=0x7fa2000aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fdd9b640 1 -- 172.21.15.3:0/423624520 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 271076184 0 0) 0x7fa20015f150 con 0x7fa2000cc300 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fe59c640 1 -- 172.21.15.3:0/423624520 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20015ee70 con 0x7fa20006f6e0 2024-09-12T12:20:03.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fed9d640 1 --2- 172.21.15.3:0/423624520 >> v2:172.21.15.142:3300/0 conn(0x7fa20015b2d0 0x7fa2000b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:03.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fe59c640 1 --2- 172.21.15.3:0/423624520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20006f6e0 0x7fa200072f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa20006fe30 tx=0x7fa1e801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12b24934a32d00f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fdd9b640 1 -- 172.21.15.3:0/423624520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80b2070 con 0x7fa20006f6e0 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fdd9b640 1 -- 172.21.15.3:0/423624520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e80a9070 con 0x7fa20006f6e0 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa1fdd9b640 1 -- 172.21.15.3:0/423624520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80ad880 con 0x7fa20006f6e0 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa2051e6640 1 -- 172.21.15.3:0/423624520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20006f6e0 msgr2=0x7fa200072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.259+0000 7fa2051e6640 1 --2- 172.21.15.3:0/423624520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20006f6e0 0x7fa200072f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa20006fe30 tx=0x7fa1e801c450 comp rx=0 tx=0).stop 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- 172.21.15.3:0/423624520 shutdown_connections 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 --2- 172.21.15.3:0/423624520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa20006f6e0 0x7fa200072f30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 --2- 172.21.15.3:0/423624520 >> v2:172.21.15.142:3300/0 conn(0x7fa20015b2d0 0x7fa2000b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- 172.21.15.3:0/423624520 >> 172.21.15.3:0/423624520 conn(0x7fa2000b1d80 msgr2=0x7fa2000b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- 172.21.15.3:0/423624520 shutdown_connections 2024-09-12T12:20:03.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- 172.21.15.3:0/423624520 wait complete. 2024-09-12T12:20:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 Processor -- start 2024-09-12T12:20:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- start start 2024-09-12T12:20:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa20006f6e0 0x7fa2001509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2000cc300 0x7fa20009cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20015b2d0 0x7fa200165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa20006d8b0 con 0x7fa2000cc300 2024-09-12T12:20:03.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa200073880 con 0x7fa20015b2d0 2024-09-12T12:20:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa2000b35c0 con 0x7fa20006f6e0 2024-09-12T12:20:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2000cc300 0x7fa20009cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2000cc300 0x7fa20009cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41288/0 (socket says 172.21.15.3:41288) 2024-09-12T12:20:03.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 -- 172.21.15.3:0/742689586 learned_addr learned my addr 172.21.15.3:0/742689586 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 -- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa20006f6e0 msgr2=0x7fa2001509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa20006f6e0 0x7fa2001509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 -- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20015b2d0 msgr2=0x7fa200165440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20015b2d0 0x7fa200165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 -- 172.21.15.3:0/742689586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20009d460 con 0x7fa2000cc300 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1fe59c640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2000cc300 0x7fa20009cad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8019060 tx=0x7fa1e801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1f77fe640 1 -- 172.21.15.3:0/742689586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80b2050 con 0x7fa2000cc300 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1f77fe640 1 -- 172.21.15.3:0/742689586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e80be070 con 0x7fa2000cc300 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa1f77fe640 1 -- 172.21.15.3:0/742689586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80b98b0 con 0x7fa2000cc300 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20009d740 con 0x7fa2000cc300 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20009dd00 con 0x7fa2000cc300 2024-09-12T12:20:03.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.263+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa200155da0 con 0x7fa2000cc300 2024-09-12T12:20:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.267+0000 7fa1f77fe640 1 -- 172.21.15.3:0/742689586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa1e80ab070 con 0x7fa2000cc300 2024-09-12T12:20:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.267+0000 7fa1f77fe640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa1e00678f0 0x7fa1e0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.267+0000 7fa1f77fe640 1 -- 172.21.15.3:0/742689586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e81202a0 con 0x7fa2000cc300 2024-09-12T12:20:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.267+0000 7fa1fed9d640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa1e00678f0 0x7fa1e0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.267+0000 7fa1fed9d640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa1e00678f0 0x7fa1e0069db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000a8d0 tx=0x7fa1f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:03.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.271+0000 7fa1f77fe640 1 -- 172.21.15.3:0/742689586 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa1e8016070 con 0x7fa2000cc300 2024-09-12T12:20:03.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.431+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2001557e0 con 0x7fa2000cc300 2024-09-12T12:20:03.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.435+0000 7fa1f77fe640 1 -- 172.21.15.3:0/742689586 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fa1e80f0f00 con 0x7fa2000cc300 2024-09-12T12:20:03.438 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:03.439 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:49.738546+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.435+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa1e00678f0 msgr2=0x7fa1e0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.435+0000 7fa2051e6640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa1e00678f0 0x7fa1e0069db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000a8d0 tx=0x7fa1f0008040 comp rx=0 tx=0).stop 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.435+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2000cc300 msgr2=0x7fa20009cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.435+0000 7fa2051e6640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2000cc300 0x7fa20009cad0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8019060 tx=0x7fa1e801ac90 comp rx=0 tx=0).stop 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 shutdown_connections 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa1e00678f0 0x7fa1e0069db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa20015b2d0 0x7fa200165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2000cc300 0x7fa20009cad0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 --2- 172.21.15.3:0/742689586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa20006f6e0 0x7fa2001509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 >> 172.21.15.3:0/742689586 conn(0x7fa2000b1d80 msgr2=0x7fa20015f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 shutdown_connections 2024-09-12T12:20:03.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.439+0000 7fa2051e6640 1 -- 172.21.15.3:0/742689586 wait complete. 2024-09-12T12:20:03.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:03.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.563+0000 7f82ac842640 1 Processor -- start 2024-09-12T12:20:03.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.563+0000 7f82ac842640 1 -- start start 2024-09-12T12:20:03.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82ac842640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82a815e800 0x7f82a80b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82ac842640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a80b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82ac842640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82a806bcc0 0x7f82a806c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82ac842640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f82a815f940 con 0x7f82a815e800 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82ac842640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f82a815fc20 con 0x7f82a806bcc0 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82ac842640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82a815ff00 con 0x7f82a80b5d50 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82a815e800 0x7f82a80b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82a815e800 0x7f82a80b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41292/0 (socket says 172.21.15.3:41292) 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 -- 172.21.15.3:0/3396123088 learned_addr learned my addr 172.21.15.3:0/3396123088 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a6d9d640 1 --2- 172.21.15.3:0/3396123088 >> v2:172.21.15.142:3300/0 conn(0x7f82a806bcc0 0x7f82a806c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a5d9b640 1 --2- 172.21.15.3:0/3396123088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a80b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 -- 172.21.15.3:0/3396123088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 msgr2=0x7f82a80b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:03.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 --2- 172.21.15.3:0/3396123088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a80b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 -- 172.21.15.3:0/3396123088 >> v2:172.21.15.142:3300/0 conn(0x7f82a806bcc0 msgr2=0x7f82a806c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:03.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 --2- 172.21.15.3:0/3396123088 >> v2:172.21.15.142:3300/0 conn(0x7f82a806bcc0 0x7f82a806c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.567+0000 7f82a659c640 1 -- 172.21.15.3:0/3396123088 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a81601e0 con 0x7f82a815e800 2024-09-12T12:20:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a659c640 1 --2- 172.21.15.3:0/3396123088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a815e800 0x7f82a80b42a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82a8073fe0 tx=0x7f829801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b4f5dd6e20c3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a559a640 1 -- 172.21.15.3:0/3396123088 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980a5840 con 0x7f82a815e800 2024-09-12T12:20:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a559a640 1 -- 172.21.15.3:0/3396123088 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82980a59e0 con 0x7f82a815e800 2024-09-12T12:20:03.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a559a640 1 -- 172.21.15.3:0/3396123088 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980a5db0 con 0x7f82a815e800 2024-09-12T12:20:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- 172.21.15.3:0/3396123088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a815e800 msgr2=0x7f82a80b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 --2- 172.21.15.3:0/3396123088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a815e800 0x7f82a80b42a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f82a8073fe0 tx=0x7f829801c9c0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- 172.21.15.3:0/3396123088 shutdown_connections 2024-09-12T12:20:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 --2- 172.21.15.3:0/3396123088 >> v2:172.21.15.142:3300/0 conn(0x7f82a806bcc0 0x7f82a806c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 --2- 172.21.15.3:0/3396123088 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a80b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 --2- 172.21.15.3:0/3396123088 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a815e800 0x7f82a80b42a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- 172.21.15.3:0/3396123088 >> 172.21.15.3:0/3396123088 conn(0x7f82a8073500 msgr2=0x7f82a80b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- 172.21.15.3:0/3396123088 shutdown_connections 2024-09-12T12:20:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- 172.21.15.3:0/3396123088 wait complete. 2024-09-12T12:20:03.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 Processor -- start 2024-09-12T12:20:03.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- start start 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a806bcc0 0x7f82a815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a814fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a815e800 0x7f82a8150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82a80b47e0 con 0x7f82a806bcc0 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82a80b66d0 con 0x7f82a815e800 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82ac842640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82a80b2b10 con 0x7f82a80b5d50 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a806bcc0 0x7f82a815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a806bcc0 0x7f82a815ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41294/0 (socket says 172.21.15.3:41294) 2024-09-12T12:20:03.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a659c640 1 -- 172.21.15.3:0/1167708573 learned_addr learned my addr 172.21.15.3:0/1167708573 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a6d9d640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a815e800 0x7f82a8150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a5d9b640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a814fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.571+0000 7f82a659c640 1 -- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 msgr2=0x7f82a814fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a659c640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a814fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a659c640 1 -- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a815e800 msgr2=0x7f82a8150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a659c640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a815e800 0x7f82a8150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a659c640 1 -- 172.21.15.3:0/1167708573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a81508f0 con 0x7f82a806bcc0 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a5d9b640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a814fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a659c640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a806bcc0 0x7f82a815ba20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82980af040 tx=0x7f8298019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f8296ffd640 1 -- 172.21.15.3:0/1167708573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980bc020 con 0x7f82a806bcc0 2024-09-12T12:20:03.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f8296ffd640 1 -- 172.21.15.3:0/1167708573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82980b8680 con 0x7f82a806bcc0 2024-09-12T12:20:03.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a8150bd0 con 0x7f82a806bcc0 2024-09-12T12:20:03.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f8296ffd640 1 -- 172.21.15.3:0/1167708573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980b8a50 con 0x7f82a806bcc0 2024-09-12T12:20:03.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a8151190 con 0x7f82a806bcc0 2024-09-12T12:20:03.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a80b6190 con 0x7f82a806bcc0 2024-09-12T12:20:03.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f8296ffd640 1 -- 172.21.15.3:0/1167708573 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f82980bf020 con 0x7f82a806bcc0 2024-09-12T12:20:03.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f8296ffd640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82880678f0 0x7f8288069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:03.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f8296ffd640 1 -- 172.21.15.3:0/1167708573 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8298120b10 con 0x7f82a806bcc0 2024-09-12T12:20:03.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a5d9b640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82880678f0 0x7f8288069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:03.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.575+0000 7f82a5d9b640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82880678f0 0x7f8288069db0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f82a80d0a60 tx=0x7f829c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:03.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.579+0000 7f8296ffd640 1 -- 172.21.15.3:0/1167708573 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f82980165e0 con 0x7f82a806bcc0 2024-09-12T12:20:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.775+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82a806edd0 con 0x7f82a806bcc0 2024-09-12T12:20:03.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.775+0000 7f8296ffd640 1 -- 172.21.15.3:0/1167708573 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f82980f1800 con 0x7f82a806bcc0 2024-09-12T12:20:03.780 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:03.780 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:20:03.781 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:20:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82880678f0 msgr2=0x7f8288069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82880678f0 0x7f8288069db0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f82a80d0a60 tx=0x7f829c008040 comp rx=0 tx=0).stop 2024-09-12T12:20:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a806bcc0 msgr2=0x7f82a815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a806bcc0 0x7f82a815ba20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82980af040 tx=0x7f8298019760 comp rx=0 tx=0).stop 2024-09-12T12:20:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 shutdown_connections 2024-09-12T12:20:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82880678f0 0x7f8288069db0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a815e800 0x7f82a8150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a80b5d50 0x7f82a814fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 --2- 172.21.15.3:0/1167708573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a806bcc0 0x7f82a815ba20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 >> 172.21.15.3:0/1167708573 conn(0x7f82a8073500 msgr2=0x7f82a8158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 shutdown_connections 2024-09-12T12:20:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:03.779+0000 7f82ac842640 1 -- 172.21.15.3:0/1167708573 wait complete. 2024-09-12T12:20:04.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:19:48.098984+0000', 'modified': '2024-09-12T12:19:51.114095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6837/2900613310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2900613310}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2900613310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4306, 'qdb_cluster': [4306]} 2024-09-12T12:20:04.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:20:04.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-12T12:20:04.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0caa158640 1 Processor -- start 2024-09-12T12:20:04.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0caa158640 1 -- start start 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0caa158640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ca40d0a90 0x7f0ca4072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0caa158640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca406f010 con 0x7f0ca4073320 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0caa158640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca406f2f0 con 0x7f0ca40af550 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0caa158640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca4073ad0 con 0x7f0ca40d0a90 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0ca2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ca40d0a90 0x7f0ca4072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0ca3fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0ca4073320 0x7f0ca415b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36266/0 (socket says 172.21.15.3:36266) 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0ca2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0ca40d0a90 0x7f0ca4072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41300/0 (socket says 172.21.15.3:41300) 2024-09-12T12:20:04.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0ca3fff640 1 -- 172.21.15.3:0/121818402 learned_addr learned my addr 172.21.15.3:0/121818402 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:04.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/121818402 >> v1:172.21.15.142:6789/0 conn(0x7f0ca40af550 legacy=0x7f0ca40d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/121818402 >> v1:172.21.15.3:6789/0 conn(0x7f0ca4073320 legacy=0x7f0ca415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.331+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/121818402 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca406f5d0 con 0x7f0ca40d0a90 2024-09-12T12:20:04.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/121818402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4072de0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c980097c0 tx=0x7f0c9801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc84b1430513e8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0ca27fc640 1 -- 172.21.15.3:0/121818402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c980ab5f0 con 0x7f0ca40d0a90 2024-09-12T12:20:04.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0ca27fc640 1 -- 172.21.15.3:0/121818402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c980b0070 con 0x7f0ca40d0a90 2024-09-12T12:20:04.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0ca27fc640 1 -- 172.21.15.3:0/121818402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c980ab9b0 con 0x7f0ca40d0a90 2024-09-12T12:20:04.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 -- 172.21.15.3:0/121818402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 msgr2=0x7f0ca4072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 --2- 172.21.15.3:0/121818402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4072de0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c980097c0 tx=0x7f0c9801c3d0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 -- 172.21.15.3:0/121818402 shutdown_connections 2024-09-12T12:20:04.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 --2- 172.21.15.3:0/121818402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4072de0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 -- 172.21.15.3:0/121818402 >> 172.21.15.3:0/121818402 conn(0x7f0ca406be90 msgr2=0x7f0ca406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 -- 172.21.15.3:0/121818402 shutdown_connections 2024-09-12T12:20:04.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 -- 172.21.15.3:0/121818402 wait complete. 2024-09-12T12:20:04.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 Processor -- start 2024-09-12T12:20:04.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.335+0000 7f0caa158640 1 -- start start 2024-09-12T12:20:04.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ca4073320 0x7f0ca40cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ca40af550 0x7f0ca41586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca415eda0 con 0x7f0ca40d0a90 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca415ec20 con 0x7f0ca4073320 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca4168d80 con 0x7f0ca40af550 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41304/0 (socket says 172.21.15.3:41304) 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 -- 172.21.15.3:0/2918134254 learned_addr learned my addr 172.21.15.3:0/2918134254 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ca4073320 0x7f0ca40cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ca40af550 0x7f0ca41586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 -- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ca40af550 msgr2=0x7f0ca41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ca40af550 0x7f0ca41586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 -- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ca4073320 msgr2=0x7f0ca40cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ca4073320 0x7f0ca40cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 -- 172.21.15.3:0/2918134254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca40cccd0 con 0x7f0ca40d0a90 2024-09-12T12:20:04.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0ca3fff640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4164e80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00cbf0 tx=0x7f0c9c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0c8bfff640 1 -- 172.21.15.3:0/2918134254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c017070 con 0x7f0ca40d0a90 2024-09-12T12:20:04.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0c8bfff640 1 -- 172.21.15.3:0/2918134254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c9c00ad40 con 0x7f0ca40d0a90 2024-09-12T12:20:04.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca40ccfb0 con 0x7f0ca40d0a90 2024-09-12T12:20:04.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0c8bfff640 1 -- 172.21.15.3:0/2918134254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c012600 con 0x7f0ca40d0a90 2024-09-12T12:20:04.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca4164960 con 0x7f0ca40d0a90 2024-09-12T12:20:04.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0c8bfff640 1 -- 172.21.15.3:0/2918134254 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0c9c01a070 con 0x7f0ca40d0a90 2024-09-12T12:20:04.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.339+0000 7f0c8bfff640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0c900678c0 0x7f0c90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0c8bfff640 1 -- 172.21.15.3:0/2918134254 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0c9c085990 con 0x7f0ca40d0a90 2024-09-12T12:20:04.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0c900678c0 0x7f0c90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 Processor -- start 2024-09-12T12:20:04.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- start start 2024-09-12T12:20:04.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0c900678c0 0x7f0c90069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c94007fd0 tx=0x7f0c94007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca95f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca95a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95aa290 0x563ca95aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563ca9491b60 con 0x563ca95f3b20 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ca93f8c70 con 0x563ca95aa290 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563ca939eed0 con 0x563ca95f4440 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca95f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca95f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41308/0 (socket says 172.21.15.3:41308) 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 -- 172.21.15.3:0/1807056078 learned_addr learned my addr 172.21.15.3:0/1807056078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca3fff640 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95aa290 0x563ca95aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca95a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 -- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 msgr2=0x563ca95a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca95a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 -- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95aa290 msgr2=0x563ca95aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95aa290 0x563ca95aa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 -- 172.21.15.3:0/1807056078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca93b0e50 con 0x563ca95f3b20 2024-09-12T12:20:04.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca95f3f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400fb20 tx=0x7f0c94012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba919f4df29fc31c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0c897fa640 1 -- 172.21.15.3:0/1807056078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401b050 con 0x563ca95f3b20 2024-09-12T12:20:04.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0c897fa640 1 -- 172.21.15.3:0/1807056078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c94025070 con 0x563ca95f3b20 2024-09-12T12:20:04.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0c897fa640 1 -- 172.21.15.3:0/1807056078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94020440 con 0x563ca95f3b20 2024-09-12T12:20:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- 172.21.15.3:0/1807056078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 msgr2=0x563ca95f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca95f3f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400fb20 tx=0x7f0c94012000 comp rx=0 tx=0).stop 2024-09-12T12:20:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- 172.21.15.3:0/1807056078 shutdown_connections 2024-09-12T12:20:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95aa290 0x563ca95aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca95a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 --2- 172.21.15.3:0/1807056078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca95f3f00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- 172.21.15.3:0/1807056078 >> 172.21.15.3:0/1807056078 conn(0x563ca95b0600 msgr2=0x563ca95a8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- 172.21.15.3:0/1807056078 shutdown_connections 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.343+0000 7f0cad214000 1 -- 172.21.15.3:0/1807056078 wait complete. 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 -- start start 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 Processor -- start 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 0x563ca95aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4440 0x563ca95b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca95b5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563ca945ab00 con 0x563ca95f4440 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ca93d3360 con 0x563ca95af390 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563ca93d02d0 con 0x563ca95f3b20 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 10 client.0 _ll_register_callbacks cb 0x563ca9595820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:04.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4440 0x563ca95b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4440 0x563ca95b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41318/0 (socket says 172.21.15.3:41318) 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/2761830996 learned_addr learned my addr 172.21.15.3:0/2761830996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 0x563ca95aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca3fff640 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca95b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 msgr2=0x563ca95aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 0x563ca95aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 msgr2=0x563ca95b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca95b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/2761830996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca939f120 con 0x563ca95f4440 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4440 0x563ca95b4cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c98018fe0 tx=0x7f0c9801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58c545ca9be6f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0c827fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0c827fc640 1 -- 172.21.15.3:0/2761830996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c980b6cf0 con 0x563ca95f4440 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0c827fc640 1 -- 172.21.15.3:0/2761830996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0c980c03f0 con 0x563ca95f4440 2024-09-12T12:20:04.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c94025070 con 0x563ca95f4440 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0c827fc640 1 -- 172.21.15.3:0/2761830996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f0c980c7050 con 0x563ca95f4440 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 10 client.4566 fetch_fsmap learned FSMap version 7 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563ca9398450 con 0x563ca95f4440 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0c827fc640 1 -- 172.21.15.3:0/2761830996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c94025070 con 0x563ca95f4440 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0c827fc640 1 -- 172.21.15.3:0/2761830996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f0c980ca020 con 0x563ca95f4440 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 10 client.4566 fetch_fsmap finished waiting for FSMap version 7 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 10 client.4566 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 20 client.4566 populate_metadata read hostname 'smithi003' 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca9602280 0x563ca9604670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 4 client.4566 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 --> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca9392c00 con 0x563ca9602280 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca9602280 0x563ca9604670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0c827fc640 10 client.4566 ms_handle_connect on v2:172.21.15.142:6836/2900613310 2024-09-12T12:20:04.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.347+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca9602280 0x563ca9604670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d12142147876611 server_cookie=d051cd8893d5a921 in_seq=0 out_seq=0 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0c827fc640 1 -- 172.21.15.3:0/2761830996 <== mds.0 v2:172.21.15.142:6836/2900613310 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ca9392c00 con 0x563ca9602280 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0c827fc640 10 client.4566 handle_command_reply: tid=0 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 client.4566 shutdown 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4440 msgr2=0x563ca95b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4440 0x563ca95b4cf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0c98018fe0 tx=0x7f0c9801ac10 comp rx=0 tx=0).stop 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 shutdown_connections 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca9602280 0x563ca9604670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca95b5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f4440 0x563ca95b4cf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0c827fc640 0 client.4566 ms_handle_reset on v2:172.21.15.142:6836/2900613310 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 --2- 172.21.15.3:0/2761830996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 0x563ca95aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 >> 172.21.15.3:0/2761830996 conn(0x563ca95aa290 msgr2=0x563ca95a8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 shutdown_connections 2024-09-12T12:20:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 1 -- 172.21.15.3:0/2761830996 wait complete. 2024-09-12T12:20:04.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.351+0000 7f0cad214000 20 client.4566 trim_cache size 0 max 16384 2024-09-12T12:20:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 Processor -- start 2024-09-12T12:20:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- start start 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95af390 0x563ca95f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 0x563ca95f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95f4440 0x563ca9604670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4202ac0 con 0x563ca95af390 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40b2040 con 0x563ca95f4440 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca40b2690 con 0x563ca95f3b20 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95af390 0x563ca95f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95af390 0x563ca95f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41330/0 (socket says 172.21.15.3:41330) 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 -- 172.21.15.3:0/577997334 learned_addr learned my addr 172.21.15.3:0/577997334 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 -- 172.21.15.3:0/577997334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 msgr2=0x563ca95f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/577997334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 0x563ca95f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 -- 172.21.15.3:0/577997334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95f4440 msgr2=0x563ca9604670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/577997334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95f4440 0x563ca9604670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 -- 172.21.15.3:0/577997334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca937ffd0 con 0x563ca95af390 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/577997334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95af390 0x563ca95f2cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0c94007b80 tx=0x7f0c94014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ef0cd8aaad19e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0c88ff9640 1 -- 172.21.15.3:0/577997334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9401b040 con 0x563ca95af390 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0c88ff9640 1 -- 172.21.15.3:0/577997334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c94025050 con 0x563ca95af390 2024-09-12T12:20:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0c88ff9640 1 -- 172.21.15.3:0/577997334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94020580 con 0x563ca95af390 2024-09-12T12:20:04.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- 172.21.15.3:0/577997334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95af390 msgr2=0x563ca95f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 --2- 172.21.15.3:0/577997334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95af390 0x563ca95f2cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0c94007b80 tx=0x7f0c94014000 comp rx=0 tx=0).stop 2024-09-12T12:20:04.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- 172.21.15.3:0/577997334 shutdown_connections 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 --2- 172.21.15.3:0/577997334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95f4440 0x563ca9604670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 --2- 172.21.15.3:0/577997334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f3b20 0x563ca95f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 --2- 172.21.15.3:0/577997334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95af390 0x563ca95f2cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- 172.21.15.3:0/577997334 >> 172.21.15.3:0/577997334 conn(0x563ca95aa290 msgr2=0x563ca95ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- 172.21.15.3:0/577997334 shutdown_connections 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- 172.21.15.3:0/577997334 wait complete. 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 -- start start 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.371+0000 7f0cad214000 1 Processor -- start 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca9607590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca9622960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca9622ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563ca93f7f80 con 0x563ca95f3b20 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ca93b0ca0 con 0x563ca95af390 2024-09-12T12:20:04.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x563ca93fd2a0 con 0x563ca95f4440 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 10 client.0 _ll_register_callbacks cb 0x7f0c98009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca9622960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca9622960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41336/0 (socket says 172.21.15.3:41336) 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/3586674468 learned_addr learned my addr 172.21.15.3:0/3586674468 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca9607590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 msgr2=0x563ca9622ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca3fff640 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca9622ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca9622ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 msgr2=0x563ca9607590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca9607590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 -- 172.21.15.3:0/3586674468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca947b750 con 0x563ca95f3b20 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca2ffd640 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca9622960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c9801c690 tx=0x7f0c98019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fea862e3c757ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 1 -- 172.21.15.3:0/3586674468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c980b69b0 con 0x563ca95f3b20 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 1 -- 172.21.15.3:0/3586674468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0c980b6b50 con 0x563ca95f3b20 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c94025050 con 0x563ca95f3b20 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 1 -- 172.21.15.3:0/3586674468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f0c980093e0 con 0x563ca95f3b20 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 10 client.4572 fetch_fsmap learned FSMap version 7 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563ca9386a30 con 0x563ca95f3b20 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 1 -- 172.21.15.3:0/3586674468 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c94025050 con 0x563ca95f3b20 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 1 -- 172.21.15.3:0/3586674468 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f0c980ca040 con 0x563ca95f3b20 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 10 client.4572 fetch_fsmap finished waiting for FSMap version 7 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 10 client.4572 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 20 client.4572 populate_metadata read hostname 'smithi003' 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca96233e0 0x563ca9666ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 4 client.4572 mds_command: new command op to 4306 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:20:04.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 --> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] -- command(tid 0: {"prefix": "status"}) -- 0x563ca94063a0 con 0x563ca96233e0 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca96233e0 0x563ca9666ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0ca37fe640 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca96233e0 0x563ca9666ef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c451495d580d25be server_cookie=e90117586946ca39 in_seq=0 out_seq=0 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 10 client.4572 ms_handle_connect on v2:172.21.15.142:6836/2900613310 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 1 -- 172.21.15.3:0/3586674468 <== mds.0 v2:172.21.15.142:6836/2900613310 1 ==== command_reply(tid 0: 0 ) ==== 8+0+317 (crc 0 0 0) 0x563ca94063a0 con 0x563ca96233e0 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 10 client.4572 handle_command_reply: tid=0 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 client.4572 shutdown 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 msgr2=0x563ca9622960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca9622960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c9801c690 tx=0x7f0c98019170 comp rx=0 tx=0).stop 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 shutdown_connections 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:6836/2900613310,v1:172.21.15.142:6837/2900613310] conn(0x563ca96233e0 0x563ca9666ef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x563ca95f4440 0x563ca9622ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563ca95f3b20 0x563ca9622960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 --2- 172.21.15.3:0/3586674468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ca95af390 0x563ca9607590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 >> 172.21.15.3:0/3586674468 conn(0x563ca95aa290 msgr2=0x563ca95f1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0c827fc640 0 client.4572 ms_handle_reset on v2:172.21.15.142:6836/2900613310 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 shutdown_connections 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 1 -- 172.21.15.3:0/3586674468 wait complete. 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.375+0000 7f0cad214000 20 client.4572 trim_cache size 0 max 16384 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4306, 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:20:04.385 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:20:04.386 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 15.63726512, 2024-09-12T12:20:04.386 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 7, 2024-09-12T12:20:04.386 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 21, 2024-09-12T12:20:04.386 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 21, 2024-09-12T12:20:04.386 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 21.154504519 2024-09-12T12:20:04.386 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0c900678c0 msgr2=0x7f0c90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0c900678c0 0x7f0c90069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0c94007fd0 tx=0x7f0c94007e40 comp rx=0 tx=0).stop 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 msgr2=0x7f0ca4164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4164e80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c00cbf0 tx=0x7f0c9c00a430 comp rx=0 tx=0).stop 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 shutdown_connections 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0c900678c0 0x7f0c90069d80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ca40d0a90 0x7f0ca4164e80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ca40af550 0x7f0ca41586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 --2- 172.21.15.3:0/2918134254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ca4073320 0x7f0ca40cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 >> 172.21.15.3:0/2918134254 conn(0x7f0ca406be90 msgr2=0x7f0ca406c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 shutdown_connections 2024-09-12T12:20:04.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.379+0000 7f0caa158640 1 -- 172.21.15.3:0/2918134254 wait complete. 2024-09-12T12:20:04.397 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T12:20:04.406 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T12:20:04.407 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T12:20:04.407 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T12:20:04.407 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-12T12:20:04.489 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-12T12:20:04.489 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-12T12:20:04.489 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-12T12:20:04.489 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T12:20:04.490 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:04.490 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-12T12:20:04.490 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T12:20:04.490 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:04.490 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-12T12:20:04.490 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T12:20:04.490 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:04.490 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-12T12:20:04.490 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T12:20:04.490 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:04.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:04.491 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:20:04.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:04.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-09-12T12:20:04.557 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-09-12T12:20:04.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:20:04.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:04.558 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:20:04.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:04.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-12T12:20:04.625 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-12T12:20:04.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:20:04.627 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-12T12:20:04.627 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T12:20:04.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:20:04.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.799+0000 7f84ba6f0640 1 Processor -- start 2024-09-12T12:20:04.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.799+0000 7f84ba6f0640 1 -- start start 2024-09-12T12:20:04.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84ba6f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 0x7f84b406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84ba6f0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b406fbb0 con 0x7f84b4159fb0 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84ba6f0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b406e400 con 0x7f84b415d2d0 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84ba6f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84b406f010 con 0x7f84b4159bd0 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b37fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f84b4159fb0 0x7f84b406f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36270/0 (socket says 172.21.15.3:36270) 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b37fe640 1 -- 172.21.15.3:0/2521421254 learned_addr learned my addr 172.21.15.3:0/2521421254 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2583524325 0 0) 0x7f84b406fbb0 con 0x7f84b4159fb0 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f849c003530 con 0x7f84b4159fb0 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b3fff640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 0x7f84b406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3880682106 0 0) 0x7f849c003530 con 0x7f84b4159fb0 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 msgr2=0x7f84b406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 0x7f84b406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 >> v1:172.21.15.142:6789/0 conn(0x7f84b415d2d0 legacy=0x7f84b40ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b406e6e0 con 0x7f84b4159fb0 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f84a8001680 con 0x7f84b4159fb0 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 >> v1:172.21.15.3:6789/0 conn(0x7f84b4159fb0 legacy=0x7f84b406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 0x7f849c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f849c004040 0x7f849c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849c004d50 0x7f849c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f84b406f010 con 0x7f849c004d50 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f849c00bb90 con 0x7f849c004040 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f849c00bd90 con 0x7f84b4159bd0 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b8c8d640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849c004d50 0x7f849c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b37fe640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f849c004040 0x7f849c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b3fff640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 0x7f849c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b8c8d640 1 -- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 msgr2=0x7f849c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b8c8d640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b4159bd0 0x7f849c003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b8c8d640 1 -- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f849c004040 msgr2=0x7f849c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b8c8d640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f849c004040 0x7f849c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b8c8d640 1 -- 172.21.15.3:0/2521421254 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c003530 con 0x7f849c004d50 2024-09-12T12:20:04.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.803+0000 7f84b37fe640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f849c004040 0x7f849c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b8c8d640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849c004d50 0x7f849c0051d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84ac001e70 tx=0x7f84ac01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b363bdf1e510aac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac001a90 con 0x7f849c004d50 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ac003b80 con 0x7f849c004d50 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b2ffd640 1 -- 172.21.15.3:0/2521421254 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac005080 con 0x7f849c004d50 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- 172.21.15.3:0/2521421254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849c004d50 msgr2=0x7f849c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 --2- 172.21.15.3:0/2521421254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f849c004d50 0x7f849c0051d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84ac001e70 tx=0x7f84ac01c780 comp rx=0 tx=0).stop 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b8c8d640 1 -- 172.21.15.3:0/2521421254 reap_dead start 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- 172.21.15.3:0/2521421254 shutdown_connections 2024-09-12T12:20:04.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- 172.21.15.3:0/2521421254 >> 172.21.15.3:0/2521421254 conn(0x7f84b40b1d80 msgr2=0x7f84b40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- 172.21.15.3:0/2521421254 shutdown_connections 2024-09-12T12:20:04.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- 172.21.15.3:0/2521421254 wait complete. 2024-09-12T12:20:04.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 Processor -- start 2024-09-12T12:20:04.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- start start 2024-09-12T12:20:04.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b4099940 0x7f84b4150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4146880 0x7f84b40cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40cf780 0x7f84b41535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f84b415d710 con 0x7f84b4099940 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84b406eca0 con 0x7f84b4146880 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84ba6f0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f84b40ad770 con 0x7f84b40cf780 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b4099940 0x7f84b4150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b4099940 0x7f84b4150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41356/0 (socket says 172.21.15.3:41356) 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 -- 172.21.15.3:0/1613315881 learned_addr learned my addr 172.21.15.3:0/1613315881 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 -- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40cf780 msgr2=0x7f84b41535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40cf780 0x7f84b41535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 -- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4146880 msgr2=0x7f84b40cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4146880 0x7f84b40cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 -- 172.21.15.3:0/1613315881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b4150580 con 0x7f84b4099940 2024-09-12T12:20:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.807+0000 7f84b3fff640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b4099940 0x7f84b4150040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f84a400f5b0 tx=0x7f84a400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b0ff9640 1 -- 172.21.15.3:0/1613315881 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4019020 con 0x7f84b4099940 2024-09-12T12:20:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b0ff9640 1 -- 172.21.15.3:0/1613315881 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a4014870 con 0x7f84b4099940 2024-09-12T12:20:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b4152e80 con 0x7f84b4099940 2024-09-12T12:20:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b0ff9640 1 -- 172.21.15.3:0/1613315881 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4014c40 con 0x7f84b4099940 2024-09-12T12:20:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b4154f70 con 0x7f84b4099940 2024-09-12T12:20:04.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b40cfba0 con 0x7f84b4099940 2024-09-12T12:20:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b0ff9640 1 -- 172.21.15.3:0/1613315881 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f84a401c020 con 0x7f84b4099940 2024-09-12T12:20:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b0ff9640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84a00678c0 0x7f84a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b0ff9640 1 -- 172.21.15.3:0/1613315881 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f84a4086440 con 0x7f84b4099940 2024-09-12T12:20:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b37fe640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84a00678c0 0x7f84a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.811+0000 7f84b37fe640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84a00678c0 0x7f84a0069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001b90 tx=0x7f84a80011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:04.815+0000 7f84b0ff9640 1 -- 172.21.15.3:0/1613315881 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f84a4023020 con 0x7f84b4099940 2024-09-12T12:20:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f84b41533a0 con 0x7f84b4099940 2024-09-12T12:20:05.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84b0ff9640 1 -- 172.21.15.3:0/1613315881 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f84a4026350 con 0x7f84b4099940 2024-09-12T12:20:05.021 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84a00678c0 msgr2=0x7f84a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84a00678c0 0x7f84a0069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84a8001b90 tx=0x7f84a80011a0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b4099940 msgr2=0x7f84b4150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b4099940 0x7f84b4150040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f84a400f5b0 tx=0x7f84a400fa70 comp rx=0 tx=0).stop 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 shutdown_connections 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84a00678c0 0x7f84a0069d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84b40cf780 0x7f84b41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84b4146880 0x7f84b40cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 --2- 172.21.15.3:0/1613315881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84b4099940 0x7f84b4150040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 >> 172.21.15.3:0/1613315881 conn(0x7f84b40b1d80 msgr2=0x7f84b415b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 shutdown_connections 2024-09-12T12:20:05.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05.019+0000 7f84ba6f0640 1 -- 172.21.15.3:0/1613315881 wait complete. 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:20:05.042 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:20:05.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:05.043 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:05.049 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42481sec preferred_lft 42481sec 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:20:05.050 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:20:05.051 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:05.051 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:20:05.051 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:05.051 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T12:20:05.051 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:05.052 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:05.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-09-12T12:20:05.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-09-12T12:20:05.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-09-12T12:20:05.052 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T12:20:05.052 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:05.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:05.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:05.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:05.184 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:20:05.193 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:20:05.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:05.194 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:20:05.257 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:20:05.257 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:20:05.257 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:20:05.257 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:20:05.257 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:20:05.257 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:20:05.258 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:05.258 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:05.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:20:05.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:20:05.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:20:05.258 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:05.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:05.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:05.380 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:20:05.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:05.386 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-12T12:20:05.438 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:05.438 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:05.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:05.438 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T12:20:05.438 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:05.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:05.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:05.564 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T12:20:05.565 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:05.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:05.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:05.671 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:05.671 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:05.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:20:05.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T12:20:05.671 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:05.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:05.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:05.729 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:20:05.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:05.730 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:05.777 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:20:05.777 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:05.777 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:05.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:05.883 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:05.932 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:05.933 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:05.934 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:05.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:05.935 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:05.981 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:20:05.981 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:20:06.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:06.039 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:06.086 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:06.087 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:06.088 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:06.088 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:06.088 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:06.088 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:06.088 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:06.088 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:06.088 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:06.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:06.089 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:06.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:20:06.099+0000 7f1638145580 -1 init, newargv = 0x55c4e9def310 newargc=15 2024-09-12T12:20:06.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[35033]: starting ceph client 2024-09-12T12:20:06.112 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[35033]: starting fuse 2024-09-12T12:20:07.210 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:07.227 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:07.274 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:07.274 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:07.275 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:07.276 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:07.276 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:07.276 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:07.276 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:07.276 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:07.276 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:07.277 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:07.278 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:07.278 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:07.278 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:07.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:07.279 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:07.325 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:20:07.326 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:20:07.326 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:20:07.327 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:20:07.328 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:20:07.328 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:07.328 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:20:07.328 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:20:07.453 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.35033.asok 2024-09-12T12:20:07.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:07.466 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35033.asok 2024-09-12T12:20:07.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:07.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35033.asok status 2024-09-12T12:20:07.627 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:20:07.627 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:20:07.627 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:20:07.627 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "35033", 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4587, 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:20:07.628 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 4587 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3861899055 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3861899055 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.4587 192.168.0.1:0/3861899055", 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3861899055", 2024-09-12T12:20:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:20:07.630 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 7, 2024-09-12T12:20:07.630 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 21, 2024-09-12T12:20:07.630 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:20:07.630 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:20:07.630 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:20:07.630 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:20:07.641 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T12:20:07.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:20:07.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f907d409640 1 Processor -- start 2024-09-12T12:20:07.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f907d409640 1 -- start start 2024-09-12T12:20:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f907d409640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90780d5050 0x7f90780cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f907d409640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90780cfe20 con 0x7f907815e5d0 2024-09-12T12:20:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f907d409640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90780d31a0 con 0x7f90780cfa10 2024-09-12T12:20:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f907d409640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f90780b1060 con 0x7f90780d5050 2024-09-12T12:20:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f90767fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90780d5050 0x7f90780cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f9076ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f907815e5d0 0x7f90780ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:36278/0 (socket says 172.21.15.3:36278) 2024-09-12T12:20:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f9076ffd640 1 -- 172.21.15.3:0/774414901 learned_addr learned my addr 172.21.15.3:0/774414901 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f90767fc640 1 -- 172.21.15.3:0/774414901 >> v1:172.21.15.142:6789/0 conn(0x7f90780cfa10 legacy=0x7f90780b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f90767fc640 1 -- 172.21.15.3:0/774414901 >> v1:172.21.15.3:6789/0 conn(0x7f907815e5d0 legacy=0x7f90780ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f9075ffb640 1 -- 172.21.15.3:0/774414901 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555775555 0 0) 0x7f9068001120 con 0x7f907815e5d0 2024-09-12T12:20:07.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.767+0000 7f90767fc640 1 -- 172.21.15.3:0/774414901 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90780d3480 con 0x7f90780d5050 2024-09-12T12:20:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f90767fc640 1 --2- 172.21.15.3:0/774414901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780d5050 0x7f90780cf4d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90600097c0 tx=0x7f906001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d31ff5da63289b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9075ffb640 1 -- 172.21.15.3:0/774414901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90600ab5f0 con 0x7f90780d5050 2024-09-12T12:20:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9075ffb640 1 -- 172.21.15.3:0/774414901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90600b0070 con 0x7f90780d5050 2024-09-12T12:20:07.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9075ffb640 1 -- 172.21.15.3:0/774414901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90600ab9b0 con 0x7f90780d5050 2024-09-12T12:20:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- 172.21.15.3:0/774414901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780d5050 msgr2=0x7f90780cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 --2- 172.21.15.3:0/774414901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780d5050 0x7f90780cf4d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90600097c0 tx=0x7f906001c3d0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- 172.21.15.3:0/774414901 shutdown_connections 2024-09-12T12:20:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 --2- 172.21.15.3:0/774414901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780d5050 0x7f90780cf4d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- 172.21.15.3:0/774414901 >> 172.21.15.3:0/774414901 conn(0x7f907808c000 msgr2=0x7f90780d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- 172.21.15.3:0/774414901 shutdown_connections 2024-09-12T12:20:07.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- 172.21.15.3:0/774414901 wait complete. 2024-09-12T12:20:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 Processor -- start 2024-09-12T12:20:07.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- start start 2024-09-12T12:20:07.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780cfa10 0x7f90780c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90780d5050 0x7f9078153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f907815e5d0 0x7f907806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f90780b2820 con 0x7f90780cfa10 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90780b26a0 con 0x7f90780d5050 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f907d409640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f907816c5d0 con 0x7f907815e5d0 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780cfa10 0x7f90780c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780cfa10 0x7f90780c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41368/0 (socket says 172.21.15.3:41368) 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 -- 172.21.15.3:0/58270774 learned_addr learned my addr 172.21.15.3:0/58270774 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f90777fe640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f907815e5d0 0x7f907806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f90767fc640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90780d5050 0x7f9078153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 -- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f907815e5d0 msgr2=0x7f907806d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:07.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f907815e5d0 0x7f907806d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 -- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90780d5050 msgr2=0x7f9078153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90780d5050 0x7f9078153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 -- 172.21.15.3:0/58270774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90780c96a0 con 0x7f90780cfa10 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f90777fe640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f907815e5d0 0x7f907806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f90767fc640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90780d5050 0x7f9078153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f9076ffd640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780cfa10 0x7f90780c8fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f906800c480 tx=0x7f906800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f906f7fe640 1 -- 172.21.15.3:0/58270774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068018070 con 0x7f90780cfa10 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.771+0000 7f906f7fe640 1 -- 172.21.15.3:0/58270774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9068007500 con 0x7f90780cfa10 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90780c9980 con 0x7f90780cfa10 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f906f7fe640 1 -- 172.21.15.3:0/58270774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004a30 con 0x7f90780cfa10 2024-09-12T12:20:07.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90780c9f40 con 0x7f90780cfa10 2024-09-12T12:20:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90780cf4d0 con 0x7f90780cfa10 2024-09-12T12:20:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f906f7fe640 1 -- 172.21.15.3:0/58270774 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f906801b070 con 0x7f90780cfa10 2024-09-12T12:20:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f906f7fe640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90580678c0 0x7f9058069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f906f7fe640 1 -- 172.21.15.3:0/58270774 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9068084f70 con 0x7f90780cfa10 2024-09-12T12:20:07.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f90767fc640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90580678c0 0x7f9058069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:07.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.775+0000 7f90767fc640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90580678c0 0x7f9058069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f90600af000 tx=0x7f9060008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:07.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.779+0000 7f906f7fe640 1 -- 172.21.15.3:0/58270774 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f90780cfe20 con 0x7f90780cfa10 2024-09-12T12:20:07.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.979+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90780ba490 con 0x7f90780cfa10 2024-09-12T12:20:07.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f906f7fe640 1 -- 172.21.15.3:0/58270774 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f9068055b40 con 0x7f90780cfa10 2024-09-12T12:20:07.985 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:20:07.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90580678c0 msgr2=0x7f9058069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:07.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90580678c0 0x7f9058069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f90600af000 tx=0x7f9060008040 comp rx=0 tx=0).stop 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780cfa10 msgr2=0x7f90780c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780cfa10 0x7f90780c8fc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f906800c480 tx=0x7f906800c940 comp rx=0 tx=0).stop 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 shutdown_connections 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90580678c0 0x7f9058069d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f907815e5d0 0x7f907806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90780d5050 0x7f9078153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 --2- 172.21.15.3:0/58270774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90780cfa10 0x7f90780c8fc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 >> 172.21.15.3:0/58270774 conn(0x7f907808c000 msgr2=0x7f9078156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 shutdown_connections 2024-09-12T12:20:07.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:07.983+0000 7f907d409640 1 -- 172.21.15.3:0/58270774 wait complete. 2024-09-12T12:20:08.004 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:20:08.004 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:20:08.004 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:20:08.005 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-12T12:20:08.005 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:08.005 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:20:08.005 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:20:08.005 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:20:08.005 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-12T12:20:08.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.005 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:20:08.011 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:08.011 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:08.011 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:08.011 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.011 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42478sec preferred_lft 42478sec 2024-09-12T12:20:08.012 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:20:08.013 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.014 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:20:08.014 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:08.014 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:20:08.014 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.014 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:20:08.062 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:20:08.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.064 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:20:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:20:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:20:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:20:08.111 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:20:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:20:08.112 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:20:08.112 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:08.112 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:08.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:20:08.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:20:08.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:20:08.112 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:08.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:08.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:08.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.238 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:20:08.243 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:20:08.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.244 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-12T12:20:08.289 INFO:teuthology.orchestra.run.smithi003.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-12T12:20:08.289 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:08.289 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:08.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:08.289 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-12T12:20:08.289 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:08.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:08.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:08.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 1a:96:f1:31:85:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::1896:f1ff:fe31:859b/64 scope link 2024-09-12T12:20:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-12T12:20:08.480 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 1a:96:f1:31:85:9b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::1896:f1ff:fe31:859b/64 scope link 2024-09-12T12:20:08.481 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:08.482 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-12T12:20:08.483 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:08.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:08.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:08.623 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:08.623 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:08.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.1 up 2024-09-12T12:20:08.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.1 master ceph-brx 2024-09-12T12:20:08.623 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:08.683 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-12T12:20:08.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:08.683 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:08.729 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-12T12:20:08.729 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:08.729 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:08.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:08.831 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:08.882 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:08.882 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:08.882 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:08.882 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:08.882 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:08.882 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:08.883 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:08.884 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:08.885 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:08.885 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:08.885 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:08.885 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:08.885 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:08.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:08.887 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:08.933 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:20:08.934 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-12T12:20:08.934 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-12T12:20:08.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:08.992 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:09.037 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:09.038 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:09.039 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:09.040 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:09.040 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:09.040 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:09.040 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:09.040 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:09.040 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:09.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:09.041 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:09.057 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi003.stderr:2024-09-12T12:20:09.055+0000 7f499ee14580 -1 init, newargv = 0x5586199b1310 newargc=15 2024-09-12T12:20:09.058 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi003.stderr:ceph-fuse[35177]: starting ceph client 2024-09-12T12:20:09.062 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:20:09.064 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi003.stderr:ceph-fuse[35177]: starting fuse 2024-09-12T12:20:10.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:10.180 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:10.232 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:10.232 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:10.232 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:10.232 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:10.232 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:10.233 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:10.234 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:10.235 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:10.235 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:10.235 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:10.235 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:10.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:10.236 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:10.286 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:20:10.286 INFO:teuthology.orchestra.run.smithi003.stdout:46 2024-09-12T12:20:10.286 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:20:10.287 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.2")) 2024-09-12T12:20:10.288 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:20:10.413 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.2.35177.asok 2024-09-12T12:20:10.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:10.425 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35177.asok 2024-09-12T12:20:10.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:10.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35177.asok status 2024-09-12T12:20:10.584 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:20:10.584 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:20:10.584 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:20:10.584 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:20:10.584 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "2", 2024-09-12T12:20:10.584 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:20:10.584 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "35177", 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4602, 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:20:10.585 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 4602 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.2:0", 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3894249459 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:20:10.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.2:0", 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3894249459 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.4602 192.168.0.2:0/3894249459", 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.2:0/3894249459", 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 7, 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 21, 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:20:10.587 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:20:10.588 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:20:10.588 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:20:10.598 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T12:20:10.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:20:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbf557640 1 Processor -- start 2024-09-12T12:20:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbf557640 1 -- start start 2024-09-12T12:20:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbf557640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fefb8159fb0 0x7fefb806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbf557640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb806fbb0 con 0x7fefb815d2d0 2024-09-12T12:20:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbf557640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb815d760 con 0x7fefb8159bd0 2024-09-12T12:20:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbf557640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fefb806f010 con 0x7fefb8159fb0 2024-09-12T12:20:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbdaf4640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fefb815d2d0 0x7fefb80ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44048/0 (socket says 172.21.15.3:44048) 2024-09-12T12:20:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.727+0000 7fefbdaf4640 1 -- 172.21.15.3:0/2187572443 learned_addr learned my addr 172.21.15.3:0/2187572443 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbcaf2640 1 --2- 172.21.15.3:0/2187572443 >> v2:172.21.15.142:3300/0 conn(0x7fefb8159fb0 0x7fefb806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069032704 0 0) 0x7fefb806fbb0 con 0x7fefb815d2d0 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefa4003530 con 0x7fefb815d2d0 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2511956615 0 0) 0x7fefa4003530 con 0x7fefb815d2d0 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 >> v2:172.21.15.142:3300/0 conn(0x7fefb8159fb0 msgr2=0x7fefb806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 --2- 172.21.15.3:0/2187572443 >> v2:172.21.15.142:3300/0 conn(0x7fefb8159fb0 0x7fefb806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 >> v1:172.21.15.142:6789/0 conn(0x7fefb8159bd0 legacy=0x7fefb806f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb806e400 con 0x7fefb815d2d0 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fefb0001a00 con 0x7fefb815d2d0 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 >> v1:172.21.15.3:6789/0 conn(0x7fefb815d2d0 legacy=0x7fefb80ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefa40039c0 0x7fefa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa4004810 0x7fefa4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefa40051d0 0x7fefa400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fefb806f010 con 0x7fefa40039c0 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefa40104b0 con 0x7fefa4004810 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefa40106b0 con 0x7fefa40051d0 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbd2f3640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefa40039c0 0x7fefa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbdaf4640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefa40051d0 0x7fefa400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbcaf2640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa4004810 0x7fefa4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbd2f3640 1 -- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefa40051d0 msgr2=0x7fefa400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbd2f3640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefa40051d0 0x7fefa400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbd2f3640 1 -- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa4004810 msgr2=0x7fefa4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbd2f3640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa4004810 0x7fefa4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbd2f3640 1 -- 172.21.15.3:0/2187572443 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa4003530 con 0x7fefa40039c0 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbdaf4640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefa40051d0 0x7fefa400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbcaf2640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa4004810 0x7fefa4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbdaf4640 1 -- 172.21.15.3:0/2187572443 reap_dead start 2024-09-12T12:20:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefbd2f3640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefa40039c0 0x7fefa4003e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fefb4001780 tx=0x7fefb401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7729c226d06dd6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb40013a0 con 0x7fefa40039c0 2024-09-12T12:20:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb40052f0 con 0x7fefa40039c0 2024-09-12T12:20:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.731+0000 7fefaffff640 1 -- 172.21.15.3:0/2187572443 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4003840 con 0x7fefa40039c0 2024-09-12T12:20:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- 172.21.15.3:0/2187572443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefa40039c0 msgr2=0x7fefa4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefa40039c0 0x7fefa4003e00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fefb4001780 tx=0x7fefb401c3d0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- 172.21.15.3:0/2187572443 shutdown_connections 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 --2- 172.21.15.3:0/2187572443 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefa40039c0 0x7fefa4003e00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- 172.21.15.3:0/2187572443 >> 172.21.15.3:0/2187572443 conn(0x7fefb80b1d80 msgr2=0x7fefb80b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- 172.21.15.3:0/2187572443 shutdown_connections 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- 172.21.15.3:0/2187572443 wait complete. 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 Processor -- start 2024-09-12T12:20:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- start start 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa40039c0 0x7fefb8156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb8145b30 0x7fefb8099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefb81431b0 0x7fefb80b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fefb806eca0 con 0x7fefb8145b30 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8073db0 con 0x7fefa40039c0 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefb806f890 con 0x7fefb81431b0 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb8145b30 0x7fefb8099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb8145b30 0x7fefb8099920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46590/0 (socket says 172.21.15.3:46590) 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 -- 172.21.15.3:0/2376792637 learned_addr learned my addr 172.21.15.3:0/2376792637 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbd2f3640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa40039c0 0x7fefb8156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbdaf4640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefb81431b0 0x7fefb80b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 -- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefb81431b0 msgr2=0x7fefb80b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefb81431b0 0x7fefb80b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 -- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa40039c0 msgr2=0x7fefb8156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa40039c0 0x7fefb8156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 -- 172.21.15.3:0/2376792637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb80ada60 con 0x7fefb8145b30 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbcaf2640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb8145b30 0x7fefb8099920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefa80107f0 tx=0x7fefa8010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefadffb640 1 -- 172.21.15.3:0/2376792637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa800f040 con 0x7fefb8145b30 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefadffb640 1 -- 172.21.15.3:0/2376792637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa8004220 con 0x7fefb8145b30 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefadffb640 1 -- 172.21.15.3:0/2376792637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8010ea0 con 0x7fefb8145b30 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb8143aa0 con 0x7fefb8145b30 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.735+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb8155e50 con 0x7fefb8145b30 2024-09-12T12:20:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.739+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb806fb90 con 0x7fefb8145b30 2024-09-12T12:20:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.739+0000 7fefadffb640 1 -- 172.21.15.3:0/2376792637 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fefa800e960 con 0x7fefb8145b30 2024-09-12T12:20:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.739+0000 7fefadffb640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fef9c0678f0 0x7fef9c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.739+0000 7fefadffb640 1 -- 172.21.15.3:0/2376792637 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fefa8085a40 con 0x7fefb8145b30 2024-09-12T12:20:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.739+0000 7fefbd2f3640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fef9c0678f0 0x7fef9c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.739+0000 7fefbd2f3640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fef9c0678f0 0x7fef9c069db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fefb40ac000 tx=0x7fefb40011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.743+0000 7fefadffb640 1 -- 172.21.15.3:0/2376792637 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fefa8018030 con 0x7fefb8145b30 2024-09-12T12:20:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.943+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fefb80b2b00 con 0x7fefb8145b30 2024-09-12T12:20:10.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefadffb640 1 -- 172.21.15.3:0/2376792637 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fefa80565f0 con 0x7fefb8145b30 2024-09-12T12:20:10.949 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:20:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fef9c0678f0 msgr2=0x7fef9c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fef9c0678f0 0x7fef9c069db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fefb40ac000 tx=0x7fefb40011a0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb8145b30 msgr2=0x7fefb8099920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb8145b30 0x7fefb8099920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefa80107f0 tx=0x7fefa8010cb0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 shutdown_connections 2024-09-12T12:20:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fef9c0678f0 0x7fef9c069db0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefb81431b0 0x7fefb80b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb8145b30 0x7fefb8099920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 --2- 172.21.15.3:0/2376792637 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefa40039c0 0x7fefb8156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:10.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 >> 172.21.15.3:0/2376792637 conn(0x7fefb80b1d80 msgr2=0x7fefb80750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:10.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 shutdown_connections 2024-09-12T12:20:10.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:10.947+0000 7fefbf557640 1 -- 172.21.15.3:0/2376792637 wait complete. 2024-09-12T12:20:10.969 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:20:10.969 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:20:10.969 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:20:10.970 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:20:10.970 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:10.970 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:20:10.970 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:20:10.970 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:20:10.970 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:20:10.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:10.970 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:10.977 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42473sec preferred_lft 42473sec 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:20:10.978 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:10.979 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:20:10.979 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:10.979 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T12:20:10.979 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:10.979 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:10.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add name ceph-brx type bridge 2024-09-12T12:20:10.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr flush dev ceph-brx 2024-09-12T12:20:10.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set ceph-brx up 2024-09-12T12:20:10.979 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T12:20:10.979 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:11.066 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:11.103 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:11.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:11.110 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:20:11.120 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:20:11.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:11.121 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:20:11.194 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:20:11.194 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:11.195 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:20:11.195 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:11.196 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:11.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:20:11.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:20:11.196 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:20:11.196 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:11.284 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:11.312 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:11.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:11.319 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:20:11.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:11.325 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-09-12T12:20:11.372 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:11.373 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:11.373 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:11.373 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-12T12:20:11.373 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:11.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:11.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:11.489 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-12T12:20:11.490 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:11.535 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:11.593 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:11.600 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:11.600 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:11.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:20:11.600 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T12:20:11.601 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:11.682 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:11.701 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:11.707 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:20:11.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:11.707 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:11.752 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:20:11.752 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:11.752 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:11.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:11.858 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:11.906 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:11.907 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:11.908 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:11.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:11.909 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:11.956 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:20:11.957 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:20:12.005 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:12.019 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:12.065 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:12.066 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:12.067 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:12.067 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:12.067 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:12.067 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:12.067 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:12.067 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:12.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:12.068 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:12.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:20:12.079+0000 7efd5d429580 -1 init, newargv = 0x55861c3ceee0 newargc=15 2024-09-12T12:20:12.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35216]: starting ceph client 2024-09-12T12:20:12.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35216]: starting fuse 2024-09-12T12:20:13.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:13.203 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:13.250 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:13.251 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:13.252 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:13.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:13.253 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:13.301 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:20:13.301 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:20:13.301 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:20:13.301 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:20:13.301 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:20:13.301 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:20:13.302 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:20:13.303 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:20:13.303 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:20:13.303 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:20:13.303 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:20:13.303 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:13.303 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:20:13.303 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:20:13.426 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.35216.asok 2024-09-12T12:20:13.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:13.437 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35216.asok 2024-09-12T12:20:13.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:13.437 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35216.asok status 2024-09-12T12:20:13.590 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:20:13.590 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:20:13.590 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:20:13.590 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:20:13.590 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:20:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:20:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:20:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "35216", 2024-09-12T12:20:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:20:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:13.591 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:20:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:20:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 4549, 2024-09-12T12:20:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:20:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:20:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:20:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 4549 2024-09-12T12:20:13.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1797657568 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1797657568 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.4549 192.168.0.1:0/1797657568", 2024-09-12T12:20:13.593 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/1797657568", 2024-09-12T12:20:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:20:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 7, 2024-09-12T12:20:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 21, 2024-09-12T12:20:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:20:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:20:13.594 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:20:13.594 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:20:13.603 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-12T12:20:13.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:20:13.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.735+0000 7fbb6c23b640 1 Processor -- start 2024-09-12T12:20:13.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.735+0000 7fbb6c23b640 1 -- start start 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6c23b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159bd0 0x7fbb6406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6c23b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb6415d2d0 0x7fbb640ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6c23b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6406fbb0 con 0x7fbb64159fb0 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6c23b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6406f010 con 0x7fbb6415d2d0 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6c23b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6406e400 con 0x7fbb64159bd0 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb6415d2d0 0x7fbb640ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb6415d2d0 0x7fbb640ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46594/0 (socket says 172.21.15.3:46594) 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 -- 172.21.15.3:0/1346696462 learned_addr learned my addr 172.21.15.3:0/1346696462 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:13.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb69fd7640 1 --2- 172.21.15.3:0/1346696462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159bd0 0x7fbb6406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 -- 172.21.15.3:0/1346696462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159bd0 msgr2=0x7fbb6406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 --2- 172.21.15.3:0/1346696462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159bd0 0x7fbb6406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 -- 172.21.15.3:0/1346696462 >> v1:172.21.15.142:6789/0 conn(0x7fbb64159fb0 legacy=0x7fbb6406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:13.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 -- 172.21.15.3:0/1346696462 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6406e6e0 con 0x7fbb6415d2d0 2024-09-12T12:20:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb6a7d8640 1 --2- 172.21.15.3:0/1346696462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6415d2d0 0x7fbb640ad060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed73534a26f8b717 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:13.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb68fd5640 1 -- 172.21.15.3:0/1346696462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0ab640 con 0x7fbb6415d2d0 2024-09-12T12:20:13.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb68fd5640 1 -- 172.21.15.3:0/1346696462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c0b0070 con 0x7fbb6415d2d0 2024-09-12T12:20:13.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.739+0000 7fbb68fd5640 1 -- 172.21.15.3:0/1346696462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0aba00 con 0x7fbb6415d2d0 2024-09-12T12:20:13.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- 172.21.15.3:0/1346696462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6415d2d0 msgr2=0x7fbb640ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:13.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/1346696462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6415d2d0 0x7fbb640ad060 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009870 tx=0x7fbb5c01c530 comp rx=0 tx=0).stop 2024-09-12T12:20:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- 172.21.15.3:0/1346696462 shutdown_connections 2024-09-12T12:20:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/1346696462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb6415d2d0 0x7fbb640ad060 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/1346696462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159bd0 0x7fbb6406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- 172.21.15.3:0/1346696462 >> 172.21.15.3:0/1346696462 conn(0x7fbb640b1d80 msgr2=0x7fbb640b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- 172.21.15.3:0/1346696462 shutdown_connections 2024-09-12T12:20:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- 172.21.15.3:0/1346696462 wait complete. 2024-09-12T12:20:13.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 Processor -- start 2024-09-12T12:20:13.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- start start 2024-09-12T12:20:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb64156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:13.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64159fb0 0x7fbb6414b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb6415d2d0 0x7fbb64072520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64169360 con 0x7fbb64159bd0 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641691e0 con 0x7fbb64159fb0 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb641694e0 con 0x7fbb6415d2d0 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb64156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb64156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46600/0 (socket says 172.21.15.3:46600) 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 -- 172.21.15.3:0/3199636425 learned_addr learned my addr 172.21.15.3:0/3199636425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 -- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb6415d2d0 msgr2=0x7fbb64072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb6415d2d0 0x7fbb64072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb697d6640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64159fb0 0x7fbb6414b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 -- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64159fb0 msgr2=0x7fbb6414b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64159fb0 0x7fbb6414b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 -- 172.21.15.3:0/3199636425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6414b830 con 0x7fbb64159bd0 2024-09-12T12:20:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb69fd7640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb64156840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb60007ef0 tx=0x7fbb6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb5a7fc640 1 -- 172.21.15.3:0/3199636425 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60014620 con 0x7fbb64159bd0 2024-09-12T12:20:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb5a7fc640 1 -- 172.21.15.3:0/3199636425 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb60019070 con 0x7fbb64159bd0 2024-09-12T12:20:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb5a7fc640 1 -- 172.21.15.3:0/3199636425 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60014a00 con 0x7fbb64159bd0 2024-09-12T12:20:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb641f8000 con 0x7fbb64159bd0 2024-09-12T12:20:13.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.743+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641f85c0 con 0x7fbb64159bd0 2024-09-12T12:20:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.747+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb640ad060 con 0x7fbb64159bd0 2024-09-12T12:20:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.747+0000 7fbb5a7fc640 1 -- 172.21.15.3:0/3199636425 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbb60014ba0 con 0x7fbb64159bd0 2024-09-12T12:20:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.747+0000 7fbb5a7fc640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb480678c0 0x7fbb48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.747+0000 7fbb5a7fc640 1 -- 172.21.15.3:0/3199636425 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbb600865c0 con 0x7fbb64159bd0 2024-09-12T12:20:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.747+0000 7fbb697d6640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb480678c0 0x7fbb48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:13.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.747+0000 7fbb697d6640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb480678c0 0x7fbb48069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbb54007fd0 tx=0x7fbb54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:13.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.751+0000 7fbb5a7fc640 1 -- 172.21.15.3:0/3199636425 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbb60057100 con 0x7fbb64159bd0 2024-09-12T12:20:13.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.951+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbb640b2b00 con 0x7fbb64159bd0 2024-09-12T12:20:13.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.951+0000 7fbb5a7fc640 1 -- 172.21.15.3:0/3199636425 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fbb60056a00 con 0x7fbb64159bd0 2024-09-12T12:20:13.955 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:20:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb480678c0 msgr2=0x7fbb48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb480678c0 0x7fbb48069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbb54007fd0 tx=0x7fbb54009040 comp rx=0 tx=0).stop 2024-09-12T12:20:13.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 msgr2=0x7fbb64156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb64156840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbb60007ef0 tx=0x7fbb6000e4b0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 shutdown_connections 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb480678c0 0x7fbb48069d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb6415d2d0 0x7fbb64072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb64159fb0 0x7fbb6414b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 --2- 172.21.15.3:0/3199636425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb64156840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 >> 172.21.15.3:0/3199636425 conn(0x7fbb640b1d80 msgr2=0x7fbb64158790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 shutdown_connections 2024-09-12T12:20:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:13.955+0000 7fbb6c23b640 1 -- 172.21.15.3:0/3199636425 wait complete. 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:20:13.974 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-12T12:20:13.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:13.975 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:20:13.982 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42470sec preferred_lft 42470sec 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:13.983 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:13.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:13.984 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:20:14.036 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:20:14.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:14.038 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:20:14.090 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:20:14.090 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:20:14.090 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:20:14.090 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:20:14.090 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:20:14.091 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:20:14.091 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:20:14.091 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:14.091 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:14.091 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:14.091 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:20:14.092 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:14.092 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:14.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:20:14.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:20:14.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:20:14.092 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:14.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:14.208 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:14.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:14.215 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:20:14.221 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:20:14.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:14.221 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-09-12T12:20:14.269 INFO:teuthology.orchestra.run.smithi142.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-12T12:20:14.269 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:14.269 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:14.270 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:14.270 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-12T12:20:14.270 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:14.357 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:14.378 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:14.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:14.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-12T12:20:14.397 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 9a:a0:72:4f:1c:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T12:20:14.398 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.399 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::98a0:72ff:fe4f:1c70/64 scope link 2024-09-12T12:20:14.399 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:14.400 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:14.455 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 9a:a0:72:4f:1c:70 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-12T12:20:14.456 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-12T12:20:14.456 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.456 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::98a0:72ff:fe4f:1c70/64 scope link 2024-09-12T12:20:14.456 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:14.457 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-12T12:20:14.457 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:14.546 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:14.591 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:14.598 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:14.598 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:14.598 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.1 up 2024-09-12T12:20:14.598 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.1 master ceph-brx 2024-09-12T12:20:14.598 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:14.637 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:14.652 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:14.658 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-12T12:20:14.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:14.658 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:14.708 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-12T12:20:14.709 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:14.709 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:14.756 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:14.814 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:14.867 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:14.867 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:14.867 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:14.868 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:14.869 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:14.870 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:14.870 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:14.870 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:14.870 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:14.870 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:14.870 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:14.870 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:14.871 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:14.871 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:14.871 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:14.871 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:14.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:14.872 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:14.925 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:20:14.925 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-12T12:20:14.926 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-12T12:20:14.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:14.984 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:15.033 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:15.034 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:15.034 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:15.034 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:15.034 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:15.034 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:15.035 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:15.035 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:15.035 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:15.035 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:15.035 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:15.036 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:15.036 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:15.036 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:15.036 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:15.036 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:15.037 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:15.037 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:15.037 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:15.037 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:15.037 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:15.037 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:15.038 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:15.038 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:15.038 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:15.038 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:15.038 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:15.039 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:15.039 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:15.039 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:15.039 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:15.039 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:15.040 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:15.040 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:15.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:15.042 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:15.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi142.stderr:2024-09-12T12:20:15.046+0000 7fc44a8eb580 -1 init, newargv = 0x5565dec7f0b0 newargc=15 2024-09-12T12:20:15.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi142.stderr:ceph-fuse[35335]: starting ceph client 2024-09-12T12:20:15.053 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:20:15.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi142.stderr:ceph-fuse[35335]: starting fuse 2024-09-12T12:20:16.154 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:16.170 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:16.219 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:16.219 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:16.219 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:16.219 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:16.219 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:16.220 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:16.221 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:16.222 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:16.222 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:16.222 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:16.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.223 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:16.268 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:20:16.269 INFO:teuthology.orchestra.run.smithi142.stdout:46 2024-09-12T12:20:16.269 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-12T12:20:16.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:20:16.269 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:20:16.269 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:20:16.269 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:16.270 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.3")) 2024-09-12T12:20:16.271 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:20:16.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:20:16.289 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:20:16.390 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.3.35335.asok 2024-09-12T12:20:16.394 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:16.401 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.35335.asok 2024-09-12T12:20:16.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.402 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.35335.asok status 2024-09-12T12:20:16.557 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:20:16.557 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:20:16.557 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "3", 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "35335", 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 4409, 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:20:16.558 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 4409 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.2:0", 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 655462855 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.2:0", 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 655462855 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.4409 192.168.0.2:0/655462855", 2024-09-12T12:20:16.559 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.2:0/655462855", 2024-09-12T12:20:16.560 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:20:16.560 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 7, 2024-09-12T12:20:16.560 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 21, 2024-09-12T12:20:16.560 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:20:16.560 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:20:16.560 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:20:16.560 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:20:16.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.571 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:20:16.571 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:16.579 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:20:16.579 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:16.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:16.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.639 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:20:16.639 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:16.687 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:20:16.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:16.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.687 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:16.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.749 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:20:16.749 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:16.757 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:20:16.757 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:16.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.757 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:16.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.814 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:20:16.814 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:16.865 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:20:16.866 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:16.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:16.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:16.921 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-12T12:20:16.959 INFO:tasks.cephfs_test_runner:Starting test: test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a client unmounts, the thread count on the MDS goes back 2024-09-12T12:20:16.959 INFO:tasks.cephfs_test_runner:test_mount_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-12T12:20:16.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:20:17.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f156b2f7640 1 Processor -- start 2024-09-12T12:20:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f156b2f7640 1 -- start start 2024-09-12T12:20:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f156b2f7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 0x7f156406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f156b2f7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f156406fbb0 con 0x7f1564159fb0 2024-09-12T12:20:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f156b2f7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f156406e400 con 0x7f1564159bd0 2024-09-12T12:20:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f156b2f7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f156406f010 con 0x7f15640ad770 2024-09-12T12:20:17.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1564159fb0 0x7f15640ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44058/0 (socket says 172.21.15.3:44058) 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 -- 172.21.15.3:0/1812074279 learned_addr learned my addr 172.21.15.3:0/1812074279 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1569894640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 0x7f156406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2913081589 0 0) 0x7f156406fbb0 con 0x7f1564159fb0 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1550003530 con 0x7f1564159fb0 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2088929607 0 0) 0x7f1550003530 con 0x7f1564159fb0 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 msgr2=0x7f156406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 0x7f156406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 >> v1:172.21.15.142:6789/0 conn(0x7f1564159bd0 legacy=0x7f156406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156406e6e0 con 0x7f1564159fb0 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1554001680 con 0x7f1564159fb0 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 >> v1:172.21.15.3:6789/0 conn(0x7f1564159fb0 legacy=0x7f15640ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 0x7f15500039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550003f70 0x7f1550004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1550004d50 0x7f15500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f156406f010 con 0x7f1550003f70 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f155000bab0 con 0x7f1550004d50 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f155000bcb0 con 0x7f15640ad770 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550003f70 0x7f1550004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1569894640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1550004d50 0x7f15500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1569093640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 0x7f15500039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 -- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 msgr2=0x7f15500039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 0x7f15500039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 -- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1550004d50 msgr2=0x7f15500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1550004d50 0x7f15500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1568892640 1 -- 172.21.15.3:0/1812074279 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1550003530 con 0x7f1550003f70 2024-09-12T12:20:17.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.091+0000 7f1569093640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15640ad770 0x7f15500039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f1568892640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550003f70 0x7f1550004810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f156415fda0 tx=0x7f155401f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6658e767941e8ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554007d20 con 0x7f1550003f70 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1554005ae0 con 0x7f1550003f70 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f155bfff640 1 -- 172.21.15.3:0/1812074279 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554007070 con 0x7f1550003f70 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 -- 172.21.15.3:0/1812074279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550003f70 msgr2=0x7f1550004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 --2- 172.21.15.3:0/1812074279 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1550003f70 0x7f1550004810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f156415fda0 tx=0x7f155401f390 comp rx=0 tx=0).stop 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f1569894640 1 -- 172.21.15.3:0/1812074279 reap_dead start 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 -- 172.21.15.3:0/1812074279 shutdown_connections 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 -- 172.21.15.3:0/1812074279 >> 172.21.15.3:0/1812074279 conn(0x7f15640b1d80 msgr2=0x7f15640b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 -- 172.21.15.3:0/1812074279 shutdown_connections 2024-09-12T12:20:17.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 -- 172.21.15.3:0/1812074279 wait complete. 2024-09-12T12:20:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 Processor -- start 2024-09-12T12:20:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.095+0000 7f156b2f7640 1 -- start start 2024-09-12T12:20:17.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1550003f70 0x7f15641452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15641e61d0 0x7f1564099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1564155880 0x7f1564155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f156406dd40 con 0x7f15641e61d0 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f156406eca0 con 0x7f1564155880 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f156406dec0 con 0x7f1550003f70 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15641e61d0 0x7f1564099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15641e61d0 0x7f1564099ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46618/0 (socket says 172.21.15.3:46618) 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 -- 172.21.15.3:0/603026281 learned_addr learned my addr 172.21.15.3:0/603026281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:17.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1569093640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1550003f70 0x7f15641452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1569894640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1564155880 0x7f1564155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 -- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1550003f70 msgr2=0x7f15641452e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1550003f70 0x7f15641452e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 -- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1564155880 msgr2=0x7f1564155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1564155880 0x7f1564155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 -- 172.21.15.3:0/603026281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15641e6740 con 0x7f15641e61d0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1568892640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15641e61d0 0x7f1564099ab0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f155401f1c0 tx=0x7f1550005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1559ffb640 1 -- 172.21.15.3:0/603026281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15540047f0 con 0x7f15641e61d0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1559ffb640 1 -- 172.21.15.3:0/603026281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1554004990 con 0x7f15641e61d0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1564154da0 con 0x7f15641e61d0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1559ffb640 1 -- 172.21.15.3:0/603026281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554004d40 con 0x7f15641e61d0 2024-09-12T12:20:17.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1564145980 con 0x7f15641e61d0 2024-09-12T12:20:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156415d2d0 con 0x7f15641e61d0 2024-09-12T12:20:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1559ffb640 1 -- 172.21.15.3:0/603026281 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1554007070 con 0x7f15641e61d0 2024-09-12T12:20:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1559ffb640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f15480678f0 0x7f1548069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1559ffb640 1 -- 172.21.15.3:0/603026281 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f15541232a0 con 0x7f15641e61d0 2024-09-12T12:20:17.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.099+0000 7f1569093640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f15480678f0 0x7f1548069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.103+0000 7f1569093640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f15480678f0 0x7f1548069db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1560000c10 tx=0x7f1560009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.103+0000 7f1559ffb640 1 -- 172.21.15.3:0/603026281 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f15540b7070 con 0x7f15641e61d0 2024-09-12T12:20:17.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.271+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1564073ee0 con 0x7f15641e61d0 2024-09-12T12:20:17.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.275+0000 7f1559ffb640 1 -- 172.21.15.3:0/603026281 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f15540190a0 con 0x7f15641e61d0 2024-09-12T12:20:17.279 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:17.279 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:49.738546+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.275+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f15480678f0 msgr2=0x7f1548069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.275+0000 7f156b2f7640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f15480678f0 0x7f1548069db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1560000c10 tx=0x7f1560009040 comp rx=0 tx=0).stop 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.275+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15641e61d0 msgr2=0x7f1564099ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.275+0000 7f156b2f7640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15641e61d0 0x7f1564099ab0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f155401f1c0 tx=0x7f1550005840 comp rx=0 tx=0).stop 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.275+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 shutdown_connections 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.275+0000 7f156b2f7640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f15480678f0 0x7f1548069db0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.279+0000 7f156b2f7640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1564155880 0x7f1564155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.279+0000 7f156b2f7640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15641e61d0 0x7f1564099ab0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.279+0000 7f156b2f7640 1 --2- 172.21.15.3:0/603026281 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1550003f70 0x7f15641452e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.279+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 >> 172.21.15.3:0/603026281 conn(0x7f15640b1d80 msgr2=0x7f15640b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.279+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 shutdown_connections 2024-09-12T12:20:17.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.279+0000 7f156b2f7640 1 -- 172.21.15.3:0/603026281 wait complete. 2024-09-12T12:20:17.298 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:19:37.735209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T12:19:46.505915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T12:19:47.135597+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:20:17.298 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T12:20:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487c09c640 1 Processor -- start 2024-09-12T12:20:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487c09c640 1 -- start start 2024-09-12T12:20:17.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487c09c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f487415c430 0x7f48740cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487c09c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48740cf3c0 0x7f48740b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487c09c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48740d0880 con 0x7f48740d03b0 2024-09-12T12:20:17.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487c09c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f48740d0b60 con 0x7f487415c430 2024-09-12T12:20:17.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487c09c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f48740d0e40 con 0x7f48740cf3c0 2024-09-12T12:20:17.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4879e38640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f487415c430 0x7f48740cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4879e38640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f487415c430 0x7f48740cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46626/0 (socket says 172.21.15.3:46626) 2024-09-12T12:20:17.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4879e38640 1 -- 172.21.15.3:0/4180588969 learned_addr learned my addr 172.21.15.3:0/4180588969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:17.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f487a639640 1 --2- 172.21.15.3:0/4180588969 >> v2:172.21.15.142:3300/0 conn(0x7f48740cf3c0 0x7f48740b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4879e38640 1 -- 172.21.15.3:0/4180588969 >> v2:172.21.15.142:3300/0 conn(0x7f48740cf3c0 msgr2=0x7f48740b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4879e38640 1 --2- 172.21.15.3:0/4180588969 >> v2:172.21.15.142:3300/0 conn(0x7f48740cf3c0 0x7f48740b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4879e38640 1 -- 172.21.15.3:0/4180588969 >> v1:172.21.15.3:6789/0 conn(0x7f48740d03b0 legacy=0x7f48740cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4878e36640 1 -- 172.21.15.3:0/4180588969 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3483274647 0 0) 0x7f48740d0880 con 0x7f48740d03b0 2024-09-12T12:20:17.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.419+0000 7f4879e38640 1 -- 172.21.15.3:0/4180588969 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48740d1120 con 0x7f487415c430 2024-09-12T12:20:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f4879e38640 1 --2- 172.21.15.3:0/4180588969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487415c430 0x7f48740cfe70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f48740d1400 tx=0x7f486401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fba992e0889c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f4878e36640 1 -- 172.21.15.3:0/4180588969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640ab430 con 0x7f487415c430 2024-09-12T12:20:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f4878e36640 1 -- 172.21.15.3:0/4180588969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48640b0070 con 0x7f487415c430 2024-09-12T12:20:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f4878e36640 1 -- 172.21.15.3:0/4180588969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640ab7f0 con 0x7f487415c430 2024-09-12T12:20:17.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- 172.21.15.3:0/4180588969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487415c430 msgr2=0x7f48740cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 --2- 172.21.15.3:0/4180588969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487415c430 0x7f48740cfe70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f48740d1400 tx=0x7f486401c520 comp rx=0 tx=0).stop 2024-09-12T12:20:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- 172.21.15.3:0/4180588969 shutdown_connections 2024-09-12T12:20:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 --2- 172.21.15.3:0/4180588969 >> v2:172.21.15.142:3300/0 conn(0x7f48740cf3c0 0x7f48740b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 --2- 172.21.15.3:0/4180588969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f487415c430 0x7f48740cfe70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- 172.21.15.3:0/4180588969 >> 172.21.15.3:0/4180588969 conn(0x7f48740b3f50 msgr2=0x7f48740b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- 172.21.15.3:0/4180588969 shutdown_connections 2024-09-12T12:20:17.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- 172.21.15.3:0/4180588969 wait complete. 2024-09-12T12:20:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 Processor -- start 2024-09-12T12:20:17.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- start start 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48740cf3c0 0x7f4874152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48740d03b0 0x7f487413d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f487415c430 0x7f487413a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f48740b6850 con 0x7f48740cf3c0 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48740b66d0 con 0x7f487415c430 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f487c09c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48740b69d0 con 0x7f48740d03b0 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.423+0000 7f4879e38640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48740cf3c0 0x7f4874152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48740cf3c0 0x7f4874152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46638/0 (socket says 172.21.15.3:46638) 2024-09-12T12:20:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 -- 172.21.15.3:0/1806901766 learned_addr learned my addr 172.21.15.3:0/1806901766 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f487a639640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f487415c430 0x7f487413a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879637640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48740d03b0 0x7f487413d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 -- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48740d03b0 msgr2=0x7f487413d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48740d03b0 0x7f487413d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 -- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f487415c430 msgr2=0x7f487413a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f487415c430 0x7f487413a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 -- 172.21.15.3:0/1806901766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48740cecb0 con 0x7f48740cf3c0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879e38640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48740cf3c0 0x7f4874152e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4864019130 tx=0x7f486401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f48727fc640 1 -- 172.21.15.3:0/1806901766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640be5f0 con 0x7f48740cf3c0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f48727fc640 1 -- 172.21.15.3:0/1806901766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48640be790 con 0x7f48740cf3c0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f48727fc640 1 -- 172.21.15.3:0/1806901766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640beb80 con 0x7f48740cf3c0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4874154080 con 0x7f48740cf3c0 2024-09-12T12:20:17.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4874153500 con 0x7f48740cf3c0 2024-09-12T12:20:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48740b5770 con 0x7f48740cf3c0 2024-09-12T12:20:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f48727fc640 1 -- 172.21.15.3:0/1806901766 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f48640b43f0 con 0x7f48740cf3c0 2024-09-12T12:20:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f48727fc640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48600678f0 0x7f4860069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f48727fc640 1 -- 172.21.15.3:0/1806901766 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f48640a8030 con 0x7f48740cf3c0 2024-09-12T12:20:17.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.427+0000 7f4879637640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48600678f0 0x7f4860069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.431+0000 7f4879637640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48600678f0 0x7f4860069db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4868007fd0 tx=0x7f4868009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.431+0000 7f48727fc640 1 -- 172.21.15.3:0/1806901766 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4864016110 con 0x7f48740cf3c0 2024-09-12T12:20:17.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4874157040 con 0x7f48740cf3c0 2024-09-12T12:20:17.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f48727fc640 1 -- 172.21.15.3:0/1806901766 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f48640f0e40 con 0x7f48740cf3c0 2024-09-12T12:20:17.598 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T12:20:17.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48600678f0 msgr2=0x7f4860069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48600678f0 0x7f4860069db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4868007fd0 tx=0x7f4868009040 comp rx=0 tx=0).stop 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48740cf3c0 msgr2=0x7f4874152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48740cf3c0 0x7f4874152e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4864019130 tx=0x7f486401ad60 comp rx=0 tx=0).stop 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 shutdown_connections 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48600678f0 0x7f4860069db0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f487415c430 0x7f487413a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48740d03b0 0x7f487413d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 --2- 172.21.15.3:0/1806901766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48740cf3c0 0x7f4874152e40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 >> 172.21.15.3:0/1806901766 conn(0x7f48740b3f50 msgr2=0x7f48740b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 shutdown_connections 2024-09-12T12:20:17.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.595+0000 7f487c09c640 1 -- 172.21.15.3:0/1806901766 wait complete. 2024-09-12T12:20:17.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T12:20:17.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a792d640 1 Processor -- start 2024-09-12T12:20:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a792d640 1 -- start start 2024-09-12T12:20:17.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a792d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79a0159dd0 0x7f79a015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a792d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79a006f150 0x7f79a00ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a792d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a015d830 con 0x7f79a006f5b0 2024-09-12T12:20:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a792d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f79a006e0b0 con 0x7f79a006f150 2024-09-12T12:20:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a792d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79a006e390 con 0x7f79a0159dd0 2024-09-12T12:20:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a4ec8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f79a006f150 0x7f79a00ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a5eca640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79a006f5b0 0x7f79a00b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44068/0 (socket says 172.21.15.3:44068) 2024-09-12T12:20:17.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a5eca640 1 -- 172.21.15.3:0/2266931583 learned_addr learned my addr 172.21.15.3:0/2266931583 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:17.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a56c9640 1 --2- 172.21.15.3:0/2266931583 >> v2:172.21.15.142:3300/0 conn(0x7f79a0159dd0 0x7f79a015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a4ec8640 1 -- 172.21.15.3:0/2266931583 >> v2:172.21.15.142:3300/0 conn(0x7f79a0159dd0 msgr2=0x7f79a015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a4ec8640 1 --2- 172.21.15.3:0/2266931583 >> v2:172.21.15.142:3300/0 conn(0x7f79a0159dd0 0x7f79a015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a4ec8640 1 -- 172.21.15.3:0/2266931583 >> v1:172.21.15.3:6789/0 conn(0x7f79a006f5b0 legacy=0x7f79a00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f79a4ec8640 1 -- 172.21.15.3:0/2266931583 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a006e670 con 0x7f79a006f150 2024-09-12T12:20:17.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.731+0000 7f7997fff640 1 -- 172.21.15.3:0/2266931583 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149339179 0 0) 0x7f79980014a0 con 0x7f79a006f5b0 2024-09-12T12:20:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a4ec8640 1 --2- 172.21.15.3:0/2266931583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a006f150 0x7f79a00ad060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f79a00702b0 tx=0x7f799c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c722923dc0d71b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f7997fff640 1 -- 172.21.15.3:0/2266931583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0ab650 con 0x7f79a006f150 2024-09-12T12:20:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f7997fff640 1 -- 172.21.15.3:0/2266931583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c0b0070 con 0x7f79a006f150 2024-09-12T12:20:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f7997fff640 1 -- 172.21.15.3:0/2266931583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c0aba10 con 0x7f79a006f150 2024-09-12T12:20:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- 172.21.15.3:0/2266931583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a006f150 msgr2=0x7f79a00ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 --2- 172.21.15.3:0/2266931583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a006f150 0x7f79a00ad060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f79a00702b0 tx=0x7f799c01c450 comp rx=0 tx=0).stop 2024-09-12T12:20:17.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- 172.21.15.3:0/2266931583 shutdown_connections 2024-09-12T12:20:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 --2- 172.21.15.3:0/2266931583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a006f150 0x7f79a00ad060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 --2- 172.21.15.3:0/2266931583 >> v2:172.21.15.142:3300/0 conn(0x7f79a0159dd0 0x7f79a015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- 172.21.15.3:0/2266931583 >> 172.21.15.3:0/2266931583 conn(0x7f79a00b1d80 msgr2=0x7f79a00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- 172.21.15.3:0/2266931583 shutdown_connections 2024-09-12T12:20:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- 172.21.15.3:0/2266931583 wait complete. 2024-09-12T12:20:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 Processor -- start 2024-09-12T12:20:17.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- start start 2024-09-12T12:20:17.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a006f150 0x7f79a0145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a006f5b0 0x7f79a00733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a0159dd0 0x7f79a015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79a015a1b0 con 0x7f79a0159dd0 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79a00ad9b0 con 0x7f79a006f150 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a792d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79a00b2a40 con 0x7f79a006f5b0 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a0159dd0 0x7f79a015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a0159dd0 0x7f79a015e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46668/0 (socket says 172.21.15.3:46668) 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 -- 172.21.15.3:0/1795911238 learned_addr learned my addr 172.21.15.3:0/1795911238 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:17.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a56c9640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a006f150 0x7f79a0145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a4ec8640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a006f5b0 0x7f79a00733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 -- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a006f5b0 msgr2=0x7f79a00733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a006f5b0 0x7f79a00733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 -- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a006f150 msgr2=0x7f79a0145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a006f150 0x7f79a0145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 -- 172.21.15.3:0/1795911238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a01572f0 con 0x7f79a0159dd0 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a4ec8640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a006f5b0 0x7f79a00733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.735+0000 7f79a5eca640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a0159dd0 0x7f79a015e1b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f799800c7f0 tx=0x7f799800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f7995ffb640 1 -- 172.21.15.3:0/1795911238 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998018070 con 0x7f79a0159dd0 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f7995ffb640 1 -- 172.21.15.3:0/1795911238 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799800a640 con 0x7f79a0159dd0 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a0072310 con 0x7f79a0159dd0 2024-09-12T12:20:17.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f7995ffb640 1 -- 172.21.15.3:0/1795911238 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799800aa30 con 0x7f79a0159dd0 2024-09-12T12:20:17.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a00728d0 con 0x7f79a0159dd0 2024-09-12T12:20:17.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a00ad060 con 0x7f79a0159dd0 2024-09-12T12:20:17.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f7995ffb640 1 -- 172.21.15.3:0/1795911238 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f799801b070 con 0x7f79a0159dd0 2024-09-12T12:20:17.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f7995ffb640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79840678c0 0x7f7984069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f7995ffb640 1 -- 172.21.15.3:0/1795911238 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7998085570 con 0x7f79a0159dd0 2024-09-12T12:20:17.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f79a56c9640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79840678c0 0x7f7984069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.739+0000 7f79a56c9640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79840678c0 0x7f7984069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f799000b9d0 tx=0x7f7990008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.743+0000 7f7995ffb640 1 -- 172.21.15.3:0/1795911238 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f79a015d830 con 0x7f79a0159dd0 2024-09-12T12:20:17.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.899+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f79a0157010 con 0x7f79a0159dd0 2024-09-12T12:20:17.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.899+0000 7f7995ffb640 1 -- 172.21.15.3:0/1795911238 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f7998056190 con 0x7f79a0159dd0 2024-09-12T12:20:17.904 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79840678c0 msgr2=0x7f7984069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79840678c0 0x7f7984069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f799000b9d0 tx=0x7f7990008040 comp rx=0 tx=0).stop 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a0159dd0 msgr2=0x7f79a015e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a0159dd0 0x7f79a015e1b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f799800c7f0 tx=0x7f799800ccb0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 shutdown_connections 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79840678c0 0x7f7984069d80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79a0159dd0 0x7f79a015e1b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79a006f5b0 0x7f79a00733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 --2- 172.21.15.3:0/1795911238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79a006f150 0x7f79a0145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 >> 172.21.15.3:0/1795911238 conn(0x7f79a00b1d80 msgr2=0x7f79a00aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 shutdown_connections 2024-09-12T12:20:17.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:17.903+0000 7f79a792d640 1 -- 172.21.15.3:0/1795911238 wait complete. 2024-09-12T12:20:17.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T12:20:18.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d93023640 1 Processor -- start 2024-09-12T12:20:18.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d93023640 1 -- start start 2024-09-12T12:20:18.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d93023640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8d8c06f6e0 0x7f8d8c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d93023640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d8c15f0b0 con 0x7f8d8c15b130 2024-09-12T12:20:18.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d93023640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d8c15edd0 con 0x7f8d8c0cc270 2024-09-12T12:20:18.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d93023640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8d8c0b3790 con 0x7f8d8c06f6e0 2024-09-12T12:20:18.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d915c0640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8d8c0cc270 0x7f8d8c0aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:57586/0 (socket says 172.21.15.3:57586) 2024-09-12T12:20:18.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d915c0640 1 -- 172.21.15.3:0/1520648216 learned_addr learned my addr 172.21.15.3:0/1520648216 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:18.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d90dbf640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8d8c15b130 0x7f8d8c0b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34646/0 (socket says 172.21.15.3:34646) 2024-09-12T12:20:18.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8bfff640 1 --2- 172.21.15.3:0/1520648216 >> v2:172.21.15.142:3300/0 conn(0x7f8d8c06f6e0 0x7f8d8c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139006925 0 0) 0x7f8d8c15f0b0 con 0x7f8d8c15b130 2024-09-12T12:20:18.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d74003530 con 0x7f8d8c15b130 2024-09-12T12:20:18.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2843447498 0 0) 0x7f8d8c15edd0 con 0x7f8d8c0cc270 2024-09-12T12:20:18.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d8c15f0b0 con 0x7f8d8c0cc270 2024-09-12T12:20:18.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500249645 0 0) 0x7f8d74003530 con 0x7f8d8c15b130 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 >> v2:172.21.15.142:3300/0 conn(0x7f8d8c06f6e0 msgr2=0x7f8d8c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 --2- 172.21.15.3:0/1520648216 >> v2:172.21.15.142:3300/0 conn(0x7f8d8c06f6e0 0x7f8d8c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 >> v1:172.21.15.142:6789/0 conn(0x7f8d8c0cc270 legacy=0x7f8d8c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8c0adff0 con 0x7f8d8c15b130 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f8d7c001680 con 0x7f8d8c15b130 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 >> v1:172.21.15.3:6789/0 conn(0x7f8d8c15b130 legacy=0x7f8d8c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d74003ba0 0x7f8d74003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d74004810 0x7f8d74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d740051d0 0x7f8d7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c0b3790 con 0x7f8d740051d0 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7400c0a0 con 0x7f8d74003ba0 2024-09-12T12:20:18.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7400c2a0 con 0x7f8d74004810 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d915c0640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d740051d0 0x7f8d7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.047+0000 7f8d90dbf640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d74003ba0 0x7f8d74003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d8bfff640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d74004810 0x7f8d74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d915c0640 1 -- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d74004810 msgr2=0x7f8d74004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d915c0640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d74004810 0x7f8d74004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d915c0640 1 -- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d74003ba0 msgr2=0x7f8d74003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d915c0640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d74003ba0 0x7f8d74003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d915c0640 1 -- 172.21.15.3:0/1520648216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d74003530 con 0x7f8d740051d0 2024-09-12T12:20:18.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d8bfff640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d74004810 0x7f8d74004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:18.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d915c0640 1 -- 172.21.15.3:0/1520648216 reap_dead start 2024-09-12T12:20:18.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d915c0640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d740051d0 0x7f8d7400b9d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c240 tx=0x7f8d8401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7593ed1aaf467f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84001fa0 con 0x7f8d740051d0 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d840a9070 con 0x7f8d740051d0 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d8b7fe640 1 -- 172.21.15.3:0/1520648216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840a5bc0 con 0x7f8d740051d0 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d93023640 1 -- 172.21.15.3:0/1520648216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d740051d0 msgr2=0x7f8d7400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d93023640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d740051d0 0x7f8d7400b9d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c240 tx=0x7f8d8401cca0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d93023640 1 -- 172.21.15.3:0/1520648216 shutdown_connections 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d93023640 1 --2- 172.21.15.3:0/1520648216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d740051d0 0x7f8d7400b9d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d93023640 1 -- 172.21.15.3:0/1520648216 >> 172.21.15.3:0/1520648216 conn(0x7f8d8c0b1d80 msgr2=0x7f8d8c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d93023640 1 -- 172.21.15.3:0/1520648216 shutdown_connections 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.051+0000 7f8d93023640 1 -- 172.21.15.3:0/1520648216 wait complete. 2024-09-12T12:20:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 Processor -- start 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 -- start start 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d740051d0 0x7f8d8c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d8c15c3c0 0x7f8d8c0c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d8c1f3b80 0x7f8d8c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c0ae5b0 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c0ae920 con 0x7f8d740051d0 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8c06d860 con 0x7f8d8c15c3c0 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d8c1f3b80 0x7f8d8c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d8c1f3b80 0x7f8d8c1509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49470/0 (socket says 172.21.15.3:49470) 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 -- 172.21.15.3:0/1899071195 learned_addr learned my addr 172.21.15.3:0/1899071195 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d90dbf640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d740051d0 0x7f8d8c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d8bfff640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d8c15c3c0 0x7f8d8c0c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 -- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d8c15c3c0 msgr2=0x7f8d8c0c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d8c15c3c0 0x7f8d8c0c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 -- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d740051d0 msgr2=0x7f8d8c0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d740051d0 0x7f8d8c0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 -- 172.21.15.3:0/1899071195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8c0b0830 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d8bfff640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d8c15c3c0 0x7f8d8c0c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d915c0640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d8c1f3b80 0x7f8d8c1509e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d8401cad0 tx=0x7f8d74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d897fa640 1 -- 172.21.15.3:0/1899071195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84001fa0 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d897fa640 1 -- 172.21.15.3:0/1899071195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d840ad620 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8c15d0c0 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d897fa640 1 -- 172.21.15.3:0/1899071195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840ada10 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8c0c9580 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8c15b510 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d897fa640 1 -- 172.21.15.3:0/1899071195 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8d840ab030 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.055+0000 7f8d897fa640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8d780678c0 0x7f8d78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.059+0000 7f8d897fa640 1 -- 172.21.15.3:0/1899071195 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8d8411f0f0 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.059+0000 7f8d90dbf640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8d780678c0 0x7f8d78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.059+0000 7f8d90dbf640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8d780678c0 0x7f8d78069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c001060 tx=0x7f8d7c001e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.063+0000 7f8d897fa640 1 -- 172.21.15.3:0/1899071195 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8d840bd070 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d8c0b0d80 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d897fa640 1 -- 172.21.15.3:0/1899071195 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f8d840169b0 con 0x7f8d8c1f3b80 2024-09-12T12:20:18.230 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8d780678c0 msgr2=0x7f8d78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8d780678c0 0x7f8d78069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c001060 tx=0x7f8d7c001e40 comp rx=0 tx=0).stop 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d8c1f3b80 msgr2=0x7f8d8c1509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d8c1f3b80 0x7f8d8c1509e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d8401cad0 tx=0x7f8d74005840 comp rx=0 tx=0).stop 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 shutdown_connections 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8d780678c0 0x7f8d78069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d8c1f3b80 0x7f8d8c1509e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d8c15c3c0 0x7f8d8c0c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 --2- 172.21.15.3:0/1899071195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d740051d0 0x7f8d8c0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 >> 172.21.15.3:0/1899071195 conn(0x7f8d8c0b1d80 msgr2=0x7f8d8c15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:18.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 shutdown_connections 2024-09-12T12:20:18.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.227+0000 7f8d93023640 1 -- 172.21.15.3:0/1899071195 wait complete. 2024-09-12T12:20:18.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-09-12T12:20:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba9ba0640 1 Processor -- start 2024-09-12T12:20:18.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba9ba0640 1 -- start start 2024-09-12T12:20:18.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba9ba0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffba4159fb0 0x7ffba406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba9ba0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba406fbb0 con 0x7ffba4159bd0 2024-09-12T12:20:18.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba9ba0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba415d760 con 0x7ffba415d2d0 2024-09-12T12:20:18.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba9ba0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffba406f010 con 0x7ffba4159fb0 2024-09-12T12:20:18.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba37fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffba4159bd0 0x7ffba406f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34660/0 (socket says 172.21.15.3:34660) 2024-09-12T12:20:18.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba37fe640 1 -- 172.21.15.3:0/919175236 learned_addr learned my addr 172.21.15.3:0/919175236 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/919175236 >> v2:172.21.15.142:3300/0 conn(0x7ffba4159fb0 0x7ffba406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 431730783 0 0) 0x7ffba406fbb0 con 0x7ffba4159bd0 2024-09-12T12:20:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb8c003530 con 0x7ffba4159bd0 2024-09-12T12:20:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2235962039 0 0) 0x7ffb8c003530 con 0x7ffba4159bd0 2024-09-12T12:20:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 >> v2:172.21.15.142:3300/0 conn(0x7ffba4159fb0 msgr2=0x7ffba406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 --2- 172.21.15.3:0/919175236 >> v2:172.21.15.142:3300/0 conn(0x7ffba4159fb0 0x7ffba406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 >> v1:172.21.15.142:6789/0 conn(0x7ffba415d2d0 legacy=0x7ffba40ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba406e400 con 0x7ffba4159bd0 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7ffb94001680 con 0x7ffba4159bd0 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 >> v1:172.21.15.3:6789/0 conn(0x7ffba4159bd0 legacy=0x7ffba406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c0039c0 0x7ffb8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb8c0051d0 0x7ffb8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406f010 con 0x7ffb8c004810 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c00c0a0 con 0x7ffb8c0051d0 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c00c2a0 con 0x7ffb8c0039c0 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba37fe640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c0039c0 0x7ffb8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba3fff640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb8c0051d0 0x7ffb8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba2ffd640 1 -- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c0039c0 msgr2=0x7ffb8c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c0039c0 0x7ffb8c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba2ffd640 1 -- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb8c0051d0 msgr2=0x7ffb8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb8c0051d0 0x7ffb8c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba3fff640 1 -- 172.21.15.3:0/919175236 reap_dead start 2024-09-12T12:20:18.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.375+0000 7ffba2ffd640 1 -- 172.21.15.3:0/919175236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c003530 con 0x7ffb8c004810 2024-09-12T12:20:18.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffba415fda0 tx=0x7ffb9801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7078a2fe3e4cabd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:18.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb980096b0 con 0x7ffb8c004810 2024-09-12T12:20:18.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb980a6500 con 0x7ffb8c004810 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba27fc640 1 -- 172.21.15.3:0/919175236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb980a68b0 con 0x7ffb8c004810 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba9ba0640 1 -- 172.21.15.3:0/919175236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c004810 msgr2=0x7ffb8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffba415fda0 tx=0x7ffb9801ca10 comp rx=0 tx=0).stop 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba9ba0640 1 -- 172.21.15.3:0/919175236 shutdown_connections 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/919175236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba9ba0640 1 -- 172.21.15.3:0/919175236 >> 172.21.15.3:0/919175236 conn(0x7ffba40b1d80 msgr2=0x7ffba40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba9ba0640 1 -- 172.21.15.3:0/919175236 shutdown_connections 2024-09-12T12:20:18.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.379+0000 7ffba9ba0640 1 -- 172.21.15.3:0/919175236 wait complete. 2024-09-12T12:20:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 Processor -- start 2024-09-12T12:20:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 -- start start 2024-09-12T12:20:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c004810 0x7ffba41f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffba40cc380 0x7ffba40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba40b1310 0x7ffba4153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406eca0 con 0x7ffba40cc380 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406e9c0 con 0x7ffba40b1310 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffba406dd40 con 0x7ffb8c004810 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffba40cc380 0x7ffba40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffba40cc380 0x7ffba40b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49490/0 (socket says 172.21.15.3:49490) 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 -- 172.21.15.3:0/1708156776 learned_addr learned my addr 172.21.15.3:0/1708156776 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba3fff640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba40b1310 0x7ffba4153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba37fe640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c004810 0x7ffba41f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 -- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c004810 msgr2=0x7ffba41f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c004810 0x7ffba41f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 -- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba40b1310 msgr2=0x7ffba4153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba40b1310 0x7ffba4153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 -- 172.21.15.3:0/1708156776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba41f5700 con 0x7ffba40cc380 2024-09-12T12:20:18.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba2ffd640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffba40cc380 0x7ffba40b0dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb9801c840 tx=0x7ffb980197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:18.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffb8bfff640 1 -- 172.21.15.3:0/1708156776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb980a6980 con 0x7ffba40cc380 2024-09-12T12:20:18.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffb8bfff640 1 -- 172.21.15.3:0/1708156776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb980be070 con 0x7ffba40cc380 2024-09-12T12:20:18.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffb8bfff640 1 -- 172.21.15.3:0/1708156776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb980a6d40 con 0x7ffba40cc380 2024-09-12T12:20:18.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba4155420 con 0x7ffba40cc380 2024-09-12T12:20:18.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba40ccce0 con 0x7ffba40cc380 2024-09-12T12:20:18.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.383+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba4155940 con 0x7ffba40cc380 2024-09-12T12:20:18.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.387+0000 7ffb8bfff640 1 -- 172.21.15.3:0/1708156776 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb98006000 con 0x7ffba40cc380 2024-09-12T12:20:18.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.387+0000 7ffb8bfff640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffb900678c0 0x7ffb90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:18.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.387+0000 7ffb8bfff640 1 -- 172.21.15.3:0/1708156776 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffb98121330 con 0x7ffba40cc380 2024-09-12T12:20:18.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.387+0000 7ffba37fe640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffb900678c0 0x7ffb90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:18.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.387+0000 7ffba37fe640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffb900678c0 0x7ffb90069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb940052c0 tx=0x7ffb94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:18.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.391+0000 7ffb8bfff640 1 -- 172.21.15.3:0/1708156776 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffba4155940 con 0x7ffba40cc380 2024-09-12T12:20:18.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.615+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7ffba406e0d0 con 0x7ffba40cc380 2024-09-12T12:20:18.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.895+0000 7ffb8bfff640 1 -- 172.21.15.3:0/1708156776 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v47) ==== 147+0+0 (secure 0 0 0) 0x7ffb98016720 con 0x7ffba40cc380 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffb900678c0 msgr2=0x7ffb90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffb900678c0 0x7ffb90069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb940052c0 tx=0x7ffb94009040 comp rx=0 tx=0).stop 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffba40cc380 msgr2=0x7ffba40b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffba40cc380 0x7ffba40b0dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb9801c840 tx=0x7ffb980197b0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 shutdown_connections 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffb900678c0 0x7ffb90069d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffba40b1310 0x7ffba4153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffba40cc380 0x7ffba40b0dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 --2- 172.21.15.3:0/1708156776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffb8c004810 0x7ffba41f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 >> 172.21.15.3:0/1708156776 conn(0x7ffba40b1d80 msgr2=0x7ffba40ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:18.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 shutdown_connections 2024-09-12T12:20:18.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:18.899+0000 7ffba9ba0640 1 -- 172.21.15.3:0/1708156776 wait complete. 2024-09-12T12:20:18.922 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T12:20:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.047+0000 7f48fc569640 1 Processor -- start 2024-09-12T12:20:19.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.047+0000 7f48fc569640 1 -- start start 2024-09-12T12:20:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.047+0000 7f48fc569640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48f415e520 0x7f48f40d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.047+0000 7f48fc569640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40b3a20 0x7f48f40d2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.047+0000 7f48fc569640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f4163040 con 0x7f48f40d0ea0 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.047+0000 7f48fc569640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f48f4162430 con 0x7f48f415e520 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.047+0000 7f48fc569640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48f4162710 con 0x7f48f40b3a20 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48f415e520 0x7f48f40d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f48f415e520 0x7f48f40d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49506/0 (socket says 172.21.15.3:49506) 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fab06640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f48f40d0ea0 0x7f48f40d40e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34670/0 (socket says 172.21.15.3:34670) 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 -- 172.21.15.3:0/1810049204 learned_addr learned my addr 172.21.15.3:0/1810049204 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48f9b04640 1 --2- 172.21.15.3:0/1810049204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40b3a20 0x7f48f40d2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 -- 172.21.15.3:0/1810049204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40b3a20 msgr2=0x7f48f40d2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 --2- 172.21.15.3:0/1810049204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40b3a20 0x7f48f40d2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 -- 172.21.15.3:0/1810049204 >> v1:172.21.15.3:6789/0 conn(0x7f48f40d0ea0 legacy=0x7f48f40d40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 -- 172.21.15.3:0/1810049204 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f41629f0 con 0x7f48f415e520 2024-09-12T12:20:19.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48f9b04640 1 --2- 172.21.15.3:0/1810049204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40b3a20 0x7f48f40d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:19.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fa305640 1 --2- 172.21.15.3:0/1810049204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f415e520 0x7f48f40d2710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009870 tx=0x7f48e401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19cbd6fcc4403d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48f9303640 1 -- 172.21.15.3:0/1810049204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40b0070 con 0x7f48f415e520 2024-09-12T12:20:19.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48f9303640 1 -- 172.21.15.3:0/1810049204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4004230 con 0x7f48f415e520 2024-09-12T12:20:19.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48f9303640 1 -- 172.21.15.3:0/1810049204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003620 con 0x7f48f415e520 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fc569640 1 -- 172.21.15.3:0/1810049204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f415e520 msgr2=0x7f48f40d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fc569640 1 --2- 172.21.15.3:0/1810049204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f415e520 0x7f48f40d2710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009870 tx=0x7f48e401c3f0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fc569640 1 -- 172.21.15.3:0/1810049204 shutdown_connections 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fc569640 1 --2- 172.21.15.3:0/1810049204 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40b3a20 0x7f48f40d2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fc569640 1 --2- 172.21.15.3:0/1810049204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f415e520 0x7f48f40d2710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.051+0000 7f48fc569640 1 -- 172.21.15.3:0/1810049204 >> 172.21.15.3:0/1810049204 conn(0x7f48f409d820 msgr2=0x7f48f408c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- 172.21.15.3:0/1810049204 shutdown_connections 2024-09-12T12:20:19.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- 172.21.15.3:0/1810049204 wait complete. 2024-09-12T12:20:19.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 Processor -- start 2024-09-12T12:20:19.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- start start 2024-09-12T12:20:19.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f40b3a20 0x7f48f41f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40d0ea0 0x7f48f40b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f415e520 0x7f48f40cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f48f4162100 con 0x7f48f40b3a20 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48f415e900 con 0x7f48f415e520 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48f40d0350 con 0x7f48f40d0ea0 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f40b3a20 0x7f48f41f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f40b3a20 0x7f48f41f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49514/0 (socket says 172.21.15.3:49514) 2024-09-12T12:20:19.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 -- 172.21.15.3:0/1477567900 learned_addr learned my addr 172.21.15.3:0/1477567900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fab06640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f415e520 0x7f48f40cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48f9b04640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40d0ea0 0x7f48f40b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 -- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40d0ea0 msgr2=0x7f48f40b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40d0ea0 0x7f48f40b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 -- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f415e520 msgr2=0x7f48f40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f415e520 0x7f48f40cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 -- 172.21.15.3:0/1477567900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f41f5830 con 0x7f48f40b3a20 2024-09-12T12:20:19.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fab06640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f415e520 0x7f48f40cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fa305640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f40b3a20 0x7f48f41f52f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48e40ae040 tx=0x7f48e4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48eaffd640 1 -- 172.21.15.3:0/1477567900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40b0040 con 0x7f48f40b3a20 2024-09-12T12:20:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48eaffd640 1 -- 172.21.15.3:0/1477567900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e40ab640 con 0x7f48f40b3a20 2024-09-12T12:20:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48eaffd640 1 -- 172.21.15.3:0/1477567900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40aba30 con 0x7f48f40b3a20 2024-09-12T12:20:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f40cc5a0 con 0x7f48f40b3a20 2024-09-12T12:20:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.055+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f40cc840 con 0x7f48f40b3a20 2024-09-12T12:20:19.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.059+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f40d3620 con 0x7f48f40b3a20 2024-09-12T12:20:19.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.059+0000 7f48eaffd640 1 -- 172.21.15.3:0/1477567900 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f48e40abbd0 con 0x7f48f40b3a20 2024-09-12T12:20:19.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.059+0000 7f48eaffd640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48d80678f0 0x7f48d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.059+0000 7f48eaffd640 1 -- 172.21.15.3:0/1477567900 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f48e4120430 con 0x7f48f40b3a20 2024-09-12T12:20:19.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.059+0000 7f48f9b04640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48d80678f0 0x7f48d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.059+0000 7f48f9b04640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48d80678f0 0x7f48d8069db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.063+0000 7f48eaffd640 1 -- 172.21.15.3:0/1477567900 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f48f41f5830 con 0x7f48f40b3a20 2024-09-12T12:20:19.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.227+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f48f406dd50 con 0x7f48f40b3a20 2024-09-12T12:20:19.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.235+0000 7f48eaffd640 1 -- 172.21.15.3:0/1477567900 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f48e4016070 con 0x7f48f40b3a20 2024-09-12T12:20:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48d80678f0 msgr2=0x7f48d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48d80678f0 0x7f48d8069db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0008040 comp rx=0 tx=0).stop 2024-09-12T12:20:19.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f40b3a20 msgr2=0x7f48f41f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f40b3a20 0x7f48f41f52f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f48e40ae040 tx=0x7f48e4019190 comp rx=0 tx=0).stop 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 shutdown_connections 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48d80678f0 0x7f48d8069db0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f415e520 0x7f48f40cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48f40d0ea0 0x7f48f40b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 --2- 172.21.15.3:0/1477567900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48f40b3a20 0x7f48f41f52f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 >> 172.21.15.3:0/1477567900 conn(0x7f48f409d820 msgr2=0x7f48f406cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 shutdown_connections 2024-09-12T12:20:19.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.239+0000 7f48fc569640 1 -- 172.21.15.3:0/1477567900 wait complete. 2024-09-12T12:20:19.258 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:20:19.258 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:20:19.286 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:20:19.346 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:20:19.346 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:19.346 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:19.351 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:19.351 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:19.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:19.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cf9c2640 1 Processor -- start 2024-09-12T12:20:19.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cf9c2640 1 -- start start 2024-09-12T12:20:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cf9c2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff2c80d26f0 0x7ff2c815f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cf9c2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c80d2d90 con 0x7ff2c80ced00 2024-09-12T12:20:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cf9c2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c80d3070 con 0x7ff2c80b5fc0 2024-09-12T12:20:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cf9c2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c80d2b60 con 0x7ff2c80d26f0 2024-09-12T12:20:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cdf5f640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff2c80ced00 0x7ff2c80cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34676/0 (socket says 172.21.15.3:34676) 2024-09-12T12:20:19.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cdf5f640 1 -- 172.21.15.3:0/1961936630 learned_addr learned my addr 172.21.15.3:0/1961936630 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962295607 0 0) 0x7ff2c80d2d90 con 0x7ff2c80ced00 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2ccf5d640 1 --2- 172.21.15.3:0/1961936630 >> v2:172.21.15.142:3300/0 conn(0x7ff2c80d26f0 0x7ff2c815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2b4003530 con 0x7ff2c80ced00 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1246168049 0 0) 0x7ff2b4003530 con 0x7ff2c80ced00 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 >> v2:172.21.15.142:3300/0 conn(0x7ff2c80d26f0 msgr2=0x7ff2c815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 --2- 172.21.15.3:0/1961936630 >> v2:172.21.15.142:3300/0 conn(0x7ff2c80d26f0 0x7ff2c815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 >> v1:172.21.15.142:6789/0 conn(0x7ff2c80b5fc0 legacy=0x7ff2c80b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c80d3350 con 0x7ff2c80ced00 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7ff2c0001680 con 0x7ff2c80ced00 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 >> v1:172.21.15.3:6789/0 conn(0x7ff2c80ced00 legacy=0x7ff2c80cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2b40039c0 0x7ff2b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2b4004810 0x7ff2b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40051d0 0x7ff2b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b400c0f0 con 0x7ff2b40039c0 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b400c2a0 con 0x7ff2b4004810 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b400c4a0 con 0x7ff2b40051d0 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cd75e640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2b40039c0 0x7ff2b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2ccf5d640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2b4004810 0x7ff2b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cd75e640 1 -- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40051d0 msgr2=0x7ff2b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cdf5f640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40051d0 0x7ff2b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cd75e640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40051d0 0x7ff2b400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cd75e640 1 -- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2b4004810 msgr2=0x7ff2b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cd75e640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2b4004810 0x7ff2b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cd75e640 1 -- 172.21.15.3:0/1961936630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b4003530 con 0x7ff2b40039c0 2024-09-12T12:20:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.579+0000 7ff2cdf5f640 1 -- 172.21.15.3:0/1961936630 reap_dead start 2024-09-12T12:20:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cd75e640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2b40039c0 0x7ff2b4003de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4001e70 tx=0x7ff2c401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179ea298010f21d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4001a90 con 0x7ff2b40039c0 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c4003b80 con 0x7ff2b40039c0 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2bffff640 1 -- 172.21.15.3:0/1961936630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4005080 con 0x7ff2b40039c0 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1961936630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2b40039c0 msgr2=0x7ff2b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2b40039c0 0x7ff2b4003de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4001e70 tx=0x7ff2c401c730 comp rx=0 tx=0).stop 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1961936630 shutdown_connections 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1961936630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2b40039c0 0x7ff2b4003de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1961936630 >> 172.21.15.3:0/1961936630 conn(0x7ff2c808c000 msgr2=0x7ff2c80d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1961936630 shutdown_connections 2024-09-12T12:20:19.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1961936630 wait complete. 2024-09-12T12:20:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 Processor -- start 2024-09-12T12:20:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.583+0000 7ff2cf9c2640 1 -- start start 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40039c0 0x7ff2c8154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c80ba020 0x7ff2c80b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c80c9a50 0x7ff2c80b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c80b6930 con 0x7ff2c80ba020 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c80b67b0 con 0x7ff2c80c9a50 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c815e8b0 con 0x7ff2b40039c0 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c80ba020 0x7ff2c80b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c80ba020 0x7ff2c80b9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49520/0 (socket says 172.21.15.3:49520) 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 -- 172.21.15.3:0/1988950080 learned_addr learned my addr 172.21.15.3:0/1988950080 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cdf5f640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c80c9a50 0x7ff2c80b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cd75e640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40039c0 0x7ff2c8154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 -- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40039c0 msgr2=0x7ff2c8154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40039c0 0x7ff2c8154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 -- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c80c9a50 msgr2=0x7ff2c80b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c80c9a50 0x7ff2c80b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 -- 172.21.15.3:0/1988950080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c8155130 con 0x7ff2c80ba020 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2ccf5d640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c80ba020 0x7ff2c80b9010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800f590 tx=0x7ff2b800fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2bdffb640 1 -- 172.21.15.3:0/1988950080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8019030 con 0x7ff2c80ba020 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2bdffb640 1 -- 172.21.15.3:0/1988950080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b8004020 con 0x7ff2c80ba020 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c80c8c90 con 0x7ff2c80ba020 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2bdffb640 1 -- 172.21.15.3:0/1988950080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80043f0 con 0x7ff2c80ba020 2024-09-12T12:20:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c80c9250 con 0x7ff2c80ba020 2024-09-12T12:20:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2bdffb640 1 -- 172.21.15.3:0/1988950080 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2b801c070 con 0x7ff2c80ba020 2024-09-12T12:20:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8162920 con 0x7ff2c80ba020 2024-09-12T12:20:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2bdffb640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2ac0678f0 0x7ff2ac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2bdffb640 1 -- 172.21.15.3:0/1988950080 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2b8085fc0 con 0x7ff2c80ba020 2024-09-12T12:20:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.587+0000 7ff2cd75e640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2ac0678f0 0x7ff2ac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.591+0000 7ff2cd75e640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2ac0678f0 0x7ff2ac069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4005000 tx=0x7ff2c4001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.591+0000 7ff2bdffb640 1 -- 172.21.15.3:0/1988950080 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff2b80569d0 con 0x7ff2c80ba020 2024-09-12T12:20:19.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.763+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2c80d42b0 con 0x7ff2c80ba020 2024-09-12T12:20:19.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.763+0000 7ff2bdffb640 1 -- 172.21.15.3:0/1988950080 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7ff2b80562d0 con 0x7ff2c80ba020 2024-09-12T12:20:19.767 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:19.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2ac0678f0 msgr2=0x7ff2ac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2ac0678f0 0x7ff2ac069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4005000 tx=0x7ff2c4001520 comp rx=0 tx=0).stop 2024-09-12T12:20:19.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c80ba020 msgr2=0x7ff2c80b9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c80ba020 0x7ff2c80b9010 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800f590 tx=0x7ff2b800fa50 comp rx=0 tx=0).stop 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 shutdown_connections 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2ac0678f0 0x7ff2ac069db0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2c80c9a50 0x7ff2c80b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c80ba020 0x7ff2c80b9010 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 --2- 172.21.15.3:0/1988950080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2b40039c0 0x7ff2c8154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 >> 172.21.15.3:0/1988950080 conn(0x7ff2c808c000 msgr2=0x7ff2c80cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 shutdown_connections 2024-09-12T12:20:19.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.767+0000 7ff2cf9c2640 1 -- 172.21.15.3:0/1988950080 wait complete. 2024-09-12T12:20:19.785 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:19.785 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:19.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.939+0000 7f67f1279640 1 Processor -- start 2024-09-12T12:20:19.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67f1279640 1 -- start start 2024-09-12T12:20:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67f1279640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67ec15a9d0 0x7f67ec0ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67f1279640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67ec0b4190 0x7f67ec072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67f1279640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67ec0b4980 con 0x7f67ec15a5f0 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67f1279640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec0b46a0 con 0x7f67ec15a9d0 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67f1279640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec06de50 con 0x7f67ec0b4190 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67ea7fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67ec15a9d0 0x7f67ec0ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67eaffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f67ec15a5f0 0x7f67ec071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34690/0 (socket says 172.21.15.3:34690) 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67eaffd640 1 -- 172.21.15.3:0/3483620717 learned_addr learned my addr 172.21.15.3:0/3483620717 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67ea7fc640 1 --2- 172.21.15.3:0/3483620717 >> v2:172.21.15.3:3300/0 conn(0x7f67ec15a9d0 0x7f67ec0ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49534/0 (socket says 172.21.15.3:49534) 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67eb7fe640 1 --2- 172.21.15.3:0/3483620717 >> v2:172.21.15.142:3300/0 conn(0x7f67ec0b4190 0x7f67ec072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67ea7fc640 1 -- 172.21.15.3:0/3483620717 >> v2:172.21.15.142:3300/0 conn(0x7f67ec0b4190 msgr2=0x7f67ec072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67ea7fc640 1 --2- 172.21.15.3:0/3483620717 >> v2:172.21.15.142:3300/0 conn(0x7f67ec0b4190 0x7f67ec072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67ea7fc640 1 -- 172.21.15.3:0/3483620717 >> v1:172.21.15.3:6789/0 conn(0x7f67ec15a5f0 legacy=0x7f67ec071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67ea7fc640 1 -- 172.21.15.3:0/3483620717 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec06e130 con 0x7f67ec15a9d0 2024-09-12T12:20:19.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67e9ffb640 1 -- 172.21.15.3:0/3483620717 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389067205 0 0) 0x7f67ec0b4980 con 0x7f67ec15a5f0 2024-09-12T12:20:19.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67eb7fe640 1 --2- 172.21.15.3:0/3483620717 >> v2:172.21.15.142:3300/0 conn(0x7f67ec0b4190 0x7f67ec072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:19.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67ea7fc640 1 --2- 172.21.15.3:0/3483620717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a9d0 0x7f67ec0ad460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67ec073c30 tx=0x7f67d401e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b12c6d90f91af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67e9ffb640 1 -- 172.21.15.3:0/3483620717 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40a86e0 con 0x7f67ec15a9d0 2024-09-12T12:20:19.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67e9ffb640 1 -- 172.21.15.3:0/3483620717 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d40a8880 con 0x7f67ec15a9d0 2024-09-12T12:20:19.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.943+0000 7f67e9ffb640 1 -- 172.21.15.3:0/3483620717 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40a8c50 con 0x7f67ec15a9d0 2024-09-12T12:20:19.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- 172.21.15.3:0/3483620717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a9d0 msgr2=0x7f67ec0ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 --2- 172.21.15.3:0/3483620717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a9d0 0x7f67ec0ad460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f67ec073c30 tx=0x7f67d401e880 comp rx=0 tx=0).stop 2024-09-12T12:20:19.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- 172.21.15.3:0/3483620717 shutdown_connections 2024-09-12T12:20:19.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 --2- 172.21.15.3:0/3483620717 >> v2:172.21.15.142:3300/0 conn(0x7f67ec0b4190 0x7f67ec072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 --2- 172.21.15.3:0/3483620717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a9d0 0x7f67ec0ad460 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- 172.21.15.3:0/3483620717 >> 172.21.15.3:0/3483620717 conn(0x7f67ec06be90 msgr2=0x7f67ec06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:19.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- 172.21.15.3:0/3483620717 shutdown_connections 2024-09-12T12:20:19.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- 172.21.15.3:0/3483620717 wait complete. 2024-09-12T12:20:19.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 Processor -- start 2024-09-12T12:20:19.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- start start 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0b4190 0x7f67ec153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a5f0 0x7f67ec1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec15a9d0 0x7f67ec1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec06ecc0 con 0x7f67ec15a5f0 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec06eb40 con 0x7f67ec15a9d0 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec06ee40 con 0x7f67ec0b4190 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a5f0 0x7f67ec1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a5f0 0x7f67ec1590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49548/0 (socket says 172.21.15.3:49548) 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 -- 172.21.15.3:0/4166113634 learned_addr learned my addr 172.21.15.3:0/4166113634 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67eaffd640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0b4190 0x7f67ec153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 -- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0b4190 msgr2=0x7f67ec153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0b4190 0x7f67ec153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 -- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec15a9d0 msgr2=0x7f67ec1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec15a9d0 0x7f67ec1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 -- 172.21.15.3:0/4166113634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec14c990 con 0x7f67ec15a5f0 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67ea7fc640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a5f0 0x7f67ec1590a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67d401c4b0 tx=0x7f67d401b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67e37fe640 1 -- 172.21.15.3:0/4166113634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40b2840 con 0x7f67ec15a5f0 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67e37fe640 1 -- 172.21.15.3:0/4166113634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67d40b29e0 con 0x7f67ec15a5f0 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec14cc70 con 0x7f67ec15a5f0 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67e37fe640 1 -- 172.21.15.3:0/4166113634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40b2bd0 con 0x7f67ec15a5f0 2024-09-12T12:20:19.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.947+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec09bc60 con 0x7f67ec15a5f0 2024-09-12T12:20:19.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.951+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ec072720 con 0x7f67ec15a5f0 2024-09-12T12:20:19.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.951+0000 7f67e37fe640 1 -- 172.21.15.3:0/4166113634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67d40c2020 con 0x7f67ec15a5f0 2024-09-12T12:20:19.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.951+0000 7f67e37fe640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67cc0678c0 0x7f67cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.951+0000 7f67e37fe640 1 -- 172.21.15.3:0/4166113634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67d4122300 con 0x7f67ec15a5f0 2024-09-12T12:20:19.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.951+0000 7f67eaffd640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67cc0678c0 0x7f67cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.951+0000 7f67eaffd640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67cc0678c0 0x7f67cc069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000f70 tx=0x7f67dc007b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:19.955+0000 7f67e37fe640 1 -- 172.21.15.3:0/4166113634 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f67d40c9020 con 0x7f67ec15a5f0 2024-09-12T12:20:20.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f67ec14d0c0 con 0x7f67ec15a5f0 2024-09-12T12:20:20.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67e37fe640 1 -- 172.21.15.3:0/4166113634 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f67d4018500 con 0x7f67ec15a5f0 2024-09-12T12:20:20.134 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:20.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67cc0678c0 msgr2=0x7f67cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67cc0678c0 0x7f67cc069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000f70 tx=0x7f67dc007b80 comp rx=0 tx=0).stop 2024-09-12T12:20:20.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a5f0 msgr2=0x7f67ec1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a5f0 0x7f67ec1590a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f67d401c4b0 tx=0x7f67d401b620 comp rx=0 tx=0).stop 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 shutdown_connections 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67cc0678c0 0x7f67cc069d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ec15a9d0 0x7f67ec1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67ec15a5f0 0x7f67ec1590a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 --2- 172.21.15.3:0/4166113634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ec0b4190 0x7f67ec153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 >> 172.21.15.3:0/4166113634 conn(0x7f67ec06be90 msgr2=0x7f67ec14a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 shutdown_connections 2024-09-12T12:20:20.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.131+0000 7f67f1279640 1 -- 172.21.15.3:0/4166113634 wait complete. 2024-09-12T12:20:20.153 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:20:20.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:20.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:20.175 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:20.175 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:20:20.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:20.175 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:20:20.227 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:20.227 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:20.234 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:20.234 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:20.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:20.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f32a1e0c640 1 Processor -- start 2024-09-12T12:20:20.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f32a1e0c640 1 -- start start 2024-09-12T12:20:20.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f32a1e0c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 0x7f329c0ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f32a1e0c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f329c0b4980 con 0x7f329c15a5f0 2024-09-12T12:20:20.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f32a1e0c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f329c0b46a0 con 0x7f329c0b4190 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f32a1e0c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f329c06de50 con 0x7f329c15a9d0 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f329c15a5f0 0x7f329c071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34704/0 (socket says 172.21.15.3:34704) 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 -- 172.21.15.3:0/3403041271 learned_addr learned my addr 172.21.15.3:0/3403041271 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329affd640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 0x7f329c0ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859047550 0 0) 0x7f329c0b4980 con 0x7f329c15a5f0 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3288003530 con 0x7f329c15a5f0 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3924283762 0 0) 0x7f3288003530 con 0x7f329c15a5f0 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 msgr2=0x7f329c0ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 0x7f329c0ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 >> v1:172.21.15.142:6789/0 conn(0x7f329c0b4190 legacy=0x7f329c072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c06e130 con 0x7f329c15a5f0 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f328c001680 con 0x7f329c15a5f0 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 >> v1:172.21.15.3:6789/0 conn(0x7f329c15a5f0 legacy=0x7f329c071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32880039f0 0x7f3288003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3288004810 0x7f3288004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 0x7f32880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f329c06de50 con 0x7f32880039f0 2024-09-12T12:20:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f328800ba00 con 0x7f3288004810 2024-09-12T12:20:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f328800bc00 con 0x7f329c15a9d0 2024-09-12T12:20:20.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32880039f0 0x7f3288003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329affd640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3288004810 0x7f3288004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329bfff640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 0x7f32880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 -- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 msgr2=0x7f32880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 0x7f32880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 -- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3288004810 msgr2=0x7f3288004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3288004810 0x7f3288004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329b7fe640 1 -- 172.21.15.3:0/3403041271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3288003530 con 0x7f32880039f0 2024-09-12T12:20:20.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.407+0000 7f329bfff640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c15a9d0 0x7f32880051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f329b7fe640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32880039f0 0x7f3288003e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f328c00c210 tx=0x7f328c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6fd26150fc09b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c007d20 con 0x7f32880039f0 2024-09-12T12:20:20.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328c0a9070 con 0x7f32880039f0 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f329a7fc640 1 -- 172.21.15.3:0/3403041271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c0a5d50 con 0x7f32880039f0 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 -- 172.21.15.3:0/3403041271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32880039f0 msgr2=0x7f3288003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 --2- 172.21.15.3:0/3403041271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32880039f0 0x7f3288003e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f328c00c210 tx=0x7f328c01cc10 comp rx=0 tx=0).stop 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f329bfff640 1 -- 172.21.15.3:0/3403041271 reap_dead start 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 -- 172.21.15.3:0/3403041271 shutdown_connections 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 -- 172.21.15.3:0/3403041271 >> 172.21.15.3:0/3403041271 conn(0x7f329c06be90 msgr2=0x7f329c06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 -- 172.21.15.3:0/3403041271 shutdown_connections 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 -- 172.21.15.3:0/3403041271 wait complete. 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 Processor -- start 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.411+0000 7f32a1e0c640 1 -- start start 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32880039f0 0x7f329c09be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c0b2760 0x7f329c09c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c0b1150 0x7f329c149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f329c06eab0 con 0x7f329c0b1150 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f329c06e930 con 0x7f32880039f0 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f329c06ec30 con 0x7f329c0b2760 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c0b1150 0x7f329c149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c0b1150 0x7f329c149830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49566/0 (socket says 172.21.15.3:49566) 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 -- 172.21.15.3:0/4186919928 learned_addr learned my addr 172.21.15.3:0/4186919928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329b7fe640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32880039f0 0x7f329c09be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329affd640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c0b2760 0x7f329c09c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 -- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c0b2760 msgr2=0x7f329c09c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c0b2760 0x7f329c09c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 -- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32880039f0 msgr2=0x7f329c09be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32880039f0 0x7f329c09be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 -- 172.21.15.3:0/4186919928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c14a650 con 0x7f329c0b1150 2024-09-12T12:20:20.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329b7fe640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32880039f0 0x7f329c09be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329bfff640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c0b1150 0x7f329c149830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f329c0b3330 tx=0x7f329400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f3293fff640 1 -- 172.21.15.3:0/4186919928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294006d60 con 0x7f329c0b1150 2024-09-12T12:20:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f3293fff640 1 -- 172.21.15.3:0/4186919928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3294003980 con 0x7f329c0b1150 2024-09-12T12:20:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f3293fff640 1 -- 172.21.15.3:0/4186919928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294003d70 con 0x7f329c0b1150 2024-09-12T12:20:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c0b16c0 con 0x7f329c0b1150 2024-09-12T12:20:20.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c0b1c80 con 0x7f329c0b1150 2024-09-12T12:20:20.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c072720 con 0x7f329c0b1150 2024-09-12T12:20:20.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f3293fff640 1 -- 172.21.15.3:0/4186919928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3294005070 con 0x7f329c0b1150 2024-09-12T12:20:20.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f3293fff640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f327c0678f0 0x7f327c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f3293fff640 1 -- 172.21.15.3:0/4186919928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3294085a90 con 0x7f329c0b1150 2024-09-12T12:20:20.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.415+0000 7f329b7fe640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f327c0678f0 0x7f327c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.419+0000 7f329b7fe640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f327c0678f0 0x7f327c069db0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f328c0ab000 tx=0x7f328c0b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:20.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.419+0000 7f3293fff640 1 -- 172.21.15.3:0/4186919928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f32940564b0 con 0x7f329c0b1150 2024-09-12T12:20:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.595+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f329c0b3360 con 0x7f329c0b1150 2024-09-12T12:20:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.595+0000 7f3293fff640 1 -- 172.21.15.3:0/4186919928 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f3294055db0 con 0x7f329c0b1150 2024-09-12T12:20:20.599 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f327c0678f0 msgr2=0x7f327c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f327c0678f0 0x7f327c069db0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f328c0ab000 tx=0x7f328c0b1040 comp rx=0 tx=0).stop 2024-09-12T12:20:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c0b1150 msgr2=0x7f329c149830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c0b1150 0x7f329c149830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f329c0b3330 tx=0x7f329400dc00 comp rx=0 tx=0).stop 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 shutdown_connections 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f327c0678f0 0x7f327c069db0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f329c0b1150 0x7f329c149830 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f329c0b2760 0x7f329c09c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 --2- 172.21.15.3:0/4186919928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32880039f0 0x7f329c09be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 >> 172.21.15.3:0/4186919928 conn(0x7f329c06be90 msgr2=0x7f329c0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 shutdown_connections 2024-09-12T12:20:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.599+0000 7f32a1e0c640 1 -- 172.21.15.3:0/4186919928 wait complete. 2024-09-12T12:20:20.619 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:20.619 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:20.625 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b291f640 1 Processor -- start 2024-09-12T12:20:20.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b291f640 1 -- start start 2024-09-12T12:20:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b291f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8ac0af550 0x7ff8ac0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b291f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0734f0 0x7ff8ac15b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b291f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8ac06f010 con 0x7ff8ac0d08c0 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b291f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff8ac0d0d10 con 0x7ff8ac0af550 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b291f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ac073ad0 con 0x7ff8ac0734f0 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8abfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8ac0af550 0x7ff8ac0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8abfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8ac0af550 0x7ff8ac0d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49578/0 (socket says 172.21.15.3:49578) 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8abfff640 1 -- 172.21.15.3:0/1288580846 learned_addr learned my addr 172.21.15.3:0/1288580846 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8abfff640 1 -- 172.21.15.3:0/1288580846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0734f0 msgr2=0x7ff8ac15b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b0ebc640 1 --2- 172.21.15.3:0/1288580846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0734f0 0x7ff8ac15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8abfff640 1 --2- 172.21.15.3:0/1288580846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0734f0 0x7ff8ac15b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8abfff640 1 -- 172.21.15.3:0/1288580846 >> v1:172.21.15.142:6789/0 conn(0x7ff8ac0d08c0 legacy=0x7ff8ac072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8abfff640 1 -- 172.21.15.3:0/1288580846 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac06f2f0 con 0x7ff8ac0af550 2024-09-12T12:20:20.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.739+0000 7ff8b0ebc640 1 --2- 172.21.15.3:0/1288580846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0734f0 0x7ff8ac15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8abfff640 1 --2- 172.21.15.3:0/1288580846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac0d0380 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009e80 tx=0x7ff89c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c6c200e8cf982 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8aaffd640 1 -- 172.21.15.3:0/1288580846 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0a56d0 con 0x7ff8ac0af550 2024-09-12T12:20:20.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8aaffd640 1 -- 172.21.15.3:0/1288580846 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89c0a5870 con 0x7ff8ac0af550 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8aaffd640 1 -- 172.21.15.3:0/1288580846 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0a5c40 con 0x7ff8ac0af550 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 -- 172.21.15.3:0/1288580846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 msgr2=0x7ff8ac0d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 --2- 172.21.15.3:0/1288580846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac0d0380 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009e80 tx=0x7ff89c01c940 comp rx=0 tx=0).stop 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 -- 172.21.15.3:0/1288580846 shutdown_connections 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 --2- 172.21.15.3:0/1288580846 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0734f0 0x7ff8ac15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 --2- 172.21.15.3:0/1288580846 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac0d0380 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 -- 172.21.15.3:0/1288580846 >> 172.21.15.3:0/1288580846 conn(0x7ff8ac06be90 msgr2=0x7ff8ac06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:20.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 -- 172.21.15.3:0/1288580846 shutdown_connections 2024-09-12T12:20:20.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 -- 172.21.15.3:0/1288580846 wait complete. 2024-09-12T12:20:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 Processor -- start 2024-09-12T12:20:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.743+0000 7ff8b291f640 1 -- start start 2024-09-12T12:20:20.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac0734f0 0x7ff8ac0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0d08c0 0x7ff8ac156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac15eda0 con 0x7ff8ac0af550 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac15ec20 con 0x7ff8ac0734f0 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ac167d70 con 0x7ff8ac0d08c0 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49584/0 (socket says 172.21.15.3:49584) 2024-09-12T12:20:20.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 -- 172.21.15.3:0/3731690892 learned_addr learned my addr 172.21.15.3:0/3731690892 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 -- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0d08c0 msgr2=0x7ff8ac156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b0ebc640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0d08c0 0x7ff8ac156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8abfff640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac0734f0 0x7ff8ac0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0d08c0 0x7ff8ac156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 -- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac0734f0 msgr2=0x7ff8ac0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac0734f0 0x7ff8ac0b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 -- 172.21.15.3:0/3731690892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac06e530 con 0x7ff8ac0af550 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8ab7fe640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac150570 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000b410 tx=0x7ff8a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:20.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8a8ff9640 1 -- 172.21.15.3:0/3731690892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0015030 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8a8ff9640 1 -- 172.21.15.3:0/3731690892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a0004020 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8a8ff9640 1 -- 172.21.15.3:0/3731690892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0004450 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac0b1b30 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac14fc20 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac0afd00 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8a8ff9640 1 -- 172.21.15.3:0/3731690892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff8a0018070 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8a8ff9640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8980678c0 0x7ff898069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8a8ff9640 1 -- 172.21.15.3:0/3731690892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff8a00857b0 con 0x7ff8ac0af550 2024-09-12T12:20:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.747+0000 7ff8abfff640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8980678c0 0x7ff898069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.751+0000 7ff8abfff640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8980678c0 0x7ff898069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0ae000 tx=0x7ff89c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:20.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.751+0000 7ff8a8ff9640 1 -- 172.21.15.3:0/3731690892 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff8ac0afd00 con 0x7ff8ac0af550 2024-09-12T12:20:20.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.923+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff8ac15b630 con 0x7ff8ac0af550 2024-09-12T12:20:20.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.923+0000 7ff8a8ff9640 1 -- 172.21.15.3:0/3731690892 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7ff8ac0b1b30 con 0x7ff8ac0af550 2024-09-12T12:20:20.929 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8980678c0 msgr2=0x7ff898069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8980678c0 0x7ff898069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0ae000 tx=0x7ff89c0a7070 comp rx=0 tx=0).stop 2024-09-12T12:20:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 msgr2=0x7ff8ac150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac150570 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000b410 tx=0x7ff8a000b8d0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 shutdown_connections 2024-09-12T12:20:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8980678c0 0x7ff898069d80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff8ac0d08c0 0x7ff8ac156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ac0af550 0x7ff8ac150570 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 --2- 172.21.15.3:0/3731690892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ac0734f0 0x7ff8ac0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 >> 172.21.15.3:0/3731690892 conn(0x7ff8ac06be90 msgr2=0x7ff8ac06de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 shutdown_connections 2024-09-12T12:20:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:20.927+0000 7ff8b291f640 1 -- 172.21.15.3:0/3731690892 wait complete. 2024-09-12T12:20:20.947 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:20:20.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:20.947 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:20.970 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:20.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:20:20.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:20.971 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:20:21.022 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:21.023 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:21.029 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:21.029 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:21.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:21.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 Processor -- start 2024-09-12T12:20:21.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 -- start start 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f824c15e5d0 0x7f824c073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f824c06fdd0 0x7f824c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06d210 0x7f824c06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f824c0aeb20 con 0x7f824c15e5d0 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f824c0d0680 con 0x7f824c06fdd0 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f825224a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0ae7f0 con 0x7f824c06d210 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f824c15e5d0 0x7f824c073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f824c15e5d0 0x7f824c073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49594/0 (socket says 172.21.15.3:49594) 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 -- 172.21.15.3:0/2985493164 learned_addr learned my addr 172.21.15.3:0/2985493164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:21.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824bfff640 1 --2- 172.21.15.3:0/2985493164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06d210 0x7f824c06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824affd640 1 --2- 172.21.15.3:0/2985493164 >> v2:172.21.15.142:3300/0 conn(0x7f824c06fdd0 0x7f824c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 -- 172.21.15.3:0/2985493164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06d210 msgr2=0x7f824c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 --2- 172.21.15.3:0/2985493164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06d210 0x7f824c06e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 -- 172.21.15.3:0/2985493164 >> v2:172.21.15.142:3300/0 conn(0x7f824c06fdd0 msgr2=0x7f824c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 --2- 172.21.15.3:0/2985493164 >> v2:172.21.15.142:3300/0 conn(0x7f824c06fdd0 0x7f824c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824b7fe640 1 -- 172.21.15.3:0/2985493164 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c0d00c0 con 0x7f824c15e5d0 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824affd640 1 --2- 172.21.15.3:0/2985493164 >> v2:172.21.15.142:3300/0 conn(0x7f824c06fdd0 0x7f824c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:21.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.247+0000 7f824bfff640 1 --2- 172.21.15.3:0/2985493164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06d210 0x7f824c06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:21.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f824b7fe640 1 --2- 172.21.15.3:0/2985493164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c073260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f824c0adfa0 tx=0x7f823c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535fc4b59b591915 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f824a7fc640 1 -- 172.21.15.3:0/2985493164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0b0070 con 0x7f824c15e5d0 2024-09-12T12:20:21.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f824a7fc640 1 -- 172.21.15.3:0/2985493164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823c0ab600 con 0x7f824c15e5d0 2024-09-12T12:20:21.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f824a7fc640 1 -- 172.21.15.3:0/2985493164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0ab9d0 con 0x7f824c15e5d0 2024-09-12T12:20:21.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 -- 172.21.15.3:0/2985493164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 msgr2=0x7f824c073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 --2- 172.21.15.3:0/2985493164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c073260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f824c0adfa0 tx=0x7f823c01c260 comp rx=0 tx=0).stop 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 -- 172.21.15.3:0/2985493164 shutdown_connections 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 --2- 172.21.15.3:0/2985493164 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06d210 0x7f824c06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 --2- 172.21.15.3:0/2985493164 >> v2:172.21.15.142:3300/0 conn(0x7f824c06fdd0 0x7f824c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 --2- 172.21.15.3:0/2985493164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c073260 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 -- 172.21.15.3:0/2985493164 >> 172.21.15.3:0/2985493164 conn(0x7f824c072740 msgr2=0x7f824c072b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 -- 172.21.15.3:0/2985493164 shutdown_connections 2024-09-12T12:20:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 -- 172.21.15.3:0/2985493164 wait complete. 2024-09-12T12:20:21.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 Processor -- start 2024-09-12T12:20:21.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.251+0000 7f825224a640 1 -- start start 2024-09-12T12:20:21.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824c06d210 0x7f824c0b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06fdd0 0x7f824c1500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0af380 con 0x7f824c15e5d0 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0af200 con 0x7f824c06d210 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0af500 con 0x7f824c06fdd0 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c0b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49600/0 (socket says 172.21.15.3:49600) 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 -- 172.21.15.3:0/3305484052 learned_addr learned my addr 172.21.15.3:0/3305484052 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824b7fe640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824c06d210 0x7f824c0b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824affd640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06fdd0 0x7f824c1500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 -- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06fdd0 msgr2=0x7f824c1500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06fdd0 0x7f824c1500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 -- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824c06d210 msgr2=0x7f824c0b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824c06d210 0x7f824c0b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 -- 172.21.15.3:0/3305484052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c0b3ec0 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824bfff640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c0b4f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f824400cef0 tx=0x7f824400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f8233fff640 1 -- 172.21.15.3:0/3305484052 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824401a070 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f8233fff640 1 -- 172.21.15.3:0/3305484052 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824400d860 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c0b5a50 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f8233fff640 1 -- 172.21.15.3:0/3305484052 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400dc50 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c0b4440 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824c06ccd0 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f8233fff640 1 -- 172.21.15.3:0/3305484052 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f824401d070 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f8233fff640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82380678f0 0x7f8238069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f8233fff640 1 -- 172.21.15.3:0/3305484052 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8244086440 con 0x7f824c15e5d0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.255+0000 7f824b7fe640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82380678f0 0x7f8238069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.259+0000 7f824b7fe640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82380678f0 0x7f8238069db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ad000 tx=0x7f823c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.259+0000 7f8233fff640 1 -- 172.21.15.3:0/3305484052 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8244057010 con 0x7f824c15e5d0 2024-09-12T12:20:21.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.435+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f824c0afab0 con 0x7f824c15e5d0 2024-09-12T12:20:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f8233fff640 1 -- 172.21.15.3:0/3305484052 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f8244056910 con 0x7f824c15e5d0 2024-09-12T12:20:21.441 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82380678f0 msgr2=0x7f8238069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82380678f0 0x7f8238069db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ad000 tx=0x7f823c008040 comp rx=0 tx=0).stop 2024-09-12T12:20:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 msgr2=0x7f824c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c0b4f00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f824400cef0 tx=0x7f824400a710 comp rx=0 tx=0).stop 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 shutdown_connections 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82380678f0 0x7f8238069db0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f824c15e5d0 0x7f824c0b4f00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f824c06fdd0 0x7f824c1500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 --2- 172.21.15.3:0/3305484052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f824c06d210 0x7f824c0b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 >> 172.21.15.3:0/3305484052 conn(0x7f824c072740 msgr2=0x7f824c155470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 shutdown_connections 2024-09-12T12:20:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.439+0000 7f825224a640 1 -- 172.21.15.3:0/3305484052 wait complete. 2024-09-12T12:20:21.460 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:21.460 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:21.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:21.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f669a378640 1 Processor -- start 2024-09-12T12:20:21.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f669a378640 1 -- start start 2024-09-12T12:20:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f669a378640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f66941568f0 0x7f66940aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f669a378640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66940d0380 0x7f66940d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f669a378640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66940afb00 con 0x7f66940b47a0 2024-09-12T12:20:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f669a378640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f66940d0cf0 con 0x7f66940d0380 2024-09-12T12:20:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f669a378640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f66940b31b0 con 0x7f66941568f0 2024-09-12T12:20:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f6698915640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66940d0380 0x7f66940d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f6698915640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f66940d0380 0x7f66940d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49606/0 (socket says 172.21.15.3:49606) 2024-09-12T12:20:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.631+0000 7f6698915640 1 -- 172.21.15.3:0/1269570848 learned_addr learned my addr 172.21.15.3:0/1269570848 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6693fff640 1 --2- 172.21.15.3:0/1269570848 >> v2:172.21.15.142:3300/0 conn(0x7f66941568f0 0x7f66940aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6698915640 1 -- 172.21.15.3:0/1269570848 >> v2:172.21.15.142:3300/0 conn(0x7f66941568f0 msgr2=0x7f66940aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6698915640 1 --2- 172.21.15.3:0/1269570848 >> v2:172.21.15.142:3300/0 conn(0x7f66941568f0 0x7f66940aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6698915640 1 -- 172.21.15.3:0/1269570848 >> v1:172.21.15.142:6789/0 conn(0x7f66940b47a0 legacy=0x7f66940b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6698915640 1 -- 172.21.15.3:0/1269570848 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66940af540 con 0x7f66940d0380 2024-09-12T12:20:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6698915640 1 --2- 172.21.15.3:0/1269570848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f66940d07b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f669406cc60 tx=0x7f668c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316613a3838f0dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6692ffd640 1 -- 172.21.15.3:0/1269570848 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c0a55e0 con 0x7f66940d0380 2024-09-12T12:20:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6692ffd640 1 -- 172.21.15.3:0/1269570848 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668c0a5780 con 0x7f66940d0380 2024-09-12T12:20:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f6692ffd640 1 -- 172.21.15.3:0/1269570848 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c0a5b50 con 0x7f66940d0380 2024-09-12T12:20:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 -- 172.21.15.3:0/1269570848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 msgr2=0x7f66940d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 --2- 172.21.15.3:0/1269570848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f66940d07b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f669406cc60 tx=0x7f668c01c920 comp rx=0 tx=0).stop 2024-09-12T12:20:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 -- 172.21.15.3:0/1269570848 shutdown_connections 2024-09-12T12:20:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 --2- 172.21.15.3:0/1269570848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f66940d07b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 --2- 172.21.15.3:0/1269570848 >> v2:172.21.15.142:3300/0 conn(0x7f66941568f0 0x7f66940aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 -- 172.21.15.3:0/1269570848 >> 172.21.15.3:0/1269570848 conn(0x7f669406be90 msgr2=0x7f669406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 -- 172.21.15.3:0/1269570848 shutdown_connections 2024-09-12T12:20:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.635+0000 7f669a378640 1 -- 172.21.15.3:0/1269570848 wait complete. 2024-09-12T12:20:21.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 Processor -- start 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 -- start start 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b47a0 0x7f669414e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f669409c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66941568f0 0x7f669414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f669415d220 con 0x7f66940d0380 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f669415d0a0 con 0x7f66941568f0 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f669415d3a0 con 0x7f66940b47a0 2024-09-12T12:20:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f669409c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f669409c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49608/0 (socket says 172.21.15.3:49608) 2024-09-12T12:20:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 -- 172.21.15.3:0/2402668329 learned_addr learned my addr 172.21.15.3:0/2402668329 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6693fff640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b47a0 0x7f669414e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6698915640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66941568f0 0x7f669414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 -- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b47a0 msgr2=0x7f669414e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b47a0 0x7f669414e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 -- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66941568f0 msgr2=0x7f669414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66941568f0 0x7f669414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 -- 172.21.15.3:0/2402668329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941f5230 con 0x7f66940d0380 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6698915640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66941568f0 0x7f669414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f66937fe640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f669409c460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f668800b410 tx=0x7f668800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6690ff9640 1 -- 172.21.15.3:0/2402668329 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688015030 con 0x7f66940d0380 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6690ff9640 1 -- 172.21.15.3:0/2402668329 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6688004020 con 0x7f66940d0380 2024-09-12T12:20:21.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66941f5510 con 0x7f66940d0380 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6690ff9640 1 -- 172.21.15.3:0/2402668329 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688004450 con 0x7f66940d0380 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941f5db0 con 0x7f66940d0380 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66941f56e0 con 0x7f66940d0380 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6690ff9640 1 -- 172.21.15.3:0/2402668329 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6688018070 con 0x7f66940d0380 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.639+0000 7f6690ff9640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66800678c0 0x7f6680069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.643+0000 7f6690ff9640 1 -- 172.21.15.3:0/2402668329 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6688085830 con 0x7f66940d0380 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.643+0000 7f6693fff640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66800678c0 0x7f6680069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.643+0000 7f6693fff640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66800678c0 0x7f6680069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6684009790 tx=0x7f6684008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.647+0000 7f6690ff9640 1 -- 172.21.15.3:0/2402668329 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6688056370 con 0x7f66940d0380 2024-09-12T12:20:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66941f5fb0 con 0x7f66940d0380 2024-09-12T12:20:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f6690ff9640 1 -- 172.21.15.3:0/2402668329 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f66941f5fb0 con 0x7f66940d0380 2024-09-12T12:20:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66800678c0 msgr2=0x7f6680069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66800678c0 0x7f6680069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6684009790 tx=0x7f6684008040 comp rx=0 tx=0).stop 2024-09-12T12:20:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 msgr2=0x7f669409c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f669409c460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f668800b410 tx=0x7f668800b8d0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 shutdown_connections 2024-09-12T12:20:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66800678c0 0x7f6680069d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66941568f0 0x7f669414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66940d0380 0x7f669409c460 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 --2- 172.21.15.3:0/2402668329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66940b47a0 0x7f669414e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 >> 172.21.15.3:0/2402668329 conn(0x7f669406be90 msgr2=0x7f669415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 shutdown_connections 2024-09-12T12:20:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:21.823+0000 7f669a378640 1 -- 172.21.15.3:0/2402668329 wait complete. 2024-09-12T12:20:21.847 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:20:21.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:21.848 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-12T12:20:21.860 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:21.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:20:21.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:21.861 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T12:20:21.910 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:21.910 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:21.918 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:21.918 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:21.967 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:22.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abf6b0640 1 Processor -- start 2024-09-12T12:20:22.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abf6b0640 1 -- start start 2024-09-12T12:20:22.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abf6b0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab80d0380 0x7f2ab80d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abf6b0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab806fc30 con 0x7f2ab80ad140 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abf6b0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ab80d0ce0 con 0x7f2ab815e250 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abf6b0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80b4df0 con 0x7f2ab80d0380 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abdc4d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2ab80ad140 0x7f2ab80ad5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34712/0 (socket says 172.21.15.3:34712) 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abdc4d640 1 -- 172.21.15.3:0/388866712 learned_addr learned my addr 172.21.15.3:0/388866712 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2abcc4b640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab80d0380 0x7f2ab80d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894307094 0 0) 0x7f2ab806fc30 con 0x7f2ab80ad140 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aa0003530 con 0x7f2ab80ad140 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643482297 0 0) 0x7f2ab80d0ce0 con 0x7f2ab815e250 2024-09-12T12:20:22.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.087+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab806fc30 con 0x7f2ab815e250 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1536401402 0 0) 0x7f2aa0003530 con 0x7f2ab80ad140 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab80d0380 msgr2=0x7f2ab80d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab80d0380 0x7f2ab80d07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 >> v1:172.21.15.142:6789/0 conn(0x7f2ab815e250 legacy=0x7f2ab806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab80b3420 con 0x7f2ab80ad140 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f2ab0001680 con 0x7f2ab80ad140 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 >> v1:172.21.15.3:6789/0 conn(0x7f2ab80ad140 legacy=0x7f2ab80ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aa0003b80 0x7f2aa0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa0004810 0x7f2aa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa00051d0 0x7f2aa000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80b4df0 con 0x7f2aa00051d0 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa000c040 con 0x7f2aa0004810 2024-09-12T12:20:22.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa000c240 con 0x7f2aa0003b80 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abcc4b640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab80d0380 0x7f2ab80d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa00051d0 0x7f2aa000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abd44c640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aa0003b80 0x7f2aa0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abcc4b640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa0004810 0x7f2aa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 -- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aa0003b80 msgr2=0x7f2aa0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2aa0003b80 0x7f2aa0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 -- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa0004810 msgr2=0x7f2aa0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2aa0004810 0x7f2aa0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 -- 172.21.15.3:0/388866712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa0003530 con 0x7f2aa00051d0 2024-09-12T12:20:22.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa00051d0 0x7f2aa000b9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000c1c0 tx=0x7f2ab001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335c47e891797ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0007cf0 con 0x7f2aa00051d0 2024-09-12T12:20:22.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab00a9020 con 0x7f2aa00051d0 2024-09-12T12:20:22.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2ab7fff640 1 -- 172.21.15.3:0/388866712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00a5b00 con 0x7f2aa00051d0 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abf6b0640 1 -- 172.21.15.3:0/388866712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa00051d0 msgr2=0x7f2aa000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abf6b0640 1 --2- 172.21.15.3:0/388866712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2aa00051d0 0x7f2aa000b9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000c1c0 tx=0x7f2ab001cc00 comp rx=0 tx=0).stop 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abdc4d640 1 -- 172.21.15.3:0/388866712 reap_dead start 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abf6b0640 1 -- 172.21.15.3:0/388866712 shutdown_connections 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abf6b0640 1 -- 172.21.15.3:0/388866712 >> 172.21.15.3:0/388866712 conn(0x7f2ab806be90 msgr2=0x7f2ab806c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abf6b0640 1 -- 172.21.15.3:0/388866712 shutdown_connections 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.091+0000 7f2abf6b0640 1 -- 172.21.15.3:0/388866712 wait complete. 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 Processor -- start 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 -- start start 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab81511a0 0x7f2ab815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8151580 0x7f2ab81582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8151a00 0x7f2ab8158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80b3ca0 con 0x7f2ab8151580 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab80b39c0 con 0x7f2ab8151a00 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab80b3f80 con 0x7f2ab81511a0 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8151580 0x7f2ab81582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8151580 0x7f2ab81582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49618/0 (socket says 172.21.15.3:49618) 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 -- 172.21.15.3:0/979419784 learned_addr learned my addr 172.21.15.3:0/979419784 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abdc4d640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8151a00 0x7f2ab8158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abd44c640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab81511a0 0x7f2ab815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 -- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab81511a0 msgr2=0x7f2ab815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab81511a0 0x7f2ab815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 -- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8151a00 msgr2=0x7f2ab8158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8151a00 0x7f2ab8158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 -- 172.21.15.3:0/979419784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab815fd50 con 0x7f2ab8151580 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abdc4d640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8151a00 0x7f2ab8158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:22.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abcc4b640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8151580 0x7f2ab81582f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8007f20 tx=0x7f2aa8011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/979419784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa801a020 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/979419784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa8016470 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab8160030 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/979419784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8016860 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab8155a70 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.095+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab815e630 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.099+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/979419784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2aa801d020 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.099+0000 7f2ab5ffb640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa40678f0 0x7f2aa4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.099+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/979419784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2aa8086bf0 con 0x7f2ab8151580 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.099+0000 7f2abd44c640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa40678f0 0x7f2aa4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.099+0000 7f2abd44c640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa40678f0 0x7f2aa4069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2ab81522b0 tx=0x7f2aac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.103+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/979419784 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2aa8024030 con 0x7f2ab8151580 2024-09-12T12:20:22.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ab8161c80 con 0x7f2ab8151580 2024-09-12T12:20:22.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2ab5ffb640 1 -- 172.21.15.3:0/979419784 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f2aa80577c0 con 0x7f2ab8151580 2024-09-12T12:20:22.275 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa40678f0 msgr2=0x7f2aa4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa40678f0 0x7f2aa4069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2ab81522b0 tx=0x7f2aac009040 comp rx=0 tx=0).stop 2024-09-12T12:20:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8151580 msgr2=0x7f2ab81582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8151580 0x7f2ab81582f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8007f20 tx=0x7f2aa8011660 comp rx=0 tx=0).stop 2024-09-12T12:20:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 shutdown_connections 2024-09-12T12:20:22.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa40678f0 0x7f2aa4069db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ab8151a00 0x7f2ab8158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ab8151580 0x7f2ab81582f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.271+0000 7f2abf6b0640 1 --2- 172.21.15.3:0/979419784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ab81511a0 0x7f2ab815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.275+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 >> 172.21.15.3:0/979419784 conn(0x7f2ab806be90 msgr2=0x7f2ab815c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.275+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 shutdown_connections 2024-09-12T12:20:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.275+0000 7f2abf6b0640 1 -- 172.21.15.3:0/979419784 wait complete. 2024-09-12T12:20:22.292 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:22.293 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:22.299 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.407+0000 7f34a964d640 1 Processor -- start 2024-09-12T12:20:22.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.407+0000 7f34a964d640 1 -- start start 2024-09-12T12:20:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a964d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a40ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a964d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f34a40b4190 0x7f34a4072720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a964d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a40b4980 con 0x7f34a415a5f0 2024-09-12T12:20:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a964d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f34a40b46a0 con 0x7f34a40b4190 2024-09-12T12:20:22.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a964d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34a406de50 con 0x7f34a415a9d0 2024-09-12T12:20:22.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a40ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a37fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f34a40b4190 0x7f34a4072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a2ffd640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f34a415a5f0 0x7f34a4071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:57662/0 (socket says 172.21.15.3:57662) 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a2ffd640 1 -- 172.21.15.3:0/2947750798 learned_addr learned my addr 172.21.15.3:0/2947750798 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a37fe640 1 -- 172.21.15.3:0/2947750798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 msgr2=0x7f34a40ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a37fe640 1 --2- 172.21.15.3:0/2947750798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a40ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a37fe640 1 -- 172.21.15.3:0/2947750798 >> v1:172.21.15.142:6789/0 conn(0x7f34a415a5f0 legacy=0x7f34a4071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a1ffb640 1 -- 172.21.15.3:0/2947750798 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409150900 0 0) 0x7f34a40b4980 con 0x7f34a415a5f0 2024-09-12T12:20:22.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.411+0000 7f34a37fe640 1 -- 172.21.15.3:0/2947750798 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a406e130 con 0x7f34a40b4190 2024-09-12T12:20:22.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a37fe640 1 --2- 172.21.15.3:0/2947750798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 0x7f34a4072720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f349c00aea0 tx=0x7f349c01da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d72a211ec2c34c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a1ffb640 1 -- 172.21.15.3:0/2947750798 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0a66d0 con 0x7f34a40b4190 2024-09-12T12:20:22.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a1ffb640 1 -- 172.21.15.3:0/2947750798 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c0a6870 con 0x7f34a40b4190 2024-09-12T12:20:22.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a1ffb640 1 -- 172.21.15.3:0/2947750798 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0a6c40 con 0x7f34a40b4190 2024-09-12T12:20:22.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- 172.21.15.3:0/2947750798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 msgr2=0x7f34a4072720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 --2- 172.21.15.3:0/2947750798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 0x7f34a4072720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f349c00aea0 tx=0x7f349c01da70 comp rx=0 tx=0).stop 2024-09-12T12:20:22.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- 172.21.15.3:0/2947750798 shutdown_connections 2024-09-12T12:20:22.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 --2- 172.21.15.3:0/2947750798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a40ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 --2- 172.21.15.3:0/2947750798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 0x7f34a4072720 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- 172.21.15.3:0/2947750798 >> 172.21.15.3:0/2947750798 conn(0x7f34a406be90 msgr2=0x7f34a406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:22.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- 172.21.15.3:0/2947750798 shutdown_connections 2024-09-12T12:20:22.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- 172.21.15.3:0/2947750798 wait complete. 2024-09-12T12:20:22.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 Processor -- start 2024-09-12T12:20:22.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- start start 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 0x7f34a4147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a415a5f0 0x7f34a414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a409ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34a406ecc0 con 0x7f34a415a5f0 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34a406eb40 con 0x7f34a40b4190 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a964d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34a406ee40 con 0x7f34a415a9d0 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a415a5f0 0x7f34a414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a415a5f0 0x7f34a414c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49624/0 (socket says 172.21.15.3:49624) 2024-09-12T12:20:22.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 -- 172.21.15.3:0/2591240413 learned_addr learned my addr 172.21.15.3:0/2591240413 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a37fe640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a409ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a2ffd640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 0x7f34a4147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 -- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 msgr2=0x7f34a409ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a409ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 -- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 msgr2=0x7f34a4147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 0x7f34a4147700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 -- 172.21.15.3:0/2591240413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a4148070 con 0x7f34a415a5f0 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.415+0000 7f34a27fc640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a415a5f0 0x7f34a414c760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f349800e840 tx=0x7f349800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f348b7fe640 1 -- 172.21.15.3:0/2591240413 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498017070 con 0x7f34a415a5f0 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f348b7fe640 1 -- 172.21.15.3:0/2591240413 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349800ee90 con 0x7f34a415a5f0 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a4148350 con 0x7f34a415a5f0 2024-09-12T12:20:22.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f348b7fe640 1 -- 172.21.15.3:0/2591240413 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349800cb80 con 0x7f34a415a5f0 2024-09-12T12:20:22.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41549f0 con 0x7f34a415a5f0 2024-09-12T12:20:22.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a4148520 con 0x7f34a415a5f0 2024-09-12T12:20:22.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f348b7fe640 1 -- 172.21.15.3:0/2591240413 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f349801a070 con 0x7f34a415a5f0 2024-09-12T12:20:22.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f348b7fe640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34900678c0 0x7f3490069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f348b7fe640 1 -- 172.21.15.3:0/2591240413 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3498085e90 con 0x7f34a415a5f0 2024-09-12T12:20:22.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f34a2ffd640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34900678c0 0x7f3490069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.419+0000 7f34a2ffd640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34900678c0 0x7f3490069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3494007fd0 tx=0x7f3494007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.423+0000 7f348b7fe640 1 -- 172.21.15.3:0/2591240413 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3498056a60 con 0x7f34a415a5f0 2024-09-12T12:20:22.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34a4072720 con 0x7f34a415a5f0 2024-09-12T12:20:22.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f348b7fe640 1 -- 172.21.15.3:0/2591240413 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f3498056360 con 0x7f34a415a5f0 2024-09-12T12:20:22.598 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34900678c0 msgr2=0x7f3490069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34900678c0 0x7f3490069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3494007fd0 tx=0x7f3494007ba0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a415a5f0 msgr2=0x7f34a414c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a415a5f0 0x7f34a414c760 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f349800e840 tx=0x7f349800ec40 comp rx=0 tx=0).stop 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 shutdown_connections 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34900678c0 0x7f3490069d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34a415a9d0 0x7f34a409ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34a415a5f0 0x7f34a414c760 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 --2- 172.21.15.3:0/2591240413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34a40b4190 0x7f34a4147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 >> 172.21.15.3:0/2591240413 conn(0x7f34a406be90 msgr2=0x7f34a4157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 shutdown_connections 2024-09-12T12:20:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.595+0000 7f34a964d640 1 -- 172.21.15.3:0/2591240413 wait complete. 2024-09-12T12:20:22.616 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:20:22.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:22.616 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-12T12:20:22.631 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:22.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:20:22.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:22.632 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T12:20:22.677 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T12:20:22.677 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:22.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b8f0a640 1 Processor -- start 2024-09-12T12:20:22.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b8f0a640 1 -- start start 2024-09-12T12:20:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b8f0a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f31b40704f0 0x7f31b40708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b8f0a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 0x7f31b406cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b8f0a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b40d0390 con 0x7f31b406e2d0 2024-09-12T12:20:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b8f0a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f31b4070e10 con 0x7f31b40704f0 2024-09-12T12:20:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b8f0a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f31b409bc60 con 0x7f31b406e810 2024-09-12T12:20:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b1d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f31b406e2d0 0x7f31b409b430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34724/0 (socket says 172.21.15.3:34724) 2024-09-12T12:20:22.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b1d9b640 1 -- 172.21.15.3:0/3014301597 learned_addr learned my addr 172.21.15.3:0/3014301597 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b259c640 1 --2- 172.21.15.3:0/3014301597 >> v2:172.21.15.142:3300/0 conn(0x7f31b40704f0 0x7f31b40708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 0x7f31b406cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099033608 0 0) 0x7f31b40d0390 con 0x7f31b406e2d0 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f319c003530 con 0x7f31b406e2d0 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1234028341 0 0) 0x7f319c003530 con 0x7f31b406e2d0 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 msgr2=0x7f31b406cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 0x7f31b406cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 >> v2:172.21.15.142:3300/0 conn(0x7f31b40704f0 msgr2=0x7f31b40708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 --2- 172.21.15.3:0/3014301597 >> v2:172.21.15.142:3300/0 conn(0x7f31b40704f0 0x7f31b40708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b40d0630 con 0x7f31b406e2d0 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f31a8001680 con 0x7f31b406e2d0 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 >> v1:172.21.15.3:6789/0 conn(0x7f31b406e2d0 legacy=0x7f31b409b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319c0039c0 0x7f319c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 0x7f319c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319c004d50 0x7f319c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4070e10 con 0x7f319c004d50 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31b409bc60 con 0x7f319c0039c0 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f319c00b9d0 con 0x7f31b406e810 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319c004d50 0x7f319c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b259c640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319c0039c0 0x7f319c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b1d9b640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 0x7f319c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b2d9d640 1 -- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 msgr2=0x7f319c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 0x7f319c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b2d9d640 1 -- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319c0039c0 msgr2=0x7f319c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319c0039c0 0x7f319c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b2d9d640 1 -- 172.21.15.3:0/3014301597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c003530 con 0x7f319c004d50 2024-09-12T12:20:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b259c640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f319c0039c0 0x7f319c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:22.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.799+0000 7f31b1d9b640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b406e810 0x7f319c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319c004d50 0x7f319c0051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009fc0 tx=0x7f31ac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3aec2da3e4926e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0b2070 con 0x7f319c004d50 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac0a5c90 con 0x7f319c004d50 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b159a640 1 -- 172.21.15.3:0/3014301597 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac009c10 con 0x7f319c004d50 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 -- 172.21.15.3:0/3014301597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319c004d50 msgr2=0x7f319c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 --2- 172.21.15.3:0/3014301597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f319c004d50 0x7f319c0051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009fc0 tx=0x7f31ac01cc80 comp rx=0 tx=0).stop 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b2d9d640 1 -- 172.21.15.3:0/3014301597 reap_dead start 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 -- 172.21.15.3:0/3014301597 shutdown_connections 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 -- 172.21.15.3:0/3014301597 >> 172.21.15.3:0/3014301597 conn(0x7f31b40b3e60 msgr2=0x7f31b40b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 -- 172.21.15.3:0/3014301597 shutdown_connections 2024-09-12T12:20:22.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 -- 172.21.15.3:0/3014301597 wait complete. 2024-09-12T12:20:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 Processor -- start 2024-09-12T12:20:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.803+0000 7f31b8f0a640 1 -- start start 2024-09-12T12:20:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b415a810 0x7f31b41f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31b40c7cc0 0x7f31b415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b415fbf0 0x7f31b41f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31b406cdd0 con 0x7f31b415fbf0 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31b40d0bf0 con 0x7f31b40c7cc0 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f31b406cf50 con 0x7f31b415a810 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b415fbf0 0x7f31b41f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b415fbf0 0x7f31b41f7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49648/0 (socket says 172.21.15.3:49648) 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 -- 172.21.15.3:0/225380445 learned_addr learned my addr 172.21.15.3:0/225380445 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b259c640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b415a810 0x7f31b41f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 -- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b415a810 msgr2=0x7f31b41f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b1d9b640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31b40c7cc0 0x7f31b415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b415a810 0x7f31b41f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 -- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31b40c7cc0 msgr2=0x7f31b415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:22.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31b40c7cc0 0x7f31b415f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:22.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 -- 172.21.15.3:0/225380445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b41f5040 con 0x7f31b415fbf0 2024-09-12T12:20:22.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b2d9d640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b415fbf0 0x7f31b41f7670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f31ac002240 tx=0x7f31ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31a2ffd640 1 -- 172.21.15.3:0/225380445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0bd070 con 0x7f31b415fbf0 2024-09-12T12:20:22.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31a2ffd640 1 -- 172.21.15.3:0/225380445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ac0a5a90 con 0x7f31b415fbf0 2024-09-12T12:20:22.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b40c5f20 con 0x7f31b415fbf0 2024-09-12T12:20:22.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31a2ffd640 1 -- 172.21.15.3:0/225380445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0b2040 con 0x7f31b415fbf0 2024-09-12T12:20:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b40c64e0 con 0x7f31b415fbf0 2024-09-12T12:20:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b406d2d0 con 0x7f31b415fbf0 2024-09-12T12:20:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31a2ffd640 1 -- 172.21.15.3:0/225380445 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f31ac0c0070 con 0x7f31b415fbf0 2024-09-12T12:20:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31a2ffd640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f31940678f0 0x7f3194069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:22.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31a2ffd640 1 -- 172.21.15.3:0/225380445 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f31ac11f9f0 con 0x7f31b415fbf0 2024-09-12T12:20:22.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.807+0000 7f31b259c640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f31940678f0 0x7f3194069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:22.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.811+0000 7f31b259c640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f31940678f0 0x7f3194069db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f31b4160390 tx=0x7f31a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:22.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:22.811+0000 7f31a2ffd640 1 -- 172.21.15.3:0/225380445 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f31ac121e20 con 0x7f31b415fbf0 2024-09-12T12:20:23.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.015+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31b40b4a50 con 0x7f31b415fbf0 2024-09-12T12:20:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.015+0000 7f31a2ffd640 1 -- 172.21.15.3:0/225380445 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f31b41f5040 con 0x7f31b415fbf0 2024-09-12T12:20:23.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:23.021 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:20:23.021 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:20:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f31940678f0 msgr2=0x7f3194069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f31940678f0 0x7f3194069db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f31b4160390 tx=0x7f31a4008040 comp rx=0 tx=0).stop 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b415fbf0 msgr2=0x7f31b41f7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b415fbf0 0x7f31b41f7670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f31ac002240 tx=0x7f31ac009750 comp rx=0 tx=0).stop 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 shutdown_connections 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f31940678f0 0x7f3194069db0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31b415fbf0 0x7f31b41f7670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31b40c7cc0 0x7f31b415f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 --2- 172.21.15.3:0/225380445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31b415a810 0x7f31b41f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 >> 172.21.15.3:0/225380445 conn(0x7f31b40b3e60 msgr2=0x7f31b40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 shutdown_connections 2024-09-12T12:20:23.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.019+0000 7f31b8f0a640 1 -- 172.21.15.3:0/225380445 wait complete. 2024-09-12T12:20:23.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:23.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff84354b640 1 Processor -- start 2024-09-12T12:20:23.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff84354b640 1 -- start start 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff84354b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff83c15e5d0 0x7ff83c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff84354b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 0x7ff83c0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff84354b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83c0d31a0 con 0x7ff83c0cfa10 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff84354b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff83c0b1060 con 0x7ff83c15e5d0 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff84354b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c0cfe40 con 0x7ff83c0d5050 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff8412e7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff83c15e5d0 0x7ff83c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff8412e7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff83c15e5d0 0x7ff83c0ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49656/0 (socket says 172.21.15.3:49656) 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff8412e7640 1 -- 172.21.15.3:0/1175188103 learned_addr learned my addr 172.21.15.3:0/1175188103 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff840ae6640 1 --2- 172.21.15.3:0/1175188103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 0x7ff83c0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff8412e7640 1 -- 172.21.15.3:0/1175188103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 msgr2=0x7ff83c0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff8412e7640 1 --2- 172.21.15.3:0/1175188103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 0x7ff83c0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff8412e7640 1 -- 172.21.15.3:0/1175188103 >> v1:172.21.15.142:6789/0 conn(0x7ff83c0cfa10 legacy=0x7ff83c162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.159+0000 7ff8412e7640 1 -- 172.21.15.3:0/1175188103 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c0d3480 con 0x7ff83c15e5d0 2024-09-12T12:20:23.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff8412e7640 1 --2- 172.21.15.3:0/1175188103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c0ceb60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff838009870 tx=0x7ff83801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b332c7d948f4d660 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff833fff640 1 -- 172.21.15.3:0/1175188103 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380a9400 con 0x7ff83c15e5d0 2024-09-12T12:20:23.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff833fff640 1 -- 172.21.15.3:0/1175188103 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8380a95a0 con 0x7ff83c15e5d0 2024-09-12T12:20:23.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff833fff640 1 -- 172.21.15.3:0/1175188103 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8380a9970 con 0x7ff83c15e5d0 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 -- 172.21.15.3:0/1175188103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 msgr2=0x7ff83c0ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 --2- 172.21.15.3:0/1175188103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c0ceb60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff838009870 tx=0x7ff83801c560 comp rx=0 tx=0).stop 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 -- 172.21.15.3:0/1175188103 shutdown_connections 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 --2- 172.21.15.3:0/1175188103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 0x7ff83c0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 --2- 172.21.15.3:0/1175188103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c0ceb60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 -- 172.21.15.3:0/1175188103 >> 172.21.15.3:0/1175188103 conn(0x7ff83c08c000 msgr2=0x7ff83c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 -- 172.21.15.3:0/1175188103 shutdown_connections 2024-09-12T12:20:23.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 -- 172.21.15.3:0/1175188103 wait complete. 2024-09-12T12:20:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.163+0000 7ff84354b640 1 Processor -- start 2024-09-12T12:20:23.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 -- start start 2024-09-12T12:20:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c0cfa10 0x7ff83c14f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 0x7ff83c156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c15d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c16b5c0 con 0x7ff83c15e5d0 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c15ebc0 con 0x7ff83c0cfa10 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c16b740 con 0x7ff83c0d5050 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c15d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c15d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49670/0 (socket says 172.21.15.3:49670) 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 -- 172.21.15.3:0/2019212153 learned_addr learned my addr 172.21.15.3:0/2019212153 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 -- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 msgr2=0x7ff83c156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 0x7ff83c156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 -- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c0cfa10 msgr2=0x7ff83c14f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c0cfa10 0x7ff83c14f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 -- 172.21.15.3:0/2019212153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c14f800 con 0x7ff83c15e5d0 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff841ae8640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c15d050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff83400c6f0 tx=0x7ff83400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff831ffb640 1 -- 172.21.15.3:0/2019212153 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834003980 con 0x7ff83c15e5d0 2024-09-12T12:20:23.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff831ffb640 1 -- 172.21.15.3:0/2019212153 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff834003b20 con 0x7ff83c15e5d0 2024-09-12T12:20:23.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c150d40 con 0x7ff83c15e5d0 2024-09-12T12:20:23.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff831ffb640 1 -- 172.21.15.3:0/2019212153 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83400a850 con 0x7ff83c15e5d0 2024-09-12T12:20:23.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c150780 con 0x7ff83c15e5d0 2024-09-12T12:20:23.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c0cf4d0 con 0x7ff83c15e5d0 2024-09-12T12:20:23.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff831ffb640 1 -- 172.21.15.3:0/2019212153 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff83400a9f0 con 0x7ff83c15e5d0 2024-09-12T12:20:23.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff831ffb640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8200678c0 0x7ff820069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff831ffb640 1 -- 172.21.15.3:0/2019212153 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff834085a80 con 0x7ff83c15e5d0 2024-09-12T12:20:23.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.167+0000 7ff8412e7640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8200678c0 0x7ff820069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.171+0000 7ff8412e7640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8200678c0 0x7ff820069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ae000 tx=0x7ff838004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.171+0000 7ff831ffb640 1 -- 172.21.15.3:0/2019212153 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff834056650 con 0x7ff83c15e5d0 2024-09-12T12:20:23.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff83c0ba490 con 0x7ff83c15e5d0 2024-09-12T12:20:23.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff831ffb640 1 -- 172.21.15.3:0/2019212153 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7ff834055f50 con 0x7ff83c15e5d0 2024-09-12T12:20:23.375 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:23.375 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:20:23.376 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:20:23.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8200678c0 msgr2=0x7ff820069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8200678c0 0x7ff820069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8380ae000 tx=0x7ff838004740 comp rx=0 tx=0).stop 2024-09-12T12:20:23.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 msgr2=0x7ff83c15d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c15d050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff83400c6f0 tx=0x7ff83400cbb0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 shutdown_connections 2024-09-12T12:20:23.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8200678c0 0x7ff820069d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff83c15e5d0 0x7ff83c15d050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff83c0d5050 0x7ff83c156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 --2- 172.21.15.3:0/2019212153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff83c0cfa10 0x7ff83c14f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.371+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 >> 172.21.15.3:0/2019212153 conn(0x7ff83c08c000 msgr2=0x7ff83c0b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:23.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.375+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 shutdown_connections 2024-09-12T12:20:23.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.375+0000 7ff84354b640 1 -- 172.21.15.3:0/2019212153 wait complete. 2024-09-12T12:20:23.393 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:20:23.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.511+0000 7f82c42c3640 1 Processor -- start 2024-09-12T12:20:23.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.511+0000 7f82c42c3640 1 -- start start 2024-09-12T12:20:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c42c3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82bc144110 0x7f82bc06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c42c3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82bc0727b0 0x7f82bc072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c42c3640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82bc073100 con 0x7f82bc15ac30 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c42c3640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f82bc072490 con 0x7f82bc144110 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c42c3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f82bc15b010 con 0x7f82bc0727b0 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c185e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82bc144110 0x7f82bc06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c205f640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f82bc15ac30 0x7f82bc0b4a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34734/0 (socket says 172.21.15.3:34734) 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c205f640 1 -- 172.21.15.3:0/2426261180 learned_addr learned my addr 172.21.15.3:0/2426261180 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c2860640 1 --2- 172.21.15.3:0/2426261180 >> v2:172.21.15.142:3300/0 conn(0x7f82bc0727b0 0x7f82bc072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c185e640 1 -- 172.21.15.3:0/2426261180 >> v2:172.21.15.142:3300/0 conn(0x7f82bc0727b0 msgr2=0x7f82bc072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c185e640 1 --2- 172.21.15.3:0/2426261180 >> v2:172.21.15.142:3300/0 conn(0x7f82bc0727b0 0x7f82bc072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c185e640 1 -- 172.21.15.3:0/2426261180 >> v1:172.21.15.3:6789/0 conn(0x7f82bc15ac30 legacy=0x7f82bc0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c105d640 1 -- 172.21.15.3:0/2426261180 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650645984 0 0) 0x7f82b8001120 con 0x7f82bc15ac30 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c185e640 1 -- 172.21.15.3:0/2426261180 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc071ed0 con 0x7f82bc144110 2024-09-12T12:20:23.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c2860640 1 --2- 172.21.15.3:0/2426261180 >> v2:172.21.15.142:3300/0 conn(0x7f82bc0727b0 0x7f82bc072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:23.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.515+0000 7f82c185e640 1 --2- 172.21.15.3:0/2426261180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc06f6b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0b4510 tx=0x7f82ac01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d573ca67f579c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c105d640 1 -- 172.21.15.3:0/2426261180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0b0070 con 0x7f82bc144110 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c105d640 1 -- 172.21.15.3:0/2426261180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ac0ab600 con 0x7f82bc144110 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c105d640 1 -- 172.21.15.3:0/2426261180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0ab9d0 con 0x7f82bc144110 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- 172.21.15.3:0/2426261180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 msgr2=0x7f82bc06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2426261180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc06f6b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0b4510 tx=0x7f82ac01c360 comp rx=0 tx=0).stop 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- 172.21.15.3:0/2426261180 shutdown_connections 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2426261180 >> v2:172.21.15.142:3300/0 conn(0x7f82bc0727b0 0x7f82bc072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2426261180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc06f6b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- 172.21.15.3:0/2426261180 >> 172.21.15.3:0/2426261180 conn(0x7f82bc0b1d80 msgr2=0x7f82bc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- 172.21.15.3:0/2426261180 shutdown_connections 2024-09-12T12:20:23.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- 172.21.15.3:0/2426261180 wait complete. 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 Processor -- start 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- start start 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82bc0727b0 0x7f82bc143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc15ac30 0x7f82bc141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc0ae120 con 0x7f82bc144110 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc0adfa0 con 0x7f82bc15ac30 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc06dd40 con 0x7f82bc0727b0 2024-09-12T12:20:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49694/0 (socket says 172.21.15.3:49694) 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 -- 172.21.15.3:0/2907616094 learned_addr learned my addr 172.21.15.3:0/2907616094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 -- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82bc0727b0 msgr2=0x7f82bc143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82bc0727b0 0x7f82bc143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 -- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc15ac30 msgr2=0x7f82bc141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc15ac30 0x7f82bc141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 -- 172.21.15.3:0/2907616094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc0cd2f0 con 0x7f82bc144110 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c185e640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc144870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f82ac019f90 tx=0x7f82ac01aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82b2ffd640 1 -- 172.21.15.3:0/2907616094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0b0050 con 0x7f82bc144110 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82b2ffd640 1 -- 172.21.15.3:0/2907616094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ac0b1040 con 0x7f82bc144110 2024-09-12T12:20:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82b2ffd640 1 -- 172.21.15.3:0/2907616094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac004220 con 0x7f82bc144110 2024-09-12T12:20:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.519+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc147120 con 0x7f82bc144110 2024-09-12T12:20:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.523+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc0cd590 con 0x7f82bc144110 2024-09-12T12:20:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.523+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc06e100 con 0x7f82bc144110 2024-09-12T12:20:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.523+0000 7f82b2ffd640 1 -- 172.21.15.3:0/2907616094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f82ac0be060 con 0x7f82bc144110 2024-09-12T12:20:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.523+0000 7f82b2ffd640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82a00678c0 0x7f82a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.523+0000 7f82b2ffd640 1 -- 172.21.15.3:0/2907616094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f82ac11f420 con 0x7f82bc144110 2024-09-12T12:20:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.523+0000 7f82c205f640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82a00678c0 0x7f82a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.523+0000 7f82c205f640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82a00678c0 0x7f82a0069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f82b80015a0 tx=0x7f82b8007970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.527+0000 7f82b2ffd640 1 -- 172.21.15.3:0/2907616094 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f82ac016070 con 0x7f82bc144110 2024-09-12T12:20:23.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.683+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82bc0b1820 con 0x7f82bc144110 2024-09-12T12:20:23.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.687+0000 7f82b2ffd640 1 -- 172.21.15.3:0/2907616094 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f82ac0efff0 con 0x7f82bc144110 2024-09-12T12:20:23.692 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:23.692 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:49.738546+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:20:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82a00678c0 msgr2=0x7f82a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82a00678c0 0x7f82a0069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f82b80015a0 tx=0x7f82b8007970 comp rx=0 tx=0).stop 2024-09-12T12:20:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 msgr2=0x7f82bc144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc144870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f82ac019f90 tx=0x7f82ac01aba0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 shutdown_connections 2024-09-12T12:20:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82a00678c0 0x7f82a0069d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82bc15ac30 0x7f82bc141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82bc144110 0x7f82bc144870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 --2- 172.21.15.3:0/2907616094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82bc0727b0 0x7f82bc143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 >> 172.21.15.3:0/2907616094 conn(0x7f82bc0b1d80 msgr2=0x7f82bc154570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 shutdown_connections 2024-09-12T12:20:23.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.691+0000 7f82c42c3640 1 -- 172.21.15.3:0/2907616094 wait complete. 2024-09-12T12:20:23.713 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:20:23.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb6a646640 1 Processor -- start 2024-09-12T12:20:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb6a646640 1 -- start start 2024-09-12T12:20:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb6a646640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb640ad770 0x7fbb6406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb6a646640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6406fbb0 con 0x7fbb64159fb0 2024-09-12T12:20:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb6a646640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6406e400 con 0x7fbb64159bd0 2024-09-12T12:20:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb6a646640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6406f010 con 0x7fbb640ad770 2024-09-12T12:20:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb637fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbb64159fb0 0x7fbb640ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34746/0 (socket says 172.21.15.3:34746) 2024-09-12T12:20:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb637fe640 1 -- 172.21.15.3:0/2925526984 learned_addr learned my addr 172.21.15.3:0/2925526984 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb68be3640 1 --2- 172.21.15.3:0/2925526984 >> v2:172.21.15.3:3300/0 conn(0x7fbb640ad770 0x7fbb6406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb68be3640 1 -- 172.21.15.3:0/2925526984 >> v1:172.21.15.142:6789/0 conn(0x7fbb64159bd0 legacy=0x7fbb6406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb68be3640 1 -- 172.21.15.3:0/2925526984 >> v1:172.21.15.3:6789/0 conn(0x7fbb64159fb0 legacy=0x7fbb640ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.831+0000 7fbb68be3640 1 -- 172.21.15.3:0/2925526984 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6406e6e0 con 0x7fbb640ad770 2024-09-12T12:20:23.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb68be3640 1 --2- 172.21.15.3:0/2925526984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb640ad770 0x7fbb6406dd40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009d90 tx=0x7fbb5c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1572aa19e065e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb62ffd640 1 -- 172.21.15.3:0/2925526984 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0b2070 con 0x7fbb640ad770 2024-09-12T12:20:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb62ffd640 1 -- 172.21.15.3:0/2925526984 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c0a5a00 con 0x7fbb640ad770 2024-09-12T12:20:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb62ffd640 1 -- 172.21.15.3:0/2925526984 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0a5dd0 con 0x7fbb640ad770 2024-09-12T12:20:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 -- 172.21.15.3:0/2925526984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb640ad770 msgr2=0x7fbb6406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 --2- 172.21.15.3:0/2925526984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb640ad770 0x7fbb6406dd40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009d90 tx=0x7fbb5c01c940 comp rx=0 tx=0).stop 2024-09-12T12:20:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 -- 172.21.15.3:0/2925526984 shutdown_connections 2024-09-12T12:20:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 --2- 172.21.15.3:0/2925526984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb640ad770 0x7fbb6406dd40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 -- 172.21.15.3:0/2925526984 >> 172.21.15.3:0/2925526984 conn(0x7fbb640b1d80 msgr2=0x7fbb640b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 -- 172.21.15.3:0/2925526984 shutdown_connections 2024-09-12T12:20:23.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 -- 172.21.15.3:0/2925526984 wait complete. 2024-09-12T12:20:23.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 Processor -- start 2024-09-12T12:20:23.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.835+0000 7fbb6a646640 1 -- start start 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb640ad770 0x7fbb6414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb640b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159fb0 0x7fbb640b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6416a370 con 0x7fbb64159bd0 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6416a1f0 con 0x7fbb640ad770 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6416a4f0 con 0x7fbb64159fb0 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb640b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb640b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49714/0 (socket says 172.21.15.3:49714) 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 -- 172.21.15.3:0/51379141 learned_addr learned my addr 172.21.15.3:0/51379141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 -- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159fb0 msgr2=0x7fbb640b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159fb0 0x7fbb640b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 -- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb640ad770 msgr2=0x7fbb6414cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb640ad770 0x7fbb6414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 -- 172.21.15.3:0/51379141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641565f0 con 0x7fbb64159bd0 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb637fe640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb640b0970 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c3a0 tx=0x7fbb5800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb60ff9640 1 -- 172.21.15.3:0/51379141 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58017070 con 0x7fbb64159bd0 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb60ff9640 1 -- 172.21.15.3:0/51379141 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb58005cf0 con 0x7fbb64159bd0 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb60ff9640 1 -- 172.21.15.3:0/51379141 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580072d0 con 0x7fbb64159bd0 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb641568d0 con 0x7fbb64159bd0 2024-09-12T12:20:23.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb64156e90 con 0x7fbb64159bd0 2024-09-12T12:20:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6406dd40 con 0x7fbb64159bd0 2024-09-12T12:20:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb60ff9640 1 -- 172.21.15.3:0/51379141 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbb5801a070 con 0x7fbb64159bd0 2024-09-12T12:20:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb60ff9640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb500678c0 0x7fbb50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:23.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb60ff9640 1 -- 172.21.15.3:0/51379141 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbb58084eb0 con 0x7fbb64159bd0 2024-09-12T12:20:23.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.839+0000 7fbb63fff640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb500678c0 0x7fbb50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:23.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.843+0000 7fbb63fff640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb500678c0 0x7fbb50069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbb54000f70 tx=0x7fbb540079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:23.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:23.843+0000 7fbb60ff9640 1 -- 172.21.15.3:0/51379141 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbb580553a0 con 0x7fbb64159bd0 2024-09-12T12:20:24.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.039+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb6414cb50 con 0x7fbb64159bd0 2024-09-12T12:20:24.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb60ff9640 1 -- 172.21.15.3:0/51379141 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7fbb580596a0 con 0x7fbb64159bd0 2024-09-12T12:20:24.045 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:24.045 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:20:24.045 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:20:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb500678c0 msgr2=0x7fbb50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb500678c0 0x7fbb50069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbb54000f70 tx=0x7fbb540079c0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 msgr2=0x7fbb640b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb640b0970 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c3a0 tx=0x7fbb5800c860 comp rx=0 tx=0).stop 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 shutdown_connections 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbb500678c0 0x7fbb50069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159fb0 0x7fbb640b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb64159bd0 0x7fbb640b0970 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 --2- 172.21.15.3:0/51379141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb640ad770 0x7fbb6414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 >> 172.21.15.3:0/51379141 conn(0x7fbb640b1d80 msgr2=0x7fbb6415b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 shutdown_connections 2024-09-12T12:20:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.043+0000 7fbb6a646640 1 -- 172.21.15.3:0/51379141 wait complete. 2024-09-12T12:20:24.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.171+0000 7f0f96217640 1 Processor -- start 2024-09-12T12:20:24.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.171+0000 7f0f96217640 1 -- start start 2024-09-12T12:20:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.171+0000 7f0f96217640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0f900b6860 0x7f0f901629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.171+0000 7f0f96217640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0f900af330 0x7f0f900afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.171+0000 7f0f96217640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f900af760 con 0x7f0f900aff40 2024-09-12T12:20:24.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.171+0000 7f0f96217640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0f900b03b0 con 0x7f0f900b6860 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.171+0000 7f0f96217640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9015f640 con 0x7f0f900af330 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0f900aff40 0x7f0f900addb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34750/0 (socket says 172.21.15.3:34750) 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8ffff640 1 -- 172.21.15.3:0/1587026847 learned_addr learned my addr 172.21.15.3:0/1587026847 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8f7fe640 1 --2- 172.21.15.3:0/1587026847 >> v2:172.21.15.3:3300/0 conn(0x7f0f900b6860 0x7f0f901629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8effd640 1 --2- 172.21.15.3:0/1587026847 >> v2:172.21.15.142:3300/0 conn(0x7f0f900af330 0x7f0f900afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8e7fc640 1 -- 172.21.15.3:0/1587026847 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144129596 0 0) 0x7f0f900af760 con 0x7f0f900aff40 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8e7fc640 1 -- 172.21.15.3:0/1587026847 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f7c003530 con 0x7f0f900aff40 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8f7fe640 1 -- 172.21.15.3:0/1587026847 >> v2:172.21.15.142:3300/0 conn(0x7f0f900af330 msgr2=0x7f0f900afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8f7fe640 1 --2- 172.21.15.3:0/1587026847 >> v2:172.21.15.142:3300/0 conn(0x7f0f900af330 0x7f0f900afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8f7fe640 1 -- 172.21.15.3:0/1587026847 >> v1:172.21.15.3:6789/0 conn(0x7f0f900aff40 legacy=0x7f0f900addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8f7fe640 1 -- 172.21.15.3:0/1587026847 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9015ed10 con 0x7f0f900b6860 2024-09-12T12:20:24.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8f7fe640 1 --2- 172.21.15.3:0/1587026847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901629e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0f90070280 tx=0x7f0f8001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0da15f8784bea0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8e7fc640 1 -- 172.21.15.3:0/1587026847 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800b2070 con 0x7f0f900b6860 2024-09-12T12:20:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8e7fc640 1 -- 172.21.15.3:0/1587026847 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f800a5930 con 0x7f0f900b6860 2024-09-12T12:20:24.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f8e7fc640 1 -- 172.21.15.3:0/1587026847 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800a5d00 con 0x7f0f900b6860 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 -- 172.21.15.3:0/1587026847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 msgr2=0x7f0f901629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 --2- 172.21.15.3:0/1587026847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901629e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0f90070280 tx=0x7f0f8001c870 comp rx=0 tx=0).stop 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 -- 172.21.15.3:0/1587026847 shutdown_connections 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 --2- 172.21.15.3:0/1587026847 >> v2:172.21.15.142:3300/0 conn(0x7f0f900af330 0x7f0f900afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 --2- 172.21.15.3:0/1587026847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901629e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 -- 172.21.15.3:0/1587026847 >> 172.21.15.3:0/1587026847 conn(0x7f0f900736b0 msgr2=0x7f0f9006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 -- 172.21.15.3:0/1587026847 shutdown_connections 2024-09-12T12:20:24.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.175+0000 7f0f96217640 1 -- 172.21.15.3:0/1587026847 wait complete. 2024-09-12T12:20:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 Processor -- start 2024-09-12T12:20:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 -- start start 2024-09-12T12:20:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f900af330 0x7f0f9006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f900aff40 0x7f0f9014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9006f780 con 0x7f0f900b6860 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f900b6c40 con 0x7f0f900af330 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9006f900 con 0x7f0f900aff40 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901613a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49730/0 (socket says 172.21.15.3:49730) 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 -- 172.21.15.3:0/1108947745 learned_addr learned my addr 172.21.15.3:0/1108947745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 -- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f900aff40 msgr2=0x7f0f9014eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f900aff40 0x7f0f9014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 -- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f900af330 msgr2=0x7f0f9006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f900af330 0x7f0f9006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 -- 172.21.15.3:0/1108947745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9014fdb0 con 0x7f0f900b6860 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f8ffff640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901613a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c450 tx=0x7f0f8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f87fff640 1 -- 172.21.15.3:0/1108947745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88017070 con 0x7f0f900b6860 2024-09-12T12:20:24.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f87fff640 1 -- 172.21.15.3:0/1108947745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f88005cf0 con 0x7f0f900b6860 2024-09-12T12:20:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9014f3f0 con 0x7f0f900b6860 2024-09-12T12:20:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f87fff640 1 -- 172.21.15.3:0/1108947745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88007310 con 0x7f0f900b6860 2024-09-12T12:20:24.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f90150020 con 0x7f0f900b6860 2024-09-12T12:20:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f901628e0 con 0x7f0f900b6860 2024-09-12T12:20:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f87fff640 1 -- 172.21.15.3:0/1108947745 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0f8801a070 con 0x7f0f900b6860 2024-09-12T12:20:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f87fff640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0f700678c0 0x7f0f70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.179+0000 7f0f87fff640 1 -- 172.21.15.3:0/1108947745 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0f88085080 con 0x7f0f900b6860 2024-09-12T12:20:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.183+0000 7f0f8f7fe640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0f700678c0 0x7f0f70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.183+0000 7f0f8f7fe640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0f700678c0 0x7f0f70069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f800af000 tx=0x7f0f80004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.187+0000 7f0f87fff640 1 -- 172.21.15.3:0/1108947745 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0f88055b30 con 0x7f0f900b6860 2024-09-12T12:20:24.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.379+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f90152dc0 con 0x7f0f900b6860 2024-09-12T12:20:24.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f87fff640 1 -- 172.21.15.3:0/1108947745 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5208 (secure 0 0 0) 0x7f0f88055430 con 0x7f0f900b6860 2024-09-12T12:20:24.385 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:24.386 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":7,"btime":"2024-09-12T12:19:51:785071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/401759651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":401759651},{"type":"v1","addr":"172.21.15.3:6833","nonce":401759651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/619839395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":619839395},{"type":"v1","addr":"172.21.15.3:6835","nonce":619839395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/494176982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":494176982},{"type":"v1","addr":"172.21.15.142:6835","nonce":494176982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:19:48.098984+0000","modified":"2024-09-12T12:19:51.114095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2900613310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2900613310},{"type":"v1","addr":"172.21.15.142:6837","nonce":2900613310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4306,"qdb_cluster":[4306]},"id":1}]} 2024-09-12T12:20:24.386 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 7 2024-09-12T12:20:24.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0f700678c0 msgr2=0x7f0f70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0f700678c0 0x7f0f70069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f800af000 tx=0x7f0f80004ac0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 msgr2=0x7f0f901613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901613a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c450 tx=0x7f0f8800c910 comp rx=0 tx=0).stop 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 shutdown_connections 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0f700678c0 0x7f0f70069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f900b6860 0x7f0f901613a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0f900aff40 0x7f0f9014eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 --2- 172.21.15.3:0/1108947745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f900af330 0x7f0f9006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 >> 172.21.15.3:0/1108947745 conn(0x7f0f900736b0 msgr2=0x7f0f9006fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 shutdown_connections 2024-09-12T12:20:24.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.383+0000 7f0f96217640 1 -- 172.21.15.3:0/1108947745 wait complete. 2024-09-12T12:20:24.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:20:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb9794640 1 Processor -- start 2024-09-12T12:20:24.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb9794640 1 -- start start 2024-09-12T12:20:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb9794640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcdb415e5d0 0x7fcdb40ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb9794640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdb40cfbe0 0x7fcdb40b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb9794640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb40d31a0 con 0x7fcdb40d5050 2024-09-12T12:20:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb9794640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb40b1060 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb9794640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb40d3480 con 0x7fcdb415e5d0 2024-09-12T12:20:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb37fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcdb40cfbe0 0x7fcdb40b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb27fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcdb40d5050 0x7fcdb40cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34752/0 (socket says 172.21.15.3:34752) 2024-09-12T12:20:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb27fc640 1 -- 172.21.15.3:0/3105782698 learned_addr learned my addr 172.21.15.3:0/3105782698 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb2ffd640 1 --2- 172.21.15.3:0/3105782698 >> v2:172.21.15.142:3300/0 conn(0x7fcdb415e5d0 0x7fcdb40ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb37fe640 1 -- 172.21.15.3:0/3105782698 >> v2:172.21.15.142:3300/0 conn(0x7fcdb415e5d0 msgr2=0x7fcdb40ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb37fe640 1 --2- 172.21.15.3:0/3105782698 >> v2:172.21.15.142:3300/0 conn(0x7fcdb415e5d0 0x7fcdb40ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb37fe640 1 -- 172.21.15.3:0/3105782698 >> v1:172.21.15.3:6789/0 conn(0x7fcdb40d5050 legacy=0x7fcdb40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.523+0000 7fcdb37fe640 1 -- 172.21.15.3:0/3105782698 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb40d3760 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb37fe640 1 --2- 172.21.15.3:0/3105782698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb40b14f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4163340 tx=0x7fcdac01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a6aedf15ae08cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb1ffb640 1 -- 172.21.15.3:0/3105782698 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0b2070 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb1ffb640 1 -- 172.21.15.3:0/3105782698 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac0a5980 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb1ffb640 1 -- 172.21.15.3:0/3105782698 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0a5d50 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- 172.21.15.3:0/3105782698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 msgr2=0x7fcdb40b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 --2- 172.21.15.3:0/3105782698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb40b14f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4163340 tx=0x7fcdac01c8a0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- 172.21.15.3:0/3105782698 shutdown_connections 2024-09-12T12:20:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 --2- 172.21.15.3:0/3105782698 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb40b14f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 --2- 172.21.15.3:0/3105782698 >> v2:172.21.15.142:3300/0 conn(0x7fcdb415e5d0 0x7fcdb40ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- 172.21.15.3:0/3105782698 >> 172.21.15.3:0/3105782698 conn(0x7fcdb408c000 msgr2=0x7fcdb40d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- 172.21.15.3:0/3105782698 shutdown_connections 2024-09-12T12:20:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- 172.21.15.3:0/3105782698 wait complete. 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 Processor -- start 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- start start 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb41fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdb40d5050 0x7fcdb4161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdb415e5d0 0x7fcdb4151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb40b25d0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb40b2450 con 0x7fcdb40d5050 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb9794640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb40b2750 con 0x7fcdb415e5d0 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb41fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb41fa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49744/0 (socket says 172.21.15.3:49744) 2024-09-12T12:20:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 -- 172.21.15.3:0/1886330384 learned_addr learned my addr 172.21.15.3:0/1886330384 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb27fc640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdb40d5050 0x7fcdb4161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb37fe640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdb415e5d0 0x7fcdb4151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 -- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdb415e5d0 msgr2=0x7fcdb4151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdb415e5d0 0x7fcdb4151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 -- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdb40d5050 msgr2=0x7fcdb4161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdb40d5050 0x7fcdb4161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 -- 172.21.15.3:0/1886330384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb41522e0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcdb2ffd640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb41fa9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcda400d5f0 tx=0x7fcda400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.527+0000 7fcd9b7fe640 1 -- 172.21.15.3:0/1886330384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40125f0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcd9b7fe640 1 -- 172.21.15.3:0/1886330384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda4019070 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb41fb020 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcd9b7fe640 1 -- 172.21.15.3:0/1886330384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40129b0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb41fb5e0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb40b14f0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcd9b7fe640 1 -- 172.21.15.3:0/1886330384 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcda4012b50 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcd9b7fe640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcda00678c0 0x7fcda0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcd9b7fe640 1 -- 172.21.15.3:0/1886330384 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcda4085fd0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcdb27fc640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcda00678c0 0x7fcda0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.531+0000 7fcdb27fc640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcda00678c0 0x7fcda0069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4161690 tx=0x7fcda8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.535+0000 7fcd9b7fe640 1 -- 172.21.15.3:0/1886330384 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcda4056ba0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.691+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdb415c180 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.695+0000 7fcd9b7fe640 1 -- 172.21.15.3:0/1886330384 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fcda40564a0 con 0x7fcdb40cfbe0 2024-09-12T12:20:24.700 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:24.700 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:19:49.738546+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:19:46.505915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:19:47.135597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:20:24.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcda00678c0 msgr2=0x7fcda0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcda00678c0 0x7fcda0069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4161690 tx=0x7fcda8009040 comp rx=0 tx=0).stop 2024-09-12T12:20:24.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 msgr2=0x7fcdb41fa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb41fa9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcda400d5f0 tx=0x7fcda400dab0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 shutdown_connections 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcda00678c0 0x7fcda0069d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcdb415e5d0 0x7fcdb4151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcdb40d5050 0x7fcdb4161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 --2- 172.21.15.3:0/1886330384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcdb40cfbe0 0x7fcdb41fa9b0 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcda400d5f0 tx=0x7fcda400dab0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 >> 172.21.15.3:0/1886330384 conn(0x7fcdb408c000 msgr2=0x7fcdb40b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 shutdown_connections 2024-09-12T12:20:24.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.699+0000 7fcdb9794640 1 -- 172.21.15.3:0/1886330384 wait complete. 2024-09-12T12:20:24.718 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T12:20:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f100e059640 1 Processor -- start 2024-09-12T12:20:24.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f100e059640 1 -- start start 2024-09-12T12:20:24.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f100e059640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10080d26f0 0x7f100815f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f100e059640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10080d2d90 con 0x7f10080b5fc0 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f100e059640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10080d3070 con 0x7f10080ced00 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f100e059640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f10080d2b60 con 0x7f10080d26f0 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10077fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f10080b5fc0 0x7f10080b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34758/0 (socket says 172.21.15.3:34758) 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10077fe640 1 -- 172.21.15.3:0/683736305 learned_addr learned my addr 172.21.15.3:0/683736305 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f1006ffd640 1 --2- 172.21.15.3:0/683736305 >> v2:172.21.15.142:3300/0 conn(0x7f10080d26f0 0x7f100815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3169250370 0 0) 0x7f10080d2d90 con 0x7f10080b5fc0 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ff0003530 con 0x7f10080b5fc0 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1181995340 0 0) 0x7f0ff0003530 con 0x7f10080b5fc0 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 >> v2:172.21.15.142:3300/0 conn(0x7f10080d26f0 msgr2=0x7f100815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 --2- 172.21.15.3:0/683736305 >> v2:172.21.15.142:3300/0 conn(0x7f10080d26f0 0x7f100815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 >> v1:172.21.15.142:6789/0 conn(0x7f10080ced00 legacy=0x7f10080cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10080d3350 con 0x7f10080b5fc0 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f0ff8001680 con 0x7f10080b5fc0 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 >> v1:172.21.15.3:6789/0 conn(0x7f10080b5fc0 legacy=0x7f10080b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ff00039c0 0x7f0ff0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff0004810 0x7f0ff0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ff00051d0 0x7f0ff000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff000c040 con 0x7f0ff00051d0 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff000c1f0 con 0x7f0ff0004810 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.831+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff000c3f0 con 0x7f0ff00039c0 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ff00051d0 0x7f0ff000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f10077fe640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ff00039c0 0x7f0ff0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1006ffd640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff0004810 0x7f0ff0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 -- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ff00039c0 msgr2=0x7f0ff0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ff00039c0 0x7f0ff0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 -- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff0004810 msgr2=0x7f0ff0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff0004810 0x7f0ff0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 -- 172.21.15.3:0/683736305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff0003530 con 0x7f0ff00051d0 2024-09-12T12:20:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1006ffd640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff0004810 0x7f0ff0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 -- 172.21.15.3:0/683736305 reap_dead start 2024-09-12T12:20:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f1007fff640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ff00051d0 0x7f0ff000b9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1000001e70 tx=0x7f100001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ff35d25c69e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000001a90 con 0x7f0ff00051d0 2024-09-12T12:20:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1000003bc0 con 0x7f0ff00051d0 2024-09-12T12:20:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f10067fc640 1 -- 172.21.15.3:0/683736305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000005020 con 0x7f0ff00051d0 2024-09-12T12:20:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f100e059640 1 -- 172.21.15.3:0/683736305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ff00051d0 msgr2=0x7f0ff000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f100e059640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ff00051d0 0x7f0ff000b9d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1000001e70 tx=0x7f100001c780 comp rx=0 tx=0).stop 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f100e059640 1 -- 172.21.15.3:0/683736305 shutdown_connections 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f100e059640 1 --2- 172.21.15.3:0/683736305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ff00051d0 0x7f0ff000b9d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f100e059640 1 -- 172.21.15.3:0/683736305 >> 172.21.15.3:0/683736305 conn(0x7f100808c000 msgr2=0x7f10080d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f100e059640 1 -- 172.21.15.3:0/683736305 shutdown_connections 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.835+0000 7f100e059640 1 -- 172.21.15.3:0/683736305 wait complete. 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 Processor -- start 2024-09-12T12:20:24.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 -- start start 2024-09-12T12:20:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00051d0 0x7f100815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10080b95b0 0x7f10080b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10080b9a30 0x7f10081f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f100815e8b0 con 0x7f10080b9a30 2024-09-12T12:20:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10080d3910 con 0x7f0ff00051d0 2024-09-12T12:20:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f100815ea30 con 0x7f10080b95b0 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10080b9a30 0x7f10081f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10080b9a30 0x7f10081f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49760/0 (socket says 172.21.15.3:49760) 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 -- 172.21.15.3:0/602071076 learned_addr learned my addr 172.21.15.3:0/602071076 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1006ffd640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10080b95b0 0x7f10080b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f10077fe640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00051d0 0x7f100815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 -- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10080b95b0 msgr2=0x7f10080b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10080b95b0 0x7f10080b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 -- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00051d0 msgr2=0x7f100815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00051d0 0x7f100815ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 -- 172.21.15.3:0/602071076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100815ddf0 con 0x7f10080b9a30 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f10077fe640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00051d0 0x7f100815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f1007fff640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10080b9a30 0x7f10081f1470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f100001cc60 tx=0x7f0ff0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f0feffff640 1 -- 172.21.15.3:0/602071076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000007dc0 con 0x7f10080b9a30 2024-09-12T12:20:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f0feffff640 1 -- 172.21.15.3:0/602071076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10000012e0 con 0x7f10080b9a30 2024-09-12T12:20:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10081f47e0 con 0x7f10080b9a30 2024-09-12T12:20:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f0feffff640 1 -- 172.21.15.3:0/602071076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10000057f0 con 0x7f10080b9a30 2024-09-12T12:20:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10081f4da0 con 0x7f10080b9a30 2024-09-12T12:20:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100815f620 con 0x7f10080b9a30 2024-09-12T12:20:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f0feffff640 1 -- 172.21.15.3:0/602071076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f10000a6480 con 0x7f10080b9a30 2024-09-12T12:20:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.839+0000 7f0feffff640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0ff40678c0 0x7f0ff4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.843+0000 7f0feffff640 1 -- 172.21.15.3:0/602071076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f100011f270 con 0x7f10080b9a30 2024-09-12T12:20:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.843+0000 7f10077fe640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0ff40678c0 0x7f0ff4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.843+0000 7f10077fe640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0ff40678c0 0x7f0ff4069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8000c10 tx=0x7f0ff8000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:24.847+0000 7f0feffff640 1 -- 172.21.15.3:0/602071076 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f10000b8060 con 0x7f10080b9a30 2024-09-12T12:20:25.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.043+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f100806edc0 con 0x7f10080b9a30 2024-09-12T12:20:25.049 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:20:25.047+0000 7f0a5d435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:20:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.047+0000 7f0feffff640 1 -- 172.21.15.3:0/602071076 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f1000016490 con 0x7f10080b9a30 2024-09-12T12:20:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0ff40678c0 msgr2=0x7f0ff4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0ff40678c0 0x7f0ff4069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8000c10 tx=0x7f0ff8000ee0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10080b9a30 msgr2=0x7f10081f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10080b9a30 0x7f10081f1470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f100001cc60 tx=0x7f0ff0005840 comp rx=0 tx=0).stop 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 shutdown_connections 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0ff40678c0 0x7f0ff4069d80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10080b9a30 0x7f10081f1470 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10080b95b0 0x7f10080b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 --2- 172.21.15.3:0/602071076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff00051d0 0x7f100815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 >> 172.21.15.3:0/602071076 conn(0x7f100808c000 msgr2=0x7f10080b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 shutdown_connections 2024-09-12T12:20:25.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.051+0000 7f100e059640 1 -- 172.21.15.3:0/602071076 wait complete. 2024-09-12T12:20:25.065 INFO:tasks.ceph.mds.d.smithi142.stderr: -2299> 2024-09-12T12:19:54.243+0000 7f1933365640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:20:25.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:20:25.126 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:20:25.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc6230b2640 1 Processor -- start 2024-09-12T12:20:25.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc6230b2640 1 -- start start 2024-09-12T12:20:25.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc6230b2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc61c06f150 0x7fc61c0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:25.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc6230b2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c06f5b0 0x7fc61c0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:25.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc6230b2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61c06e0b0 con 0x7fc61c159dd0 2024-09-12T12:20:25.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc6230b2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc61c06e390 con 0x7fc61c06f150 2024-09-12T12:20:25.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc6230b2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c06e670 con 0x7fc61c06f5b0 2024-09-12T12:20:25.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc61bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc61c06f150 0x7fc61c0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:25.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc620e4e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc61c159dd0 0x7fc61c15d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34770/0 (socket says 172.21.15.3:34770) 2024-09-12T12:20:25.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc620e4e640 1 -- 172.21.15.3:0/2396460940 learned_addr learned my addr 172.21.15.3:0/2396460940 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:25.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc62164f640 1 --2- 172.21.15.3:0/2396460940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c06f5b0 0x7fc61c0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc61bfff640 1 -- 172.21.15.3:0/2396460940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c06f5b0 msgr2=0x7fc61c0ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc61bfff640 1 --2- 172.21.15.3:0/2396460940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c06f5b0 0x7fc61c0ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc61bfff640 1 -- 172.21.15.3:0/2396460940 >> v1:172.21.15.3:6789/0 conn(0x7fc61c159dd0 legacy=0x7fc61c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc61bfff640 1 -- 172.21.15.3:0/2396460940 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c06e950 con 0x7fc61c06f150 2024-09-12T12:20:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.191+0000 7fc61b7fe640 1 -- 172.21.15.3:0/2396460940 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069458299 0 0) 0x7fc61c06e0b0 con 0x7fc61c159dd0 2024-09-12T12:20:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc61bfff640 1 --2- 172.21.15.3:0/2396460940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c0ad060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6100097c0 tx=0x7fc61001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8270c3276abe10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc61b7fe640 1 -- 172.21.15.3:0/2396460940 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100b2070 con 0x7fc61c06f150 2024-09-12T12:20:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc61b7fe640 1 -- 172.21.15.3:0/2396460940 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6100ad680 con 0x7fc61c06f150 2024-09-12T12:20:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc61b7fe640 1 -- 172.21.15.3:0/2396460940 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100ada50 con 0x7fc61c06f150 2024-09-12T12:20:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 -- 172.21.15.3:0/2396460940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 msgr2=0x7fc61c0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2396460940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c0ad060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6100097c0 tx=0x7fc61001c470 comp rx=0 tx=0).stop 2024-09-12T12:20:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 -- 172.21.15.3:0/2396460940 shutdown_connections 2024-09-12T12:20:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2396460940 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c06f5b0 0x7fc61c0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2396460940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c0ad060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 -- 172.21.15.3:0/2396460940 >> 172.21.15.3:0/2396460940 conn(0x7fc61c0b1d80 msgr2=0x7fc61c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 -- 172.21.15.3:0/2396460940 shutdown_connections 2024-09-12T12:20:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 -- 172.21.15.3:0/2396460940 wait complete. 2024-09-12T12:20:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 Processor -- start 2024-09-12T12:20:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.195+0000 7fc6230b2640 1 -- start start 2024-09-12T12:20:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc61c06f5b0 0x7fc61c15e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c159dd0 0x7fc61c15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0b2bc0 con 0x7fc61c06f150 2024-09-12T12:20:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0b2a40 con 0x7fc61c06f5b0 2024-09-12T12:20:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c169200 con 0x7fc61c159dd0 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49786/0 (socket says 172.21.15.3:49786) 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 -- 172.21.15.3:0/2667814306 learned_addr learned my addr 172.21.15.3:0/2667814306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc62164f640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c159dd0 0x7fc61c15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc61bfff640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc61c06f5b0 0x7fc61c15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 -- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c159dd0 msgr2=0x7fc61c15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c159dd0 0x7fc61c15ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 -- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc61c06f5b0 msgr2=0x7fc61c15e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc61c06f5b0 0x7fc61c15e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 -- 172.21.15.3:0/2667814306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c0b0860 con 0x7fc61c06f150 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc620e4e640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c158aa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00c7f0 tx=0x7fc60c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6197fa640 1 -- 172.21.15.3:0/2667814306 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c018070 con 0x7fc61c06f150 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6197fa640 1 -- 172.21.15.3:0/2667814306 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc60c00a640 con 0x7fc61c06f150 2024-09-12T12:20:25.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c0b0b40 con 0x7fc61c06f150 2024-09-12T12:20:25.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6197fa640 1 -- 172.21.15.3:0/2667814306 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c00aa30 con 0x7fc61c06f150 2024-09-12T12:20:25.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c073cf0 con 0x7fc61c06f150 2024-09-12T12:20:25.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c15d2f0 con 0x7fc61c06f150 2024-09-12T12:20:25.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6197fa640 1 -- 172.21.15.3:0/2667814306 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc60c01b070 con 0x7fc61c06f150 2024-09-12T12:20:25.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6197fa640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6080678c0 0x7fc608069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:25.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc6197fa640 1 -- 172.21.15.3:0/2667814306 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7fc60c085380 con 0x7fc61c06f150 2024-09-12T12:20:25.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc61bfff640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6080678c0 0x7fc608069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:25.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.199+0000 7fc61bfff640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6080678c0 0x7fc608069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6100af000 tx=0x7fc610008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:25.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.203+0000 7fc6197fa640 1 -- 172.21.15.3:0/2667814306 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc60c055e50 con 0x7fc61c06f150 2024-09-12T12:20:25.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:25.407+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc61c15fd90 con 0x7fc61c06f150 2024-09-12T12:20:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.055+0000 7fc6197fa640 1 -- 172.21.15.3:0/2667814306 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fc60c055750 con 0x7fc61c06f150 2024-09-12T12:20:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6080678c0 msgr2=0x7fc608069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6080678c0 0x7fc608069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc6100af000 tx=0x7fc610008040 comp rx=0 tx=0).stop 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 msgr2=0x7fc61c158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c158aa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00c7f0 tx=0x7fc60c00ccb0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 shutdown_connections 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6080678c0 0x7fc608069d80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc61c159dd0 0x7fc61c15ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc61c06f5b0 0x7fc61c15e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 --2- 172.21.15.3:0/2667814306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc61c06f150 0x7fc61c158aa0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 >> 172.21.15.3:0/2667814306 conn(0x7fc61c0b1d80 msgr2=0x7fc61c0aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 shutdown_connections 2024-09-12T12:20:26.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.059+0000 7fc6230b2640 1 -- 172.21.15.3:0/2667814306 wait complete. 2024-09-12T12:20:26.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:20:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.219+0000 7fbc0ad09640 1 Processor -- start 2024-09-12T12:20:26.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.219+0000 7fbc0ad09640 1 -- start start 2024-09-12T12:20:26.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc0ad09640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040cff10 0x7fbc040d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc0ad09640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbc040cf640 0x7fbc040717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc0ad09640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc040d0830 con 0x7fbc04071d00 2024-09-12T12:20:26.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc0ad09640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbc040cf1b0 con 0x7fbc040cf640 2024-09-12T12:20:26.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc0ad09640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbc040721a0 con 0x7fbc040cff10 2024-09-12T12:20:26.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc092a6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbc04071d00 0x7fbc040c73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34772/0 (socket says 172.21.15.3:34772) 2024-09-12T12:20:26.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc092a6640 1 -- 172.21.15.3:0/3865126396 learned_addr learned my addr 172.21.15.3:0/3865126396 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671319599 0 0) 0x7fbc040d0830 con 0x7fbc04071d00 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3865126396 >> v2:172.21.15.142:3300/0 conn(0x7fbc040cf640 0x7fbc040717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbf4003530 con 0x7fbc04071d00 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc08aa5640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040cff10 0x7fbc040d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4106130084 0 0) 0x7fbbf4003530 con 0x7fbc04071d00 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040cff10 msgr2=0x7fbc040d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040cff10 0x7fbc040d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 >> v2:172.21.15.142:3300/0 conn(0x7fbc040cf640 msgr2=0x7fbc040717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 --2- 172.21.15.3:0/3865126396 >> v2:172.21.15.142:3300/0 conn(0x7fbc040cf640 0x7fbc040717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc04072480 con 0x7fbc04071d00 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fbbfc001a00 con 0x7fbc04071d00 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 >> v1:172.21.15.3:6789/0 conn(0x7fbc04071d00 legacy=0x7fbc040c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:26.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbf40039c0 0x7fbbf4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbbf4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf40051d0 0x7fbbf400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc040cf1b0 con 0x7fbbf4004810 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc040721a0 con 0x7fbbf40051d0 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf400c0a0 con 0x7fbbf40039c0 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc08aa5640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040cff10 0x7fbc040d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3865126396 >> v2:172.21.15.142:3300/0 conn(0x7fbc040cf640 0x7fbc040717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbbf4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc092a6640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf40051d0 0x7fbbf400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc08aa5640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbf40039c0 0x7fbbf4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 -- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbf40039c0 msgr2=0x7fbbf4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbf40039c0 0x7fbbf4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 -- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf40051d0 msgr2=0x7fbbf400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:26.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbf40051d0 0x7fbbf400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 -- 172.21.15.3:0/3865126396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf4003530 con 0x7fbbf4004810 2024-09-12T12:20:26.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc08aa5640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbf40039c0 0x7fbbf4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbbf4004c90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098a0 tx=0x7fbbf001c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bf90e5c487a9a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000f040 con 0x7fbbf4004810 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf00af070 con 0x7fbbf4004810 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.223+0000 7fbc037fe640 1 -- 172.21.15.3:0/3865126396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00aa650 con 0x7fbbf4004810 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3865126396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 msgr2=0x7fbbf4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 --2- 172.21.15.3:0/3865126396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbbf4004c90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098a0 tx=0x7fbbf001c610 comp rx=0 tx=0).stop 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc092a6640 1 -- 172.21.15.3:0/3865126396 reap_dead start 2024-09-12T12:20:26.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3865126396 shutdown_connections 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3865126396 >> 172.21.15.3:0/3865126396 conn(0x7fbc0406ffd0 msgr2=0x7fbc040703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3865126396 shutdown_connections 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3865126396 wait complete. 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 Processor -- start 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- start start 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbc040ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040c9410 0x7fbc04150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc040cbf10 0x7fbc041554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc04156c60 con 0x7fbbf4004810 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0415cb70 con 0x7fbc040cbf10 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbc040c7ae0 con 0x7fbc040c9410 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbc040ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:26.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbc040ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49802/0 (socket says 172.21.15.3:49802) 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 -- 172.21.15.3:0/3761717690 learned_addr learned my addr 172.21.15.3:0/3761717690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 -- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040c9410 msgr2=0x7fbc04150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040c9410 0x7fbc04150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 -- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc040cbf10 msgr2=0x7fbc041554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc040cbf10 0x7fbc041554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 -- 172.21.15.3:0/3761717690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc040cae70 con 0x7fbbf4004810 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc08aa5640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbc040ca480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800f4c0 tx=0x7fbbf800f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc017fa640 1 -- 172.21.15.3:0/3761717690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8018020 con 0x7fbbf4004810 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc017fa640 1 -- 172.21.15.3:0/3761717690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf8014890 con 0x7fbbf4004810 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc040cb150 con 0x7fbbf4004810 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc017fa640 1 -- 172.21.15.3:0/3761717690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8014c40 con 0x7fbbf4004810 2024-09-12T12:20:26.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.227+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc040cb710 con 0x7fbbf4004810 2024-09-12T12:20:26.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.231+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc040d0700 con 0x7fbbf4004810 2024-09-12T12:20:26.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.231+0000 7fbc017fa640 1 -- 172.21.15.3:0/3761717690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbf801b020 con 0x7fbbf4004810 2024-09-12T12:20:26.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.231+0000 7fbc017fa640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbe80678c0 0x7fbbe8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:26.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.231+0000 7fbc017fa640 1 -- 172.21.15.3:0/3761717690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7fbbf80861a0 con 0x7fbbf4004810 2024-09-12T12:20:26.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.231+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbe80678c0 0x7fbbe8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:26.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.231+0000 7fbc03fff640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbe80678c0 0x7fbbe8069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00ae000 tx=0x7fbbf0004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:26.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.235+0000 7fbc017fa640 1 -- 172.21.15.3:0/3761717690 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbbf8022020 con 0x7fbbf4004810 2024-09-12T12:20:26.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:26.399+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbc040c77e0 con 0x7fbbf4004810 2024-09-12T12:20:27.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.059+0000 7fbc017fa640 1 -- 172.21.15.3:0/3761717690 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7fbbf8024300 con 0x7fbbf4004810 2024-09-12T12:20:27.063 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbe80678c0 msgr2=0x7fbbe8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbe80678c0 0x7fbbe8069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00ae000 tx=0x7fbbf0004740 comp rx=0 tx=0).stop 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 msgr2=0x7fbc040ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbc040ca480 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800f4c0 tx=0x7fbbf800f980 comp rx=0 tx=0).stop 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 shutdown_connections 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbe80678c0 0x7fbbe8069d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc040cbf10 0x7fbc041554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbc040c9410 0x7fbc04150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 --2- 172.21.15.3:0/3761717690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbf4004810 0x7fbc040ca480 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 >> 172.21.15.3:0/3761717690 conn(0x7fbc0406ffd0 msgr2=0x7fbc040750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:27.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 shutdown_connections 2024-09-12T12:20:27.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.063+0000 7fbc0ad09640 1 -- 172.21.15.3:0/3761717690 wait complete. 2024-09-12T12:20:27.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:20:27.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59e4d6f640 1 Processor -- start 2024-09-12T12:20:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59e4d6f640 1 -- start start 2024-09-12T12:20:27.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59e4d6f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f59e006dc80 0x7f59e00720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59e4d6f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59e006d090 0x7f59e00d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59e4d6f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59e006e060 con 0x7f59e00d2750 2024-09-12T12:20:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59e4d6f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f59e00d2ba0 con 0x7f59e006d090 2024-09-12T12:20:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59e4d6f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f59e0162e60 con 0x7f59e006dc80 2024-09-12T12:20:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59ddd9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59e006d090 0x7f59e00d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59ded9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59e00d2750 0x7f59e00cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34774/0 (socket says 172.21.15.3:34774) 2024-09-12T12:20:27.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59ded9d640 1 -- 172.21.15.3:0/3543540655 learned_addr learned my addr 172.21.15.3:0/3543540655 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59de59c640 1 --2- 172.21.15.3:0/3543540655 >> v2:172.21.15.142:3300/0 conn(0x7f59e006dc80 0x7f59e00720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59ddd9b640 1 -- 172.21.15.3:0/3543540655 >> v2:172.21.15.142:3300/0 conn(0x7f59e006dc80 msgr2=0x7f59e00720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59ddd9b640 1 --2- 172.21.15.3:0/3543540655 >> v2:172.21.15.142:3300/0 conn(0x7f59e006dc80 0x7f59e00720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59ddd9b640 1 -- 172.21.15.3:0/3543540655 >> v1:172.21.15.3:6789/0 conn(0x7f59e00d2750 legacy=0x7f59e00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59ddd9b640 1 -- 172.21.15.3:0/3543540655 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e0162250 con 0x7f59e006d090 2024-09-12T12:20:27.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.271+0000 7f59dd59a640 1 -- 172.21.15.3:0/3543540655 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297310974 0 0) 0x7f59e006e060 con 0x7f59e00d2750 2024-09-12T12:20:27.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59ddd9b640 1 --2- 172.21.15.3:0/3543540655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00d2210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59d40097c0 tx=0x7f59d401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63a580f47f0cf37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59dd59a640 1 -- 172.21.15.3:0/3543540655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b0070 con 0x7f59e006d090 2024-09-12T12:20:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59dd59a640 1 -- 172.21.15.3:0/3543540655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d40ab600 con 0x7f59e006d090 2024-09-12T12:20:27.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59dd59a640 1 -- 172.21.15.3:0/3543540655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40ab9d0 con 0x7f59e006d090 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- 172.21.15.3:0/3543540655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 msgr2=0x7f59e00d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/3543540655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00d2210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59d40097c0 tx=0x7f59d401c460 comp rx=0 tx=0).stop 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- 172.21.15.3:0/3543540655 shutdown_connections 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/3543540655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00d2210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/3543540655 >> v2:172.21.15.142:3300/0 conn(0x7f59e006dc80 0x7f59e00720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- 172.21.15.3:0/3543540655 >> 172.21.15.3:0/3543540655 conn(0x7f59e0070060 msgr2=0x7f59e0070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- 172.21.15.3:0/3543540655 shutdown_connections 2024-09-12T12:20:27.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- 172.21.15.3:0/3543540655 wait complete. 2024-09-12T12:20:27.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 Processor -- start 2024-09-12T12:20:27.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- start start 2024-09-12T12:20:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e006dc80 0x7f59e00aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:27.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e00d2750 0x7f59e00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59e006eba0 con 0x7f59e006d090 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59e006ea20 con 0x7f59e006dc80 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59e4d6f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f59e0072610 con 0x7f59e00d2750 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59de59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59de59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49820/0 (socket says 172.21.15.3:49820) 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59de59c640 1 -- 172.21.15.3:0/2192184683 learned_addr learned my addr 172.21.15.3:0/2192184683 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59ddd9b640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e006dc80 0x7f59e00aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.275+0000 7f59ded9d640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e00d2750 0x7f59e00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59de59c640 1 -- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e00d2750 msgr2=0x7f59e00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59de59c640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e00d2750 0x7f59e00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59de59c640 1 -- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e006dc80 msgr2=0x7f59e00aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59de59c640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e006dc80 0x7f59e00aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:27.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59de59c640 1 -- 172.21.15.3:0/2192184683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e00adf40 con 0x7f59e006d090 2024-09-12T12:20:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59de59c640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00ad8f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d830 tx=0x7f59d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ceffd640 1 -- 172.21.15.3:0/2192184683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d000b6c0 con 0x7f59e006d090 2024-09-12T12:20:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ceffd640 1 -- 172.21.15.3:0/2192184683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d0019070 con 0x7f59e006d090 2024-09-12T12:20:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e00ae220 con 0x7f59e006d090 2024-09-12T12:20:27.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ceffd640 1 -- 172.21.15.3:0/2192184683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d000ba60 con 0x7f59e006d090 2024-09-12T12:20:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e00ae7e0 con 0x7f59e006d090 2024-09-12T12:20:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e00d2210 con 0x7f59e006d090 2024-09-12T12:20:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ceffd640 1 -- 172.21.15.3:0/2192184683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f59d000bc00 con 0x7f59e006d090 2024-09-12T12:20:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ceffd640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59c00678c0 0x7f59c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ceffd640 1 -- 172.21.15.3:0/2192184683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7f59d0086020 con 0x7f59e006d090 2024-09-12T12:20:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ddd9b640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59c00678c0 0x7f59c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:27.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.279+0000 7f59ddd9b640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59c00678c0 0x7f59c0069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ad000 tx=0x7f59d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:27.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.283+0000 7f59ceffd640 1 -- 172.21.15.3:0/2192184683 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f59d0056cc0 con 0x7f59e006d090 2024-09-12T12:20:27.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:27.447+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f59e00d1280 con 0x7f59e006d090 2024-09-12T12:20:28.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.059+0000 7f59ceffd640 1 -- 172.21.15.3:0/2192184683 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f59d00565c0 con 0x7f59e006d090 2024-09-12T12:20:28.066 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T12:20:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59c00678c0 msgr2=0x7f59c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59c00678c0 0x7f59c0069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ad000 tx=0x7f59d4008040 comp rx=0 tx=0).stop 2024-09-12T12:20:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 msgr2=0x7f59e00ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00ad8f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d830 tx=0x7f59d000dcf0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 shutdown_connections 2024-09-12T12:20:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59c00678c0 0x7f59c0069d80 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f59d40ad000 tx=0x7f59d4008040 comp rx=0 tx=0).stop 2024-09-12T12:20:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59e00d2750 0x7f59e00c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59e006dc80 0x7f59e00aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 --2- 172.21.15.3:0/2192184683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e006d090 0x7f59e00ad8f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 >> 172.21.15.3:0/2192184683 conn(0x7f59e0070060 msgr2=0x7f59e0160130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:28.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 shutdown_connections 2024-09-12T12:20:28.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.067+0000 7f59e4d6f640 1 -- 172.21.15.3:0/2192184683 wait complete. 2024-09-12T12:20:28.096 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:20:28.096 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T12:20:28.096 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:20:28.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:28.096 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:20:28.096 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T12:20:28.097 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:20:28.097 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:28.097 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:20:28.097 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T12:20:28.097 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:20:28.097 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:28.097 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:20:28.098 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T12:20:28.098 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:20:28.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:28.110 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:20:28.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:20:28.111 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:20:28.112 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:20:28.112 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:20:28.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:20:28.117 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:20:28.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:20:28.155 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:20:28.155 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:28.155 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:20:28.156 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:28.161 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:20:28.161 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:28.163 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:20:28.163 DEBUG:teuthology.parallel:result is None 2024-09-12T12:20:28.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T12:20:28.223 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:20:28.224 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:20:28.233 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:20:28.234 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:20:28.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.283+0000 7f87985da640 1 Processor -- start 2024-09-12T12:20:28.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.283+0000 7f87985da640 1 -- start start 2024-09-12T12:20:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.283+0000 7f87985da640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8790159e20 0x7f879015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.283+0000 7f87985da640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 0x7f87900d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.283+0000 7f87985da640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879015a200 con 0x7f879015d9e0 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.283+0000 7f87985da640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f87900d1220 con 0x7f8790159e20 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.283+0000 7f87985da640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f879015ce30 con 0x7f879015de20 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795b75640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f879015d9e0 0x7f87900d16b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51676/0 (socket says 172.21.15.3:51676) 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795b75640 1 -- 172.21.15.3:0/2982663934 learned_addr learned my addr 172.21.15.3:0/2982663934 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796b77640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 0x7f87900d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 --2- 172.21.15.3:0/2982663934 >> v2:172.21.15.142:3300/0 conn(0x7f8790159e20 0x7f879015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283694658 0 0) 0x7f879015a200 con 0x7f879015d9e0 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f877c003530 con 0x7f879015d9e0 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3485663601 0 0) 0x7f877c003530 con 0x7f879015d9e0 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 msgr2=0x7f87900d1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 0x7f87900d1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 >> v2:172.21.15.142:3300/0 conn(0x7f8790159e20 msgr2=0x7f879015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 --2- 172.21.15.3:0/2982663934 >> v2:172.21.15.142:3300/0 conn(0x7f8790159e20 0x7f879015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879015d110 con 0x7f879015d9e0 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f8780001680 con 0x7f879015d9e0 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 --2- 172.21.15.3:0/2982663934 >> v2:172.21.15.142:3300/0 conn(0x7f8790159e20 0x7f879015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 >> v1:172.21.15.3:6789/0 conn(0x7f879015d9e0 legacy=0x7f87900d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f877c0039c0 0x7f877c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c004810 0x7f877c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 0x7f877c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87900d1220 con 0x7f877c0039c0 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f879015ce30 con 0x7f877c004810 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f877c00b9d0 con 0x7f879015de20 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f877c0039c0 0x7f877c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795b75640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c004810 0x7f877c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796b77640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 0x7f877c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 -- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 msgr2=0x7f877c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f879015de20 0x7f877c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 -- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c004810 msgr2=0x7f877c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f877c004810 0x7f877c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 -- 172.21.15.3:0/2982663934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c003530 con 0x7f877c0039c0 2024-09-12T12:20:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796376640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f877c0039c0 0x7f877c003de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f878c009ce0 tx=0x7f878c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557d6922630e69c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0b2070 con 0x7f877c0039c0 2024-09-12T12:20:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c0a5710 con 0x7f877c0039c0 2024-09-12T12:20:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8795374640 1 -- 172.21.15.3:0/2982663934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c009880 con 0x7f877c0039c0 2024-09-12T12:20:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f87985da640 1 -- 172.21.15.3:0/2982663934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f877c0039c0 msgr2=0x7f877c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f87985da640 1 --2- 172.21.15.3:0/2982663934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f877c0039c0 0x7f877c003de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f878c009ce0 tx=0x7f878c01cc40 comp rx=0 tx=0).stop 2024-09-12T12:20:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.287+0000 7f8796b77640 1 -- 172.21.15.3:0/2982663934 reap_dead start 2024-09-12T12:20:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- 172.21.15.3:0/2982663934 shutdown_connections 2024-09-12T12:20:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- 172.21.15.3:0/2982663934 >> 172.21.15.3:0/2982663934 conn(0x7f879009aa70 msgr2=0x7f87900b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- 172.21.15.3:0/2982663934 shutdown_connections 2024-09-12T12:20:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- 172.21.15.3:0/2982663934 wait complete. 2024-09-12T12:20:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 Processor -- start 2024-09-12T12:20:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- start start 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f877c0039c0 0x7f87900cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f879015e250 0x7f8790147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87900d0350 0x7f87900b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f879006c650 con 0x7f87900d0350 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f879006c4d0 con 0x7f879015e250 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f879006c7d0 con 0x7f877c0039c0 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87900d0350 0x7f87900b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87900d0350 0x7f87900b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36276/0 (socket says 172.21.15.3:36276) 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 -- 172.21.15.3:0/3302651143 learned_addr learned my addr 172.21.15.3:0/3302651143 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8795b75640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f879015e250 0x7f8790147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796376640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f877c0039c0 0x7f87900cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 -- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f877c0039c0 msgr2=0x7f87900cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f877c0039c0 0x7f87900cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 -- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f879015e250 msgr2=0x7f8790147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f879015e250 0x7f8790147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 -- 172.21.15.3:0/3302651143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900cf2b0 con 0x7f87900d0350 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8796b77640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87900d0350 0x7f87900b0a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8790074220 tx=0x7f87880115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8786ffd640 1 -- 172.21.15.3:0/3302651143 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788011e10 con 0x7f87900d0350 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8786ffd640 1 -- 172.21.15.3:0/3302651143 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878801b070 con 0x7f87900d0350 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f8786ffd640 1 -- 172.21.15.3:0/3302651143 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87880039a0 con 0x7f87900d0350 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87900cf590 con 0x7f87900d0350 2024-09-12T12:20:28.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.291+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87900cfb50 con 0x7f87900d0350 2024-09-12T12:20:28.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.295+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879015c820 con 0x7f87900d0350 2024-09-12T12:20:28.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.295+0000 7f8786ffd640 1 -- 172.21.15.3:0/3302651143 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8788003b40 con 0x7f87900d0350 2024-09-12T12:20:28.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.295+0000 7f8786ffd640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87740678c0 0x7f8774069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:28.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.295+0000 7f8786ffd640 1 -- 172.21.15.3:0/3302651143 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f87880861f0 con 0x7f87900d0350 2024-09-12T12:20:28.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.295+0000 7f8796376640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87740678c0 0x7f8774069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:28.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.295+0000 7f8796376640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87740678c0 0x7f8774069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f878c0af000 tx=0x7f878c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:28.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.299+0000 7f8786ffd640 1 -- 172.21.15.3:0/3302651143 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8788025020 con 0x7f87900d0350 2024-09-12T12:20:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:28.474+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f87900aec00 con 0x7f87900d0350 2024-09-12T12:20:29.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f8786ffd640 1 -- 172.21.15.3:0/3302651143 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f8788057030 con 0x7f87900d0350 2024-09-12T12:20:29.097 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87740678c0 msgr2=0x7f8774069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87740678c0 0x7f8774069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f878c0af000 tx=0x7f878c0046d0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87900d0350 msgr2=0x7f87900b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87900d0350 0x7f87900b0a10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8790074220 tx=0x7f87880115a0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 shutdown_connections 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87740678c0 0x7f8774069d80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87900d0350 0x7f87900b0a10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f879015e250 0x7f8790147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 --2- 172.21.15.3:0/3302651143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f877c0039c0 0x7f87900cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 >> 172.21.15.3:0/3302651143 conn(0x7f879009aa70 msgr2=0x7f8790070390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:29.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 shutdown_connections 2024-09-12T12:20:29.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.094+0000 7f87985da640 1 -- 172.21.15.3:0/3302651143 wait complete. 2024-09-12T12:20:29.125 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T12:20:29.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ef2e640 1 Processor -- start 2024-09-12T12:20:29.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ef2e640 1 -- start start 2024-09-12T12:20:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ef2e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed5815e5d0 0x7fed580ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ef2e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fed580d5050 0x7fed580cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ef2e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed580cfe20 con 0x7fed580cfa10 2024-09-12T12:20:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ef2e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fed580b1060 con 0x7fed5815e5d0 2024-09-12T12:20:29.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ef2e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fed580d31a0 con 0x7fed580d5050 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ccca640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fed5815e5d0 0x7fed580ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5d4cb640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed580cfa10 0x7fed58162280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51688/0 (socket says 172.21.15.3:51688) 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5d4cb640 1 -- 172.21.15.3:0/2116050941 learned_addr learned my addr 172.21.15.3:0/2116050941 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed57fff640 1 --2- 172.21.15.3:0/2116050941 >> v2:172.21.15.142:3300/0 conn(0x7fed580d5050 0x7fed580cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ccca640 1 -- 172.21.15.3:0/2116050941 >> v2:172.21.15.142:3300/0 conn(0x7fed580d5050 msgr2=0x7fed580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ccca640 1 --2- 172.21.15.3:0/2116050941 >> v2:172.21.15.142:3300/0 conn(0x7fed580d5050 0x7fed580cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ccca640 1 -- 172.21.15.3:0/2116050941 >> v1:172.21.15.3:6789/0 conn(0x7fed580cfa10 legacy=0x7fed58162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed5ccca640 1 -- 172.21.15.3:0/2116050941 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed580d3480 con 0x7fed5815e5d0 2024-09-12T12:20:29.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.250+0000 7fed577fe640 1 -- 172.21.15.3:0/2116050941 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651446211 0 0) 0x7fed500014a0 con 0x7fed580cfa10 2024-09-12T12:20:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ccca640 1 --2- 172.21.15.3:0/2116050941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580ceb60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fed580c7ff0 tx=0x7fed4801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56754c8504118cba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed577fe640 1 -- 172.21.15.3:0/2116050941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed480b1070 con 0x7fed5815e5d0 2024-09-12T12:20:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed577fe640 1 -- 172.21.15.3:0/2116050941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed480ac630 con 0x7fed5815e5d0 2024-09-12T12:20:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed577fe640 1 -- 172.21.15.3:0/2116050941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed480a8050 con 0x7fed5815e5d0 2024-09-12T12:20:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 -- 172.21.15.3:0/2116050941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 msgr2=0x7fed580ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/2116050941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580ceb60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fed580c7ff0 tx=0x7fed4801c3f0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 -- 172.21.15.3:0/2116050941 shutdown_connections 2024-09-12T12:20:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/2116050941 >> v2:172.21.15.142:3300/0 conn(0x7fed580d5050 0x7fed580cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/2116050941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580ceb60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 -- 172.21.15.3:0/2116050941 >> 172.21.15.3:0/2116050941 conn(0x7fed5808c000 msgr2=0x7fed580d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 -- 172.21.15.3:0/2116050941 shutdown_connections 2024-09-12T12:20:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.254+0000 7fed5ef2e640 1 -- 172.21.15.3:0/2116050941 wait complete. 2024-09-12T12:20:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 Processor -- start 2024-09-12T12:20:29.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 -- start start 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed580cfa10 0x7fed580b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed580d5050 0x7fed580cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed5816b5c0 con 0x7fed5815e5d0 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed5815ebc0 con 0x7fed580d5050 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fed5816b740 con 0x7fed580cfa10 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36292/0 (socket says 172.21.15.3:36292) 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 -- 172.21.15.3:0/1645339683 learned_addr learned my addr 172.21.15.3:0/1645339683 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ccca640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed580cfa10 0x7fed580b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed57fff640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed580d5050 0x7fed580cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 -- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed580cfa10 msgr2=0x7fed580b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed580cfa10 0x7fed580b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 -- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed580d5050 msgr2=0x7fed580cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed580d5050 0x7fed580cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 -- 172.21.15.3:0/1645339683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5814dd90 con 0x7fed5815e5d0 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ccca640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed580cfa10 0x7fed580b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed57fff640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed580d5050 0x7fed580cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5d4cb640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580cbb20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fed5000c780 tx=0x7fed5000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed557fa640 1 -- 172.21.15.3:0/1645339683 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50018070 con 0x7fed5815e5d0 2024-09-12T12:20:29.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed557fa640 1 -- 172.21.15.3:0/1645339683 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed50004d70 con 0x7fed5815e5d0 2024-09-12T12:20:29.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5814e070 con 0x7fed5815e5d0 2024-09-12T12:20:29.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed557fa640 1 -- 172.21.15.3:0/1645339683 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5000a880 con 0x7fed5815e5d0 2024-09-12T12:20:29.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed58155b20 con 0x7fed5815e5d0 2024-09-12T12:20:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed58155d20 con 0x7fed5815e5d0 2024-09-12T12:20:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed557fa640 1 -- 172.21.15.3:0/1645339683 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fed5001b070 con 0x7fed5815e5d0 2024-09-12T12:20:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed557fa640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed440678c0 0x7fed44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.258+0000 7fed557fa640 1 -- 172.21.15.3:0/1645339683 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fed50084f50 con 0x7fed5815e5d0 2024-09-12T12:20:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.262+0000 7fed5ccca640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed440678c0 0x7fed44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.262+0000 7fed5ccca640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed440678c0 0x7fed44069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed480ae000 tx=0x7fed48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:29.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.266+0000 7fed557fa640 1 -- 172.21.15.3:0/1645339683 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fed58155d20 con 0x7fed5815e5d0 2024-09-12T12:20:29.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.486+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fed580cf4d0 con 0x7fed5815e5d0 2024-09-12T12:20:29.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed557fa640 1 -- 172.21.15.3:0/1645339683 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fed580cfe20 con 0x7fed5815e5d0 2024-09-12T12:20:29.493 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:29.493 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:20:29.493 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg==", 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.494 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA==", 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg==", 2024-09-12T12:20:29.495 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw==", 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:20:29.496 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w==", 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.497 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA==", 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw==", 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.498 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg==", 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.499 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA==", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ==", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.500 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ==", 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.501 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA==", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA==", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:20:29.502 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw==", 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.503 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw==", 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA==", 2024-09-12T12:20:29.504 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKNVOAxAAMmejHiBys/ZYsGdB0oiroQ==", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mds", 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBM3OJmMJwWOhAAgn9I9KumbgWyIXTG2bSWZQ==", 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mds" 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-mgr", 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBM3OJmerEWOhAAzzZJ0pdNyuwZ77e7sVnO5Q==", 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.506 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-osd", 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBM3OJm/cIWOhAA2rH7gIdJmgfS+RgQcz94hw==", 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-osd" 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd", 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBM3OJmktMWOhAASYFgGCFuI+dme4NBIMDePw==", 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.507 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBM3OJmuuQWOhAAURntaT3i7DtshjVuwHCd7w==", 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.bootstrap-rgw", 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBM3OJmr/YWOhAABO8tUgVZTXNG/4SS1oiH/Q==", 2024-09-12T12:20:29.508 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA==", 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:29.509 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg==", 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:20:29.510 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:20:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed440678c0 msgr2=0x7fed44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed440678c0 0x7fed44069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fed480ae000 tx=0x7fed48008040 comp rx=0 tx=0).stop 2024-09-12T12:20:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 msgr2=0x7fed580cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580cbb20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fed5000c780 tx=0x7fed5000cc40 comp rx=0 tx=0).stop 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 shutdown_connections 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed440678c0 0x7fed44069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed5815e5d0 0x7fed580cbb20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed580d5050 0x7fed580cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 --2- 172.21.15.3:0/1645339683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed580cfa10 0x7fed580b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 >> 172.21.15.3:0/1645339683 conn(0x7fed5808c000 msgr2=0x7fed580b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 shutdown_connections 2024-09-12T12:20:29.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.490+0000 7fed5ef2e640 1 -- 172.21.15.3:0/1645339683 wait complete. 2024-09-12T12:20:29.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-12T12:20:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.634+0000 7fc0ba725640 1 Processor -- start 2024-09-12T12:20:29.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.634+0000 7fc0ba725640 1 -- start start 2024-09-12T12:20:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc0b41568f0 0x7fc0b40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0b40d0380 0x7fc0b40d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b40afb00 con 0x7fc0b40b47a0 2024-09-12T12:20:29.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b40d0cf0 con 0x7fc0b40d0380 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b40b31b0 con 0x7fc0b41568f0 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0b40d0380 0x7fc0b40d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc0b40d0380 0x7fc0b40d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36298/0 (socket says 172.21.15.3:36298) 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/3781173188 learned_addr learned my addr 172.21.15.3:0/3781173188 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b3fff640 1 --2- 172.21.15.3:0/3781173188 >> v2:172.21.15.142:3300/0 conn(0x7fc0b41568f0 0x7fc0b40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/3781173188 >> v2:172.21.15.142:3300/0 conn(0x7fc0b41568f0 msgr2=0x7fc0b40aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 --2- 172.21.15.3:0/3781173188 >> v2:172.21.15.142:3300/0 conn(0x7fc0b41568f0 0x7fc0b40aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/3781173188 >> v1:172.21.15.142:6789/0 conn(0x7fc0b40b47a0 legacy=0x7fc0b40b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/3781173188 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b40af540 con 0x7fc0b40d0380 2024-09-12T12:20:29.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b8cc2640 1 --2- 172.21.15.3:0/3781173188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b40d0380 0x7fc0b40d07b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0b406cc60 tx=0x7fc0ac01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62de1e663dfac50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b2ffd640 1 -- 172.21.15.3:0/3781173188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0a55e0 con 0x7fc0b40d0380 2024-09-12T12:20:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b2ffd640 1 -- 172.21.15.3:0/3781173188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0a5780 con 0x7fc0b40d0380 2024-09-12T12:20:29.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0b2ffd640 1 -- 172.21.15.3:0/3781173188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0a5b50 con 0x7fc0b40d0380 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 -- 172.21.15.3:0/3781173188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b40d0380 msgr2=0x7fc0b40d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 --2- 172.21.15.3:0/3781173188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b40d0380 0x7fc0b40d07b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc0b406cc60 tx=0x7fc0ac01c920 comp rx=0 tx=0).stop 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 -- 172.21.15.3:0/3781173188 shutdown_connections 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 --2- 172.21.15.3:0/3781173188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b40d0380 0x7fc0b40d07b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 --2- 172.21.15.3:0/3781173188 >> v2:172.21.15.142:3300/0 conn(0x7fc0b41568f0 0x7fc0b40aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.638+0000 7fc0ba725640 1 -- 172.21.15.3:0/3781173188 >> 172.21.15.3:0/3781173188 conn(0x7fc0b406be90 msgr2=0x7fc0b406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- 172.21.15.3:0/3781173188 shutdown_connections 2024-09-12T12:20:29.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- 172.21.15.3:0/3781173188 wait complete. 2024-09-12T12:20:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 Processor -- start 2024-09-12T12:20:29.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- start start 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b40b47a0 0x7fc0b414f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0b40d0380 0x7fc0b4073860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b41568f0 0x7fc0b409afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b415d220 con 0x7fc0b41568f0 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b415d0a0 con 0x7fc0b40b47a0 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b415d3a0 con 0x7fc0b40d0380 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b41568f0 0x7fc0b409afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b41568f0 0x7fc0b409afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36310/0 (socket says 172.21.15.3:36310) 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/1941917482 learned_addr learned my addr 172.21.15.3:0/1941917482 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:29.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b3fff640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b40b47a0 0x7fc0b414f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b37fe640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0b40d0380 0x7fc0b4073860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0b40d0380 msgr2=0x7fc0b4073860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0b40d0380 0x7fc0b4073860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b40b47a0 msgr2=0x7fc0b414f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b40b47a0 0x7fc0b414f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 -- 172.21.15.3:0/1941917482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b4073da0 con 0x7fc0b41568f0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b8cc2640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b41568f0 0x7fc0b409afd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0af040 tx=0x7fc0ac0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b0ff9640 1 -- 172.21.15.3:0/1941917482 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac009510 con 0x7fc0b41568f0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b0ff9640 1 -- 172.21.15.3:0/1941917482 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0be070 con 0x7fc0b41568f0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b414ff20 con 0x7fc0b41568f0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0b0ff9640 1 -- 172.21.15.3:0/1941917482 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0b88d0 con 0x7fc0b41568f0 2024-09-12T12:20:29.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b414d770 con 0x7fc0b41568f0 2024-09-12T12:20:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.642+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b40d07b0 con 0x7fc0b41568f0 2024-09-12T12:20:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.646+0000 7fc0b0ff9640 1 -- 172.21.15.3:0/1941917482 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc0ac0b8a70 con 0x7fc0b41568f0 2024-09-12T12:20:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.646+0000 7fc0b0ff9640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0a00678c0 0x7fc0a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.646+0000 7fc0b0ff9640 1 -- 172.21.15.3:0/1941917482 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fc0ac1209d0 con 0x7fc0b41568f0 2024-09-12T12:20:29.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.646+0000 7fc0b3fff640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0a00678c0 0x7fc0a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:29.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.646+0000 7fc0b3fff640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0a00678c0 0x7fc0a0069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b9d0 tx=0x7fc0a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:29.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.650+0000 7fc0b0ff9640 1 -- 172.21.15.3:0/1941917482 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc0ac0165a0 con 0x7fc0b41568f0 2024-09-12T12:20:29.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.870+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc0b40d07b0 con 0x7fc0b41568f0 2024-09-12T12:20:29.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.874+0000 7fc0b0ff9640 1 -- 172.21.15.3:0/1941917482 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fc0ac0f17d0 con 0x7fc0b41568f0 2024-09-12T12:20:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0a00678c0 msgr2=0x7fc0a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0a00678c0 0x7fc0a0069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400b9d0 tx=0x7fc0a4008040 comp rx=0 tx=0).stop 2024-09-12T12:20:29.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b41568f0 msgr2=0x7fc0b409afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b41568f0 0x7fc0b409afd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0af040 tx=0x7fc0ac0196c0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 shutdown_connections 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0a00678c0 0x7fc0a0069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc0b41568f0 0x7fc0b409afd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc0b40d0380 0x7fc0b4073860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 --2- 172.21.15.3:0/1941917482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0b40b47a0 0x7fc0b414f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 >> 172.21.15.3:0/1941917482 conn(0x7fc0b406be90 msgr2=0x7fc0b415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 shutdown_connections 2024-09-12T12:20:29.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:29.878+0000 7fc0ba725640 1 -- 172.21.15.3:0/1941917482 wait complete. 2024-09-12T12:20:29.897 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-12T12:20:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d52441640 1 Processor -- start 2024-09-12T12:20:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d52441640 1 -- start start 2024-09-12T12:20:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d52441640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d4c0d4c00 0x7f6d4c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d52441640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6d4c0cfa90 0x7f6d4c0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d52441640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d4c0d5170 con 0x7f6d4c15e7e0 2024-09-12T12:20:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d52441640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4c0b6ac0 con 0x7f6d4c0d4c00 2024-09-12T12:20:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d52441640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4c0b1a70 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d4c15e7e0 0x7f6d4c0cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51690/0 (socket says 172.21.15.3:51690) 2024-09-12T12:20:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4bfff640 1 -- 172.21.15.3:0/1153817074 learned_addr learned my addr 172.21.15.3:0/1153817074 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4b7fe640 1 --2- 172.21.15.3:0/1153817074 >> v2:172.21.15.3:3300/0 conn(0x7f6d4c0d4c00 0x7f6d4c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d509de640 1 --2- 172.21.15.3:0/1153817074 >> v2:172.21.15.142:3300/0 conn(0x7f6d4c0cfa90 0x7f6d4c0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4affd640 1 -- 172.21.15.3:0/1153817074 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141719709 0 0) 0x7f6d4c0d5170 con 0x7f6d4c15e7e0 2024-09-12T12:20:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4affd640 1 -- 172.21.15.3:0/1153817074 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d34003530 con 0x7f6d4c15e7e0 2024-09-12T12:20:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4b7fe640 1 -- 172.21.15.3:0/1153817074 >> v2:172.21.15.142:3300/0 conn(0x7f6d4c0cfa90 msgr2=0x7f6d4c0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4affd640 1 -- 172.21.15.3:0/1153817074 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3747139087 0 0) 0x7f6d34003530 con 0x7f6d4c15e7e0 2024-09-12T12:20:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4b7fe640 1 --2- 172.21.15.3:0/1153817074 >> v2:172.21.15.142:3300/0 conn(0x7f6d4c0cfa90 0x7f6d4c0b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4b7fe640 1 -- 172.21.15.3:0/1153817074 >> v1:172.21.15.3:6789/0 conn(0x7f6d4c15e7e0 legacy=0x7f6d4c0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.006+0000 7f6d4b7fe640 1 -- 172.21.15.3:0/1153817074 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c0b1cc0 con 0x7f6d4c0d4c00 2024-09-12T12:20:30.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d4b7fe640 1 --2- 172.21.15.3:0/1153817074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0d4c00 0x7f6d4c0cf550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cb550 tx=0x7f6d4001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644512d6d49ce499 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d4affd640 1 -- 172.21.15.3:0/1153817074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d400b0070 con 0x7f6d4c0d4c00 2024-09-12T12:20:30.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d4affd640 1 -- 172.21.15.3:0/1153817074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d400ab400 con 0x7f6d4c0d4c00 2024-09-12T12:20:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d4affd640 1 -- 172.21.15.3:0/1153817074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c001680 con 0x7f6d4c0d4c00 2024-09-12T12:20:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 -- 172.21.15.3:0/1153817074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0d4c00 msgr2=0x7f6d4c0cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 --2- 172.21.15.3:0/1153817074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0d4c00 0x7f6d4c0cf550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0cb550 tx=0x7f6d4001c3a0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 -- 172.21.15.3:0/1153817074 shutdown_connections 2024-09-12T12:20:30.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 --2- 172.21.15.3:0/1153817074 >> v2:172.21.15.142:3300/0 conn(0x7f6d4c0cfa90 0x7f6d4c0b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 --2- 172.21.15.3:0/1153817074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0d4c00 0x7f6d4c0cf550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 -- 172.21.15.3:0/1153817074 >> 172.21.15.3:0/1153817074 conn(0x7f6d4c08c000 msgr2=0x7f6d4c0d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 -- 172.21.15.3:0/1153817074 shutdown_connections 2024-09-12T12:20:30.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.010+0000 7f6d52441640 1 -- 172.21.15.3:0/1153817074 wait complete. 2024-09-12T12:20:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 Processor -- start 2024-09-12T12:20:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 -- start start 2024-09-12T12:20:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0cfa90 0x7f6d4c15d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d4c0d4c00 0x7f6d4c161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d4c15e7e0 0x7f6d4c0b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c0b3670 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c0b34f0 con 0x7f6d4c15e7e0 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4c0b37f0 con 0x7f6d4c0d4c00 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0cfa90 0x7f6d4c15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0cfa90 0x7f6d4c15d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36338/0 (socket says 172.21.15.3:36338) 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 -- 172.21.15.3:0/3137914246 learned_addr learned my addr 172.21.15.3:0/3137914246 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 -- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d4c0d4c00 msgr2=0x7f6d4c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4b7fe640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d4c0d4c00 0x7f6d4c161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d509de640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d4c15e7e0 0x7f6d4c0b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d4c0d4c00 0x7f6d4c161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 -- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d4c15e7e0 msgr2=0x7f6d4c0b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d4c15e7e0 0x7f6d4c0b7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 -- 172.21.15.3:0/3137914246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c15dd30 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d4bfff640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0cfa90 0x7f6d4c15d650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0017f0 tx=0x7f6d3c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d48ff9640 1 -- 172.21.15.3:0/3137914246 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c005ae0 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d48ff9640 1 -- 172.21.15.3:0/3137914246 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3c005ca0 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c15e010 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d48ff9640 1 -- 172.21.15.3:0/3137914246 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0072b0 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c15ff20 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4c0b27a0 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d48ff9640 1 -- 172.21.15.3:0/3137914246 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d3c007450 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d48ff9640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d380678c0 0x7f6d38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.014+0000 7f6d48ff9640 1 -- 172.21.15.3:0/3137914246 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f6d3c085790 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.018+0000 7f6d4b7fe640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d380678c0 0x7f6d38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.018+0000 7f6d4b7fe640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d380678c0 0x7f6d38069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d400ad000 tx=0x7f6d40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.022+0000 7f6d48ff9640 1 -- 172.21.15.3:0/3137914246 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6d3c010300 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.246+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f6d4c150250 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.250+0000 7f6d48ff9640 1 -- 172.21.15.3:0/3137914246 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f6d3c056620 con 0x7f6d4c0cfa90 2024-09-12T12:20:30.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d380678c0 msgr2=0x7f6d38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d380678c0 0x7f6d38069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d400ad000 tx=0x7f6d40008040 comp rx=0 tx=0).stop 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0cfa90 msgr2=0x7f6d4c15d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0cfa90 0x7f6d4c15d650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0017f0 tx=0x7f6d3c00cbe0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 shutdown_connections 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d380678c0 0x7f6d38069d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d4c15e7e0 0x7f6d4c0b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d4c0d4c00 0x7f6d4c161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 --2- 172.21.15.3:0/3137914246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d4c0cfa90 0x7f6d4c15d650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 >> 172.21.15.3:0/3137914246 conn(0x7f6d4c08c000 msgr2=0x7f6d4c0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 shutdown_connections 2024-09-12T12:20:30.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.254+0000 7f6d52441640 1 -- 172.21.15.3:0/3137914246 wait complete. 2024-09-12T12:20:30.273 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-12T12:20:30.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d9b27b640 1 Processor -- start 2024-09-12T12:20:30.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d9b27b640 1 -- start start 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d9b27b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d940aedf0 0x7f6d940b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d9b27b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d940d0380 0x7f6d940d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d9b27b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d940afb00 con 0x7f6d941568f0 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d9b27b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6d940b4ce0 con 0x7f6d940d0380 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d9b27b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6d940b31b0 con 0x7f6d940aedf0 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d99818640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d940d0380 0x7f6d940d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d99818640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d940d0380 0x7f6d940d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36344/0 (socket says 172.21.15.3:36344) 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d99818640 1 -- 172.21.15.3:0/1294755359 learned_addr learned my addr 172.21.15.3:0/1294755359 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d98816640 1 --2- 172.21.15.3:0/1294755359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d940aedf0 0x7f6d940b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d99818640 1 -- 172.21.15.3:0/1294755359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d940aedf0 msgr2=0x7f6d940b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d99818640 1 --2- 172.21.15.3:0/1294755359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d940aedf0 0x7f6d940b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d99818640 1 -- 172.21.15.3:0/1294755359 >> v1:172.21.15.142:6789/0 conn(0x7f6d941568f0 legacy=0x7f6d940b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d99818640 1 -- 172.21.15.3:0/1294755359 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d940af540 con 0x7f6d940d0380 2024-09-12T12:20:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.386+0000 7f6d98816640 1 --2- 172.21.15.3:0/1294755359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d940aedf0 0x7f6d940b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:30.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d99818640 1 --2- 172.21.15.3:0/1294755359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940d0380 0x7f6d940d0810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009e50 tx=0x7f6d8c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827b69b8dec84a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d8bfff640 1 -- 172.21.15.3:0/1294755359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c0a5660 con 0x7f6d940d0380 2024-09-12T12:20:30.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d8bfff640 1 -- 172.21.15.3:0/1294755359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8c0a5800 con 0x7f6d940d0380 2024-09-12T12:20:30.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d8bfff640 1 -- 172.21.15.3:0/1294755359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c0a5bd0 con 0x7f6d940d0380 2024-09-12T12:20:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 -- 172.21.15.3:0/1294755359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940d0380 msgr2=0x7f6d940d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/1294755359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940d0380 0x7f6d940d0810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009e50 tx=0x7f6d8c01c9a0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 -- 172.21.15.3:0/1294755359 shutdown_connections 2024-09-12T12:20:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/1294755359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940d0380 0x7f6d940d0810 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/1294755359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d940aedf0 0x7f6d940b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 -- 172.21.15.3:0/1294755359 >> 172.21.15.3:0/1294755359 conn(0x7f6d9406be90 msgr2=0x7f6d9406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 -- 172.21.15.3:0/1294755359 shutdown_connections 2024-09-12T12:20:30.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 -- 172.21.15.3:0/1294755359 wait complete. 2024-09-12T12:20:30.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 Processor -- start 2024-09-12T12:20:30.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.390+0000 7f6d9b27b640 1 -- start start 2024-09-12T12:20:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940aedf0 0x7f6d94153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d940d0380 0x7f6d9414e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d941568f0 0x7f6d94159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9415d220 con 0x7f6d940aedf0 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9415d0a0 con 0x7f6d940d0380 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9415d3a0 con 0x7f6d941568f0 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940aedf0 0x7f6d94153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940aedf0 0x7f6d94153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36348/0 (socket says 172.21.15.3:36348) 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 -- 172.21.15.3:0/3580518786 learned_addr learned my addr 172.21.15.3:0/3580518786 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99818640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d941568f0 0x7f6d94159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d98816640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d940d0380 0x7f6d9414e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 -- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d941568f0 msgr2=0x7f6d94159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d941568f0 0x7f6d94159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 -- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d940d0380 msgr2=0x7f6d9414e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d940d0380 0x7f6d9414e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 -- 172.21.15.3:0/3580518786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d94157080 con 0x7f6d940aedf0 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99818640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d941568f0 0x7f6d94159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d98816640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d940d0380 0x7f6d9414e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d99017640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940aedf0 0x7f6d94153490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c530 tx=0x7f6d9000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d89ffb640 1 -- 172.21.15.3:0/3580518786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90016020 con 0x7f6d940aedf0 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d89ffb640 1 -- 172.21.15.3:0/3580518786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d90005220 con 0x7f6d940aedf0 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d89ffb640 1 -- 172.21.15.3:0/3580518786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90007030 con 0x7f6d940aedf0 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d94157320 con 0x7f6d940aedf0 2024-09-12T12:20:30.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9414ea10 con 0x7f6d940aedf0 2024-09-12T12:20:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d940b47a0 con 0x7f6d940aedf0 2024-09-12T12:20:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d89ffb640 1 -- 172.21.15.3:0/3580518786 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d900071d0 con 0x7f6d940aedf0 2024-09-12T12:20:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d89ffb640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d780678c0 0x7f6d78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d89ffb640 1 -- 172.21.15.3:0/3580518786 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f6d90085260 con 0x7f6d940aedf0 2024-09-12T12:20:30.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.394+0000 7f6d98816640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d780678c0 0x7f6d78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.398+0000 7f6d98816640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d780678c0 0x7f6d78069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d9409dfd0 tx=0x7f6d84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.398+0000 7f6d89ffb640 1 -- 172.21.15.3:0/3580518786 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6d90020020 con 0x7f6d940aedf0 2024-09-12T12:20:30.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.622+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f6d9414d8f0 con 0x7f6d940aedf0 2024-09-12T12:20:30.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d89ffb640 1 -- 172.21.15.3:0/3580518786 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f6d900074c0 con 0x7f6d940aedf0 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d780678c0 msgr2=0x7f6d78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d780678c0 0x7f6d78069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6d9409dfd0 tx=0x7f6d84008040 comp rx=0 tx=0).stop 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940aedf0 msgr2=0x7f6d94153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940aedf0 0x7f6d94153490 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000c530 tx=0x7f6d9000c9f0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 shutdown_connections 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d780678c0 0x7f6d78069d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d941568f0 0x7f6d94159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d940d0380 0x7f6d9414e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 --2- 172.21.15.3:0/3580518786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d940aedf0 0x7f6d94153490 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 >> 172.21.15.3:0/3580518786 conn(0x7f6d9406be90 msgr2=0x7f6d94151090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 shutdown_connections 2024-09-12T12:20:30.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.626+0000 7f6d9b27b640 1 -- 172.21.15.3:0/3580518786 wait complete. 2024-09-12T12:20:30.649 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-12T12:20:30.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f9624ff4640 1 Processor -- start 2024-09-12T12:20:30.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f9624ff4640 1 -- start start 2024-09-12T12:20:30.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f9624ff4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96200d0a90 0x7f96200b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f9624ff4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96200afb00 con 0x7f96201568f0 2024-09-12T12:20:30.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f9624ff4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96200b31b0 con 0x7f96200aedf0 2024-09-12T12:20:30.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f9624ff4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f96200b4ce0 con 0x7f96200d0a90 2024-09-12T12:20:30.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f961ed9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f96200d0a90 0x7f96200b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f961e59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f96201568f0 0x7f96200b3970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51706/0 (socket says 172.21.15.3:51706) 2024-09-12T12:20:30.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f961e59c640 1 -- 172.21.15.3:0/3495038572 learned_addr learned my addr 172.21.15.3:0/3495038572 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:30.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f961ed9d640 1 -- 172.21.15.3:0/3495038572 >> v1:172.21.15.142:6789/0 conn(0x7f96200aedf0 legacy=0x7f96200d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f961ed9d640 1 -- 172.21.15.3:0/3495038572 >> v1:172.21.15.3:6789/0 conn(0x7f96201568f0 legacy=0x7f96200b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f961ed9d640 1 -- 172.21.15.3:0/3495038572 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96200af540 con 0x7f96200d0a90 2024-09-12T12:20:30.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.758+0000 7f961d59a640 1 -- 172.21.15.3:0/3495038572 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3377630701 0 0) 0x7f96200afb00 con 0x7f96201568f0 2024-09-12T12:20:30.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f961ed9d640 1 --2- 172.21.15.3:0/3495038572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200d0a90 0x7f96200b47a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f96180097c0 tx=0x7f961801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41210d5cbda2c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f961d59a640 1 -- 172.21.15.3:0/3495038572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96180b0070 con 0x7f96200d0a90 2024-09-12T12:20:30.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f961d59a640 1 -- 172.21.15.3:0/3495038572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96180ab600 con 0x7f96200d0a90 2024-09-12T12:20:30.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f961d59a640 1 -- 172.21.15.3:0/3495038572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96180a4e60 con 0x7f96200d0a90 2024-09-12T12:20:30.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 -- 172.21.15.3:0/3495038572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200d0a90 msgr2=0x7f96200b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 --2- 172.21.15.3:0/3495038572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200d0a90 0x7f96200b47a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f96180097c0 tx=0x7f961801c3d0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 -- 172.21.15.3:0/3495038572 shutdown_connections 2024-09-12T12:20:30.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 --2- 172.21.15.3:0/3495038572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200d0a90 0x7f96200b47a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 -- 172.21.15.3:0/3495038572 >> 172.21.15.3:0/3495038572 conn(0x7f962006be90 msgr2=0x7f962006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 -- 172.21.15.3:0/3495038572 shutdown_connections 2024-09-12T12:20:30.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 -- 172.21.15.3:0/3495038572 wait complete. 2024-09-12T12:20:30.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 Processor -- start 2024-09-12T12:20:30.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.762+0000 7f9624ff4640 1 -- start start 2024-09-12T12:20:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200aedf0 0x7f962014d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96200d0a90 0x7f9620145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96201568f0 0x7f9620155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f962015d220 con 0x7f96200aedf0 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f962015d0a0 con 0x7f96201568f0 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f962015d3a0 con 0x7f96200d0a90 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200aedf0 0x7f962014d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200aedf0 0x7f962014d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36372/0 (socket says 172.21.15.3:36372) 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 -- 172.21.15.3:0/2127923471 learned_addr learned my addr 172.21.15.3:0/2127923471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961dd9b640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96200d0a90 0x7f9620145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 -- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96200d0a90 msgr2=0x7f9620145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961ed9d640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96201568f0 0x7f9620155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96200d0a90 0x7f9620145370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 -- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96201568f0 msgr2=0x7f9620155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96201568f0 0x7f9620155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 -- 172.21.15.3:0/2127923471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962014dc20 con 0x7f96200aedf0 2024-09-12T12:20:30.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961e59c640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200aedf0 0x7f962014d620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f96100011f0 tx=0x7f961000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f960effd640 1 -- 172.21.15.3:0/2127923471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610018070 con 0x7f96200aedf0 2024-09-12T12:20:30.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f960effd640 1 -- 172.21.15.3:0/2127923471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96100074e0 con 0x7f96200aedf0 2024-09-12T12:20:30.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f960effd640 1 -- 172.21.15.3:0/2127923471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610004a10 con 0x7f96200aedf0 2024-09-12T12:20:30.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962014df00 con 0x7f96200aedf0 2024-09-12T12:20:30.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962014e4c0 con 0x7f96200aedf0 2024-09-12T12:20:30.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96200b47a0 con 0x7f96200aedf0 2024-09-12T12:20:30.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f960effd640 1 -- 172.21.15.3:0/2127923471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f961001b070 con 0x7f96200aedf0 2024-09-12T12:20:30.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f960effd640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f96000678c0 0x7f9600069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f960effd640 1 -- 172.21.15.3:0/2127923471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f9610084c40 con 0x7f96200aedf0 2024-09-12T12:20:30.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.766+0000 7f961dd9b640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f96000678c0 0x7f9600069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.770+0000 7f961dd9b640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f96000678c0 0x7f9600069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f96201fa920 tx=0x7f96140072f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.770+0000 7f960effd640 1 -- 172.21.15.3:0/2127923471 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9610055b60 con 0x7f96200aedf0 2024-09-12T12:20:30.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.994+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f962015d540 con 0x7f96200aedf0 2024-09-12T12:20:31.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f960effd640 1 -- 172.21.15.3:0/2127923471 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f9610055460 con 0x7f96200aedf0 2024-09-12T12:20:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f96000678c0 msgr2=0x7f9600069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f96000678c0 0x7f9600069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f96201fa920 tx=0x7f96140072f0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200aedf0 msgr2=0x7f962014d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200aedf0 0x7f962014d620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f96100011f0 tx=0x7f961000c9b0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 shutdown_connections 2024-09-12T12:20:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f96000678c0 0x7f9600069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96201568f0 0x7f9620155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f96200d0a90 0x7f9620145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 --2- 172.21.15.3:0/2127923471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f96200aedf0 0x7f962014d620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 >> 172.21.15.3:0/2127923471 conn(0x7f962006be90 msgr2=0x7f962015e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 shutdown_connections 2024-09-12T12:20:31.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:30.998+0000 7f9624ff4640 1 -- 172.21.15.3:0/2127923471 wait complete. 2024-09-12T12:20:31.020 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-12T12:20:31.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff592a2f640 1 Processor -- start 2024-09-12T12:20:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff592a2f640 1 -- start start 2024-09-12T12:20:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff592a2f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff58c1568f0 0x7ff58c0aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff592a2f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff58c0b47a0 0x7ff58c0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff592a2f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58c0afb00 con 0x7ff58c0d0380 2024-09-12T12:20:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff592a2f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff58c0b31b0 con 0x7ff58c1568f0 2024-09-12T12:20:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff592a2f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff58c0af540 con 0x7ff58c0b47a0 2024-09-12T12:20:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff590fcc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff58c0d0380 0x7ff58c15c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51710/0 (socket says 172.21.15.3:51710) 2024-09-12T12:20:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58bfff640 1 --2- 172.21.15.3:0/1747693892 >> v2:172.21.15.3:3300/0 conn(0x7ff58c1568f0 0x7ff58c0aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff590fcc640 1 -- 172.21.15.3:0/1747693892 learned_addr learned my addr 172.21.15.3:0/1747693892 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58affd640 1 -- 172.21.15.3:0/1747693892 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438528442 0 0) 0x7ff58c0afb00 con 0x7ff58c0d0380 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58b7fe640 1 --2- 172.21.15.3:0/1747693892 >> v2:172.21.15.142:3300/0 conn(0x7ff58c0b47a0 0x7ff58c0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58affd640 1 -- 172.21.15.3:0/1747693892 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff574003530 con 0x7ff58c0d0380 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58bfff640 1 -- 172.21.15.3:0/1747693892 >> v2:172.21.15.142:3300/0 conn(0x7ff58c0b47a0 msgr2=0x7ff58c0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58bfff640 1 --2- 172.21.15.3:0/1747693892 >> v2:172.21.15.142:3300/0 conn(0x7ff58c0b47a0 0x7ff58c0b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58bfff640 1 -- 172.21.15.3:0/1747693892 >> v1:172.21.15.3:6789/0 conn(0x7ff58c0d0380 legacy=0x7ff58c15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58bfff640 1 -- 172.21.15.3:0/1747693892 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c0af820 con 0x7ff58c1568f0 2024-09-12T12:20:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.142+0000 7ff58b7fe640 1 --2- 172.21.15.3:0/1747693892 >> v2:172.21.15.142:3300/0 conn(0x7ff58c0b47a0 0x7ff58c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:31.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff58bfff640 1 --2- 172.21.15.3:0/1747693892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c0aedf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0cfba0 tx=0x7ff57c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3b14e204c66d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff58affd640 1 -- 172.21.15.3:0/1747693892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0b0070 con 0x7ff58c1568f0 2024-09-12T12:20:31.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff58affd640 1 -- 172.21.15.3:0/1747693892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c0ab600 con 0x7ff58c1568f0 2024-09-12T12:20:31.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff58affd640 1 -- 172.21.15.3:0/1747693892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0ab9d0 con 0x7ff58c1568f0 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- 172.21.15.3:0/1747693892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 msgr2=0x7ff58c0aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1747693892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c0aedf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0cfba0 tx=0x7ff57c01c3f0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- 172.21.15.3:0/1747693892 shutdown_connections 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1747693892 >> v2:172.21.15.142:3300/0 conn(0x7ff58c0b47a0 0x7ff58c0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1747693892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c0aedf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- 172.21.15.3:0/1747693892 >> 172.21.15.3:0/1747693892 conn(0x7ff58c06be90 msgr2=0x7ff58c06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- 172.21.15.3:0/1747693892 shutdown_connections 2024-09-12T12:20:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- 172.21.15.3:0/1747693892 wait complete. 2024-09-12T12:20:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 Processor -- start 2024-09-12T12:20:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- start start 2024-09-12T12:20:31.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff58c0b47a0 0x7ff58c14e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c0d0380 0x7ff58c099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c072fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c0d0ba0 con 0x7ff58c1568f0 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c15d3b0 con 0x7ff58c0d0380 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c073c60 con 0x7ff58c0b47a0 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c072fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c072fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36386/0 (socket says 172.21.15.3:36386) 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 -- 172.21.15.3:0/1625619252 learned_addr learned my addr 172.21.15.3:0/1625619252 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 -- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff58c0b47a0 msgr2=0x7ff58c14e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff58bfff640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff58c0b47a0 0x7ff58c14e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff58c0b47a0 0x7ff58c14e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 -- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c0d0380 msgr2=0x7ff58c099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff58b7fe640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c0d0380 0x7ff58c099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c0d0380 0x7ff58c099880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 -- 172.21.15.3:0/1625619252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c14e690 con 0x7ff58c1568f0 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff590fcc640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c072fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c3f0 tx=0x7ff584001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff588ff9640 1 -- 172.21.15.3:0/1625619252 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584017070 con 0x7ff58c1568f0 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff588ff9640 1 -- 172.21.15.3:0/1625619252 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff584005cf0 con 0x7ff58c1568f0 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c073980 con 0x7ff58c1568f0 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff588ff9640 1 -- 172.21.15.3:0/1625619252 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840072f0 con 0x7ff58c1568f0 2024-09-12T12:20:31.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.146+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c14d950 con 0x7ff58c1568f0 2024-09-12T12:20:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.150+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c0b3950 con 0x7ff58c1568f0 2024-09-12T12:20:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.150+0000 7ff588ff9640 1 -- 172.21.15.3:0/1625619252 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff58401a070 con 0x7ff58c1568f0 2024-09-12T12:20:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.150+0000 7ff588ff9640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff5780678c0 0x7ff578069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.150+0000 7ff588ff9640 1 -- 172.21.15.3:0/1625619252 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7ff584084c00 con 0x7ff58c1568f0 2024-09-12T12:20:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.150+0000 7ff58bfff640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff5780678c0 0x7ff578069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.150+0000 7ff58bfff640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff5780678c0 0x7ff578069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0ad000 tx=0x7ff57c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.154+0000 7ff588ff9640 1 -- 172.21.15.3:0/1625619252 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff58408b080 con 0x7ff58c1568f0 2024-09-12T12:20:31.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.378+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7ff58c15f5b0 con 0x7ff58c1568f0 2024-09-12T12:20:31.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff588ff9640 1 -- 172.21.15.3:0/1625619252 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7ff584055b20 con 0x7ff58c1568f0 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff5780678c0 msgr2=0x7ff578069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff5780678c0 0x7ff578069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0ad000 tx=0x7ff57c008040 comp rx=0 tx=0).stop 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 msgr2=0x7ff58c072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c072fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c3f0 tx=0x7ff584001e80 comp rx=0 tx=0).stop 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 shutdown_connections 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff5780678c0 0x7ff578069d80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff58c1568f0 0x7ff58c072fe0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff58c0d0380 0x7ff58c099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 --2- 172.21.15.3:0/1625619252 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff58c0b47a0 0x7ff58c14e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 >> 172.21.15.3:0/1625619252 conn(0x7ff58c06be90 msgr2=0x7ff58c0b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 shutdown_connections 2024-09-12T12:20:31.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.382+0000 7ff592a2f640 1 -- 172.21.15.3:0/1625619252 wait complete. 2024-09-12T12:20:31.403 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-12T12:20:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 Processor -- start 2024-09-12T12:20:31.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 -- start start 2024-09-12T12:20:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca1815c2c0 0x7fca1809b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca180ce900 0x7fca1809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fca1809afb0 0x7fca1815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fca180d09f0 con 0x7fca1815c2c0 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fca1809b3e0 con 0x7fca1809afb0 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca1e5c6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca1809bc30 con 0x7fca180ce900 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.510+0000 7fca17fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca1815c2c0 0x7fca1809b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fca1815c2c0 0x7fca1809b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36402/0 (socket says 172.21.15.3:36402) 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 -- 172.21.15.3:0/1067119539 learned_addr learned my addr 172.21.15.3:0/1067119539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca0f7fe640 1 --2- 172.21.15.3:0/1067119539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca180ce900 0x7fca1809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1cb63640 1 --2- 172.21.15.3:0/1067119539 >> v2:172.21.15.142:3300/0 conn(0x7fca1809afb0 0x7fca1815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 -- 172.21.15.3:0/1067119539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca180ce900 msgr2=0x7fca1809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 --2- 172.21.15.3:0/1067119539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca180ce900 0x7fca1809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 -- 172.21.15.3:0/1067119539 >> v2:172.21.15.142:3300/0 conn(0x7fca1809afb0 msgr2=0x7fca1815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 --2- 172.21.15.3:0/1067119539 >> v2:172.21.15.142:3300/0 conn(0x7fca1809afb0 0x7fca1815ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 -- 172.21.15.3:0/1067119539 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca180d0c40 con 0x7fca1815c2c0 2024-09-12T12:20:31.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1cb63640 1 --2- 172.21.15.3:0/1067119539 >> v2:172.21.15.142:3300/0 conn(0x7fca1809afb0 0x7fca1815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:31.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca17fff640 1 --2- 172.21.15.3:0/1067119539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca1815c2c0 0x7fca1809b6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fca1815fe40 tx=0x7fca0801c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87a388ffe4967c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca177fe640 1 -- 172.21.15.3:0/1067119539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080b0070 con 0x7fca1815c2c0 2024-09-12T12:20:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca177fe640 1 -- 172.21.15.3:0/1067119539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca080ab600 con 0x7fca1815c2c0 2024-09-12T12:20:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca177fe640 1 -- 172.21.15.3:0/1067119539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080ab9d0 con 0x7fca1815c2c0 2024-09-12T12:20:31.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 -- 172.21.15.3:0/1067119539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca1815c2c0 msgr2=0x7fca1809b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/1067119539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca1815c2c0 0x7fca1809b6f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fca1815fe40 tx=0x7fca0801c110 comp rx=0 tx=0).stop 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 -- 172.21.15.3:0/1067119539 shutdown_connections 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/1067119539 >> v2:172.21.15.142:3300/0 conn(0x7fca1809afb0 0x7fca1815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/1067119539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca180ce900 0x7fca1809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/1067119539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca1815c2c0 0x7fca1809b6f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 -- 172.21.15.3:0/1067119539 >> 172.21.15.3:0/1067119539 conn(0x7fca180b3f00 msgr2=0x7fca180b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 -- 172.21.15.3:0/1067119539 shutdown_connections 2024-09-12T12:20:31.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.514+0000 7fca1e5c6640 1 -- 172.21.15.3:0/1067119539 wait complete. 2024-09-12T12:20:31.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 Processor -- start 2024-09-12T12:20:31.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 -- start start 2024-09-12T12:20:31.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1809afb0 0x7fca1815f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca180ce900 0x7fca180cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1815c2c0 0x7fca180c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fca180d1380 con 0x7fca180ce900 2024-09-12T12:20:31.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca180d1200 con 0x7fca1809afb0 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fca180b1d70 con 0x7fca1815c2c0 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca180ce900 0x7fca180cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca180ce900 0x7fca180cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36410/0 (socket says 172.21.15.3:36410) 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 -- 172.21.15.3:0/3860044027 learned_addr learned my addr 172.21.15.3:0/3860044027 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca17fff640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1809afb0 0x7fca1815f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 -- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1815c2c0 msgr2=0x7fca180c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1cb63640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1815c2c0 0x7fca180c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1815c2c0 0x7fca180c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 -- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1809afb0 msgr2=0x7fca1815f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1809afb0 0x7fca1815f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 -- 172.21.15.3:0/3860044027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1815f9b0 con 0x7fca180ce900 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca17fff640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1809afb0 0x7fca1815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:31.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca0f7fe640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca180ce900 0x7fca180cfcf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d620 tx=0x7fca0000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca157fa640 1 -- 172.21.15.3:0/3860044027 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0000b620 con 0x7fca180ce900 2024-09-12T12:20:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca157fa640 1 -- 172.21.15.3:0/3860044027 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca00019070 con 0x7fca180ce900 2024-09-12T12:20:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca157fa640 1 -- 172.21.15.3:0/3860044027 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0000ba20 con 0x7fca180ce900 2024-09-12T12:20:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca180c7160 con 0x7fca180ce900 2024-09-12T12:20:31.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca180b73e0 con 0x7fca180ce900 2024-09-12T12:20:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1809aa70 con 0x7fca180ce900 2024-09-12T12:20:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca157fa640 1 -- 172.21.15.3:0/3860044027 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fca0000bbc0 con 0x7fca180ce900 2024-09-12T12:20:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca157fa640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9f80678f0 0x7fc9f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.518+0000 7fca157fa640 1 -- 172.21.15.3:0/3860044027 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fca00085dd0 con 0x7fca180ce900 2024-09-12T12:20:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.522+0000 7fca17fff640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9f80678f0 0x7fc9f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.522+0000 7fca17fff640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9f80678f0 0x7fc9f8069db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca080ad000 tx=0x7fca08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.526+0000 7fca157fa640 1 -- 172.21.15.3:0/3860044027 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fca1815f9b0 con 0x7fca180ce900 2024-09-12T12:20:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.750+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fca1815fb90 con 0x7fca180ce900 2024-09-12T12:20:31.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.754+0000 7fca157fa640 1 -- 172.21.15.3:0/3860044027 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7fca00087020 con 0x7fca180ce900 2024-09-12T12:20:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9f80678f0 msgr2=0x7fc9f8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9f80678f0 0x7fc9f8069db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca080ad000 tx=0x7fca08008040 comp rx=0 tx=0).stop 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca180ce900 msgr2=0x7fca180cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca180ce900 0x7fca180cfcf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d620 tx=0x7fca0000dae0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 shutdown_connections 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9f80678f0 0x7fc9f8069db0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fca1815c2c0 0x7fca180c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca180ce900 0x7fca180cfcf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 --2- 172.21.15.3:0/3860044027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca1809afb0 0x7fca1815f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 >> 172.21.15.3:0/3860044027 conn(0x7fca180b3f00 msgr2=0x7fca180cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 shutdown_connections 2024-09-12T12:20:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.758+0000 7fca1e5c6640 1 -- 172.21.15.3:0/3860044027 wait complete. 2024-09-12T12:20:31.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T12:20:31.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f465dcf6640 1 Processor -- start 2024-09-12T12:20:31.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f465dcf6640 1 -- start start 2024-09-12T12:20:31.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f465dcf6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f465806f6e0 0x7f4658072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f465dcf6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46580cc300 0x7f4658073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f465dcf6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f465815f150 con 0x7f465815b2d0 2024-09-12T12:20:31.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f465dcf6640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f46580b3790 con 0x7f46580cc300 2024-09-12T12:20:31.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f465dcf6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46580ae920 con 0x7f465806f6e0 2024-09-12T12:20:31.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4657fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46580cc300 0x7f4658073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4657fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f46580cc300 0x7f4658073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36424/0 (socket says 172.21.15.3:36424) 2024-09-12T12:20:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4657fff640 1 -- 172.21.15.3:0/2912452035 learned_addr learned my addr 172.21.15.3:0/2912452035 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4656ffd640 1 --2- 172.21.15.3:0/2912452035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f465806f6e0 0x7f4658072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4657fff640 1 -- 172.21.15.3:0/2912452035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f465806f6e0 msgr2=0x7f4658072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4657fff640 1 --2- 172.21.15.3:0/2912452035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f465806f6e0 0x7f4658072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4657fff640 1 -- 172.21.15.3:0/2912452035 >> v1:172.21.15.142:6789/0 conn(0x7f465815b2d0 legacy=0x7f46580b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.890+0000 7f4657fff640 1 -- 172.21.15.3:0/2912452035 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465815b6b0 con 0x7f46580cc300 2024-09-12T12:20:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f4657fff640 1 --2- 172.21.15.3:0/2912452035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46580cc300 0x7f4658073470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4650009e50 tx=0x7f465001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11457c820cccd6f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f46567fc640 1 -- 172.21.15.3:0/2912452035 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500a58d0 con 0x7f46580cc300 2024-09-12T12:20:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f46567fc640 1 -- 172.21.15.3:0/2912452035 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46500a5a70 con 0x7f46580cc300 2024-09-12T12:20:31.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f46567fc640 1 -- 172.21.15.3:0/2912452035 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500a5e40 con 0x7f46580cc300 2024-09-12T12:20:31.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- 172.21.15.3:0/2912452035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46580cc300 msgr2=0x7f4658073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2912452035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46580cc300 0x7f4658073470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4650009e50 tx=0x7f465001ca00 comp rx=0 tx=0).stop 2024-09-12T12:20:31.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- 172.21.15.3:0/2912452035 shutdown_connections 2024-09-12T12:20:31.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2912452035 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f46580cc300 0x7f4658073470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2912452035 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f465806f6e0 0x7f4658072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- 172.21.15.3:0/2912452035 >> 172.21.15.3:0/2912452035 conn(0x7f46580b1d80 msgr2=0x7f46580b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:31.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- 172.21.15.3:0/2912452035 shutdown_connections 2024-09-12T12:20:31.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- 172.21.15.3:0/2912452035 wait complete. 2024-09-12T12:20:31.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 Processor -- start 2024-09-12T12:20:31.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- start start 2024-09-12T12:20:31.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f465806f6e0 0x7f4658156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46580cc300 0x7f46581454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f465815b2d0 0x7f4658158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f465806d8b0 con 0x7f465815b2d0 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f46580ae5b0 con 0x7f465806f6e0 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f465dcf6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f46580af5a0 con 0x7f46580cc300 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f4657fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f465815b2d0 0x7f4658158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f4657fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f465815b2d0 0x7f4658158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36432/0 (socket says 172.21.15.3:36432) 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.894+0000 7f4657fff640 1 -- 172.21.15.3:0/2087561659 learned_addr learned my addr 172.21.15.3:0/2087561659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f4656ffd640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46580cc300 0x7f46581454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f4657fff640 1 -- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46580cc300 msgr2=0x7f46581454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f4657fff640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46580cc300 0x7f46581454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f46577fe640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f465806f6e0 0x7f4658156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f4657fff640 1 -- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f465806f6e0 msgr2=0x7f4658156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:31.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f4657fff640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f465806f6e0 0x7f4658156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f4657fff640 1 -- 172.21.15.3:0/2087561659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4658157270 con 0x7f465815b2d0 2024-09-12T12:20:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f4657fff640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f465815b2d0 0x7f4658158f80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f465001a630 tx=0x7f46500197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f463ffff640 1 -- 172.21.15.3:0/2087561659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500bc030 con 0x7f465815b2d0 2024-09-12T12:20:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f463ffff640 1 -- 172.21.15.3:0/2087561659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4650007c70 con 0x7f465815b2d0 2024-09-12T12:20:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f463ffff640 1 -- 172.21.15.3:0/2087561659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500a58d0 con 0x7f465815b2d0 2024-09-12T12:20:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4658157550 con 0x7f465815b2d0 2024-09-12T12:20:31.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4658157b10 con 0x7f465815b2d0 2024-09-12T12:20:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4658072eb0 con 0x7f465815b2d0 2024-09-12T12:20:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f463ffff640 1 -- 172.21.15.3:0/2087561659 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46500bf070 con 0x7f465815b2d0 2024-09-12T12:20:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f463ffff640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46440678f0 0x7f4644069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f463ffff640 1 -- 172.21.15.3:0/2087561659 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f4650120470 con 0x7f465815b2d0 2024-09-12T12:20:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f46577fe640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46440678f0 0x7f4644069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:31.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.898+0000 7f46577fe640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46440678f0 0x7f4644069db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4648007fd0 tx=0x7f4648009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:31.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:31.902+0000 7f463ffff640 1 -- 172.21.15.3:0/2087561659 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4650016620 con 0x7f465815b2d0 2024-09-12T12:20:32.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.062+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f46580b3080 con 0x7f465815b2d0 2024-09-12T12:20:32.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.062+0000 7f463ffff640 1 -- 172.21.15.3:0/2087561659 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f4650007e20 con 0x7f465815b2d0 2024-09-12T12:20:32.067 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:32.067 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T12:20:32.067 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:20:32.068 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T12:20:32.068 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T12:20:32.068 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:32.068 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46440678f0 msgr2=0x7f4644069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46440678f0 0x7f4644069db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4648007fd0 tx=0x7f4648009040 comp rx=0 tx=0).stop 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f465815b2d0 msgr2=0x7f4658158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f465815b2d0 0x7f4658158f80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f465001a630 tx=0x7f46500197a0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 shutdown_connections 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46440678f0 0x7f4644069db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f465815b2d0 0x7f4658158f80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46580cc300 0x7f46581454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 --2- 172.21.15.3:0/2087561659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f465806f6e0 0x7f4658156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 >> 172.21.15.3:0/2087561659 conn(0x7f46580b1d80 msgr2=0x7f46580b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:32.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 shutdown_connections 2024-09-12T12:20:32.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.066+0000 7f465dcf6640 1 -- 172.21.15.3:0/2087561659 wait complete. 2024-09-12T12:20:32.085 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:20:32.085 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:20:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.198+0000 7f190ad52640 1 Processor -- start 2024-09-12T12:20:32.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.198+0000 7f190ad52640 1 -- start start 2024-09-12T12:20:32.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f190ad52640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 0x7f19040d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f190ad52640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190415d0b0 con 0x7f1904153480 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f190ad52640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190415d390 con 0x7f19041530a0 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f190ad52640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19040cec50 con 0x7f19040d1cf0 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1904153480 0x7f19040cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51726/0 (socket says 172.21.15.3:51726) 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 -- 172.21.15.3:0/4109321055 learned_addr learned my addr 172.21.15.3:0/4109321055 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fbfff640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 0x7f19040d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853826954 0 0) 0x7f190415d0b0 con 0x7f1904153480 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18ec003530 con 0x7f1904153480 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 182403863 0 0) 0x7f18ec003530 con 0x7f1904153480 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 msgr2=0x7f19040d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 0x7f19040d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 >> v1:172.21.15.142:6789/0 conn(0x7f19041530a0 legacy=0x7f190415c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190415d670 con 0x7f1904153480 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f18fc001a00 con 0x7f1904153480 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 >> v1:172.21.15.3:6789/0 conn(0x7f1904153480 legacy=0x7f19040cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 0x7f18ec0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec003f30 0x7f18ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec004d50 0x7f18ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f19040cec50 con 0x7f18ec004d50 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec00ba30 con 0x7f18ec003f30 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18ec00bc30 con 0x7f19040d1cf0 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec004d50 0x7f18ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fbfff640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec003f30 0x7f18ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 -- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 msgr2=0x7f18ec0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:32.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f1908aee640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 0x7f18ec0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19040d1cf0 0x7f18ec0039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 -- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec003f30 msgr2=0x7f18ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:32.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18ec003f30 0x7f18ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 -- 172.21.15.3:0/4109321055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec003530 con 0x7f18ec004d50 2024-09-12T12:20:32.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f19092ef640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec004d50 0x7f18ec0051d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00a880 tx=0x7f18fc01fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6302e2b198d8099d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:32.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00a4a0 con 0x7f18ec004d50 2024-09-12T12:20:32.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18fc0aa730 con 0x7f18ec004d50 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.202+0000 7f18fb7fe640 1 -- 172.21.15.3:0/4109321055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0aab60 con 0x7f18ec004d50 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- 172.21.15.3:0/4109321055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec004d50 msgr2=0x7f18ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 --2- 172.21.15.3:0/4109321055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18ec004d50 0x7f18ec0051d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00a880 tx=0x7f18fc01fae0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f19092ef640 1 -- 172.21.15.3:0/4109321055 reap_dead start 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- 172.21.15.3:0/4109321055 shutdown_connections 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- 172.21.15.3:0/4109321055 >> 172.21.15.3:0/4109321055 conn(0x7f1904070060 msgr2=0x7f1904070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- 172.21.15.3:0/4109321055 shutdown_connections 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- 172.21.15.3:0/4109321055 wait complete. 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 Processor -- start 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- start start 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18ec004d50 0x7f190414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040d21e0 0x7f19040d2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19040b31f0 0x7f19040b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f19040ce960 con 0x7f19040d21e0 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19040ce7e0 con 0x7f19040b31f0 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f19040cf7f0 con 0x7f18ec004d50 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040d21e0 0x7f19040d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040d21e0 0x7f19040d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36450/0 (socket says 172.21.15.3:36450) 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 -- 172.21.15.3:0/2687728437 learned_addr learned my addr 172.21.15.3:0/2687728437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f1908aee640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18ec004d50 0x7f190414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 -- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18ec004d50 msgr2=0x7f190414c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f19092ef640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19040b31f0 0x7f19040b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18ec004d50 0x7f190414c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 -- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19040b31f0 msgr2=0x7f19040b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19040b31f0 0x7f19040b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 -- 172.21.15.3:0/2687728437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19040c5ee0 con 0x7f19040d21e0 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f1908aee640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18ec004d50 0x7f190414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f19092ef640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19040b31f0 0x7f19040b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18fbfff640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040d21e0 0x7f19040d2660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f18f40115d0 tx=0x7f18f4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18f97fa640 1 -- 172.21.15.3:0/2687728437 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f400f8f0 con 0x7f19040d21e0 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18f97fa640 1 -- 172.21.15.3:0/2687728437 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f400fa90 con 0x7f19040d21e0 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19040c61c0 con 0x7f19040d21e0 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f18f97fa640 1 -- 172.21.15.3:0/2687728437 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f400fe80 con 0x7f19040d21e0 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.206+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19040c6780 con 0x7f19040d21e0 2024-09-12T12:20:32.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.210+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19040cfb50 con 0x7f19040d21e0 2024-09-12T12:20:32.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.210+0000 7f18f97fa640 1 -- 172.21.15.3:0/2687728437 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f18f4015460 con 0x7f19040d21e0 2024-09-12T12:20:32.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.210+0000 7f18f97fa640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18f00678f0 0x7f18f0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:32.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.210+0000 7f18f97fa640 1 -- 172.21.15.3:0/2687728437 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f18f40861a0 con 0x7f19040d21e0 2024-09-12T12:20:32.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.210+0000 7f1908aee640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18f00678f0 0x7f18f0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:32.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.210+0000 7f1908aee640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18f00678f0 0x7f18f0069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1900001810 tx=0x7f1900009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:32.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.214+0000 7f18f97fa640 1 -- 172.21.15.3:0/2687728437 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f18f4019030 con 0x7f19040d21e0 2024-09-12T12:20:32.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:32.374+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1904072350 con 0x7f19040d21e0 2024-09-12T12:20:33.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.110+0000 7f18f97fa640 1 -- 172.21.15.3:0/2687728437 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f18f40570c0 con 0x7f19040d21e0 2024-09-12T12:20:33.115 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T12:20:33.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18f00678f0 msgr2=0x7f18f0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:33.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18f00678f0 0x7f18f0069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1900001810 tx=0x7f1900009040 comp rx=0 tx=0).stop 2024-09-12T12:20:33.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040d21e0 msgr2=0x7f19040d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:33.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040d21e0 0x7f19040d2660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f18f40115d0 tx=0x7f18f4011a90 comp rx=0 tx=0).stop 2024-09-12T12:20:33.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 shutdown_connections 2024-09-12T12:20:33.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18f00678f0 0x7f18f0069db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19040b31f0 0x7f19040b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f19040d21e0 0x7f19040d2660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 --2- 172.21.15.3:0/2687728437 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18ec004d50 0x7f190414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 >> 172.21.15.3:0/2687728437 conn(0x7f1904070060 msgr2=0x7f19040cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:33.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 shutdown_connections 2024-09-12T12:20:33.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.114+0000 7f190ad52640 1 -- 172.21.15.3:0/2687728437 wait complete. 2024-09-12T12:20:33.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:20:33.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f2820640 1 Processor -- start 2024-09-12T12:20:33.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f2820640 1 -- start start 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f2820640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec15c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f2820640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92ec0d1cf0 0x7f92ec0d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f2820640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ec15d0b0 con 0x7f92ec153480 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f2820640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f92ec0cec50 con 0x7f92ec0d1cf0 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f2820640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec15d390 con 0x7f92ec1530a0 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92e37fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92ec0d1cf0 0x7f92ec0d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92e37fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92ec0d1cf0 0x7f92ec0d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36464/0 (socket says 172.21.15.3:36464) 2024-09-12T12:20:33.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92e37fe640 1 -- 172.21.15.3:0/26097782 learned_addr learned my addr 172.21.15.3:0/26097782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92f0dbd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f92ec153480 0x7f92ec0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51738/0 (socket says 172.21.15.3:51738) 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92ebfff640 1 --2- 172.21.15.3:0/26097782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92e37fe640 1 -- 172.21.15.3:0/26097782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 msgr2=0x7f92ec15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92e37fe640 1 --2- 172.21.15.3:0/26097782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92e37fe640 1 -- 172.21.15.3:0/26097782 >> v1:172.21.15.3:6789/0 conn(0x7f92ec153480 legacy=0x7f92ec0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92e37fe640 1 -- 172.21.15.3:0/26097782 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec15d670 con 0x7f92ec0d1cf0 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92eb7fe640 1 -- 172.21.15.3:0/26097782 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1485954240 0 0) 0x7f92ec15d0b0 con 0x7f92ec153480 2024-09-12T12:20:33.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.282+0000 7f92ebfff640 1 --2- 172.21.15.3:0/26097782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:33.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92e37fe640 1 --2- 172.21.15.3:0/26097782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec0d2210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009870 tx=0x7f92d401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db9c50b47894ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:33.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92eb7fe640 1 -- 172.21.15.3:0/26097782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d40ab430 con 0x7f92ec0d1cf0 2024-09-12T12:20:33.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92eb7fe640 1 -- 172.21.15.3:0/26097782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d40b0070 con 0x7f92ec0d1cf0 2024-09-12T12:20:33.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92eb7fe640 1 -- 172.21.15.3:0/26097782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d40ab7f0 con 0x7f92ec0d1cf0 2024-09-12T12:20:33.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 -- 172.21.15.3:0/26097782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec0d1cf0 msgr2=0x7f92ec0d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:33.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 --2- 172.21.15.3:0/26097782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec0d2210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009870 tx=0x7f92d401c560 comp rx=0 tx=0).stop 2024-09-12T12:20:33.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 -- 172.21.15.3:0/26097782 shutdown_connections 2024-09-12T12:20:33.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 --2- 172.21.15.3:0/26097782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec0d2210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 --2- 172.21.15.3:0/26097782 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 -- 172.21.15.3:0/26097782 >> 172.21.15.3:0/26097782 conn(0x7f92ec070060 msgr2=0x7f92ec070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 -- 172.21.15.3:0/26097782 shutdown_connections 2024-09-12T12:20:33.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 -- 172.21.15.3:0/26097782 wait complete. 2024-09-12T12:20:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 Processor -- start 2024-09-12T12:20:33.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.286+0000 7f92f2820640 1 -- start start 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec14a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec153480 0x7f92ec14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec0d0660 con 0x7f92ec153480 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec0d2b60 con 0x7f92ec0d1cf0 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec0d07e0 con 0x7f92ec1530a0 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec153480 0x7f92ec14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec153480 0x7f92ec14cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36472/0 (socket says 172.21.15.3:36472) 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 -- 172.21.15.3:0/951450812 learned_addr learned my addr 172.21.15.3:0/951450812 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92e37fe640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92ebfff640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec14a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:33.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 -- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 msgr2=0x7f92ec14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec14fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 -- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92ec0d1cf0 msgr2=0x7f92ec14a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec14a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 -- 172.21.15.3:0/951450812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec14bf50 con 0x7f92ec153480 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92ebfff640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec14a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f0dbd640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec153480 0x7f92ec14cae0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c730 tx=0x7f92e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92e97fa640 1 -- 172.21.15.3:0/951450812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4018070 con 0x7f92ec153480 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92e97fa640 1 -- 172.21.15.3:0/951450812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e400ce40 con 0x7f92ec153480 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92e97fa640 1 -- 172.21.15.3:0/951450812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400a860 con 0x7f92ec153480 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec14c230 con 0x7f92ec153480 2024-09-12T12:20:33.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec14f7f0 con 0x7f92ec153480 2024-09-12T12:20:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec0d2210 con 0x7f92ec153480 2024-09-12T12:20:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.290+0000 7f92e97fa640 1 -- 172.21.15.3:0/951450812 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f92e400aa00 con 0x7f92ec153480 2024-09-12T12:20:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.294+0000 7f92e97fa640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92cc0678c0 0x7f92cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:33.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.294+0000 7f92e97fa640 1 -- 172.21.15.3:0/951450812 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f92e4085110 con 0x7f92ec153480 2024-09-12T12:20:33.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.294+0000 7f92ebfff640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92cc0678c0 0x7f92cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:33.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.294+0000 7f92ebfff640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92cc0678c0 0x7f92cc069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92dc008930 tx=0x7f92dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:33.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.298+0000 7f92e97fa640 1 -- 172.21.15.3:0/951450812 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f92e4055e30 con 0x7f92ec153480 2024-09-12T12:20:33.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:33.462+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f92ec14a910 con 0x7f92ec153480 2024-09-12T12:20:34.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.122+0000 7f92e97fa640 1 -- 172.21.15.3:0/951450812 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f92e4055730 con 0x7f92ec153480 2024-09-12T12:20:34.127 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92cc0678c0 msgr2=0x7f92cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92cc0678c0 0x7f92cc069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f92dc008930 tx=0x7f92dc008040 comp rx=0 tx=0).stop 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec153480 msgr2=0x7f92ec14cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec153480 0x7f92ec14cae0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c730 tx=0x7f92e400cbf0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 shutdown_connections 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92cc0678c0 0x7f92cc069d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92ec153480 0x7f92ec14cae0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f92ec1530a0 0x7f92ec14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 --2- 172.21.15.3:0/951450812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92ec0d1cf0 0x7f92ec14a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 >> 172.21.15.3:0/951450812 conn(0x7f92ec070060 msgr2=0x7f92ec06cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 shutdown_connections 2024-09-12T12:20:34.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.126+0000 7f92f2820640 1 -- 172.21.15.3:0/951450812 wait complete. 2024-09-12T12:20:34.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:20:34.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a551be640 1 Processor -- start 2024-09-12T12:20:34.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a551be640 1 -- start start 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a551be640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a501530a0 0x7f2a5015c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a551be640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2a500d1cf0 0x7f2a500d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a551be640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5015d0b0 con 0x7f2a50153480 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a551be640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2a500cec50 con 0x7f2a501530a0 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a551be640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5015d390 con 0x7f2a500d1cf0 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4ed9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a501530a0 0x7f2a5015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4ed9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2a501530a0 0x7f2a5015c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36476/0 (socket says 172.21.15.3:36476) 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4ed9d640 1 -- 172.21.15.3:0/2953073747 learned_addr learned my addr 172.21.15.3:0/2953073747 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4f59e640 1 --1- 172.21.15.3:0/2953073747 >> v1:172.21.15.3:6789/0 conn(0x7f2a50153480 0x7f2a500cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51744/0 (socket says 172.21.15.3:51744) 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4e59c640 1 --2- 172.21.15.3:0/2953073747 >> v2:172.21.15.142:3300/0 conn(0x7f2a500d1cf0 0x7f2a500d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4ed9d640 1 -- 172.21.15.3:0/2953073747 >> v2:172.21.15.142:3300/0 conn(0x7f2a500d1cf0 msgr2=0x7f2a500d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:34.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4ed9d640 1 --2- 172.21.15.3:0/2953073747 >> v2:172.21.15.142:3300/0 conn(0x7f2a500d1cf0 0x7f2a500d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4ed9d640 1 -- 172.21.15.3:0/2953073747 >> v1:172.21.15.3:6789/0 conn(0x7f2a50153480 legacy=0x7f2a500cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:34.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4ed9d640 1 -- 172.21.15.3:0/2953073747 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5015d670 con 0x7f2a501530a0 2024-09-12T12:20:34.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.290+0000 7f2a4dd9b640 1 -- 172.21.15.3:0/2953073747 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974815888 0 0) 0x7f2a5015d0b0 con 0x7f2a50153480 2024-09-12T12:20:34.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a4ed9d640 1 --2- 172.21.15.3:0/2953073747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a501530a0 0x7f2a5015c810 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009e50 tx=0x7f2a4001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcfd40b2396f512 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:34.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a4dd9b640 1 -- 172.21.15.3:0/2953073747 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400a57d0 con 0x7f2a501530a0 2024-09-12T12:20:34.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a4dd9b640 1 -- 172.21.15.3:0/2953073747 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a400a5970 con 0x7f2a501530a0 2024-09-12T12:20:34.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a4dd9b640 1 -- 172.21.15.3:0/2953073747 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400a5d40 con 0x7f2a501530a0 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- 172.21.15.3:0/2953073747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a501530a0 msgr2=0x7f2a5015c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 --2- 172.21.15.3:0/2953073747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a501530a0 0x7f2a5015c810 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009e50 tx=0x7f2a4001c9f0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- 172.21.15.3:0/2953073747 shutdown_connections 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 --2- 172.21.15.3:0/2953073747 >> v2:172.21.15.142:3300/0 conn(0x7f2a500d1cf0 0x7f2a500d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 --2- 172.21.15.3:0/2953073747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a501530a0 0x7f2a5015c810 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- 172.21.15.3:0/2953073747 >> 172.21.15.3:0/2953073747 conn(0x7f2a50070060 msgr2=0x7f2a50070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- 172.21.15.3:0/2953073747 shutdown_connections 2024-09-12T12:20:34.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- 172.21.15.3:0/2953073747 wait complete. 2024-09-12T12:20:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 Processor -- start 2024-09-12T12:20:34.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- start start 2024-09-12T12:20:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a500d1cf0 0x7f2a5006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a501530a0 0x7f2a500b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a50153480 0x7f2a5015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500d0660 con 0x7f2a50153480 2024-09-12T12:20:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500d2b60 con 0x7f2a500d1cf0 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a551be640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2a500d07e0 con 0x7f2a501530a0 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.294+0000 7f2a4f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a50153480 0x7f2a5015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a50153480 0x7f2a5015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36490/0 (socket says 172.21.15.3:36490) 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 -- 172.21.15.3:0/3246084358 learned_addr learned my addr 172.21.15.3:0/3246084358 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4e59c640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a501530a0 0x7f2a500b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4ed9d640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a500d1cf0 0x7f2a5006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 -- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a501530a0 msgr2=0x7f2a500b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a501530a0 0x7f2a500b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 -- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a500d1cf0 msgr2=0x7f2a5006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a500d1cf0 0x7f2a5006e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 -- 172.21.15.3:0/3246084358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5015d0b0 con 0x7f2a50153480 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4e59c640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a501530a0 0x7f2a500b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4ed9d640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a500d1cf0 0x7f2a5006e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4f59e640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a50153480 0x7f2a5015b2e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c3c0 tx=0x7f2a4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a477fe640 1 -- 172.21.15.3:0/3246084358 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a480074e0 con 0x7f2a50153480 2024-09-12T12:20:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a477fe640 1 -- 172.21.15.3:0/3246084358 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a480047f0 con 0x7f2a50153480 2024-09-12T12:20:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a500cb040 con 0x7f2a50153480 2024-09-12T12:20:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a477fe640 1 -- 172.21.15.3:0/3246084358 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a48007680 con 0x7f2a50153480 2024-09-12T12:20:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a500cb600 con 0x7f2a50153480 2024-09-12T12:20:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a500d2210 con 0x7f2a50153480 2024-09-12T12:20:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a477fe640 1 -- 172.21.15.3:0/3246084358 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2a48011070 con 0x7f2a50153480 2024-09-12T12:20:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a477fe640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2a300678c0 0x7f2a30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a477fe640 1 -- 172.21.15.3:0/3246084358 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f2a48085780 con 0x7f2a50153480 2024-09-12T12:20:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4ed9d640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2a300678c0 0x7f2a30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.298+0000 7f2a4ed9d640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2a300678c0 0x7f2a30069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ae000 tx=0x7f2a400a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.302+0000 7f2a477fe640 1 -- 172.21.15.3:0/3246084358 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2a480563b0 con 0x7f2a50153480 2024-09-12T12:20:34.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:34.502+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2a50152140 con 0x7f2a50153480 2024-09-12T12:20:35.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.126+0000 7f2a477fe640 1 -- 172.21.15.3:0/3246084358 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f2a48055cb0 con 0x7f2a50153480 2024-09-12T12:20:35.130 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 4 and data pool 5 2024-09-12T12:20:35.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2a300678c0 msgr2=0x7f2a30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2a300678c0 0x7f2a30069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a400ae000 tx=0x7f2a400a7070 comp rx=0 tx=0).stop 2024-09-12T12:20:35.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a50153480 msgr2=0x7f2a5015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a50153480 0x7f2a5015b2e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c3c0 tx=0x7f2a4800c880 comp rx=0 tx=0).stop 2024-09-12T12:20:35.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 shutdown_connections 2024-09-12T12:20:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2a300678c0 0x7f2a30069d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a50153480 0x7f2a5015b2e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2a501530a0 0x7f2a500b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 --2- 172.21.15.3:0/3246084358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a500d1cf0 0x7f2a5006e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 >> 172.21.15.3:0/3246084358 conn(0x7f2a50070060 msgr2=0x7f2a500c9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 shutdown_connections 2024-09-12T12:20:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.130+0000 7f2a551be640 1 -- 172.21.15.3:0/3246084358 wait complete. 2024-09-12T12:20:35.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:20:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9ba4df3640 1 Processor -- start 2024-09-12T12:20:35.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9ba4df3640 1 -- start start 2024-09-12T12:20:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9ba4df3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba015e5d0 0x7f9ba00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9ba4df3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ba00cfbe0 0x7f9ba00b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9ba4df3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba00d31a0 con 0x7f9ba00d5050 2024-09-12T12:20:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9ba4df3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f9ba00b1060 con 0x7f9ba00cfbe0 2024-09-12T12:20:35.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9ba4df3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba00d3480 con 0x7f9ba015e5d0 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9e59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba015e5d0 0x7f9ba00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9e59c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba015e5d0 0x7f9ba00ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:43202/0 (socket says 172.21.15.3:43202) 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9e59c640 1 -- 172.21.15.3:0/3613705942 learned_addr learned my addr 172.21.15.3:0/3613705942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9ed9d640 1 --2- 172.21.15.3:0/3613705942 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00cfbe0 0x7f9ba00b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9e59c640 1 -- 172.21.15.3:0/3613705942 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00cfbe0 msgr2=0x7f9ba00b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9e59c640 1 --2- 172.21.15.3:0/3613705942 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00cfbe0 0x7f9ba00b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9e59c640 1 -- 172.21.15.3:0/3613705942 >> v1:172.21.15.142:6789/0 conn(0x7f9ba00d5050 legacy=0x7f9ba00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.286+0000 7f9b9e59c640 1 -- 172.21.15.3:0/3613705942 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba00d3760 con 0x7f9ba015e5d0 2024-09-12T12:20:35.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9b9e59c640 1 --2- 172.21.15.3:0/3613705942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba015e5d0 0x7f9ba00ceb60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009870 tx=0x7f9b9001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72a63cc2355060a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9b9d59a640 1 -- 172.21.15.3:0/3613705942 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900a9640 con 0x7f9ba015e5d0 2024-09-12T12:20:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9b9d59a640 1 -- 172.21.15.3:0/3613705942 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b900a97e0 con 0x7f9ba015e5d0 2024-09-12T12:20:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9b9d59a640 1 -- 172.21.15.3:0/3613705942 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900a9bb0 con 0x7f9ba015e5d0 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 -- 172.21.15.3:0/3613705942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba015e5d0 msgr2=0x7f9ba00ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/3613705942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba015e5d0 0x7f9ba00ceb60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009870 tx=0x7f9b9001c560 comp rx=0 tx=0).stop 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 -- 172.21.15.3:0/3613705942 shutdown_connections 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/3613705942 >> v2:172.21.15.142:3300/0 conn(0x7f9ba00cfbe0 0x7f9ba00b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/3613705942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba015e5d0 0x7f9ba00ceb60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 -- 172.21.15.3:0/3613705942 >> 172.21.15.3:0/3613705942 conn(0x7f9ba008c000 msgr2=0x7f9ba00d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 -- 172.21.15.3:0/3613705942 shutdown_connections 2024-09-12T12:20:35.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 -- 172.21.15.3:0/3613705942 wait complete. 2024-09-12T12:20:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 Processor -- start 2024-09-12T12:20:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.290+0000 7f9ba4df3640 1 -- start start 2024-09-12T12:20:35.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00cfbe0 0x7f9ba015f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00d5050 0x7f9ba00d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba015e5d0 0x7f9ba00d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba00b25d0 con 0x7f9ba00d5050 2024-09-12T12:20:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba00b2450 con 0x7f9ba015e5d0 2024-09-12T12:20:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba00b2750 con 0x7f9ba00cfbe0 2024-09-12T12:20:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00d5050 0x7f9ba00d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00d5050 0x7f9ba00d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36494/0 (socket says 172.21.15.3:36494) 2024-09-12T12:20:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 -- 172.21.15.3:0/2291285241 learned_addr learned my addr 172.21.15.3:0/2291285241 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:35.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9ed9d640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba015e5d0 0x7f9ba00d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9e59c640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00cfbe0 0x7f9ba015f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 -- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00cfbe0 msgr2=0x7f9ba015f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00cfbe0 0x7f9ba015f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 -- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba015e5d0 msgr2=0x7f9ba00d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba015e5d0 0x7f9ba00d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 -- 172.21.15.3:0/2291285241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba00cd8b0 con 0x7f9ba00d5050 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9e59c640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00cfbe0 0x7f9ba015f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9ed9d640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba015e5d0 0x7f9ba00d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b9dd9b640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00d5050 0x7f9ba00d0ca0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c450 tx=0x7f9b9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b8effd640 1 -- 172.21.15.3:0/2291285241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94016020 con 0x7f9ba00d5050 2024-09-12T12:20:35.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b8effd640 1 -- 172.21.15.3:0/2291285241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b940052d0 con 0x7f9ba00d5050 2024-09-12T12:20:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba00cdb50 con 0x7f9ba00d5050 2024-09-12T12:20:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b8effd640 1 -- 172.21.15.3:0/2291285241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94007260 con 0x7f9ba00d5050 2024-09-12T12:20:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba00ce110 con 0x7f9ba00d5050 2024-09-12T12:20:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba00b14f0 con 0x7f9ba00d5050 2024-09-12T12:20:35.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.294+0000 7f9b8effd640 1 -- 172.21.15.3:0/2291285241 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9b94007400 con 0x7f9ba00d5050 2024-09-12T12:20:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.298+0000 7f9b8effd640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9b800678c0 0x7f9b80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.298+0000 7f9b8effd640 1 -- 172.21.15.3:0/2291285241 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f9b94086480 con 0x7f9ba00d5050 2024-09-12T12:20:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.298+0000 7f9b9e59c640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9b800678c0 0x7f9b80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.298+0000 7f9b9e59c640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9b800678c0 0x7f9b80069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b900ae000 tx=0x7f9b900b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.298+0000 7f9b8effd640 1 -- 172.21.15.3:0/2291285241 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9b94020020 con 0x7f9ba00d5050 2024-09-12T12:20:35.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.458+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ba00ba490 con 0x7f9ba00d5050 2024-09-12T12:20:35.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9b8effd640 1 -- 172.21.15.3:0/2291285241 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16682 (secure 0 0 0) 0x7f9b94005480 con 0x7f9ba00d5050 2024-09-12T12:20:35.465 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:35.466 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:20:34.508721+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:20:32.417571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T12:20:33.506170+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:20:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9b800678c0 msgr2=0x7f9b80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9b800678c0 0x7f9b80069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9b900ae000 tx=0x7f9b900b3040 comp rx=0 tx=0).stop 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00d5050 msgr2=0x7f9ba00d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00d5050 0x7f9ba00d0ca0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c450 tx=0x7f9b9400c910 comp rx=0 tx=0).stop 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 shutdown_connections 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9b800678c0 0x7f9b80069d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ba015e5d0 0x7f9ba00d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ba00d5050 0x7f9ba00d0ca0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 --2- 172.21.15.3:0/2291285241 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ba00cfbe0 0x7f9ba015f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 >> 172.21.15.3:0/2291285241 conn(0x7f9ba008c000 msgr2=0x7f9ba006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:35.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 shutdown_connections 2024-09-12T12:20:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.462+0000 7f9ba4df3640 1 -- 172.21.15.3:0/2291285241 wait complete. 2024-09-12T12:20:35.484 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:20:35.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 Processor -- start 2024-09-12T12:20:35.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 -- start start 2024-09-12T12:20:35.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f225815e800 0x7f22580b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f22580b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f225806bcc0 0x7f225806c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f225815f940 con 0x7f225815e800 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f225815fc20 con 0x7f225806bcc0 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f225d15f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f225815ff00 con 0x7f22580b5d50 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f225815e800 0x7f22580b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f225815e800 0x7f22580b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36510/0 (socket says 172.21.15.3:36510) 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 -- 172.21.15.3:0/4094238001 learned_addr learned my addr 172.21.15.3:0/4094238001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 -- 172.21.15.3:0/4094238001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 msgr2=0x7f22580b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 --2- 172.21.15.3:0/4094238001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f22580b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 -- 172.21.15.3:0/4094238001 >> v2:172.21.15.142:3300/0 conn(0x7f225806bcc0 msgr2=0x7f225806c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:35.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 --2- 172.21.15.3:0/4094238001 >> v2:172.21.15.142:3300/0 conn(0x7f225806bcc0 0x7f225806c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.602+0000 7f2256d9d640 1 -- 172.21.15.3:0/4094238001 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22581601e0 con 0x7f225815e800 2024-09-12T12:20:35.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f2256d9d640 1 --2- 172.21.15.3:0/4094238001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225815e800 0x7f22580b42a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2258073fe0 tx=0x7f224801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05fa5666d4f191 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f2255d9b640 1 -- 172.21.15.3:0/4094238001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480aa610 con 0x7f225815e800 2024-09-12T12:20:35.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f2255d9b640 1 -- 172.21.15.3:0/4094238001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22480af070 con 0x7f225815e800 2024-09-12T12:20:35.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f2255d9b640 1 -- 172.21.15.3:0/4094238001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480aa9d0 con 0x7f225815e800 2024-09-12T12:20:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 -- 172.21.15.3:0/4094238001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225815e800 msgr2=0x7f22580b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 --2- 172.21.15.3:0/4094238001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225815e800 0x7f22580b42a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2258073fe0 tx=0x7f224801c4f0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 -- 172.21.15.3:0/4094238001 shutdown_connections 2024-09-12T12:20:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 --2- 172.21.15.3:0/4094238001 >> v2:172.21.15.142:3300/0 conn(0x7f225806bcc0 0x7f225806c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 --2- 172.21.15.3:0/4094238001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f22580b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 --2- 172.21.15.3:0/4094238001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225815e800 0x7f22580b42a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 -- 172.21.15.3:0/4094238001 >> 172.21.15.3:0/4094238001 conn(0x7f2258073500 msgr2=0x7f22580b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 -- 172.21.15.3:0/4094238001 shutdown_connections 2024-09-12T12:20:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 -- 172.21.15.3:0/4094238001 wait complete. 2024-09-12T12:20:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 Processor -- start 2024-09-12T12:20:35.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 -- start start 2024-09-12T12:20:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225806bcc0 0x7f2258161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.606+0000 7f225d15f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f225809c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225d15f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f225815e800 0x7f225814a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225d15f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f22580b47e0 con 0x7f225806bcc0 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225d15f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f22580b66d0 con 0x7f225815e800 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225d15f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f22580b2b10 con 0x7f22580b5d50 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225806bcc0 0x7f2258161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225806bcc0 0x7f2258161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36526/0 (socket says 172.21.15.3:36526) 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 -- 172.21.15.3:0/3959769144 learned_addr learned my addr 172.21.15.3:0/3959769144 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225759e640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f225815e800 0x7f225814a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225659c640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f225809c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 -- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 msgr2=0x7f225809c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f225809c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 -- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f225815e800 msgr2=0x7f225814a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f225815e800 0x7f225814a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 -- 172.21.15.3:0/3959769144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225814e590 con 0x7f225806bcc0 2024-09-12T12:20:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225659c640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f225809c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f2256d9d640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225806bcc0 0x7f2258161450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f224801aba0 tx=0x7f224801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f223f7fe640 1 -- 172.21.15.3:0/3959769144 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480bd5f0 con 0x7f225806bcc0 2024-09-12T12:20:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f223f7fe640 1 -- 172.21.15.3:0/3959769144 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22480bd7b0 con 0x7f225806bcc0 2024-09-12T12:20:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225814e870 con 0x7f225806bcc0 2024-09-12T12:20:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f223f7fe640 1 -- 172.21.15.3:0/3959769144 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480bdba0 con 0x7f225806bcc0 2024-09-12T12:20:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225814ee30 con 0x7f225806bcc0 2024-09-12T12:20:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22580b42a0 con 0x7f225806bcc0 2024-09-12T12:20:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f223f7fe640 1 -- 172.21.15.3:0/3959769144 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f22480b33f0 con 0x7f225806bcc0 2024-09-12T12:20:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f223f7fe640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22440678f0 0x7f2244069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f223f7fe640 1 -- 172.21.15.3:0/3959769144 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f2248120ad0 con 0x7f225806bcc0 2024-09-12T12:20:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225659c640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22440678f0 0x7f2244069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.610+0000 7f225659c640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22440678f0 0x7f2244069db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f225814fd20 tx=0x7f224c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.614+0000 7f223f7fe640 1 -- 172.21.15.3:0/3959769144 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f22480c3060 con 0x7f225806bcc0 2024-09-12T12:20:35.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.778+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22580d0ba0 con 0x7f225806bcc0 2024-09-12T12:20:35.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f223f7fe640 1 -- 172.21.15.3:0/3959769144 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16682 (secure 0 0 0) 0x7f22480a7070 con 0x7f225806bcc0 2024-09-12T12:20:35.787 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:35.787 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":31,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:20:34.508721+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:20:32.417571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T12:20:33.506170+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:20:35.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22440678f0 msgr2=0x7f2244069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22440678f0 0x7f2244069db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f225814fd20 tx=0x7f224c008040 comp rx=0 tx=0).stop 2024-09-12T12:20:35.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225806bcc0 msgr2=0x7f2258161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225806bcc0 0x7f2258161450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f224801aba0 tx=0x7f224801ad30 comp rx=0 tx=0).stop 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 shutdown_connections 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22440678f0 0x7f2244069db0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f225815e800 0x7f225814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22580b5d50 0x7f225809c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 --2- 172.21.15.3:0/3959769144 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f225806bcc0 0x7f2258161450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 >> 172.21.15.3:0/3959769144 conn(0x7f2258073500 msgr2=0x7f225815c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 shutdown_connections 2024-09-12T12:20:35.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.782+0000 7f225d15f640 1 -- 172.21.15.3:0/3959769144 wait complete. 2024-09-12T12:20:35.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:20:35.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd00c914640 1 Processor -- start 2024-09-12T12:20:35.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd00c914640 1 -- start start 2024-09-12T12:20:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd00c914640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd00806f150 0x7fd0080ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd00c914640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd00806f5b0 0x7fd0080ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd00c914640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00806e0b0 con 0x7fd008159dd0 2024-09-12T12:20:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd00c914640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd00806e390 con 0x7fd00806f150 2024-09-12T12:20:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd00c914640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd00806e670 con 0x7fd00806f5b0 2024-09-12T12:20:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd005d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd00806f150 0x7fd0080ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd005d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd00806f150 0x7fd0080ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36530/0 (socket says 172.21.15.3:36530) 2024-09-12T12:20:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd005d9b640 1 -- 172.21.15.3:0/3106999286 learned_addr learned my addr 172.21.15.3:0/3106999286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd006d9d640 1 --2- 172.21.15.3:0/3106999286 >> v2:172.21.15.142:3300/0 conn(0x7fd00806f5b0 0x7fd0080ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd005d9b640 1 -- 172.21.15.3:0/3106999286 >> v2:172.21.15.142:3300/0 conn(0x7fd00806f5b0 msgr2=0x7fd0080ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd005d9b640 1 --2- 172.21.15.3:0/3106999286 >> v2:172.21.15.142:3300/0 conn(0x7fd00806f5b0 0x7fd0080ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd005d9b640 1 -- 172.21.15.3:0/3106999286 >> v1:172.21.15.142:6789/0 conn(0x7fd008159dd0 legacy=0x7fd00815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.930+0000 7fd005d9b640 1 -- 172.21.15.3:0/3106999286 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00806e950 con 0x7fd00806f150 2024-09-12T12:20:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd005d9b640 1 --2- 172.21.15.3:0/3106999286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00806f150 0x7fd0080ad060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd0080701d0 tx=0x7fcff001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80f055dc4191f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00559a640 1 -- 172.21.15.3:0/3106999286 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00a58a0 con 0x7fd00806f150 2024-09-12T12:20:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00559a640 1 -- 172.21.15.3:0/3106999286 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff00a5a40 con 0x7fd00806f150 2024-09-12T12:20:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00559a640 1 -- 172.21.15.3:0/3106999286 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00a5e10 con 0x7fd00806f150 2024-09-12T12:20:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- 172.21.15.3:0/3106999286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00806f150 msgr2=0x7fd0080ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 --2- 172.21.15.3:0/3106999286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00806f150 0x7fd0080ad060 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd0080701d0 tx=0x7fcff001c9c0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- 172.21.15.3:0/3106999286 shutdown_connections 2024-09-12T12:20:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 --2- 172.21.15.3:0/3106999286 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd00806f150 0x7fd0080ad060 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 --2- 172.21.15.3:0/3106999286 >> v2:172.21.15.142:3300/0 conn(0x7fd00806f5b0 0x7fd0080ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- 172.21.15.3:0/3106999286 >> 172.21.15.3:0/3106999286 conn(0x7fd0080b1d80 msgr2=0x7fd0080b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- 172.21.15.3:0/3106999286 shutdown_connections 2024-09-12T12:20:35.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- 172.21.15.3:0/3106999286 wait complete. 2024-09-12T12:20:35.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 Processor -- start 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- start start 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00806f150 0x7fd008072a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd00806f5b0 0x7fd0081669e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd008159dd0 0x7fd008144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080b2bc0 con 0x7fd008159dd0 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080b2a40 con 0x7fd00806f150 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.934+0000 7fd00c914640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd008169200 con 0x7fd00806f5b0 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd008159dd0 0x7fd008144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd008159dd0 0x7fd008144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36534/0 (socket says 172.21.15.3:36534) 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 -- 172.21.15.3:0/3682867 learned_addr learned my addr 172.21.15.3:0/3682867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:35.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd005d9b640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd00806f5b0 0x7fd0081669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd00659c640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00806f150 0x7fd008072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 -- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd00806f5b0 msgr2=0x7fd0081669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd00806f5b0 0x7fd0081669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 -- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00806f150 msgr2=0x7fd008072a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00806f150 0x7fd008072a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 -- 172.21.15.3:0/3682867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd008145e90 con 0x7fd008159dd0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd006d9d640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd008159dd0 0x7fd008144f10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd00000d800 tx=0x7fd00000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fcffeffd640 1 -- 172.21.15.3:0/3682867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00000b660 con 0x7fd008159dd0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fcffeffd640 1 -- 172.21.15.3:0/3682867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd000019070 con 0x7fd008159dd0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fcffeffd640 1 -- 172.21.15.3:0/3682867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00000ba40 con 0x7fd008159dd0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd008165940 con 0x7fd008159dd0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd008165f00 con 0x7fd008159dd0 2024-09-12T12:20:35.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00815d2f0 con 0x7fd008159dd0 2024-09-12T12:20:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fcffeffd640 1 -- 172.21.15.3:0/3682867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd00000bbe0 con 0x7fd008159dd0 2024-09-12T12:20:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fcffeffd640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcfe80678c0 0x7fcfe8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fcffeffd640 1 -- 172.21.15.3:0/3682867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd000086170 con 0x7fd008159dd0 2024-09-12T12:20:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd00659c640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcfe80678c0 0x7fcfe8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:35.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.938+0000 7fd00659c640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcfe80678c0 0x7fcfe8069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcff80015b0 tx=0x7fcff8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:35.942+0000 7fcffeffd640 1 -- 172.21.15.3:0/3682867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd000056c10 con 0x7fd008159dd0 2024-09-12T12:20:36.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.142+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd008153740 con 0x7fd008159dd0 2024-09-12T12:20:36.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.166+0000 7fcffeffd640 1 -- 172.21.15.3:0/3682867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7fd000056510 con 0x7fd008159dd0 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcfe80678c0 msgr2=0x7fcfe8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcfe80678c0 0x7fcfe8069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcff80015b0 tx=0x7fcff8009040 comp rx=0 tx=0).stop 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd008159dd0 msgr2=0x7fd008144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd008159dd0 0x7fd008144f10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd00000d800 tx=0x7fd00000dcc0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 shutdown_connections 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcfe80678c0 0x7fcfe8069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd008159dd0 0x7fd008144f10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd00806f5b0 0x7fd0081669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 --2- 172.21.15.3:0/3682867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd00806f150 0x7fd008072a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 >> 172.21.15.3:0/3682867 conn(0x7fd0080b1d80 msgr2=0x7fd0080aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 shutdown_connections 2024-09-12T12:20:36.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.170+0000 7fd00c914640 1 -- 172.21.15.3:0/3682867 wait complete. 2024-09-12T12:20:36.191 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T12:20:36.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T12:20:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f3fbb640 1 Processor -- start 2024-09-12T12:20:36.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f3fbb640 1 -- start start 2024-09-12T12:20:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f3fbb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec15e5f0 0x7f26ec0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f3fbb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26ec06fe30 0x7f26ec06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f3fbb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26ec0aead0 con 0x7f26ec06bcc0 2024-09-12T12:20:36.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f3fbb640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f26ec0af940 con 0x7f26ec06fe30 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f3fbb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26ec0aed70 con 0x7f26ec15e5f0 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1556640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26ec06fe30 0x7f26ec06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1556640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26ec06fe30 0x7f26ec06e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36546/0 (socket says 172.21.15.3:36546) 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1556640 1 -- 172.21.15.3:0/4056809845 learned_addr learned my addr 172.21.15.3:0/4056809845 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1d57640 1 --2- 172.21.15.3:0/4056809845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec15e5f0 0x7f26ec0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1556640 1 -- 172.21.15.3:0/4056809845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec15e5f0 msgr2=0x7f26ec0afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1556640 1 --2- 172.21.15.3:0/4056809845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec15e5f0 0x7f26ec0afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1556640 1 -- 172.21.15.3:0/4056809845 >> v1:172.21.15.142:6789/0 conn(0x7f26ec06bcc0 legacy=0x7f26ec0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:36.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.310+0000 7f26f1556640 1 -- 172.21.15.3:0/4056809845 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec0af010 con 0x7f26ec06fe30 2024-09-12T12:20:36.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f1556640 1 --2- 172.21.15.3:0/4056809845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec06fe30 0x7f26ec06e7c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009e50 tx=0x7f26ec0b4a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe721d0d71de2589 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:36.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f0d55640 1 -- 172.21.15.3:0/4056809845 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80a5670 con 0x7f26ec06fe30 2024-09-12T12:20:36.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f0d55640 1 -- 172.21.15.3:0/4056809845 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e80a5810 con 0x7f26ec06fe30 2024-09-12T12:20:36.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f0d55640 1 -- 172.21.15.3:0/4056809845 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80a5be0 con 0x7f26ec06fe30 2024-09-12T12:20:36.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 -- 172.21.15.3:0/4056809845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec06fe30 msgr2=0x7f26ec06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:36.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/4056809845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec06fe30 0x7f26ec06e7c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009e50 tx=0x7f26ec0b4a30 comp rx=0 tx=0).stop 2024-09-12T12:20:36.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 -- 172.21.15.3:0/4056809845 shutdown_connections 2024-09-12T12:20:36.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/4056809845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec06fe30 0x7f26ec06e7c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/4056809845 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec15e5f0 0x7f26ec0afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 -- 172.21.15.3:0/4056809845 >> 172.21.15.3:0/4056809845 conn(0x7f26ec0b29e0 msgr2=0x7f26ec0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:36.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 -- 172.21.15.3:0/4056809845 shutdown_connections 2024-09-12T12:20:36.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 -- 172.21.15.3:0/4056809845 wait complete. 2024-09-12T12:20:36.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 Processor -- start 2024-09-12T12:20:36.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 -- start start 2024-09-12T12:20:36.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec06bcc0 0x7f26ec1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:36.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.314+0000 7f26f3fbb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec06fe30 0x7f26ec164020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:36.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f3fbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec15e5f0 0x7f26ec152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f3fbb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec16b3a0 con 0x7f26ec15e5f0 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f3fbb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26ec16b220 con 0x7f26ec06fe30 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f3fbb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26ec16b520 con 0x7f26ec06bcc0 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec15e5f0 0x7f26ec152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec15e5f0 0x7f26ec152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36558/0 (socket says 172.21.15.3:36558) 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 -- 172.21.15.3:0/2083843462 learned_addr learned my addr 172.21.15.3:0/2083843462 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f1556640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec06fe30 0x7f26ec164020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 -- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec06bcc0 msgr2=0x7f26ec1514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:36.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f1d57640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec06bcc0 0x7f26ec1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec06bcc0 0x7f26ec1514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 -- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec06fe30 msgr2=0x7f26ec164020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec06fe30 0x7f26ec164020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 -- 172.21.15.3:0/2083843462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ec167ed0 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f2558640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec15e5f0 0x7f26ec152bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26e400b410 tx=0x7f26e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26e27fc640 1 -- 172.21.15.3:0/2083843462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4015020 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26e27fc640 1 -- 172.21.15.3:0/2083843462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e4011870 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ec165fd0 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26e27fc640 1 -- 172.21.15.3:0/2083843462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4011c60 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ec164560 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ec06e7c0 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26e27fc640 1 -- 172.21.15.3:0/2083843462 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f26e4018020 con 0x7f26ec15e5f0 2024-09-12T12:20:36.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26e27fc640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26d00678f0 0x7f26d0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:36.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26e27fc640 1 -- 172.21.15.3:0/2083843462 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f26e4085830 con 0x7f26ec15e5f0 2024-09-12T12:20:36.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.318+0000 7f26f1d57640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26d00678f0 0x7f26d0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:36.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.322+0000 7f26f1d57640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26d00678f0 0x7f26d0069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009790 tx=0x7f26dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:36.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.322+0000 7f26e27fc640 1 -- 172.21.15.3:0/2083843462 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f26e401f030 con 0x7f26ec15e5f0 2024-09-12T12:20:36.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:36.522+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f26ec166480 con 0x7f26ec15e5f0 2024-09-12T12:20:37.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.174+0000 7f26e27fc640 1 -- 172.21.15.3:0/2083843462 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f26e4056360 con 0x7f26ec15e5f0 2024-09-12T12:20:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26d00678f0 msgr2=0x7f26d0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26d00678f0 0x7f26d0069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009790 tx=0x7f26dc008040 comp rx=0 tx=0).stop 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec15e5f0 msgr2=0x7f26ec152bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec15e5f0 0x7f26ec152bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26e400b410 tx=0x7f26e400b8d0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 shutdown_connections 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26d00678f0 0x7f26d0069db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26ec15e5f0 0x7f26ec152bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26ec06fe30 0x7f26ec164020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 --2- 172.21.15.3:0/2083843462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26ec06bcc0 0x7f26ec1514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 >> 172.21.15.3:0/2083843462 conn(0x7f26ec0b29e0 msgr2=0x7f26ec160e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 shutdown_connections 2024-09-12T12:20:37.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.178+0000 7f26f3fbb640 1 -- 172.21.15.3:0/2083843462 wait complete. 2024-09-12T12:20:37.202 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T12:20:37.202 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ed5722640 1 Processor -- start 2024-09-12T12:20:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ed5722640 1 -- start start 2024-09-12T12:20:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ed5722640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3ed006f6e0 0x7f3ed0072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ed5722640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 0x7f3ed0073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ed5722640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed015f150 con 0x7f3ed015b2d0 2024-09-12T12:20:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ed5722640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed00b3790 con 0x7f3ed006f6e0 2024-09-12T12:20:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ed5722640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed00ae920 con 0x7f3ed00cc300 2024-09-12T12:20:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3ed015b2d0 0x7f3ed00b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:51750/0 (socket says 172.21.15.3:51750) 2024-09-12T12:20:37.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 -- 172.21.15.3:0/3067163531 learned_addr learned my addr 172.21.15.3:0/3067163531 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ec67fc640 1 --2- 172.21.15.3:0/3067163531 >> v2:172.21.15.142:3300/0 conn(0x7f3ed006f6e0 0x7f3ed0072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ecf7fe640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 0x7f3ed0073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645500589 0 0) 0x7f3ed015f150 con 0x7f3ed015b2d0 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ebc003530 con 0x7f3ed015b2d0 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2323609010 0 0) 0x7f3ebc003530 con 0x7f3ed015b2d0 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 msgr2=0x7f3ed0073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 0x7f3ed0073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 >> v2:172.21.15.142:3300/0 conn(0x7f3ed006f6e0 msgr2=0x7f3ed0072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 --2- 172.21.15.3:0/3067163531 >> v2:172.21.15.142:3300/0 conn(0x7f3ed006f6e0 0x7f3ed0072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed015b6b0 con 0x7f3ed015b2d0 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f3ec0001680 con 0x7f3ed015b2d0 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 >> v1:172.21.15.3:6789/0 conn(0x7f3ed015b2d0 legacy=0x7f3ed00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ebc0039c0 0x7f3ebc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 0x7f3ebc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc004d50 0x7f3ebc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed00b3790 con 0x7f3ebc0039c0 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed00ae920 con 0x7f3ebc004d50 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ebc00b9d0 con 0x7f3ed00cc300 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ebc0039c0 0x7f3ebc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ec67fc640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 0x7f3ebc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3ecf7fe640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc004d50 0x7f3ebc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 -- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 msgr2=0x7f3ebc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed00cc300 0x7f3ebc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 -- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc004d50 msgr2=0x7f3ebc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ebc004d50 0x7f3ebc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.326+0000 7f3eceffd640 1 -- 172.21.15.3:0/3067163531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ebc003530 con 0x7f3ebc0039c0 2024-09-12T12:20:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3eceffd640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ebc0039c0 0x7f3ebc003de0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c270 tx=0x7f3ec001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2487ebb41ce7e6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0007d60 con 0x7f3ebc0039c0 2024-09-12T12:20:37.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec00a9020 con 0x7f3ebc0039c0 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ece7fc640 1 -- 172.21.15.3:0/3067163531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0005ae0 con 0x7f3ebc0039c0 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- 172.21.15.3:0/3067163531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ebc0039c0 msgr2=0x7f3ebc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 --2- 172.21.15.3:0/3067163531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ebc0039c0 0x7f3ebc003de0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c270 tx=0x7f3ec001cc90 comp rx=0 tx=0).stop 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ecf7fe640 1 -- 172.21.15.3:0/3067163531 reap_dead start 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- 172.21.15.3:0/3067163531 shutdown_connections 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- 172.21.15.3:0/3067163531 >> 172.21.15.3:0/3067163531 conn(0x7f3ed00b1d80 msgr2=0x7f3ed00b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- 172.21.15.3:0/3067163531 shutdown_connections 2024-09-12T12:20:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- 172.21.15.3:0/3067163531 wait complete. 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 Processor -- start 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- start start 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed00b4250 0x7f3ed0099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed01556f0 0x7f3ed0099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed01495d0 0x7f3ed0157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed006d8d0 con 0x7f3ed00b4250 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed00ae5b0 con 0x7f3ed01495d0 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3ed5722640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed00af5a0 con 0x7f3ed01556f0 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3eceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed00b4250 0x7f3ed0099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3eceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed00b4250 0x7f3ed0099520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36568/0 (socket says 172.21.15.3:36568) 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.330+0000 7f3eceffd640 1 -- 172.21.15.3:0/1191638985 learned_addr learned my addr 172.21.15.3:0/1191638985 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec67fc640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed01556f0 0x7f3ed0099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3eceffd640 1 -- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed01556f0 msgr2=0x7f3ed0099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3eceffd640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed01556f0 0x7f3ed0099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ecf7fe640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed01495d0 0x7f3ed0157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3eceffd640 1 -- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed01495d0 msgr2=0x7f3ed0157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3eceffd640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed01495d0 0x7f3ed0157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3eceffd640 1 -- 172.21.15.3:0/1191638985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed0155c60 con 0x7f3ed00b4250 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3eceffd640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed00b4250 0x7f3ed0099520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c190 tx=0x7f3ebc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec7fff640 1 -- 172.21.15.3:0/1191638985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0001ca0 con 0x7f3ed00b4250 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec7fff640 1 -- 172.21.15.3:0/1191638985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec0004ab0 con 0x7f3ed00b4250 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed0155f40 con 0x7f3ed00b4250 2024-09-12T12:20:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec7fff640 1 -- 172.21.15.3:0/1191638985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0004e60 con 0x7f3ed00b4250 2024-09-12T12:20:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed0156500 con 0x7f3ed00b4250 2024-09-12T12:20:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed015eb40 con 0x7f3ed00b4250 2024-09-12T12:20:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec7fff640 1 -- 172.21.15.3:0/1191638985 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3ec00bd060 con 0x7f3ed00b4250 2024-09-12T12:20:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec7fff640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3eb00678f0 0x7f3eb0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec7fff640 1 -- 172.21.15.3:0/1191638985 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3ec011ee00 con 0x7f3ed00b4250 2024-09-12T12:20:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec67fc640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3eb00678f0 0x7f3eb0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.334+0000 7f3ec67fc640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3eb00678f0 0x7f3eb0069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ed014d340 tx=0x7f3eb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:37.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.338+0000 7f3ec7fff640 1 -- 172.21.15.3:0/1191638985 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3ec00169d0 con 0x7f3ed00b4250 2024-09-12T12:20:37.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.538+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed006e910 con 0x7f3ed00b4250 2024-09-12T12:20:37.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.538+0000 7f3ec7fff640 1 -- 172.21.15.3:0/1191638985 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5220 (secure 0 0 0) 0x7f3ec00bd350 con 0x7f3ed00b4250 2024-09-12T12:20:37.543 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:37.544 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:20:34.509216+0000","modified":"2024-09-12T12:20:36.529296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4812":{"gid":4812,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4232231567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4232231567},{"type":"v1","addr":"172.21.15.3:6835","nonce":4232231567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4812,"qdb_cluster":[4812]},"id":2}]} 2024-09-12T12:20:37.544 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 14 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3eb00678f0 msgr2=0x7f3eb0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3eb00678f0 0x7f3eb0069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ed014d340 tx=0x7f3eb8008040 comp rx=0 tx=0).stop 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed00b4250 msgr2=0x7f3ed0099520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed00b4250 0x7f3ed0099520 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000c190 tx=0x7f3ebc005840 comp rx=0 tx=0).stop 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 shutdown_connections 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3eb00678f0 0x7f3eb0069db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed01495d0 0x7f3ed0157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3ed01556f0 0x7f3ed0099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 --2- 172.21.15.3:0/1191638985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ed00b4250 0x7f3ed0099520 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 >> 172.21.15.3:0/1191638985 conn(0x7f3ed00b1d80 msgr2=0x7f3ed006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 shutdown_connections 2024-09-12T12:20:37.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.542+0000 7f3ed5722640 1 -- 172.21.15.3:0/1191638985 wait complete. 2024-09-12T12:20:37.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:20:37.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9b1119640 1 Processor -- start 2024-09-12T12:20:37.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9b1119640 1 -- start start 2024-09-12T12:20:37.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9b1119640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9b1119640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9ac0cc300 0x7fc9ac073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9b1119640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9ac15f150 con 0x7fc9ac15b2d0 2024-09-12T12:20:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9b1119640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ac0b3790 con 0x7fc9ac0cc300 2024-09-12T12:20:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9b1119640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ac0ae920 con 0x7fc9ac06f6e0 2024-09-12T12:20:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9ab59e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9ac0cc300 0x7fc9ac073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9aad9d640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fc9ac15b2d0 0x7fc9ac0b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:40068/0 (socket says 172.21.15.3:40068) 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9a259c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9aad9d640 1 -- 172.21.15.3:0/326222882 learned_addr learned my addr 172.21.15.3:0/326222882 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.678+0000 7fc9ab59e640 1 --2- 172.21.15.3:0/326222882 >> v2:172.21.15.142:3300/0 conn(0x7fc9ac0cc300 0x7fc9ac073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:45114/0 (socket says 172.21.15.3:45114) 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9a259c640 1 -- 172.21.15.3:0/326222882 >> v2:172.21.15.142:3300/0 conn(0x7fc9ac0cc300 msgr2=0x7fc9ac073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9a259c640 1 --2- 172.21.15.3:0/326222882 >> v2:172.21.15.142:3300/0 conn(0x7fc9ac0cc300 0x7fc9ac073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9a259c640 1 -- 172.21.15.3:0/326222882 >> v1:172.21.15.142:6789/0 conn(0x7fc9ac15b2d0 legacy=0x7fc9ac0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9a259c640 1 -- 172.21.15.3:0/326222882 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac15b6b0 con 0x7fc9ac06f6e0 2024-09-12T12:20:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9a259c640 1 --2- 172.21.15.3:0/326222882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac072f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9940097c0 tx=0x7fc99401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f64c7144502c3ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9aa59c640 1 -- 172.21.15.3:0/326222882 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940a7070 con 0x7fc9ac06f6e0 2024-09-12T12:20:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9aa59c640 1 -- 172.21.15.3:0/326222882 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9940b0070 con 0x7fc9ac06f6e0 2024-09-12T12:20:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9aa59c640 1 -- 172.21.15.3:0/326222882 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940ab890 con 0x7fc9ac06f6e0 2024-09-12T12:20:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9b1119640 1 -- 172.21.15.3:0/326222882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 msgr2=0x7fc9ac072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.682+0000 7fc9b1119640 1 --2- 172.21.15.3:0/326222882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac072f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9940097c0 tx=0x7fc99401c470 comp rx=0 tx=0).stop 2024-09-12T12:20:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- 172.21.15.3:0/326222882 shutdown_connections 2024-09-12T12:20:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 --2- 172.21.15.3:0/326222882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac072f30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 --2- 172.21.15.3:0/326222882 >> v2:172.21.15.142:3300/0 conn(0x7fc9ac0cc300 0x7fc9ac073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- 172.21.15.3:0/326222882 >> 172.21.15.3:0/326222882 conn(0x7fc9ac0b1d80 msgr2=0x7fc9ac0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- 172.21.15.3:0/326222882 shutdown_connections 2024-09-12T12:20:37.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- 172.21.15.3:0/326222882 wait complete. 2024-09-12T12:20:37.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 Processor -- start 2024-09-12T12:20:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- start start 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac0cc300 0x7fc9ac09d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9ac15b2d0 0x7fc9ac09d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac06d8b0 con 0x7fc9ac0cc300 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac0ae5b0 con 0x7fc9ac15b2d0 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ac0af5a0 con 0x7fc9ac06f6e0 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac0cc300 0x7fc9ac09d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac0cc300 0x7fc9ac09d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36584/0 (socket says 172.21.15.3:36584) 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 -- 172.21.15.3:0/2754207477 learned_addr learned my addr 172.21.15.3:0/2754207477 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9ab59e640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9ac15b2d0 0x7fc9ac09d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9aad9d640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 -- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 msgr2=0x7fc9ac1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac1609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 -- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9ac15b2d0 msgr2=0x7fc9ac09d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9ac15b2d0 0x7fc9ac09d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 -- 172.21.15.3:0/2754207477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac162610 con 0x7fc9ac0cc300 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a259c640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac0cc300 0x7fc9ac09d240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc99401ab20 tx=0x7fc99401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a3fff640 1 -- 172.21.15.3:0/2754207477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940ab880 con 0x7fc9ac0cc300 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a3fff640 1 -- 172.21.15.3:0/2754207477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9940aba20 con 0x7fc9ac0cc300 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9a3fff640 1 -- 172.21.15.3:0/2754207477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940abe10 con 0x7fc9ac0cc300 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.686+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac1628f0 con 0x7fc9ac0cc300 2024-09-12T12:20:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.690+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac162eb0 con 0x7fc9ac0cc300 2024-09-12T12:20:37.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.690+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac0b3080 con 0x7fc9ac0cc300 2024-09-12T12:20:37.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.690+0000 7fc9a3fff640 1 -- 172.21.15.3:0/2754207477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9940c6060 con 0x7fc9ac0cc300 2024-09-12T12:20:37.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.690+0000 7fc9a3fff640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc98c0678c0 0x7fc98c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.690+0000 7fc9a3fff640 1 -- 172.21.15.3:0/2754207477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc994121bd0 con 0x7fc9ac0cc300 2024-09-12T12:20:37.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.690+0000 7fc9aad9d640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc98c0678c0 0x7fc98c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.690+0000 7fc9aad9d640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc98c0678c0 0x7fc98c069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac1551c0 tx=0x7fc99c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:37.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.694+0000 7fc9a3fff640 1 -- 172.21.15.3:0/2754207477 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc9940ba070 con 0x7fc9ac0cc300 2024-09-12T12:20:37.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.858+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9ac0afe20 con 0x7fc9ac0cc300 2024-09-12T12:20:37.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.858+0000 7fc9a3fff640 1 -- 172.21.15.3:0/2754207477 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7fc994016090 con 0x7fc9ac0cc300 2024-09-12T12:20:37.865 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:37.865 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:20:36.129741+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:20:32.417571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T12:20:33.506170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:20:37.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc98c0678c0 msgr2=0x7fc98c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc98c0678c0 0x7fc98c069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac1551c0 tx=0x7fc99c009040 comp rx=0 tx=0).stop 2024-09-12T12:20:37.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac0cc300 msgr2=0x7fc9ac09d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac0cc300 0x7fc9ac09d240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc99401ab20 tx=0x7fc99401acb0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 shutdown_connections 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc98c0678c0 0x7fc98c069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9ac15b2d0 0x7fc9ac09d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9ac0cc300 0x7fc9ac09d240 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 --2- 172.21.15.3:0/2754207477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9ac06f6e0 0x7fc9ac1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 >> 172.21.15.3:0/2754207477 conn(0x7fc9ac0b1d80 msgr2=0x7fc9ac15bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 shutdown_connections 2024-09-12T12:20:37.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.862+0000 7fc9b1119640 1 -- 172.21.15.3:0/2754207477 wait complete. 2024-09-12T12:20:37.884 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:20:37.884 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:20:37.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb3f77640 1 Processor -- start 2024-09-12T12:20:37.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb3f77640 1 -- start start 2024-09-12T12:20:37.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb3f77640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5bac159fb0 0x7f5bac06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb3f77640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bac06fbb0 con 0x7f5bac159bd0 2024-09-12T12:20:37.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb3f77640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bac15d760 con 0x7f5bac15d2d0 2024-09-12T12:20:37.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb3f77640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5bac06f010 con 0x7f5bac159fb0 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb1d13640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5bac159bd0 0x7f5bac06f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53628/0 (socket says 172.21.15.3:53628) 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb1d13640 1 -- 172.21.15.3:0/868144000 learned_addr learned my addr 172.21.15.3:0/868144000 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444580864 0 0) 0x7f5bac06fbb0 con 0x7f5bac159bd0 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b94003530 con 0x7f5bac159bd0 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb1512640 1 --2- 172.21.15.3:0/868144000 >> v2:172.21.15.142:3300/0 conn(0x7f5bac159fb0 0x7f5bac06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 511899308 0 0) 0x7f5b94003530 con 0x7f5bac159bd0 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 >> v2:172.21.15.142:3300/0 conn(0x7f5bac159fb0 msgr2=0x7f5bac06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 --2- 172.21.15.3:0/868144000 >> v2:172.21.15.142:3300/0 conn(0x7f5bac159fb0 0x7f5bac06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 >> v1:172.21.15.142:6789/0 conn(0x7f5bac15d2d0 legacy=0x7f5bac0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac06e400 con 0x7f5bac159bd0 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f5ba4001680 con 0x7f5bac159bd0 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb1512640 1 --2- 172.21.15.3:0/868144000 >> v2:172.21.15.142:3300/0 conn(0x7f5bac159fb0 0x7f5bac06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:37.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 >> v1:172.21.15.3:6789/0 conn(0x7f5bac159bd0 legacy=0x7f5bac06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940039c0 0x7f5b94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b94004810 0x7f5b94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5b9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac06f010 con 0x7f5b94005220 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9400c150 con 0x7f5b940039c0 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9400c350 con 0x7f5b94004810 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb2514640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5b9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb1d13640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940039c0 0x7f5b94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb2514640 1 -- 172.21.15.3:0/868144000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b94004810 msgr2=0x7f5b94004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb2514640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b94004810 0x7f5b94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb2514640 1 -- 172.21.15.3:0/868144000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940039c0 msgr2=0x7f5b94003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb2514640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b940039c0 0x7f5b94003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:37.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.990+0000 7f5bb2514640 1 -- 172.21.15.3:0/868144000 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b94003530 con 0x7f5b94005220 2024-09-12T12:20:38.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb2514640 1 -- 172.21.15.3:0/868144000 reap_dead start 2024-09-12T12:20:38.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb2514640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5b9400b9d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c001e70 tx=0x7f5b9c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a37531283aa72e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c001a90 con 0x7f5b94005220 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b9c003ba0 con 0x7f5b94005220 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb0d11640 1 -- 172.21.15.3:0/868144000 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c005020 con 0x7f5b94005220 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 -- 172.21.15.3:0/868144000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 msgr2=0x7f5b9400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5b9400b9d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c001e70 tx=0x7f5b9c01c780 comp rx=0 tx=0).stop 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 -- 172.21.15.3:0/868144000 shutdown_connections 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/868144000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5b9400b9d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 -- 172.21.15.3:0/868144000 >> 172.21.15.3:0/868144000 conn(0x7f5bac0b1d80 msgr2=0x7f5bac0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 -- 172.21.15.3:0/868144000 shutdown_connections 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 -- 172.21.15.3:0/868144000 wait complete. 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 Processor -- start 2024-09-12T12:20:38.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.994+0000 7f5bb3f77640 1 -- start start 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5bac0999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bac09d030 0x7f5bac0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bac09d470 0x7f5bac0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac06eca0 con 0x7f5b94005220 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac06e9c0 con 0x7f5bac09d030 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5bac06dd40 con 0x7f5bac09d470 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5bac0999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5bac0999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53666/0 (socket says 172.21.15.3:53666) 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 -- 172.21.15.3:0/3312381165 learned_addr learned my addr 172.21.15.3:0/3312381165 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1512640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bac09d030 0x7f5bac0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb2514640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bac09d470 0x7f5bac0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 -- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bac09d470 msgr2=0x7f5bac0b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bac09d470 0x7f5bac0b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 -- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bac09d030 msgr2=0x7f5bac0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bac09d030 0x7f5bac0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 -- 172.21.15.3:0/3312381165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac099ef0 con 0x7f5b94005220 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1512640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bac09d030 0x7f5bac0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1d13640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5bac0999b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400e780 tx=0x7f5ba400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5baa7fc640 1 -- 172.21.15.3:0/3312381165 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400c6c0 con 0x7f5b94005220 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5baa7fc640 1 -- 172.21.15.3:0/3312381165 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba4018070 con 0x7f5b94005220 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac09da40 con 0x7f5b94005220 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5baa7fc640 1 -- 172.21.15.3:0/3312381165 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba400caa0 con 0x7f5b94005220 2024-09-12T12:20:38.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac1f27b0 con 0x7f5b94005220 2024-09-12T12:20:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bac06e0d0 con 0x7f5b94005220 2024-09-12T12:20:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5baa7fc640 1 -- 172.21.15.3:0/3312381165 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ba400cc40 con 0x7f5b94005220 2024-09-12T12:20:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5baa7fc640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5b980678c0 0x7f5b98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5baa7fc640 1 -- 172.21.15.3:0/3312381165 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5ba40859e0 con 0x7f5b94005220 2024-09-12T12:20:38.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:37.998+0000 7f5bb1512640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5b980678c0 0x7f5b98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.002+0000 7f5bb1512640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5b980678c0 0x7f5b98069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bac145f80 tx=0x7f5ba0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.002+0000 7f5baa7fc640 1 -- 172.21.15.3:0/3312381165 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5ba4056520 con 0x7f5b94005220 2024-09-12T12:20:38.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.162+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bac15a860 con 0x7f5b980678c0 2024-09-12T12:20:38.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.182+0000 7f5baa7fc640 1 -- 172.21.15.3:0/3312381165 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237665 (secure 0 0 0) 0x7f5bac15a860 con 0x7f5b980678c0 2024-09-12T12:20:38.188 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:38.190 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:20:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5b980678c0 msgr2=0x7f5b98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5b980678c0 0x7f5b98069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bac145f80 tx=0x7f5ba0008040 comp rx=0 tx=0).stop 2024-09-12T12:20:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 msgr2=0x7f5bac0999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5bac0999b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400e780 tx=0x7f5ba400ec40 comp rx=0 tx=0).stop 2024-09-12T12:20:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 shutdown_connections 2024-09-12T12:20:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5b980678c0 0x7f5b98069d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bac09d470 0x7f5bac0b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bac09d030 0x7f5bac0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 --2- 172.21.15.3:0/3312381165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b94005220 0x7f5bac0999b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 >> 172.21.15.3:0/3312381165 conn(0x7f5bac0b1d80 msgr2=0x7f5bac15e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:38.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 shutdown_connections 2024-09-12T12:20:38.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.186+0000 7f5bb3f77640 1 -- 172.21.15.3:0/3312381165 wait complete. 2024-09-12T12:20:38.207 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-12T12:20:37.672665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":219848,"kb_used_data":3572,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754872,"statfs":{"total":773094113280,"available":772868988928,"internally_reserved":0,"allocated":3657728,"data_stored":2081881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.554066"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125281+0000","last_change":"2024-09-12T12:20:33.125281+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125281+0000","last_undegraded":"2024-09-12T12:20:33.125281+0000","last_fullsized":"2024-09-12T12:20:33.125281+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.128079+0000","last_change":"2024-09-12T12:20:33.128079+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.128079+0000","last_undegraded":"2024-09-12T12:20:33.128079+0000","last_fullsized":"2024-09-12T12:20:33.128079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125484+0000","last_change":"2024-09-12T12:20:33.125484+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125484+0000","last_undegraded":"2024-09-12T12:20:33.125484+0000","last_fullsized":"2024-09-12T12:20:33.125484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125258+0000","last_change":"2024-09-12T12:20:33.125258+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125258+0000","last_undegraded":"2024-09-12T12:20:33.125258+0000","last_fullsized":"2024-09-12T12:20:33.125258+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125313+0000","last_change":"2024-09-12T12:20:33.125313+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125313+0000","last_undegraded":"2024-09-12T12:20:33.125313+0000","last_fullsized":"2024-09-12T12:20:33.125313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125942+0000","last_change":"2024-09-12T12:20:33.125942+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125942+0000","last_undegraded":"2024-09-12T12:20:33.125942+0000","last_fullsized":"2024-09-12T12:20:33.125942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.124224+0000","last_change":"2024-09-12T12:20:33.124224+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.124224+0000","last_undegraded":"2024-09-12T12:20:33.124224+0000","last_fullsized":"2024-09-12T12:20:33.124224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-12T12:20:30.055403+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:20:30.055403+0000","last_peered":"2024-09-12T12:20:30.055403+0000","last_clean":"2024-09-12T12:20:30.055403+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:20:30.055403+0000","last_undegraded":"2024-09-12T12:20:30.055403+0000","last_fullsized":"2024-09-12T12:20:30.055403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":319488,"data_stored":117733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":901120,"data_stored":702284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27344,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":901120,"data_stored":702284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:20:38.211 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:20:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.366+0000 7f8c82359640 1 Processor -- start 2024-09-12T12:20:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.366+0000 7f8c82359640 1 -- start start 2024-09-12T12:20:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c82359640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8c7c15e4b0 0x7f8c7c0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c82359640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c0ad5a0 con 0x7f8c7c0b42b0 2024-09-12T12:20:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c82359640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c06dee0 con 0x7f8c7c0724b0 2024-09-12T12:20:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c82359640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c06e810 con 0x7f8c7c15e4b0 2024-09-12T12:20:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8c7c0b42b0 0x7f8c7c071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53632/0 (socket says 172.21.15.3:53632) 2024-09-12T12:20:38.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 -- 172.21.15.3:0/3384450773 learned_addr learned my addr 172.21.15.3:0/3384450773 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7bfff640 1 --2- 172.21.15.3:0/3384450773 >> v2:172.21.15.142:3300/0 conn(0x7f8c7c15e4b0 0x7f8c7c0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2743724072 0 0) 0x7f8c7c0ad5a0 con 0x7f8c7c0b42b0 2024-09-12T12:20:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c64003530 con 0x7f8c7c0b42b0 2024-09-12T12:20:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2500743250 0 0) 0x7f8c64003530 con 0x7f8c7c0b42b0 2024-09-12T12:20:38.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 >> v2:172.21.15.142:3300/0 conn(0x7f8c7c15e4b0 msgr2=0x7f8c7c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 --2- 172.21.15.3:0/3384450773 >> v2:172.21.15.142:3300/0 conn(0x7f8c7c15e4b0 0x7f8c7c0ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 >> v1:172.21.15.142:6789/0 conn(0x7f8c7c0724b0 legacy=0x7f8c7c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c06e1c0 con 0x7f8c7c0b42b0 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f8c70001680 con 0x7f8c7c0b42b0 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 >> v1:172.21.15.3:6789/0 conn(0x7f8c7c0b42b0 legacy=0x7f8c7c071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c640039c0 0x7f8c64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c640051d0 0x7f8c6400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c06e810 con 0x7f8c64004810 2024-09-12T12:20:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6400c170 con 0x7f8c640051d0 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8c6400c370 con 0x7f8c640039c0 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7bfff640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c640039c0 0x7f8c64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c808f6640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c640051d0 0x7f8c6400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 -- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c640039c0 msgr2=0x7f8c64003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c640039c0 0x7f8c64003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 -- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c640051d0 msgr2=0x7f8c6400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c640051d0 0x7f8c6400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7b7fe640 1 -- 172.21.15.3:0/3384450773 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c64003530 con 0x7f8c64004810 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c808f6640 1 -- 172.21.15.3:0/3384450773 reap_dead start 2024-09-12T12:20:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.370+0000 7f8c7bfff640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c640039c0 0x7f8c64003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:38.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c7b7fe640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c210 tx=0x7f8c7001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de35bc873b054a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70007d20 con 0x7f8c64004810 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c700a9020 con 0x7f8c64004810 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c7affd640 1 -- 172.21.15.3:0/3384450773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70005ae0 con 0x7f8c64004810 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- 172.21.15.3:0/3384450773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64004810 msgr2=0x7f8c64004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c210 tx=0x7f8c7001cc60 comp rx=0 tx=0).stop 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- 172.21.15.3:0/3384450773 shutdown_connections 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 --2- 172.21.15.3:0/3384450773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c64004810 0x7f8c64004c90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- 172.21.15.3:0/3384450773 >> 172.21.15.3:0/3384450773 conn(0x7f8c7c0b1d80 msgr2=0x7f8c7c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- 172.21.15.3:0/3384450773 shutdown_connections 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- 172.21.15.3:0/3384450773 wait complete. 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 Processor -- start 2024-09-12T12:20:38.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- start start 2024-09-12T12:20:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c64004810 0x7f8c7c150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c7c0cf6c0 0x7f8c7c09cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7c0cc090 0x7f8c7c0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c06f490 con 0x7f8c7c0cc090 2024-09-12T12:20:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c0b3390 con 0x7f8c7c0cf6c0 2024-09-12T12:20:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c82359640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7c06f610 con 0x7f8c64004810 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7c0cc090 0x7f8c7c0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7c0cc090 0x7f8c7c0cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53686/0 (socket says 172.21.15.3:53686) 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 -- 172.21.15.3:0/1591396389 learned_addr learned my addr 172.21.15.3:0/1591396389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c7b7fe640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c7c0cf6c0 0x7f8c7c09cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c7bfff640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c64004810 0x7f8c7c150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 -- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c64004810 msgr2=0x7f8c7c150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c64004810 0x7f8c7c150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 -- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c7c0cf6c0 msgr2=0x7f8c7c09cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c7c0cf6c0 0x7f8c7c09cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 -- 172.21.15.3:0/1591396389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c09d710 con 0x7f8c7c0cc090 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c7bfff640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c64004810 0x7f8c7c150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.374+0000 7f8c808f6640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7c0cc090 0x7f8c7c0cc520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c072ed0 tx=0x7f8c6c00e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c78ff9640 1 -- 172.21.15.3:0/1591396389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c019070 con 0x7f8c7c0cc090 2024-09-12T12:20:38.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c78ff9640 1 -- 172.21.15.3:0/1591396389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c00ee70 con 0x7f8c7c0cc090 2024-09-12T12:20:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c09d9f0 con 0x7f8c7c0cc090 2024-09-12T12:20:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c78ff9640 1 -- 172.21.15.3:0/1591396389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0054a0 con 0x7f8c7c0cc090 2024-09-12T12:20:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c0c9870 con 0x7f8c7c0cc090 2024-09-12T12:20:38.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7c0adc30 con 0x7f8c7c0cc090 2024-09-12T12:20:38.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c78ff9640 1 -- 172.21.15.3:0/1591396389 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c6c01c070 con 0x7f8c7c0cc090 2024-09-12T12:20:38.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c78ff9640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c680678f0 0x7f8c68069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c78ff9640 1 -- 172.21.15.3:0/1591396389 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8c6c084ce0 con 0x7f8c7c0cc090 2024-09-12T12:20:38.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c7bfff640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c680678f0 0x7f8c68069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.378+0000 7f8c7bfff640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c680678f0 0x7f8c68069db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0cdf60 tx=0x7f8c7400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.382+0000 7f8c78ff9640 1 -- 172.21.15.3:0/1591396389 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8c6c0551d0 con 0x7f8c7c0cc090 2024-09-12T12:20:38.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.534+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c7c0b47a0 con 0x7f8c680678f0 2024-09-12T12:20:38.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.550+0000 7f8c78ff9640 1 -- 172.21.15.3:0/1591396389 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237665 (secure 0 0 0) 0x7f8c7c0b47a0 con 0x7f8c680678f0 2024-09-12T12:20:38.557 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:38.559 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c680678f0 msgr2=0x7f8c68069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c680678f0 0x7f8c68069db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0cdf60 tx=0x7f8c7400b040 comp rx=0 tx=0).stop 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7c0cc090 msgr2=0x7f8c7c0cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7c0cc090 0x7f8c7c0cc520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c072ed0 tx=0x7f8c6c00e670 comp rx=0 tx=0).stop 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 shutdown_connections 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c680678f0 0x7f8c68069db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c7c0cc090 0x7f8c7c0cc520 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c7c0cf6c0 0x7f8c7c09cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 --2- 172.21.15.3:0/1591396389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c64004810 0x7f8c7c150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 >> 172.21.15.3:0/1591396389 conn(0x7f8c7c0b1d80 msgr2=0x7f8c7c15cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:38.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 shutdown_connections 2024-09-12T12:20:38.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.554+0000 7f8c82359640 1 -- 172.21.15.3:0/1591396389 wait complete. 2024-09-12T12:20:38.574 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-12T12:20:37.672665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":219848,"kb_used_data":3572,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754872,"statfs":{"total":773094113280,"available":772868988928,"internally_reserved":0,"allocated":3657728,"data_stored":2081881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.554066"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125281+0000","last_change":"2024-09-12T12:20:33.125281+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125281+0000","last_undegraded":"2024-09-12T12:20:33.125281+0000","last_fullsized":"2024-09-12T12:20:33.125281+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.128079+0000","last_change":"2024-09-12T12:20:33.128079+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.128079+0000","last_undegraded":"2024-09-12T12:20:33.128079+0000","last_fullsized":"2024-09-12T12:20:33.128079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125484+0000","last_change":"2024-09-12T12:20:33.125484+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125484+0000","last_undegraded":"2024-09-12T12:20:33.125484+0000","last_fullsized":"2024-09-12T12:20:33.125484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125258+0000","last_change":"2024-09-12T12:20:33.125258+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125258+0000","last_undegraded":"2024-09-12T12:20:33.125258+0000","last_fullsized":"2024-09-12T12:20:33.125258+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125313+0000","last_change":"2024-09-12T12:20:33.125313+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125313+0000","last_undegraded":"2024-09-12T12:20:33.125313+0000","last_fullsized":"2024-09-12T12:20:33.125313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125942+0000","last_change":"2024-09-12T12:20:33.125942+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125942+0000","last_undegraded":"2024-09-12T12:20:33.125942+0000","last_fullsized":"2024-09-12T12:20:33.125942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.124224+0000","last_change":"2024-09-12T12:20:33.124224+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.124224+0000","last_undegraded":"2024-09-12T12:20:33.124224+0000","last_fullsized":"2024-09-12T12:20:33.124224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-12T12:20:30.055403+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:20:30.055403+0000","last_peered":"2024-09-12T12:20:30.055403+0000","last_clean":"2024-09-12T12:20:30.055403+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:20:30.055403+0000","last_undegraded":"2024-09-12T12:20:30.055403+0000","last_fullsized":"2024-09-12T12:20:30.055403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":319488,"data_stored":117733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":901120,"data_stored":702284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27344,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":901120,"data_stored":702284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:20:38.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:20:38.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c32ba7640 1 Processor -- start 2024-09-12T12:20:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c32ba7640 1 -- start start 2024-09-12T12:20:38.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c32ba7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8c2c159bd0 0x7f8c2c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c32ba7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c159fb0 0x7f8c2c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c32ba7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c2c06e400 con 0x7f8c2c06f4a0 2024-09-12T12:20:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c32ba7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8c2c06f010 con 0x7f8c2c159bd0 2024-09-12T12:20:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c32ba7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2c06e6e0 con 0x7f8c2c159fb0 2024-09-12T12:20:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c31144640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8c2c06f4a0 0x7f8c2c0ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53642/0 (socket says 172.21.15.3:53642) 2024-09-12T12:20:38.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c31144640 1 -- 172.21.15.3:0/1044503510 learned_addr learned my addr 172.21.15.3:0/1044503510 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c23fff640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c159fb0 0x7f8c2c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c30943640 1 --2- 172.21.15.3:0/1044503510 >> v2:172.21.15.142:3300/0 conn(0x7f8c2c159bd0 0x7f8c2c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1411038924 0 0) 0x7f8c2c06e400 con 0x7f8c2c06f4a0 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.738+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c14003530 con 0x7f8c2c06f4a0 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3634944847 0 0) 0x7f8c14003530 con 0x7f8c2c06f4a0 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c159fb0 msgr2=0x7f8c2c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c159fb0 0x7f8c2c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 >> v2:172.21.15.142:3300/0 conn(0x7f8c2c159bd0 msgr2=0x7f8c2c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 --2- 172.21.15.3:0/1044503510 >> v2:172.21.15.142:3300/0 conn(0x7f8c2c159bd0 0x7f8c2c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c06e9c0 con 0x7f8c2c06f4a0 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f8c24001a00 con 0x7f8c2c06f4a0 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 >> v1:172.21.15.3:6789/0 conn(0x7f8c2c06f4a0 legacy=0x7f8c2c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c140039c0 0x7f8c14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14004810 0x7f8c14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c159fb0 0x7f8c140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c06f010 con 0x7f8c140039c0 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c06e6e0 con 0x7f8c14004810 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1400b9d0 con 0x7f8c2c159fb0 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c30943640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c140039c0 0x7f8c14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c23fff640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14004810 0x7f8c14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c30943640 1 -- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c159fb0 msgr2=0x7f8c140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c30943640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c159fb0 0x7f8c140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c30943640 1 -- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14004810 msgr2=0x7f8c14004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c30943640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c14004810 0x7f8c14004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c30943640 1 -- 172.21.15.3:0/1044503510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c14003530 con 0x7f8c140039c0 2024-09-12T12:20:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c30943640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c140039c0 0x7f8c14003de0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009ae0 tx=0x7f8c2801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d79cbd818867f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280b2070 con 0x7f8c140039c0 2024-09-12T12:20:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c280a5590 con 0x7f8c140039c0 2024-09-12T12:20:38.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c237fe640 1 -- 172.21.15.3:0/1044503510 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280096b0 con 0x7f8c140039c0 2024-09-12T12:20:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c32ba7640 1 -- 172.21.15.3:0/1044503510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c140039c0 msgr2=0x7f8c14003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c32ba7640 1 --2- 172.21.15.3:0/1044503510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c140039c0 0x7f8c14003de0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009ae0 tx=0x7f8c2801ca80 comp rx=0 tx=0).stop 2024-09-12T12:20:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c31144640 1 -- 172.21.15.3:0/1044503510 reap_dead start 2024-09-12T12:20:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c32ba7640 1 -- 172.21.15.3:0/1044503510 shutdown_connections 2024-09-12T12:20:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c32ba7640 1 -- 172.21.15.3:0/1044503510 >> 172.21.15.3:0/1044503510 conn(0x7f8c2c0b1d80 msgr2=0x7f8c2c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c32ba7640 1 -- 172.21.15.3:0/1044503510 shutdown_connections 2024-09-12T12:20:38.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.742+0000 7f8c32ba7640 1 -- 172.21.15.3:0/1044503510 wait complete. 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 Processor -- start 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 -- start start 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c140039c0 0x7f8c2c0990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c099940 0x7f8c2c156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2c15b820 0x7f8c2c1567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c15d2d0 con 0x7f8c2c15b820 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c0b3aa0 con 0x7f8c140039c0 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2c15d450 con 0x7f8c2c099940 2024-09-12T12:20:38.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2c15b820 0x7f8c2c1567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2c15b820 0x7f8c2c1567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53706/0 (socket says 172.21.15.3:53706) 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 -- 172.21.15.3:0/970206892 learned_addr learned my addr 172.21.15.3:0/970206892 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c30943640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c140039c0 0x7f8c2c0990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c23fff640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c099940 0x7f8c2c156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 -- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c099940 msgr2=0x7f8c2c156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c099940 0x7f8c2c156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 -- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c140039c0 msgr2=0x7f8c2c0990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c140039c0 0x7f8c2c0990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 -- 172.21.15.3:0/970206892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c1600e0 con 0x7f8c2c15b820 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c31144640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2c15b820 0x7f8c2c1567d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c1460b0 tx=0x7f8c2400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c217fa640 1 -- 172.21.15.3:0/970206892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400a5e0 con 0x7f8c2c15b820 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c217fa640 1 -- 172.21.15.3:0/970206892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2400a780 con 0x7f8c2c15b820 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2c1603c0 con 0x7f8c2c15b820 2024-09-12T12:20:38.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c217fa640 1 -- 172.21.15.3:0/970206892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400ab70 con 0x7f8c2c15b820 2024-09-12T12:20:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2c1e9f70 con 0x7f8c2c15b820 2024-09-12T12:20:38.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.746+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c2c06dd40 con 0x7f8c2c15b820 2024-09-12T12:20:38.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.750+0000 7f8c217fa640 1 -- 172.21.15.3:0/970206892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c2400ad10 con 0x7f8c2c15b820 2024-09-12T12:20:38.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.750+0000 7f8c217fa640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c180678c0 0x7f8c18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:38.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.750+0000 7f8c217fa640 1 -- 172.21.15.3:0/970206892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8c24085370 con 0x7f8c2c15b820 2024-09-12T12:20:38.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.750+0000 7f8c30943640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c180678c0 0x7f8c18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:38.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.750+0000 7f8c30943640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c180678c0 0x7f8c18069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c280af000 tx=0x7f8c280046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:38.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.754+0000 7f8c217fa640 1 -- 172.21.15.3:0/970206892 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8c240557d0 con 0x7f8c2c15b820 2024-09-12T12:20:38.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.914+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c2c15bda0 con 0x7f8c180678c0 2024-09-12T12:20:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.930+0000 7f8c217fa640 1 -- 172.21.15.3:0/970206892 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237665 (secure 0 0 0) 0x7f8c2c15bda0 con 0x7f8c180678c0 2024-09-12T12:20:38.937 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:38.938 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:20:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c180678c0 msgr2=0x7f8c18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c180678c0 0x7f8c18069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8c280af000 tx=0x7f8c280046d0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2c15b820 msgr2=0x7f8c2c1567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2c15b820 0x7f8c2c1567d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c1460b0 tx=0x7f8c2400cc20 comp rx=0 tx=0).stop 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 shutdown_connections 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8c180678c0 0x7f8c18069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c2c15b820 0x7f8c2c1567d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8c2c099940 0x7f8c2c156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 --2- 172.21.15.3:0/970206892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c140039c0 0x7f8c2c0990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 >> 172.21.15.3:0/970206892 conn(0x7f8c2c0b1d80 msgr2=0x7f8c2c0750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 shutdown_connections 2024-09-12T12:20:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:38.934+0000 7f8c32ba7640 1 -- 172.21.15.3:0/970206892 wait complete. 2024-09-12T12:20:38.956 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-12T12:20:37.672665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":219848,"kb_used_data":3572,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754872,"statfs":{"total":773094113280,"available":772868988928,"internally_reserved":0,"allocated":3657728,"data_stored":2081881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.554066"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125281+0000","last_change":"2024-09-12T12:20:33.125281+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125281+0000","last_undegraded":"2024-09-12T12:20:33.125281+0000","last_fullsized":"2024-09-12T12:20:33.125281+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.128079+0000","last_change":"2024-09-12T12:20:33.128079+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.128079+0000","last_undegraded":"2024-09-12T12:20:33.128079+0000","last_fullsized":"2024-09-12T12:20:33.128079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125484+0000","last_change":"2024-09-12T12:20:33.125484+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125484+0000","last_undegraded":"2024-09-12T12:20:33.125484+0000","last_fullsized":"2024-09-12T12:20:33.125484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125258+0000","last_change":"2024-09-12T12:20:33.125258+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125258+0000","last_undegraded":"2024-09-12T12:20:33.125258+0000","last_fullsized":"2024-09-12T12:20:33.125258+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125313+0000","last_change":"2024-09-12T12:20:33.125313+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125313+0000","last_undegraded":"2024-09-12T12:20:33.125313+0000","last_fullsized":"2024-09-12T12:20:33.125313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.125942+0000","last_change":"2024-09-12T12:20:33.125942+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.125942+0000","last_undegraded":"2024-09-12T12:20:33.125942+0000","last_fullsized":"2024-09-12T12:20:33.125942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-12T12:20:33.124224+0000","last_change":"2024-09-12T12:20:33.124224+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.124224+0000","last_undegraded":"2024-09-12T12:20:33.124224+0000","last_fullsized":"2024-09-12T12:20:33.124224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:34.115959+0000","last_change":"2024-09-12T12:20:34.115959+0000","last_active":"2024-09-12T12:20:34.115959+0000","last_peered":"2024-09-12T12:20:34.115959+0000","last_clean":"2024-09-12T12:20:34.115959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:34.115959+0000","last_undegraded":"2024-09-12T12:20:34.115959+0000","last_fullsized":"2024-09-12T12:20:34.115959+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:20:33.103441+0000","last_change":"2024-09-12T12:20:33.103441+0000","last_active":"2024-09-12T12:20:33.103441+0000","last_peered":"2024-09-12T12:20:33.103441+0000","last_clean":"2024-09-12T12:20:33.103441+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:20:33.103441+0000","last_undegraded":"2024-09-12T12:20:33.103441+0000","last_fullsized":"2024-09-12T12:20:33.103441+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-12T12:20:30.055403+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:20:30.055403+0000","last_peered":"2024-09-12T12:20:30.055403+0000","last_clean":"2024-09-12T12:20:30.055403+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:20:30.055403+0000","last_undegraded":"2024-09-12T12:20:30.055403+0000","last_fullsized":"2024-09-12T12:20:30.055403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":319488,"data_stored":117733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":901120,"data_stored":702284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27344,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27328,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":307200,"data_stored":111916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":901120,"data_stored":702284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:20:38.959 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3c is not active+clean 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.960 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:PG 5.36 is not active+clean 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-12T12:20:38.961 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:PG 5.32 is not active+clean 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-12T12:20:38.962 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-12T12:20:38.963 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:PG 5.27 is not active+clean 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:PG 5.26 is not active+clean 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-09-12T12:20:38.964 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-12T12:20:38.965 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1f is not active+clean 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1e is not active+clean 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1d is not active+clean 2024-09-12T12:20:38.966 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:20:33.125281+0000', 'last_change': '2024-09-12T12:20:33.125281+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.125281+0000', 'last_undegraded': '2024-09-12T12:20:33.125281+0000', 'last_fullsized': '2024-09-12T12:20:33.125281+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1d is not active+clean 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1c is not active+clean 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1b is not active+clean 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.967 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1a is not active+clean 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:PG 4.19 is not active+clean 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:20:33.128079+0000', 'last_change': '2024-09-12T12:20:33.128079+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.128079+0000', 'last_undegraded': '2024-09-12T12:20:33.128079+0000', 'last_fullsized': '2024-09-12T12:20:33.128079+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:PG 5.19 is not active+clean 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:PG 4.18 is not active+clean 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.968 INFO:tasks.ceph.ceph_manager.ceph:PG 4.17 is not active+clean 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:20:33.125484+0000', 'last_change': '2024-09-12T12:20:33.125484+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.125484+0000', 'last_undegraded': '2024-09-12T12:20:33.125484+0000', 'last_fullsized': '2024-09-12T12:20:33.125484+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:PG 4.16 is not active+clean 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:PG 4.15 is not active+clean 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-12T12:20:38.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.970 INFO:tasks.ceph.ceph_manager.ceph:PG 4.14 is not active+clean 2024-09-12T12:20:38.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:20:33.125258+0000', 'last_change': '2024-09-12T12:20:33.125258+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.125258+0000', 'last_undegraded': '2024-09-12T12:20:33.125258+0000', 'last_fullsized': '2024-09-12T12:20:33.125258+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:20:38.970 INFO:tasks.ceph.ceph_manager.ceph:PG 5.12 is not active+clean 2024-09-12T12:20:38.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.970 INFO:tasks.ceph.ceph_manager.ceph:PG 4.13 is not active+clean 2024-09-12T12:20:38.971 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.971 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-09-12T12:20:38.971 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.972 INFO:tasks.ceph.ceph_manager.ceph:PG 4.12 is not active+clean 2024-09-12T12:20:38.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.972 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-12T12:20:38.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.972 INFO:tasks.ceph.ceph_manager.ceph:PG 4.11 is not active+clean 2024-09-12T12:20:38.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:20:33.125313+0000', 'last_change': '2024-09-12T12:20:33.125313+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.125313+0000', 'last_undegraded': '2024-09-12T12:20:33.125313+0000', 'last_fullsized': '2024-09-12T12:20:33.125313+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:20:38.972 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-12T12:20:38.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.973 INFO:tasks.ceph.ceph_manager.ceph:PG 4.10 is not active+clean 2024-09-12T12:20:38.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:20:33.125942+0000', 'last_change': '2024-09-12T12:20:33.125942+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.125942+0000', 'last_undegraded': '2024-09-12T12:20:33.125942+0000', 'last_fullsized': '2024-09-12T12:20:33.125942+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:20:38.973 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-12T12:20:38.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.973 INFO:tasks.ceph.ceph_manager.ceph:PG 4.f is not active+clean 2024-09-12T12:20:38.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.974 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:PG 4.e is not active+clean 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:PG 4.d is not active+clean 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-09-12T12:20:38.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:PG 4.c is not active+clean 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:PG 4.b is not active+clean 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:PG 4.a is not active+clean 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.976 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:PG 4.9 is not active+clean 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:PG 4.8 is not active+clean 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:PG 4.7 is not active+clean 2024-09-12T12:20:38.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:PG 4.6 is not active+clean 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:PG 4.4 is not active+clean 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:PG 4.3 is not active+clean 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.978 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:PG 4.2 is not active+clean 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1 is not active+clean 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:PG 4.0 is not active+clean 2024-09-12T12:20:38.979 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:20:33.124224+0000', 'last_change': '2024-09-12T12:20:33.124224+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.124224+0000', 'last_undegraded': '2024-09-12T12:20:33.124224+0000', 'last_fullsized': '2024-09-12T12:20:33.124224+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:20:38.980 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-12T12:20:38.980 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:34.115959+0000', 'last_change': '2024-09-12T12:20:34.115959+0000', 'last_active': '2024-09-12T12:20:34.115959+0000', 'last_peered': '2024-09-12T12:20:34.115959+0000', 'last_clean': '2024-09-12T12:20:34.115959+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:34.115959+0000', 'last_undegraded': '2024-09-12T12:20:34.115959+0000', 'last_fullsized': '2024-09-12T12:20:34.115959+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:34.115959+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.980 INFO:tasks.ceph.ceph_manager.ceph:PG 4.5 is not active+clean 2024-09-12T12:20:38.980 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:20:33.103441+0000', 'last_change': '2024-09-12T12:20:33.103441+0000', 'last_active': '2024-09-12T12:20:33.103441+0000', 'last_peered': '2024-09-12T12:20:33.103441+0000', 'last_clean': '2024-09-12T12:20:33.103441+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:20:33.103441+0000', 'last_undegraded': '2024-09-12T12:20:33.103441+0000', 'last_fullsized': '2024-09-12T12:20:33.103441+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'last_clean_scrub_stamp': '2024-09-12T12:20:33.103441+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:20:38.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T12:20:39.092 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-12T12:20:39.086+0000 7feebac6b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:20:39.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.118+0000 7f2664eca640 1 Processor -- start 2024-09-12T12:20:39.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.118+0000 7f2664eca640 1 -- start start 2024-09-12T12:20:39.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.118+0000 7f2664eca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 0x7f266009bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.118+0000 7f2664eca640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26600b5d40 0x7f26600b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.118+0000 7f2664eca640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266009aa70 con 0x7f266015c420 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.118+0000 7f2664eca640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f26600b5940 con 0x7f26600b5d40 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.118+0000 7f2664eca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f266009ad50 con 0x7f266015d620 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26600b5d40 0x7f26600b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26600b5d40 0x7f26600b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53710/0 (socket says 172.21.15.3:53710) 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 -- 172.21.15.3:0/3964840956 learned_addr learned my addr 172.21.15.3:0/3964840956 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265dd9b640 1 --2- 172.21.15.3:0/3964840956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 0x7f266009bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 -- 172.21.15.3:0/3964840956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 msgr2=0x7f266009bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 --2- 172.21.15.3:0/3964840956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 0x7f266009bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 -- 172.21.15.3:0/3964840956 >> v1:172.21.15.142:6789/0 conn(0x7f266015c420 legacy=0x7f26600b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 -- 172.21.15.3:0/3964840956 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266009b030 con 0x7f26600b5d40 2024-09-12T12:20:39.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265ed9d640 1 --2- 172.21.15.3:0/3964840956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f26600b61d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2658009870 tx=0x7f265801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dc9a3b6ab15ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:39.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265d59a640 1 -- 172.21.15.3:0/3964840956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580b0070 con 0x7f26600b5d40 2024-09-12T12:20:39.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265d59a640 1 -- 172.21.15.3:0/3964840956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26580ab600 con 0x7f26600b5d40 2024-09-12T12:20:39.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.122+0000 7f265d59a640 1 -- 172.21.15.3:0/3964840956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580ab9d0 con 0x7f26600b5d40 2024-09-12T12:20:39.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- 172.21.15.3:0/3964840956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 msgr2=0x7f26600b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 --2- 172.21.15.3:0/3964840956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f26600b61d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2658009870 tx=0x7f265801c4d0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- 172.21.15.3:0/3964840956 shutdown_connections 2024-09-12T12:20:39.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 --2- 172.21.15.3:0/3964840956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f26600b61d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 --2- 172.21.15.3:0/3964840956 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 0x7f266009bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- 172.21.15.3:0/3964840956 >> 172.21.15.3:0/3964840956 conn(0x7f26600b3f30 msgr2=0x7f26600b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:39.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- 172.21.15.3:0/3964840956 shutdown_connections 2024-09-12T12:20:39.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- 172.21.15.3:0/3964840956 wait complete. 2024-09-12T12:20:39.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 Processor -- start 2024-09-12T12:20:39.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- start start 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f266015f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f266015c420 0x7f266015da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 0x7f2660141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2660070d80 con 0x7f26600b5d40 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2660070c00 con 0x7f266015c420 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2660070f00 con 0x7f266015d620 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f266015f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f266015f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53726/0 (socket says 172.21.15.3:53726) 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 -- 172.21.15.3:0/4221095971 learned_addr learned my addr 172.21.15.3:0/4221095971 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265dd9b640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f266015c420 0x7f266015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 -- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 msgr2=0x7f2660141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 0x7f2660141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 -- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f266015c420 msgr2=0x7f266015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f266015c420 0x7f266015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 -- 172.21.15.3:0/4221095971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2660159830 con 0x7f26600b5d40 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265dd9b640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f266015c420 0x7f266015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f265e59c640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f266015f270 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f265000c730 tx=0x7f265000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2656ffd640 1 -- 172.21.15.3:0/4221095971 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650016020 con 0x7f26600b5d40 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2656ffd640 1 -- 172.21.15.3:0/4221095971 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2650006d60 con 0x7f26600b5d40 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2656ffd640 1 -- 172.21.15.3:0/4221095971 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650003bc0 con 0x7f26600b5d40 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.126+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266015e1d0 con 0x7f26600b5d40 2024-09-12T12:20:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.130+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266015e790 con 0x7f26600b5d40 2024-09-12T12:20:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.130+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266015eed0 con 0x7f26600b5d40 2024-09-12T12:20:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.130+0000 7f2656ffd640 1 -- 172.21.15.3:0/4221095971 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f265000a850 con 0x7f26600b5d40 2024-09-12T12:20:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.130+0000 7f2656ffd640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26400678c0 0x7f2640069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.130+0000 7f2656ffd640 1 -- 172.21.15.3:0/4221095971 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2650085960 con 0x7f26600b5d40 2024-09-12T12:20:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.130+0000 7f265dd9b640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26400678c0 0x7f2640069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.130+0000 7f265dd9b640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26400678c0 0x7f2640069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f266006e390 tx=0x7f2648008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:39.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.134+0000 7f2656ffd640 1 -- 172.21.15.3:0/4221095971 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2650020020 con 0x7f26600b5d40 2024-09-12T12:20:39.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.362+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f26600b61d0 con 0x7f26600b5d40 2024-09-12T12:20:39.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.362+0000 7f2656ffd640 1 -- 172.21.15.3:0/4221095971 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2762 (secure 0 0 0) 0x7f26500564a0 con 0x7f26600b5d40 2024-09-12T12:20:39.367 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:39.367 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1726143578,"num_in_osds":8,"osd_in_since":1726143567,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":89},{"state_name":"creating+peering","count":7},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225124352,"bytes_avail":772868988928,"bytes_total":773094113280,"unknown_pgs_ratio":0.9175257682800293,"inactive_pgs_ratio":0.072164945304393768},"fsmap":{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4812}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-12T12:20:31.670806+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T12:20:39.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.362+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26400678c0 msgr2=0x7f2640069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.362+0000 7f2664eca640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26400678c0 0x7f2640069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f266006e390 tx=0x7f2648008040 comp rx=0 tx=0).stop 2024-09-12T12:20:39.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.362+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 msgr2=0x7f266015f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.362+0000 7f2664eca640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f266015f270 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f265000c730 tx=0x7f265000cbf0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 shutdown_connections 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26400678c0 0x7f2640069d80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f266015d620 0x7f2660141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f266015c420 0x7f266015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 --2- 172.21.15.3:0/4221095971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26600b5d40 0x7f266015f270 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 >> 172.21.15.3:0/4221095971 conn(0x7f26600b3f30 msgr2=0x7f2660158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 shutdown_connections 2024-09-12T12:20:39.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.366+0000 7f2664eca640 1 -- 172.21.15.3:0/4221095971 wait complete. 2024-09-12T12:20:39.385 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T12:20:39.385 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:20:39.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdc01ee4640 1 Processor -- start 2024-09-12T12:20:39.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdc01ee4640 1 -- start start 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdc01ee4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbfc159fb0 0x7fdbfc06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdc01ee4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbfc06fbb0 con 0x7fdbfc159bd0 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdc01ee4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbfc15d760 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdc01ee4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdbfc06f010 con 0x7fdbfc159fb0 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfaffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbfc159fb0 0x7fdbfc06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfaffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdbfc159fb0 0x7fdbfc06dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53738/0 (socket says 172.21.15.3:53738) 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfb7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdbfc159bd0 0x7fdbfc06f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53644/0 (socket says 172.21.15.3:53644) 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfaffd640 1 -- 172.21.15.3:0/2863469633 learned_addr learned my addr 172.21.15.3:0/2863469633 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfaffd640 1 -- 172.21.15.3:0/2863469633 >> v1:172.21.15.142:6789/0 conn(0x7fdbfc15d2d0 legacy=0x7fdbfc0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfaffd640 1 -- 172.21.15.3:0/2863469633 >> v1:172.21.15.3:6789/0 conn(0x7fdbfc159bd0 legacy=0x7fdbfc06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfaffd640 1 -- 172.21.15.3:0/2863469633 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc06e400 con 0x7fdbfc159fb0 2024-09-12T12:20:39.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.502+0000 7fdbfa7fc640 1 -- 172.21.15.3:0/2863469633 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259843655 0 0) 0x7fdbfc06fbb0 con 0x7fdbfc159bd0 2024-09-12T12:20:39.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdbfaffd640 1 --2- 172.21.15.3:0/2863469633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc06dd40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc06f190 tx=0x7fdbec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17453ffba9c2cc5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:39.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdbfa7fc640 1 -- 172.21.15.3:0/2863469633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0aa670 con 0x7fdbfc159fb0 2024-09-12T12:20:39.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdbfa7fc640 1 -- 172.21.15.3:0/2863469633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbec0af070 con 0x7fdbfc159fb0 2024-09-12T12:20:39.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdbfa7fc640 1 -- 172.21.15.3:0/2863469633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0aaa30 con 0x7fdbfc159fb0 2024-09-12T12:20:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 -- 172.21.15.3:0/2863469633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc159fb0 msgr2=0x7fdbfc06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/2863469633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc06dd40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc06f190 tx=0x7fdbec01c510 comp rx=0 tx=0).stop 2024-09-12T12:20:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 -- 172.21.15.3:0/2863469633 shutdown_connections 2024-09-12T12:20:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/2863469633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc06dd40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 -- 172.21.15.3:0/2863469633 >> 172.21.15.3:0/2863469633 conn(0x7fdbfc0b1d80 msgr2=0x7fdbfc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 -- 172.21.15.3:0/2863469633 shutdown_connections 2024-09-12T12:20:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 -- 172.21.15.3:0/2863469633 wait complete. 2024-09-12T12:20:39.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 Processor -- start 2024-09-12T12:20:39.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.506+0000 7fdc01ee4640 1 -- start start 2024-09-12T12:20:39.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbfc159bd0 0x7fdbfc145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc1f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc15d2d0 0x7fdbfc0b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0b3a10 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc0b3890 con 0x7fdbfc159fb0 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdbfc0ad770 con 0x7fdbfc159bd0 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc15d2d0 0x7fdbfc0b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc15d2d0 0x7fdbfc0b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53750/0 (socket says 172.21.15.3:53750) 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 -- 172.21.15.3:0/3133985445 learned_addr learned my addr 172.21.15.3:0/3133985445 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfaffd640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc1f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfb7fe640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbfc159bd0 0x7fdbfc145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 -- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbfc159bd0 msgr2=0x7fdbfc145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbfc159bd0 0x7fdbfc145900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 -- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbfc159fb0 msgr2=0x7fdbfc1f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc1f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 -- 172.21.15.3:0/3133985445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc145e40 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfaffd640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc1f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfbfff640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc15d2d0 0x7fdbfc0b1340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c730 tx=0x7fdbf400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbf3fff640 1 -- 172.21.15.3:0/3133985445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4003b80 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbf3fff640 1 -- 172.21.15.3:0/3133985445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf4003d20 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbfc0cc720 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbf3fff640 1 -- 172.21.15.3:0/3133985445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400aa30 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbfc0cd710 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbfc06dd40 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbf3fff640 1 -- 172.21.15.3:0/3133985445 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdbf400abd0 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbf3fff640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdbdc0678c0 0x7fdbdc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbf3fff640 1 -- 172.21.15.3:0/3133985445 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdbf4085be0 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.510+0000 7fdbfb7fe640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdbdc0678c0 0x7fdbdc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.514+0000 7fdbfb7fe640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdbdc0678c0 0x7fdbdc069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4000f70 tx=0x7fdbe4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.514+0000 7fdbf3fff640 1 -- 172.21.15.3:0/3133985445 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdbf4056680 con 0x7fdbfc15d2d0 2024-09-12T12:20:39.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.674+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdbfc151fd0 con 0x7fdbdc0678c0 2024-09-12T12:20:39.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.694+0000 7fdbf3fff640 1 -- 172.21.15.3:0/3133985445 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7fdbfc151fd0 con 0x7fdbdc0678c0 2024-09-12T12:20:39.700 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:39.703 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdbdc0678c0 msgr2=0x7fdbdc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdbdc0678c0 0x7fdbdc069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4000f70 tx=0x7fdbe4009040 comp rx=0 tx=0).stop 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc15d2d0 msgr2=0x7fdbfc0b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc15d2d0 0x7fdbfc0b1340 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c730 tx=0x7fdbf400cbf0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 shutdown_connections 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdbdc0678c0 0x7fdbdc069d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbfc15d2d0 0x7fdbfc0b1340 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdbfc159fb0 0x7fdbfc1f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 --2- 172.21.15.3:0/3133985445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdbfc159bd0 0x7fdbfc145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:39.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 >> 172.21.15.3:0/3133985445 conn(0x7fdbfc0b1d80 msgr2=0x7fdbfc155180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:39.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 shutdown_connections 2024-09-12T12:20:39.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:39.698+0000 7fdc01ee4640 1 -- 172.21.15.3:0/3133985445 wait complete. 2024-09-12T12:20:39.721 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-12T12:20:39.673279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220240,"kb_used_data":4008,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754480,"statfs":{"total":773094113280,"available":772868587520,"internally_reserved":0,"allocated":4104192,"data_stored":2282156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001940"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358296+0000","last_change":"2024-09-12T12:20:35.198491+0000","last_active":"2024-09-12T12:20:36.358296+0000","last_peered":"2024-09-12T12:20:36.358296+0000","last_clean":"2024-09-12T12:20:36.358296+0000","last_became_active":"2024-09-12T12:20:35.195072+0000","last_became_peered":"2024-09-12T12:20:35.195072+0000","last_unstale":"2024-09-12T12:20:36.358296+0000","last_undegraded":"2024-09-12T12:20:36.358296+0000","last_fullsized":"2024-09-12T12:20:36.358296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:31:12.496155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356079+0000","last_change":"2024-09-12T12:20:35.168650+0000","last_active":"2024-09-12T12:20:36.356079+0000","last_peered":"2024-09-12T12:20:36.356079+0000","last_clean":"2024-09-12T12:20:36.356079+0000","last_became_active":"2024-09-12T12:20:35.168048+0000","last_became_peered":"2024-09-12T12:20:35.168048+0000","last_unstale":"2024-09-12T12:20:36.356079+0000","last_undegraded":"2024-09-12T12:20:36.356079+0000","last_fullsized":"2024-09-12T12:20:36.356079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:50.278861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138388+0000","last_change":"2024-09-12T12:20:35.176652+0000","last_active":"2024-09-12T12:20:36.138388+0000","last_peered":"2024-09-12T12:20:36.138388+0000","last_clean":"2024-09-12T12:20:36.138388+0000","last_became_active":"2024-09-12T12:20:35.171653+0000","last_became_peered":"2024-09-12T12:20:35.171653+0000","last_unstale":"2024-09-12T12:20:36.138388+0000","last_undegraded":"2024-09-12T12:20:36.138388+0000","last_fullsized":"2024-09-12T12:20:36.138388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:45:44.391059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355508+0000","last_change":"2024-09-12T12:20:35.164219+0000","last_active":"2024-09-12T12:20:36.355508+0000","last_peered":"2024-09-12T12:20:36.355508+0000","last_clean":"2024-09-12T12:20:36.355508+0000","last_became_active":"2024-09-12T12:20:35.163777+0000","last_became_peered":"2024-09-12T12:20:35.163777+0000","last_unstale":"2024-09-12T12:20:36.355508+0000","last_undegraded":"2024-09-12T12:20:36.355508+0000","last_fullsized":"2024-09-12T12:20:36.355508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:21.050788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357496+0000","last_change":"2024-09-12T12:20:35.147788+0000","last_active":"2024-09-12T12:20:36.357496+0000","last_peered":"2024-09-12T12:20:36.357496+0000","last_clean":"2024-09-12T12:20:36.357496+0000","last_became_active":"2024-09-12T12:20:35.147505+0000","last_became_peered":"2024-09-12T12:20:35.147505+0000","last_unstale":"2024-09-12T12:20:36.357496+0000","last_undegraded":"2024-09-12T12:20:36.357496+0000","last_fullsized":"2024-09-12T12:20:36.357496+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:29.088616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356814+0000","last_change":"2024-09-12T12:20:35.175898+0000","last_active":"2024-09-12T12:20:36.356814+0000","last_peered":"2024-09-12T12:20:36.356814+0000","last_clean":"2024-09-12T12:20:36.356814+0000","last_became_active":"2024-09-12T12:20:35.173226+0000","last_became_peered":"2024-09-12T12:20:35.173226+0000","last_unstale":"2024-09-12T12:20:36.356814+0000","last_undegraded":"2024-09-12T12:20:36.356814+0000","last_fullsized":"2024-09-12T12:20:36.356814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:21:56.760424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359495+0000","last_change":"2024-09-12T12:20:35.191815+0000","last_active":"2024-09-12T12:20:36.359495+0000","last_peered":"2024-09-12T12:20:36.359495+0000","last_clean":"2024-09-12T12:20:36.359495+0000","last_became_active":"2024-09-12T12:20:35.191023+0000","last_became_peered":"2024-09-12T12:20:35.191023+0000","last_unstale":"2024-09-12T12:20:36.359495+0000","last_undegraded":"2024-09-12T12:20:36.359495+0000","last_fullsized":"2024-09-12T12:20:36.359495+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:12.696430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357994+0000","last_change":"2024-09-12T12:20:35.164501+0000","last_active":"2024-09-12T12:20:36.357994+0000","last_peered":"2024-09-12T12:20:36.357994+0000","last_clean":"2024-09-12T12:20:36.357994+0000","last_became_active":"2024-09-12T12:20:35.163683+0000","last_became_peered":"2024-09-12T12:20:35.163683+0000","last_unstale":"2024-09-12T12:20:36.357994+0000","last_undegraded":"2024-09-12T12:20:36.357994+0000","last_fullsized":"2024-09-12T12:20:36.357994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:50.814370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141629+0000","last_change":"2024-09-12T12:20:35.155924+0000","last_active":"2024-09-12T12:20:36.141629+0000","last_peered":"2024-09-12T12:20:36.141629+0000","last_clean":"2024-09-12T12:20:36.141629+0000","last_became_active":"2024-09-12T12:20:35.155184+0000","last_became_peered":"2024-09-12T12:20:35.155184+0000","last_unstale":"2024-09-12T12:20:36.141629+0000","last_undegraded":"2024-09-12T12:20:36.141629+0000","last_fullsized":"2024-09-12T12:20:36.141629+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:38:11.565938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355813+0000","last_change":"2024-09-12T12:20:35.155981+0000","last_active":"2024-09-12T12:20:36.355813+0000","last_peered":"2024-09-12T12:20:36.355813+0000","last_clean":"2024-09-12T12:20:36.355813+0000","last_became_active":"2024-09-12T12:20:35.155735+0000","last_became_peered":"2024-09-12T12:20:35.155735+0000","last_unstale":"2024-09-12T12:20:36.355813+0000","last_undegraded":"2024-09-12T12:20:36.355813+0000","last_fullsized":"2024-09-12T12:20:36.355813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:05.783164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359987+0000","last_change":"2024-09-12T12:20:35.171158+0000","last_active":"2024-09-12T12:20:36.359987+0000","last_peered":"2024-09-12T12:20:36.359987+0000","last_clean":"2024-09-12T12:20:36.359987+0000","last_became_active":"2024-09-12T12:20:35.170458+0000","last_became_peered":"2024-09-12T12:20:35.170458+0000","last_unstale":"2024-09-12T12:20:36.359987+0000","last_undegraded":"2024-09-12T12:20:36.359987+0000","last_fullsized":"2024-09-12T12:20:36.359987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:34.871974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138328+0000","last_change":"2024-09-12T12:20:35.167588+0000","last_active":"2024-09-12T12:20:36.138328+0000","last_peered":"2024-09-12T12:20:36.138328+0000","last_clean":"2024-09-12T12:20:36.138328+0000","last_became_active":"2024-09-12T12:20:35.157946+0000","last_became_peered":"2024-09-12T12:20:35.157946+0000","last_unstale":"2024-09-12T12:20:36.138328+0000","last_undegraded":"2024-09-12T12:20:36.138328+0000","last_fullsized":"2024-09-12T12:20:36.138328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:50.195252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357575+0000","last_change":"2024-09-12T12:20:35.154274+0000","last_active":"2024-09-12T12:20:36.357575+0000","last_peered":"2024-09-12T12:20:36.357575+0000","last_clean":"2024-09-12T12:20:36.357575+0000","last_became_active":"2024-09-12T12:20:35.153997+0000","last_became_peered":"2024-09-12T12:20:35.153997+0000","last_unstale":"2024-09-12T12:20:36.357575+0000","last_undegraded":"2024-09-12T12:20:36.357575+0000","last_fullsized":"2024-09-12T12:20:36.357575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:17.603565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356966+0000","last_change":"2024-09-12T12:20:35.173071+0000","last_active":"2024-09-12T12:20:36.356966+0000","last_peered":"2024-09-12T12:20:36.356966+0000","last_clean":"2024-09-12T12:20:36.356966+0000","last_became_active":"2024-09-12T12:20:35.172438+0000","last_became_peered":"2024-09-12T12:20:35.172438+0000","last_unstale":"2024-09-12T12:20:36.356966+0000","last_undegraded":"2024-09-12T12:20:36.356966+0000","last_fullsized":"2024-09-12T12:20:36.356966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:26.175293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137526+0000","last_change":"2024-09-12T12:20:35.190298+0000","last_active":"2024-09-12T12:20:36.137526+0000","last_peered":"2024-09-12T12:20:36.137526+0000","last_clean":"2024-09-12T12:20:36.137526+0000","last_became_active":"2024-09-12T12:20:35.189773+0000","last_became_peered":"2024-09-12T12:20:35.189773+0000","last_unstale":"2024-09-12T12:20:36.137526+0000","last_undegraded":"2024-09-12T12:20:36.137526+0000","last_fullsized":"2024-09-12T12:20:36.137526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359568+0000","last_change":"2024-09-12T12:20:35.183641+0000","last_active":"2024-09-12T12:20:36.359568+0000","last_peered":"2024-09-12T12:20:36.359568+0000","last_clean":"2024-09-12T12:20:36.359568+0000","last_became_active":"2024-09-12T12:20:35.182970+0000","last_became_peered":"2024-09-12T12:20:35.182970+0000","last_unstale":"2024-09-12T12:20:36.359568+0000","last_undegraded":"2024-09-12T12:20:36.359568+0000","last_fullsized":"2024-09-12T12:20:36.359568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:50.195252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.360209+0000","last_change":"2024-09-12T12:20:35.197004+0000","last_active":"2024-09-12T12:20:36.360209+0000","last_peered":"2024-09-12T12:20:36.360209+0000","last_clean":"2024-09-12T12:20:36.360209+0000","last_became_active":"2024-09-12T12:20:35.196392+0000","last_became_peered":"2024-09-12T12:20:35.196392+0000","last_unstale":"2024-09-12T12:20:36.360209+0000","last_undegraded":"2024-09-12T12:20:36.360209+0000","last_fullsized":"2024-09-12T12:20:36.360209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:22.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356211+0000","last_change":"2024-09-12T12:20:35.172750+0000","last_active":"2024-09-12T12:20:36.356211+0000","last_peered":"2024-09-12T12:20:36.356211+0000","last_clean":"2024-09-12T12:20:36.356211+0000","last_became_active":"2024-09-12T12:20:35.171985+0000","last_became_peered":"2024-09-12T12:20:35.171985+0000","last_unstale":"2024-09-12T12:20:36.356211+0000","last_undegraded":"2024-09-12T12:20:36.356211+0000","last_fullsized":"2024-09-12T12:20:36.356211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:21:25.885895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355243+0000","last_change":"2024-09-12T12:20:35.172308+0000","last_active":"2024-09-12T12:20:36.355243+0000","last_peered":"2024-09-12T12:20:36.355243+0000","last_clean":"2024-09-12T12:20:36.355243+0000","last_became_active":"2024-09-12T12:20:35.171866+0000","last_became_peered":"2024-09-12T12:20:35.171866+0000","last_unstale":"2024-09-12T12:20:36.355243+0000","last_undegraded":"2024-09-12T12:20:36.355243+0000","last_fullsized":"2024-09-12T12:20:36.355243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:13.240065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357475+0000","last_change":"2024-09-12T12:20:35.155045+0000","last_active":"2024-09-12T12:20:36.357475+0000","last_peered":"2024-09-12T12:20:36.357475+0000","last_clean":"2024-09-12T12:20:36.357475+0000","last_became_active":"2024-09-12T12:20:35.154834+0000","last_became_peered":"2024-09-12T12:20:35.154834+0000","last_unstale":"2024-09-12T12:20:36.357475+0000","last_undegraded":"2024-09-12T12:20:36.357475+0000","last_fullsized":"2024-09-12T12:20:36.357475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:14.773145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140725+0000","last_change":"2024-09-12T12:20:35.156240+0000","last_active":"2024-09-12T12:20:36.140725+0000","last_peered":"2024-09-12T12:20:36.140725+0000","last_clean":"2024-09-12T12:20:36.140725+0000","last_became_active":"2024-09-12T12:20:35.155462+0000","last_became_peered":"2024-09-12T12:20:35.155462+0000","last_unstale":"2024-09-12T12:20:36.140725+0000","last_undegraded":"2024-09-12T12:20:36.140725+0000","last_fullsized":"2024-09-12T12:20:36.140725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140875+0000","last_change":"2024-09-12T12:20:35.162247+0000","last_active":"2024-09-12T12:20:36.140875+0000","last_peered":"2024-09-12T12:20:36.140875+0000","last_clean":"2024-09-12T12:20:36.140875+0000","last_became_active":"2024-09-12T12:20:35.160236+0000","last_became_peered":"2024-09-12T12:20:35.160236+0000","last_unstale":"2024-09-12T12:20:36.140875+0000","last_undegraded":"2024-09-12T12:20:36.140875+0000","last_fullsized":"2024-09-12T12:20:36.140875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:17:21.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355731+0000","last_change":"2024-09-12T12:20:35.148316+0000","last_active":"2024-09-12T12:20:36.355731+0000","last_peered":"2024-09-12T12:20:36.355731+0000","last_clean":"2024-09-12T12:20:36.355731+0000","last_became_active":"2024-09-12T12:20:35.147986+0000","last_became_peered":"2024-09-12T12:20:35.147986+0000","last_unstale":"2024-09-12T12:20:36.355731+0000","last_undegraded":"2024-09-12T12:20:36.355731+0000","last_fullsized":"2024-09-12T12:20:36.355731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.587701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137141+0000","last_change":"2024-09-12T12:20:35.150689+0000","last_active":"2024-09-12T12:20:36.137141+0000","last_peered":"2024-09-12T12:20:36.137141+0000","last_clean":"2024-09-12T12:20:36.137141+0000","last_became_active":"2024-09-12T12:20:35.149930+0000","last_became_peered":"2024-09-12T12:20:35.149930+0000","last_unstale":"2024-09-12T12:20:36.137141+0000","last_undegraded":"2024-09-12T12:20:36.137141+0000","last_fullsized":"2024-09-12T12:20:36.137141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:42.454081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356283+0000","last_change":"2024-09-12T12:20:35.165003+0000","last_active":"2024-09-12T12:20:36.356283+0000","last_peered":"2024-09-12T12:20:36.356283+0000","last_clean":"2024-09-12T12:20:36.356283+0000","last_became_active":"2024-09-12T12:20:35.164704+0000","last_became_peered":"2024-09-12T12:20:35.164704+0000","last_unstale":"2024-09-12T12:20:36.356283+0000","last_undegraded":"2024-09-12T12:20:36.356283+0000","last_fullsized":"2024-09-12T12:20:36.356283+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:07.194590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356294+0000","last_change":"2024-09-12T12:20:35.176503+0000","last_active":"2024-09-12T12:20:36.356294+0000","last_peered":"2024-09-12T12:20:36.356294+0000","last_clean":"2024-09-12T12:20:36.356294+0000","last_became_active":"2024-09-12T12:20:35.175766+0000","last_became_peered":"2024-09-12T12:20:35.175766+0000","last_unstale":"2024-09-12T12:20:36.356294+0000","last_undegraded":"2024-09-12T12:20:36.356294+0000","last_fullsized":"2024-09-12T12:20:36.356294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:31:39.480802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355992+0000","last_change":"2024-09-12T12:20:35.165263+0000","last_active":"2024-09-12T12:20:36.355992+0000","last_peered":"2024-09-12T12:20:36.355992+0000","last_clean":"2024-09-12T12:20:36.355992+0000","last_became_active":"2024-09-12T12:20:35.164157+0000","last_became_peered":"2024-09-12T12:20:35.164157+0000","last_unstale":"2024-09-12T12:20:36.355992+0000","last_undegraded":"2024-09-12T12:20:36.355992+0000","last_fullsized":"2024-09-12T12:20:36.355992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:27.914148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138027+0000","last_change":"2024-09-12T12:20:35.152115+0000","last_active":"2024-09-12T12:20:36.138027+0000","last_peered":"2024-09-12T12:20:36.138027+0000","last_clean":"2024-09-12T12:20:36.138027+0000","last_became_active":"2024-09-12T12:20:35.151461+0000","last_became_peered":"2024-09-12T12:20:35.151461+0000","last_unstale":"2024-09-12T12:20:36.138027+0000","last_undegraded":"2024-09-12T12:20:36.138027+0000","last_fullsized":"2024-09-12T12:20:36.138027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:12.696430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140748+0000","last_change":"2024-09-12T12:20:35.166787+0000","last_active":"2024-09-12T12:20:36.140748+0000","last_peered":"2024-09-12T12:20:36.140748+0000","last_clean":"2024-09-12T12:20:36.140748+0000","last_became_active":"2024-09-12T12:20:35.164351+0000","last_became_peered":"2024-09-12T12:20:35.164351+0000","last_unstale":"2024-09-12T12:20:36.140748+0000","last_undegraded":"2024-09-12T12:20:36.140748+0000","last_fullsized":"2024-09-12T12:20:36.140748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:46:59.470812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359410+0000","last_change":"2024-09-12T12:20:35.190696+0000","last_active":"2024-09-12T12:20:36.359410+0000","last_peered":"2024-09-12T12:20:36.359410+0000","last_clean":"2024-09-12T12:20:36.359410+0000","last_became_active":"2024-09-12T12:20:35.190109+0000","last_became_peered":"2024-09-12T12:20:35.190109+0000","last_unstale":"2024-09-12T12:20:36.359410+0000","last_undegraded":"2024-09-12T12:20:36.359410+0000","last_fullsized":"2024-09-12T12:20:36.359410+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:25.941687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357854+0000","last_change":"2024-09-12T12:20:35.186425+0000","last_active":"2024-09-12T12:20:36.357854+0000","last_peered":"2024-09-12T12:20:36.357854+0000","last_clean":"2024-09-12T12:20:36.357854+0000","last_became_active":"2024-09-12T12:20:35.185695+0000","last_became_peered":"2024-09-12T12:20:35.185695+0000","last_unstale":"2024-09-12T12:20:36.357854+0000","last_undegraded":"2024-09-12T12:20:36.357854+0000","last_fullsized":"2024-09-12T12:20:36.357854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:31.416987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358559+0000","last_change":"2024-09-12T12:20:35.179554+0000","last_active":"2024-09-12T12:20:36.358559+0000","last_peered":"2024-09-12T12:20:36.358559+0000","last_clean":"2024-09-12T12:20:36.358559+0000","last_became_active":"2024-09-12T12:20:35.178868+0000","last_became_peered":"2024-09-12T12:20:35.178868+0000","last_unstale":"2024-09-12T12:20:36.358559+0000","last_undegraded":"2024-09-12T12:20:36.358559+0000","last_fullsized":"2024-09-12T12:20:36.358559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:42.454081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356864+0000","last_change":"2024-09-12T12:20:35.164093+0000","last_active":"2024-09-12T12:20:36.356864+0000","last_peered":"2024-09-12T12:20:36.356864+0000","last_clean":"2024-09-12T12:20:36.356864+0000","last_became_active":"2024-09-12T12:20:35.163581+0000","last_became_peered":"2024-09-12T12:20:35.163581+0000","last_unstale":"2024-09-12T12:20:36.356864+0000","last_undegraded":"2024-09-12T12:20:36.356864+0000","last_fullsized":"2024-09-12T12:20:36.356864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:47:45.135412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358445+0000","last_change":"2024-09-12T12:20:34.167240+0000","last_active":"2024-09-12T12:20:36.358445+0000","last_peered":"2024-09-12T12:20:36.358445+0000","last_clean":"2024-09-12T12:20:36.358445+0000","last_became_active":"2024-09-12T12:20:34.166461+0000","last_became_peered":"2024-09-12T12:20:34.166461+0000","last_unstale":"2024-09-12T12:20:36.358445+0000","last_undegraded":"2024-09-12T12:20:36.358445+0000","last_fullsized":"2024-09-12T12:20:36.358445+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:57.357006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358593+0000","last_change":"2024-09-12T12:20:35.197969+0000","last_active":"2024-09-12T12:20:36.358593+0000","last_peered":"2024-09-12T12:20:36.358593+0000","last_clean":"2024-09-12T12:20:36.358593+0000","last_became_active":"2024-09-12T12:20:35.193450+0000","last_became_peered":"2024-09-12T12:20:35.193450+0000","last_unstale":"2024-09-12T12:20:36.358593+0000","last_undegraded":"2024-09-12T12:20:36.358593+0000","last_fullsized":"2024-09-12T12:20:36.358593+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356917+0000","last_change":"2024-09-12T12:20:34.162205+0000","last_active":"2024-09-12T12:20:36.356917+0000","last_peered":"2024-09-12T12:20:36.356917+0000","last_clean":"2024-09-12T12:20:36.356917+0000","last_became_active":"2024-09-12T12:20:34.161881+0000","last_became_peered":"2024-09-12T12:20:34.161881+0000","last_unstale":"2024-09-12T12:20:36.356917+0000","last_undegraded":"2024-09-12T12:20:36.356917+0000","last_fullsized":"2024-09-12T12:20:36.356917+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:12:22.305382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141187+0000","last_change":"2024-09-12T12:20:35.179544+0000","last_active":"2024-09-12T12:20:36.141187+0000","last_peered":"2024-09-12T12:20:36.141187+0000","last_clean":"2024-09-12T12:20:36.141187+0000","last_became_active":"2024-09-12T12:20:35.178800+0000","last_became_peered":"2024-09-12T12:20:35.178800+0000","last_unstale":"2024-09-12T12:20:36.141187+0000","last_undegraded":"2024-09-12T12:20:36.141187+0000","last_fullsized":"2024-09-12T12:20:36.141187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:21.050788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355304+0000","last_change":"2024-09-12T12:20:34.162650+0000","last_active":"2024-09-12T12:20:36.355304+0000","last_peered":"2024-09-12T12:20:36.355304+0000","last_clean":"2024-09-12T12:20:36.355304+0000","last_became_active":"2024-09-12T12:20:34.161254+0000","last_became_peered":"2024-09-12T12:20:34.161254+0000","last_unstale":"2024-09-12T12:20:36.355304+0000","last_undegraded":"2024-09-12T12:20:36.355304+0000","last_fullsized":"2024-09-12T12:20:36.355304+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:20.125706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356159+0000","last_change":"2024-09-12T12:20:35.164640+0000","last_active":"2024-09-12T12:20:36.356159+0000","last_peered":"2024-09-12T12:20:36.356159+0000","last_clean":"2024-09-12T12:20:36.356159+0000","last_became_active":"2024-09-12T12:20:35.163578+0000","last_became_peered":"2024-09-12T12:20:35.163578+0000","last_unstale":"2024-09-12T12:20:36.356159+0000","last_undegraded":"2024-09-12T12:20:36.356159+0000","last_fullsized":"2024-09-12T12:20:36.356159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:28.976887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357588+0000","last_change":"2024-09-12T12:20:34.150212+0000","last_active":"2024-09-12T12:20:36.357588+0000","last_peered":"2024-09-12T12:20:36.357588+0000","last_clean":"2024-09-12T12:20:36.357588+0000","last_became_active":"2024-09-12T12:20:34.149775+0000","last_became_peered":"2024-09-12T12:20:34.149775+0000","last_unstale":"2024-09-12T12:20:36.357588+0000","last_undegraded":"2024-09-12T12:20:36.357588+0000","last_fullsized":"2024-09-12T12:20:36.357588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:58.418103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359648+0000","last_change":"2024-09-12T12:20:35.177876+0000","last_active":"2024-09-12T12:20:36.359648+0000","last_peered":"2024-09-12T12:20:36.359648+0000","last_clean":"2024-09-12T12:20:36.359648+0000","last_became_active":"2024-09-12T12:20:35.177096+0000","last_became_peered":"2024-09-12T12:20:35.177096+0000","last_unstale":"2024-09-12T12:20:36.359648+0000","last_undegraded":"2024-09-12T12:20:36.359648+0000","last_fullsized":"2024-09-12T12:20:36.359648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:45:44.391059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141058+0000","last_change":"2024-09-12T12:20:34.155082+0000","last_active":"2024-09-12T12:20:36.141058+0000","last_peered":"2024-09-12T12:20:36.141058+0000","last_clean":"2024-09-12T12:20:36.141058+0000","last_became_active":"2024-09-12T12:20:34.154312+0000","last_became_peered":"2024-09-12T12:20:34.154312+0000","last_unstale":"2024-09-12T12:20:36.141058+0000","last_undegraded":"2024-09-12T12:20:36.141058+0000","last_fullsized":"2024-09-12T12:20:36.141058+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:02.834445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137401+0000","last_change":"2024-09-12T12:20:35.144513+0000","last_active":"2024-09-12T12:20:36.137401+0000","last_peered":"2024-09-12T12:20:36.137401+0000","last_clean":"2024-09-12T12:20:36.137401+0000","last_became_active":"2024-09-12T12:20:35.143849+0000","last_became_peered":"2024-09-12T12:20:35.143849+0000","last_unstale":"2024-09-12T12:20:36.137401+0000","last_undegraded":"2024-09-12T12:20:36.137401+0000","last_fullsized":"2024-09-12T12:20:36.137401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:05:42.965426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140745+0000","last_change":"2024-09-12T12:20:34.162299+0000","last_active":"2024-09-12T12:20:36.140745+0000","last_peered":"2024-09-12T12:20:36.140745+0000","last_clean":"2024-09-12T12:20:36.140745+0000","last_became_active":"2024-09-12T12:20:34.160652+0000","last_became_peered":"2024-09-12T12:20:34.160652+0000","last_unstale":"2024-09-12T12:20:36.140745+0000","last_undegraded":"2024-09-12T12:20:36.140745+0000","last_fullsized":"2024-09-12T12:20:36.140745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:47.749206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.139878+0000","last_change":"2024-09-12T12:20:35.187370+0000","last_active":"2024-09-12T12:20:36.139878+0000","last_peered":"2024-09-12T12:20:36.139878+0000","last_clean":"2024-09-12T12:20:36.139878+0000","last_became_active":"2024-09-12T12:20:35.186743+0000","last_became_peered":"2024-09-12T12:20:35.186743+0000","last_unstale":"2024-09-12T12:20:36.139878+0000","last_undegraded":"2024-09-12T12:20:36.139878+0000","last_fullsized":"2024-09-12T12:20:36.139878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:16:56.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355406+0000","last_change":"2024-09-12T12:20:34.154781+0000","last_active":"2024-09-12T12:20:36.355406+0000","last_peered":"2024-09-12T12:20:36.355406+0000","last_clean":"2024-09-12T12:20:36.355406+0000","last_became_active":"2024-09-12T12:20:34.154499+0000","last_became_peered":"2024-09-12T12:20:34.154499+0000","last_unstale":"2024-09-12T12:20:36.355406+0000","last_undegraded":"2024-09-12T12:20:36.355406+0000","last_fullsized":"2024-09-12T12:20:36.355406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:07:14.564521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355549+0000","last_change":"2024-09-12T12:20:35.159085+0000","last_active":"2024-09-12T12:20:36.355549+0000","last_peered":"2024-09-12T12:20:36.355549+0000","last_clean":"2024-09-12T12:20:36.355549+0000","last_became_active":"2024-09-12T12:20:35.158722+0000","last_became_peered":"2024-09-12T12:20:35.158722+0000","last_unstale":"2024-09-12T12:20:36.355549+0000","last_undegraded":"2024-09-12T12:20:36.355549+0000","last_fullsized":"2024-09-12T12:20:36.355549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:24.785474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140218+0000","last_change":"2024-09-12T12:20:34.166953+0000","last_active":"2024-09-12T12:20:36.140218+0000","last_peered":"2024-09-12T12:20:36.140218+0000","last_clean":"2024-09-12T12:20:36.140218+0000","last_became_active":"2024-09-12T12:20:34.166192+0000","last_became_peered":"2024-09-12T12:20:34.166192+0000","last_unstale":"2024-09-12T12:20:36.140218+0000","last_undegraded":"2024-09-12T12:20:36.140218+0000","last_fullsized":"2024-09-12T12:20:36.140218+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:31.934869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138049+0000","last_change":"2024-09-12T12:20:35.150750+0000","last_active":"2024-09-12T12:20:36.138049+0000","last_peered":"2024-09-12T12:20:36.138049+0000","last_clean":"2024-09-12T12:20:36.138049+0000","last_became_active":"2024-09-12T12:20:35.150011+0000","last_became_peered":"2024-09-12T12:20:35.150011+0000","last_unstale":"2024-09-12T12:20:36.138049+0000","last_undegraded":"2024-09-12T12:20:36.138049+0000","last_fullsized":"2024-09-12T12:20:36.138049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:54:32.040523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356212+0000","last_change":"2024-09-12T12:20:34.168229+0000","last_active":"2024-09-12T12:20:36.356212+0000","last_peered":"2024-09-12T12:20:36.356212+0000","last_clean":"2024-09-12T12:20:36.356212+0000","last_became_active":"2024-09-12T12:20:34.167930+0000","last_became_peered":"2024-09-12T12:20:34.167930+0000","last_unstale":"2024-09-12T12:20:36.356212+0000","last_undegraded":"2024-09-12T12:20:36.356212+0000","last_fullsized":"2024-09-12T12:20:36.356212+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:11.575182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356095+0000","last_change":"2024-09-12T12:20:35.172164+0000","last_active":"2024-09-12T12:20:36.356095+0000","last_peered":"2024-09-12T12:20:36.356095+0000","last_clean":"2024-09-12T12:20:36.356095+0000","last_became_active":"2024-09-12T12:20:35.171857+0000","last_became_peered":"2024-09-12T12:20:35.171857+0000","last_unstale":"2024-09-12T12:20:36.356095+0000","last_undegraded":"2024-09-12T12:20:36.356095+0000","last_fullsized":"2024-09-12T12:20:36.356095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:55.826703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356964+0000","last_change":"2024-09-12T12:20:34.149095+0000","last_active":"2024-09-12T12:20:36.356964+0000","last_peered":"2024-09-12T12:20:36.356964+0000","last_clean":"2024-09-12T12:20:36.356964+0000","last_became_active":"2024-09-12T12:20:34.148258+0000","last_became_peered":"2024-09-12T12:20:34.148258+0000","last_unstale":"2024-09-12T12:20:36.356964+0000","last_undegraded":"2024-09-12T12:20:36.356964+0000","last_fullsized":"2024-09-12T12:20:36.356964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:32:34.037182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356015+0000","last_change":"2024-09-12T12:20:35.176754+0000","last_active":"2024-09-12T12:20:36.356015+0000","last_peered":"2024-09-12T12:20:36.356015+0000","last_clean":"2024-09-12T12:20:36.356015+0000","last_became_active":"2024-09-12T12:20:35.176424+0000","last_became_peered":"2024-09-12T12:20:35.176424+0000","last_unstale":"2024-09-12T12:20:36.356015+0000","last_undegraded":"2024-09-12T12:20:36.356015+0000","last_fullsized":"2024-09-12T12:20:36.356015+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:21:11.005047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138688+0000","last_change":"2024-09-12T12:20:34.174690+0000","last_active":"2024-09-12T12:20:36.138688+0000","last_peered":"2024-09-12T12:20:36.138688+0000","last_clean":"2024-09-12T12:20:36.138688+0000","last_became_active":"2024-09-12T12:20:34.173961+0000","last_became_peered":"2024-09-12T12:20:34.173961+0000","last_unstale":"2024-09-12T12:20:36.138688+0000","last_undegraded":"2024-09-12T12:20:36.138688+0000","last_fullsized":"2024-09-12T12:20:36.138688+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:20.038270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.139028+0000","last_change":"2024-09-12T12:20:35.172607+0000","last_active":"2024-09-12T12:20:36.139028+0000","last_peered":"2024-09-12T12:20:36.139028+0000","last_clean":"2024-09-12T12:20:36.139028+0000","last_became_active":"2024-09-12T12:20:35.170650+0000","last_became_peered":"2024-09-12T12:20:35.170650+0000","last_unstale":"2024-09-12T12:20:36.139028+0000","last_undegraded":"2024-09-12T12:20:36.139028+0000","last_fullsized":"2024-09-12T12:20:36.139028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:22.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355386+0000","last_change":"2024-09-12T12:20:34.165218+0000","last_active":"2024-09-12T12:20:36.355386+0000","last_peered":"2024-09-12T12:20:36.355386+0000","last_clean":"2024-09-12T12:20:36.355386+0000","last_became_active":"2024-09-12T12:20:34.161437+0000","last_became_peered":"2024-09-12T12:20:34.161437+0000","last_unstale":"2024-09-12T12:20:36.355386+0000","last_undegraded":"2024-09-12T12:20:36.355386+0000","last_fullsized":"2024-09-12T12:20:36.355386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:41.453879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355842+0000","last_change":"2024-09-12T12:20:35.158990+0000","last_active":"2024-09-12T12:20:36.355842+0000","last_peered":"2024-09-12T12:20:36.355842+0000","last_clean":"2024-09-12T12:20:36.355842+0000","last_became_active":"2024-09-12T12:20:35.156888+0000","last_became_peered":"2024-09-12T12:20:35.156888+0000","last_unstale":"2024-09-12T12:20:36.355842+0000","last_undegraded":"2024-09-12T12:20:36.355842+0000","last_fullsized":"2024-09-12T12:20:36.355842+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:55.208741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141175+0000","last_change":"2024-09-12T12:20:34.167052+0000","last_active":"2024-09-12T12:20:36.141175+0000","last_peered":"2024-09-12T12:20:36.141175+0000","last_clean":"2024-09-12T12:20:36.141175+0000","last_became_active":"2024-09-12T12:20:34.166290+0000","last_became_peered":"2024-09-12T12:20:34.166290+0000","last_unstale":"2024-09-12T12:20:36.141175+0000","last_undegraded":"2024-09-12T12:20:36.141175+0000","last_fullsized":"2024-09-12T12:20:36.141175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:19.356916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355771+0000","last_change":"2024-09-12T12:20:35.166430+0000","last_active":"2024-09-12T12:20:36.355771+0000","last_peered":"2024-09-12T12:20:36.355771+0000","last_clean":"2024-09-12T12:20:36.355771+0000","last_became_active":"2024-09-12T12:20:35.160670+0000","last_became_peered":"2024-09-12T12:20:35.160670+0000","last_unstale":"2024-09-12T12:20:36.355771+0000","last_undegraded":"2024-09-12T12:20:36.355771+0000","last_fullsized":"2024-09-12T12:20:36.355771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:37:40.029294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355937+0000","last_change":"2024-09-12T12:20:34.164222+0000","last_active":"2024-09-12T12:20:36.355937+0000","last_peered":"2024-09-12T12:20:36.355937+0000","last_clean":"2024-09-12T12:20:36.355937+0000","last_became_active":"2024-09-12T12:20:34.163943+0000","last_became_peered":"2024-09-12T12:20:34.163943+0000","last_unstale":"2024-09-12T12:20:36.355937+0000","last_undegraded":"2024-09-12T12:20:36.355937+0000","last_fullsized":"2024-09-12T12:20:36.355937+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:05.729418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359109+0000","last_change":"2024-09-12T12:20:35.172112+0000","last_active":"2024-09-12T12:20:36.359109+0000","last_peered":"2024-09-12T12:20:36.359109+0000","last_clean":"2024-09-12T12:20:36.359109+0000","last_became_active":"2024-09-12T12:20:35.171294+0000","last_became_peered":"2024-09-12T12:20:35.171294+0000","last_unstale":"2024-09-12T12:20:36.359109+0000","last_undegraded":"2024-09-12T12:20:36.359109+0000","last_fullsized":"2024-09-12T12:20:36.359109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:05:42.965426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355501+0000","last_change":"2024-09-12T12:20:34.163140+0000","last_active":"2024-09-12T12:20:36.355501+0000","last_peered":"2024-09-12T12:20:36.355501+0000","last_clean":"2024-09-12T12:20:36.355501+0000","last_became_active":"2024-09-12T12:20:34.162749+0000","last_became_peered":"2024-09-12T12:20:34.162749+0000","last_unstale":"2024-09-12T12:20:36.355501+0000","last_undegraded":"2024-09-12T12:20:36.355501+0000","last_fullsized":"2024-09-12T12:20:36.355501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:48.665029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357971+0000","last_change":"2024-09-12T12:20:35.190045+0000","last_active":"2024-09-12T12:20:36.357971+0000","last_peered":"2024-09-12T12:20:36.357971+0000","last_clean":"2024-09-12T12:20:36.357971+0000","last_became_active":"2024-09-12T12:20:35.189475+0000","last_became_peered":"2024-09-12T12:20:35.189475+0000","last_unstale":"2024-09-12T12:20:36.357971+0000","last_undegraded":"2024-09-12T12:20:36.357971+0000","last_fullsized":"2024-09-12T12:20:36.357971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:18.104578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355594+0000","last_change":"2024-09-12T12:20:34.157183+0000","last_active":"2024-09-12T12:20:36.355594+0000","last_peered":"2024-09-12T12:20:36.355594+0000","last_clean":"2024-09-12T12:20:36.355594+0000","last_became_active":"2024-09-12T12:20:34.156676+0000","last_became_peered":"2024-09-12T12:20:34.156676+0000","last_unstale":"2024-09-12T12:20:36.355594+0000","last_undegraded":"2024-09-12T12:20:36.355594+0000","last_fullsized":"2024-09-12T12:20:36.355594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:17.783533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141400+0000","last_change":"2024-09-12T12:20:35.159266+0000","last_active":"2024-09-12T12:20:36.141400+0000","last_peered":"2024-09-12T12:20:36.141400+0000","last_clean":"2024-09-12T12:20:36.141400+0000","last_became_active":"2024-09-12T12:20:35.158421+0000","last_became_peered":"2024-09-12T12:20:35.158421+0000","last_unstale":"2024-09-12T12:20:36.141400+0000","last_undegraded":"2024-09-12T12:20:36.141400+0000","last_fullsized":"2024-09-12T12:20:36.141400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:24.785474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356354+0000","last_change":"2024-09-12T12:20:34.155817+0000","last_active":"2024-09-12T12:20:36.356354+0000","last_peered":"2024-09-12T12:20:36.356354+0000","last_clean":"2024-09-12T12:20:36.356354+0000","last_became_active":"2024-09-12T12:20:34.154842+0000","last_became_peered":"2024-09-12T12:20:34.154842+0000","last_unstale":"2024-09-12T12:20:36.356354+0000","last_undegraded":"2024-09-12T12:20:36.356354+0000","last_fullsized":"2024-09-12T12:20:36.356354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:04.798090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138038+0000","last_change":"2024-09-12T12:20:35.169303+0000","last_active":"2024-09-12T12:20:36.138038+0000","last_peered":"2024-09-12T12:20:36.138038+0000","last_clean":"2024-09-12T12:20:36.138038+0000","last_became_active":"2024-09-12T12:20:35.159457+0000","last_became_peered":"2024-09-12T12:20:35.159457+0000","last_unstale":"2024-09-12T12:20:36.138038+0000","last_undegraded":"2024-09-12T12:20:36.138038+0000","last_fullsized":"2024-09-12T12:20:36.138038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:25.941687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141350+0000","last_change":"2024-09-12T12:20:34.167145+0000","last_active":"2024-09-12T12:20:36.141350+0000","last_peered":"2024-09-12T12:20:36.141350+0000","last_clean":"2024-09-12T12:20:36.141350+0000","last_became_active":"2024-09-12T12:20:34.166392+0000","last_became_peered":"2024-09-12T12:20:34.166392+0000","last_unstale":"2024-09-12T12:20:36.141350+0000","last_undegraded":"2024-09-12T12:20:36.141350+0000","last_fullsized":"2024-09-12T12:20:36.141350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:28.076097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355743+0000","last_change":"2024-09-12T12:20:35.164408+0000","last_active":"2024-09-12T12:20:36.355743+0000","last_peered":"2024-09-12T12:20:36.355743+0000","last_clean":"2024-09-12T12:20:36.355743+0000","last_became_active":"2024-09-12T12:20:35.163770+0000","last_became_peered":"2024-09-12T12:20:35.163770+0000","last_unstale":"2024-09-12T12:20:36.355743+0000","last_undegraded":"2024-09-12T12:20:36.355743+0000","last_fullsized":"2024-09-12T12:20:36.355743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:38:11.565938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141623+0000","last_change":"2024-09-12T12:20:34.149142+0000","last_active":"2024-09-12T12:20:36.141623+0000","last_peered":"2024-09-12T12:20:36.141623+0000","last_clean":"2024-09-12T12:20:36.141623+0000","last_became_active":"2024-09-12T12:20:34.148431+0000","last_became_peered":"2024-09-12T12:20:34.148431+0000","last_unstale":"2024-09-12T12:20:36.141623+0000","last_undegraded":"2024-09-12T12:20:36.141623+0000","last_fullsized":"2024-09-12T12:20:36.141623+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:13.760627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357818+0000","last_change":"2024-09-12T12:20:35.160542+0000","last_active":"2024-09-12T12:20:36.357818+0000","last_peered":"2024-09-12T12:20:36.357818+0000","last_clean":"2024-09-12T12:20:36.357818+0000","last_became_active":"2024-09-12T12:20:35.160248+0000","last_became_peered":"2024-09-12T12:20:35.160248+0000","last_unstale":"2024-09-12T12:20:36.357818+0000","last_undegraded":"2024-09-12T12:20:36.357818+0000","last_fullsized":"2024-09-12T12:20:36.357818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:20.369434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141131+0000","last_change":"2024-09-12T12:20:34.156849+0000","last_active":"2024-09-12T12:20:36.141131+0000","last_peered":"2024-09-12T12:20:36.141131+0000","last_clean":"2024-09-12T12:20:36.141131+0000","last_became_active":"2024-09-12T12:20:34.153898+0000","last_became_peered":"2024-09-12T12:20:34.153898+0000","last_unstale":"2024-09-12T12:20:36.141131+0000","last_undegraded":"2024-09-12T12:20:36.141131+0000","last_fullsized":"2024-09-12T12:20:36.141131+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:16.591047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357402+0000","last_change":"2024-09-12T12:20:35.162691+0000","last_active":"2024-09-12T12:20:36.357402+0000","last_peered":"2024-09-12T12:20:36.357402+0000","last_clean":"2024-09-12T12:20:36.357402+0000","last_became_active":"2024-09-12T12:20:35.162383+0000","last_became_peered":"2024-09-12T12:20:35.162383+0000","last_unstale":"2024-09-12T12:20:36.357402+0000","last_undegraded":"2024-09-12T12:20:36.357402+0000","last_fullsized":"2024-09-12T12:20:36.357402+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:48.761725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356597+0000","last_change":"2024-09-12T12:20:34.150352+0000","last_active":"2024-09-12T12:20:36.356597+0000","last_peered":"2024-09-12T12:20:36.356597+0000","last_clean":"2024-09-12T12:20:36.356597+0000","last_became_active":"2024-09-12T12:20:34.150067+0000","last_became_peered":"2024-09-12T12:20:34.150067+0000","last_unstale":"2024-09-12T12:20:36.356597+0000","last_undegraded":"2024-09-12T12:20:36.356597+0000","last_fullsized":"2024-09-12T12:20:36.356597+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:58.443790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357745+0000","last_change":"2024-09-12T12:20:35.144745+0000","last_active":"2024-09-12T12:20:36.357745+0000","last_peered":"2024-09-12T12:20:36.357745+0000","last_clean":"2024-09-12T12:20:36.357745+0000","last_became_active":"2024-09-12T12:20:35.144472+0000","last_became_peered":"2024-09-12T12:20:35.144472+0000","last_unstale":"2024-09-12T12:20:36.357745+0000","last_undegraded":"2024-09-12T12:20:36.357745+0000","last_fullsized":"2024-09-12T12:20:36.357745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:03.846963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357940+0000","last_change":"2024-09-12T12:20:34.167240+0000","last_active":"2024-09-12T12:20:36.357940+0000","last_peered":"2024-09-12T12:20:36.357940+0000","last_clean":"2024-09-12T12:20:36.357940+0000","last_became_active":"2024-09-12T12:20:34.163642+0000","last_became_peered":"2024-09-12T12:20:34.163642+0000","last_unstale":"2024-09-12T12:20:36.357940+0000","last_undegraded":"2024-09-12T12:20:36.357940+0000","last_fullsized":"2024-09-12T12:20:36.357940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:19.356916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357246+0000","last_change":"2024-09-12T12:20:35.144677+0000","last_active":"2024-09-12T12:20:36.357246+0000","last_peered":"2024-09-12T12:20:36.357246+0000","last_clean":"2024-09-12T12:20:36.357246+0000","last_became_active":"2024-09-12T12:20:35.144360+0000","last_became_peered":"2024-09-12T12:20:35.144360+0000","last_unstale":"2024-09-12T12:20:36.357246+0000","last_undegraded":"2024-09-12T12:20:36.357246+0000","last_fullsized":"2024-09-12T12:20:36.357246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:10.456058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140903+0000","last_change":"2024-09-12T12:20:34.147318+0000","last_active":"2024-09-12T12:20:36.140903+0000","last_peered":"2024-09-12T12:20:36.140903+0000","last_clean":"2024-09-12T12:20:36.140903+0000","last_became_active":"2024-09-12T12:20:34.146581+0000","last_became_peered":"2024-09-12T12:20:34.146581+0000","last_unstale":"2024-09-12T12:20:36.140903+0000","last_undegraded":"2024-09-12T12:20:36.140903+0000","last_fullsized":"2024-09-12T12:20:36.140903+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:43:28.217090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359297+0000","last_change":"2024-09-12T12:20:35.192139+0000","last_active":"2024-09-12T12:20:36.359297+0000","last_peered":"2024-09-12T12:20:36.359297+0000","last_clean":"2024-09-12T12:20:36.359297+0000","last_became_active":"2024-09-12T12:20:35.190075+0000","last_became_peered":"2024-09-12T12:20:35.190075+0000","last_unstale":"2024-09-12T12:20:36.359297+0000","last_undegraded":"2024-09-12T12:20:36.359297+0000","last_fullsized":"2024-09-12T12:20:36.359297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137001+0000","last_change":"2024-09-12T12:20:34.174400+0000","last_active":"2024-09-12T12:20:36.137001+0000","last_peered":"2024-09-12T12:20:36.137001+0000","last_clean":"2024-09-12T12:20:36.137001+0000","last_became_active":"2024-09-12T12:20:34.173969+0000","last_became_peered":"2024-09-12T12:20:34.173969+0000","last_unstale":"2024-09-12T12:20:36.137001+0000","last_undegraded":"2024-09-12T12:20:36.137001+0000","last_fullsized":"2024-09-12T12:20:36.137001+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:17:20.002195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356123+0000","last_change":"2024-09-12T12:20:35.156705+0000","last_active":"2024-09-12T12:20:36.356123+0000","last_peered":"2024-09-12T12:20:36.356123+0000","last_clean":"2024-09-12T12:20:36.356123+0000","last_became_active":"2024-09-12T12:20:35.155895+0000","last_became_peered":"2024-09-12T12:20:35.155895+0000","last_unstale":"2024-09-12T12:20:36.356123+0000","last_undegraded":"2024-09-12T12:20:36.356123+0000","last_fullsized":"2024-09-12T12:20:36.356123+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:43:29.229608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356355+0000","last_change":"2024-09-12T12:20:34.172819+0000","last_active":"2024-09-12T12:20:36.356355+0000","last_peered":"2024-09-12T12:20:36.356355+0000","last_clean":"2024-09-12T12:20:36.356355+0000","last_became_active":"2024-09-12T12:20:34.172454+0000","last_became_peered":"2024-09-12T12:20:34.172454+0000","last_unstale":"2024-09-12T12:20:36.356355+0000","last_undegraded":"2024-09-12T12:20:36.356355+0000","last_fullsized":"2024-09-12T12:20:36.356355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:59:35.934892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138417+0000","last_change":"2024-09-12T12:20:35.168420+0000","last_active":"2024-09-12T12:20:36.138417+0000","last_peered":"2024-09-12T12:20:36.138417+0000","last_clean":"2024-09-12T12:20:36.138417+0000","last_became_active":"2024-09-12T12:20:35.157401+0000","last_became_peered":"2024-09-12T12:20:35.157401+0000","last_unstale":"2024-09-12T12:20:36.138417+0000","last_undegraded":"2024-09-12T12:20:36.138417+0000","last_fullsized":"2024-09-12T12:20:36.138417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:34.871974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:37.139174+0000","last_change":"2024-09-12T12:20:34.151870+0000","last_active":"2024-09-12T12:20:37.139174+0000","last_peered":"2024-09-12T12:20:37.139174+0000","last_clean":"2024-09-12T12:20:37.139174+0000","last_became_active":"2024-09-12T12:20:34.150672+0000","last_became_peered":"2024-09-12T12:20:34.150672+0000","last_unstale":"2024-09-12T12:20:37.139174+0000","last_undegraded":"2024-09-12T12:20:37.139174+0000","last_fullsized":"2024-09-12T12:20:37.139174+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:33.392858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355328+0000","last_change":"2024-09-12T12:20:35.150674+0000","last_active":"2024-09-12T12:20:36.355328+0000","last_peered":"2024-09-12T12:20:36.355328+0000","last_clean":"2024-09-12T12:20:36.355328+0000","last_became_active":"2024-09-12T12:20:35.150422+0000","last_became_peered":"2024-09-12T12:20:35.150422+0000","last_unstale":"2024-09-12T12:20:36.355328+0000","last_undegraded":"2024-09-12T12:20:36.355328+0000","last_fullsized":"2024-09-12T12:20:36.355328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:21.138224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355943+0000","last_change":"2024-09-12T12:20:34.155182+0000","last_active":"2024-09-12T12:20:36.355943+0000","last_peered":"2024-09-12T12:20:36.355943+0000","last_clean":"2024-09-12T12:20:36.355943+0000","last_became_active":"2024-09-12T12:20:34.154872+0000","last_became_peered":"2024-09-12T12:20:34.154872+0000","last_unstale":"2024-09-12T12:20:36.355943+0000","last_undegraded":"2024-09-12T12:20:36.355943+0000","last_fullsized":"2024-09-12T12:20:36.355943+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:57.357006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357966+0000","last_change":"2024-09-12T12:20:35.182021+0000","last_active":"2024-09-12T12:20:36.357966+0000","last_peered":"2024-09-12T12:20:36.357966+0000","last_clean":"2024-09-12T12:20:36.357966+0000","last_became_active":"2024-09-12T12:20:35.181397+0000","last_became_peered":"2024-09-12T12:20:35.181397+0000","last_unstale":"2024-09-12T12:20:36.357966+0000","last_undegraded":"2024-09-12T12:20:36.357966+0000","last_fullsized":"2024-09-12T12:20:36.357966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:16:56.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356682+0000","last_change":"2024-09-12T12:20:34.153660+0000","last_active":"2024-09-12T12:20:36.356682+0000","last_peered":"2024-09-12T12:20:36.356682+0000","last_clean":"2024-09-12T12:20:36.356682+0000","last_became_active":"2024-09-12T12:20:34.152819+0000","last_became_peered":"2024-09-12T12:20:34.152819+0000","last_unstale":"2024-09-12T12:20:36.356682+0000","last_undegraded":"2024-09-12T12:20:36.356682+0000","last_fullsized":"2024-09-12T12:20:36.356682+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:05:27.974012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356543+0000","last_change":"2024-09-12T12:20:35.181331+0000","last_active":"2024-09-12T12:20:36.356543+0000","last_peered":"2024-09-12T12:20:36.356543+0000","last_clean":"2024-09-12T12:20:36.356543+0000","last_became_active":"2024-09-12T12:20:35.181013+0000","last_became_peered":"2024-09-12T12:20:35.181013+0000","last_unstale":"2024-09-12T12:20:36.356543+0000","last_undegraded":"2024-09-12T12:20:36.356543+0000","last_fullsized":"2024-09-12T12:20:36.356543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:42.466397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356039+0000","last_change":"2024-09-12T12:20:34.156655+0000","last_active":"2024-09-12T12:20:36.356039+0000","last_peered":"2024-09-12T12:20:36.356039+0000","last_clean":"2024-09-12T12:20:36.356039+0000","last_became_active":"2024-09-12T12:20:34.156384+0000","last_became_peered":"2024-09-12T12:20:34.156384+0000","last_unstale":"2024-09-12T12:20:36.356039+0000","last_undegraded":"2024-09-12T12:20:36.356039+0000","last_fullsized":"2024-09-12T12:20:36.356039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:08.114810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355553+0000","last_change":"2024-09-12T12:20:35.178899+0000","last_active":"2024-09-12T12:20:36.355553+0000","last_peered":"2024-09-12T12:20:36.355553+0000","last_clean":"2024-09-12T12:20:36.355553+0000","last_became_active":"2024-09-12T12:20:35.175704+0000","last_became_peered":"2024-09-12T12:20:35.175704+0000","last_unstale":"2024-09-12T12:20:36.355553+0000","last_undegraded":"2024-09-12T12:20:36.355553+0000","last_fullsized":"2024-09-12T12:20:36.355553+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:04:10.061214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141383+0000","last_change":"2024-09-12T12:20:34.164506+0000","last_active":"2024-09-12T12:20:36.141383+0000","last_peered":"2024-09-12T12:20:36.141383+0000","last_clean":"2024-09-12T12:20:36.141383+0000","last_became_active":"2024-09-12T12:20:34.163812+0000","last_became_peered":"2024-09-12T12:20:34.163812+0000","last_unstale":"2024-09-12T12:20:36.141383+0000","last_undegraded":"2024-09-12T12:20:36.141383+0000","last_fullsized":"2024-09-12T12:20:36.141383+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:09.443540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140633+0000","last_change":"2024-09-12T12:20:35.156268+0000","last_active":"2024-09-12T12:20:36.140633+0000","last_peered":"2024-09-12T12:20:36.140633+0000","last_clean":"2024-09-12T12:20:36.140633+0000","last_became_active":"2024-09-12T12:20:35.155510+0000","last_became_peered":"2024-09-12T12:20:35.155510+0000","last_unstale":"2024-09-12T12:20:36.140633+0000","last_undegraded":"2024-09-12T12:20:36.140633+0000","last_fullsized":"2024-09-12T12:20:36.140633+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:31:12.496155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355620+0000","last_change":"2024-09-12T12:20:34.162403+0000","last_active":"2024-09-12T12:20:36.355620+0000","last_peered":"2024-09-12T12:20:36.355620+0000","last_clean":"2024-09-12T12:20:36.355620+0000","last_became_active":"2024-09-12T12:20:34.161974+0000","last_became_peered":"2024-09-12T12:20:34.161974+0000","last_unstale":"2024-09-12T12:20:36.355620+0000","last_undegraded":"2024-09-12T12:20:36.355620+0000","last_fullsized":"2024-09-12T12:20:36.355620+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:22.796229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359546+0000","last_change":"2024-09-12T12:20:35.164445+0000","last_active":"2024-09-12T12:20:36.359546+0000","last_peered":"2024-09-12T12:20:36.359546+0000","last_clean":"2024-09-12T12:20:36.359546+0000","last_became_active":"2024-09-12T12:20:35.163705+0000","last_became_peered":"2024-09-12T12:20:35.163705+0000","last_unstale":"2024-09-12T12:20:36.359546+0000","last_undegraded":"2024-09-12T12:20:36.359546+0000","last_fullsized":"2024-09-12T12:20:36.359546+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:54:32.040523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358134+0000","last_change":"2024-09-12T12:20:34.164793+0000","last_active":"2024-09-12T12:20:36.358134+0000","last_peered":"2024-09-12T12:20:36.358134+0000","last_clean":"2024-09-12T12:20:36.358134+0000","last_became_active":"2024-09-12T12:20:34.164086+0000","last_became_peered":"2024-09-12T12:20:34.164086+0000","last_unstale":"2024-09-12T12:20:36.358134+0000","last_undegraded":"2024-09-12T12:20:36.358134+0000","last_fullsized":"2024-09-12T12:20:36.358134+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:02.834445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359716+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:20:36.359716+0000","last_peered":"2024-09-12T12:20:36.359716+0000","last_clean":"2024-09-12T12:20:36.359716+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:20:36.359716+0000","last_undegraded":"2024-09-12T12:20:36.359716+0000","last_fullsized":"2024-09-12T12:20:36.359716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":372736,"data_stored":138709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":727533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":137145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":138709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":727443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:20:42.726 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:20:42.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2d0da4640 1 Processor -- start 2024-09-12T12:20:42.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2d0da4640 1 -- start start 2024-09-12T12:20:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2d0da4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2cc06f6e0 0x7fa2cc072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2d0da4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc0cc300 0x7fa2cc073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2d0da4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2cc15f150 con 0x7fa2cc15b2d0 2024-09-12T12:20:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2d0da4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa2cc0b3790 con 0x7fa2cc06f6e0 2024-09-12T12:20:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2d0da4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc0ae920 con 0x7fa2cc0cc300 2024-09-12T12:20:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2ca59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa2cc15b2d0 0x7fa2cc0b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53660/0 (socket says 172.21.15.3:53660) 2024-09-12T12:20:42.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2ca59c640 1 -- 172.21.15.3:0/3264073059 learned_addr learned my addr 172.21.15.3:0/3264073059 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2cad9d640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc0cc300 0x7fa2cc073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c1d9b640 1 --2- 172.21.15.3:0/3264073059 >> v2:172.21.15.142:3300/0 conn(0x7fa2cc06f6e0 0x7fa2cc072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136050278 0 0) 0x7fa2cc15f150 con 0x7fa2cc15b2d0 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2b8003530 con 0x7fa2cc15b2d0 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3856150123 0 0) 0x7fa2b8003530 con 0x7fa2cc15b2d0 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc0cc300 msgr2=0x7fa2cc073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc0cc300 0x7fa2cc073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 >> v2:172.21.15.142:3300/0 conn(0x7fa2cc06f6e0 msgr2=0x7fa2cc072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 --2- 172.21.15.3:0/3264073059 >> v2:172.21.15.142:3300/0 conn(0x7fa2cc06f6e0 0x7fa2cc072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:42.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc15b6b0 con 0x7fa2cc15b2d0 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fa2bc001680 con 0x7fa2cc15b2d0 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 >> v1:172.21.15.3:6789/0 conn(0x7fa2cc15b2d0 legacy=0x7fa2cc0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc0cc300 0x7fa2b80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b8003f40 0x7fa2b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2b8004d50 0x7fa2b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0b3790 con 0x7fa2b8003f40 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0ae920 con 0x7fa2b8004d50 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b800b9d0 con 0x7fa2cc0cc300 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c1d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b8003f40 0x7fa2b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c1d9b640 1 -- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc0cc300 msgr2=0x7fa2b80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c1d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2cc0cc300 0x7fa2b80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:42.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c1d9b640 1 -- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2b8004d50 msgr2=0x7fa2b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:42.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c1d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2b8004d50 0x7fa2b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:42.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.858+0000 7fa2c1d9b640 1 -- 172.21.15.3:0/3264073059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b8003530 con 0x7fa2b8003f40 2024-09-12T12:20:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2c1d9b640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b8003f40 0x7fa2b8004810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ab0 tx=0x7fa2b401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418da35c6d8a966e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40b2070 con 0x7fa2b8003f40 2024-09-12T12:20:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b40a5680 con 0x7fa2b8003f40 2024-09-12T12:20:42.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2c9d9b640 1 -- 172.21.15.3:0/3264073059 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40096b0 con 0x7fa2b8003f40 2024-09-12T12:20:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 -- 172.21.15.3:0/3264073059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b8003f40 msgr2=0x7fa2b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 --2- 172.21.15.3:0/3264073059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2b8003f40 0x7fa2b8004810 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ab0 tx=0x7fa2b401ca30 comp rx=0 tx=0).stop 2024-09-12T12:20:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2cad9d640 1 -- 172.21.15.3:0/3264073059 reap_dead start 2024-09-12T12:20:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 -- 172.21.15.3:0/3264073059 shutdown_connections 2024-09-12T12:20:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 -- 172.21.15.3:0/3264073059 >> 172.21.15.3:0/3264073059 conn(0x7fa2cc0b1d80 msgr2=0x7fa2cc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 -- 172.21.15.3:0/3264073059 shutdown_connections 2024-09-12T12:20:42.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 -- 172.21.15.3:0/3264073059 wait complete. 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 Processor -- start 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.862+0000 7fa2d0da4640 1 -- start start 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2b8003f40 0x7fa2cc1f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc14a8b0 0x7fa2cc14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2cc14d2c0 0x7fa2cc14e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0ae2d0 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0adff0 con 0x7fa2cc14a8b0 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc0ae5b0 con 0x7fa2b8003f40 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2cc14d2c0 0x7fa2cc14e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2cc14d2c0 0x7fa2cc14e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53758/0 (socket says 172.21.15.3:53758) 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 -- 172.21.15.3:0/1890195293 learned_addr learned my addr 172.21.15.3:0/1890195293 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:42.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2ca59c640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2b8003f40 0x7fa2cc1f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2c1d9b640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc14a8b0 0x7fa2cc14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 -- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2b8003f40 msgr2=0x7fa2cc1f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2b8003f40 0x7fa2cc1f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 -- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc14a8b0 msgr2=0x7fa2cc14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc14a8b0 0x7fa2cc14cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 -- 172.21.15.3:0/1890195293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc14d7f0 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2cad9d640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2cc14d2c0 0x7fa2cc14e890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc06d470 tx=0x7fa2c400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2c37fe640 1 -- 172.21.15.3:0/1890195293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4018020 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2c37fe640 1 -- 172.21.15.3:0/1890195293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c400bc60 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2c37fe640 1 -- 172.21.15.3:0/1890195293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4014640 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc14dad0 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc14e090 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2cc06d8d0 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2c37fe640 1 -- 172.21.15.3:0/1890195293 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa2c401b020 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2c37fe640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2ac0678c0 0x7fa2ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:42.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.866+0000 7fa2c37fe640 1 -- 172.21.15.3:0/1890195293 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa2c4086070 con 0x7fa2cc14d2c0 2024-09-12T12:20:42.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.870+0000 7fa2ca59c640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2ac0678c0 0x7fa2ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:42.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.870+0000 7fa2ca59c640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2ac0678c0 0x7fa2ac069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc001530 tx=0x7fa2bc001440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:42.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:42.874+0000 7fa2c37fe640 1 -- 172.21.15.3:0/1890195293 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa2c4022030 con 0x7fa2cc14d2c0 2024-09-12T12:20:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.034+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2cc0b4070 con 0x7fa2ac0678c0 2024-09-12T12:20:43.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2c37fe640 1 -- 172.21.15.3:0/1890195293 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7fa2cc0b4070 con 0x7fa2ac0678c0 2024-09-12T12:20:43.058 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:43.059 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:20:43.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2ac0678c0 msgr2=0x7fa2ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2ac0678c0 0x7fa2ac069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc001530 tx=0x7fa2bc001440 comp rx=0 tx=0).stop 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2cc14d2c0 msgr2=0x7fa2cc14e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2cc14d2c0 0x7fa2cc14e890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc06d470 tx=0x7fa2c400b480 comp rx=0 tx=0).stop 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 shutdown_connections 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2ac0678c0 0x7fa2ac069d80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2cc14d2c0 0x7fa2cc14e890 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2cc14a8b0 0x7fa2cc14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 --2- 172.21.15.3:0/1890195293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa2b8003f40 0x7fa2cc1f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 >> 172.21.15.3:0/1890195293 conn(0x7fa2cc0b1d80 msgr2=0x7fa2cc0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 shutdown_connections 2024-09-12T12:20:43.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.054+0000 7fa2d0da4640 1 -- 172.21.15.3:0/1890195293 wait complete. 2024-09-12T12:20:43.077 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-12T12:20:41.674307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220240,"kb_used_data":4008,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754480,"statfs":{"total":773094113280,"available":772868587520,"internally_reserved":0,"allocated":4104192,"data_stored":2282156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.543863"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358296+0000","last_change":"2024-09-12T12:20:35.198491+0000","last_active":"2024-09-12T12:20:36.358296+0000","last_peered":"2024-09-12T12:20:36.358296+0000","last_clean":"2024-09-12T12:20:36.358296+0000","last_became_active":"2024-09-12T12:20:35.195072+0000","last_became_peered":"2024-09-12T12:20:35.195072+0000","last_unstale":"2024-09-12T12:20:36.358296+0000","last_undegraded":"2024-09-12T12:20:36.358296+0000","last_fullsized":"2024-09-12T12:20:36.358296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:31:12.496155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356079+0000","last_change":"2024-09-12T12:20:35.168650+0000","last_active":"2024-09-12T12:20:36.356079+0000","last_peered":"2024-09-12T12:20:36.356079+0000","last_clean":"2024-09-12T12:20:36.356079+0000","last_became_active":"2024-09-12T12:20:35.168048+0000","last_became_peered":"2024-09-12T12:20:35.168048+0000","last_unstale":"2024-09-12T12:20:36.356079+0000","last_undegraded":"2024-09-12T12:20:36.356079+0000","last_fullsized":"2024-09-12T12:20:36.356079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:50.278861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138388+0000","last_change":"2024-09-12T12:20:35.176652+0000","last_active":"2024-09-12T12:20:36.138388+0000","last_peered":"2024-09-12T12:20:36.138388+0000","last_clean":"2024-09-12T12:20:36.138388+0000","last_became_active":"2024-09-12T12:20:35.171653+0000","last_became_peered":"2024-09-12T12:20:35.171653+0000","last_unstale":"2024-09-12T12:20:36.138388+0000","last_undegraded":"2024-09-12T12:20:36.138388+0000","last_fullsized":"2024-09-12T12:20:36.138388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:45:44.391059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355508+0000","last_change":"2024-09-12T12:20:35.164219+0000","last_active":"2024-09-12T12:20:36.355508+0000","last_peered":"2024-09-12T12:20:36.355508+0000","last_clean":"2024-09-12T12:20:36.355508+0000","last_became_active":"2024-09-12T12:20:35.163777+0000","last_became_peered":"2024-09-12T12:20:35.163777+0000","last_unstale":"2024-09-12T12:20:36.355508+0000","last_undegraded":"2024-09-12T12:20:36.355508+0000","last_fullsized":"2024-09-12T12:20:36.355508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:21.050788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357496+0000","last_change":"2024-09-12T12:20:35.147788+0000","last_active":"2024-09-12T12:20:36.357496+0000","last_peered":"2024-09-12T12:20:36.357496+0000","last_clean":"2024-09-12T12:20:36.357496+0000","last_became_active":"2024-09-12T12:20:35.147505+0000","last_became_peered":"2024-09-12T12:20:35.147505+0000","last_unstale":"2024-09-12T12:20:36.357496+0000","last_undegraded":"2024-09-12T12:20:36.357496+0000","last_fullsized":"2024-09-12T12:20:36.357496+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:29.088616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356814+0000","last_change":"2024-09-12T12:20:35.175898+0000","last_active":"2024-09-12T12:20:36.356814+0000","last_peered":"2024-09-12T12:20:36.356814+0000","last_clean":"2024-09-12T12:20:36.356814+0000","last_became_active":"2024-09-12T12:20:35.173226+0000","last_became_peered":"2024-09-12T12:20:35.173226+0000","last_unstale":"2024-09-12T12:20:36.356814+0000","last_undegraded":"2024-09-12T12:20:36.356814+0000","last_fullsized":"2024-09-12T12:20:36.356814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:21:56.760424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359495+0000","last_change":"2024-09-12T12:20:35.191815+0000","last_active":"2024-09-12T12:20:36.359495+0000","last_peered":"2024-09-12T12:20:36.359495+0000","last_clean":"2024-09-12T12:20:36.359495+0000","last_became_active":"2024-09-12T12:20:35.191023+0000","last_became_peered":"2024-09-12T12:20:35.191023+0000","last_unstale":"2024-09-12T12:20:36.359495+0000","last_undegraded":"2024-09-12T12:20:36.359495+0000","last_fullsized":"2024-09-12T12:20:36.359495+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:12.696430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357994+0000","last_change":"2024-09-12T12:20:35.164501+0000","last_active":"2024-09-12T12:20:36.357994+0000","last_peered":"2024-09-12T12:20:36.357994+0000","last_clean":"2024-09-12T12:20:36.357994+0000","last_became_active":"2024-09-12T12:20:35.163683+0000","last_became_peered":"2024-09-12T12:20:35.163683+0000","last_unstale":"2024-09-12T12:20:36.357994+0000","last_undegraded":"2024-09-12T12:20:36.357994+0000","last_fullsized":"2024-09-12T12:20:36.357994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:50.814370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141629+0000","last_change":"2024-09-12T12:20:35.155924+0000","last_active":"2024-09-12T12:20:36.141629+0000","last_peered":"2024-09-12T12:20:36.141629+0000","last_clean":"2024-09-12T12:20:36.141629+0000","last_became_active":"2024-09-12T12:20:35.155184+0000","last_became_peered":"2024-09-12T12:20:35.155184+0000","last_unstale":"2024-09-12T12:20:36.141629+0000","last_undegraded":"2024-09-12T12:20:36.141629+0000","last_fullsized":"2024-09-12T12:20:36.141629+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:38:11.565938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355813+0000","last_change":"2024-09-12T12:20:35.155981+0000","last_active":"2024-09-12T12:20:36.355813+0000","last_peered":"2024-09-12T12:20:36.355813+0000","last_clean":"2024-09-12T12:20:36.355813+0000","last_became_active":"2024-09-12T12:20:35.155735+0000","last_became_peered":"2024-09-12T12:20:35.155735+0000","last_unstale":"2024-09-12T12:20:36.355813+0000","last_undegraded":"2024-09-12T12:20:36.355813+0000","last_fullsized":"2024-09-12T12:20:36.355813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:25:05.783164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359987+0000","last_change":"2024-09-12T12:20:35.171158+0000","last_active":"2024-09-12T12:20:36.359987+0000","last_peered":"2024-09-12T12:20:36.359987+0000","last_clean":"2024-09-12T12:20:36.359987+0000","last_became_active":"2024-09-12T12:20:35.170458+0000","last_became_peered":"2024-09-12T12:20:35.170458+0000","last_unstale":"2024-09-12T12:20:36.359987+0000","last_undegraded":"2024-09-12T12:20:36.359987+0000","last_fullsized":"2024-09-12T12:20:36.359987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:34.871974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138328+0000","last_change":"2024-09-12T12:20:35.167588+0000","last_active":"2024-09-12T12:20:36.138328+0000","last_peered":"2024-09-12T12:20:36.138328+0000","last_clean":"2024-09-12T12:20:36.138328+0000","last_became_active":"2024-09-12T12:20:35.157946+0000","last_became_peered":"2024-09-12T12:20:35.157946+0000","last_unstale":"2024-09-12T12:20:36.138328+0000","last_undegraded":"2024-09-12T12:20:36.138328+0000","last_fullsized":"2024-09-12T12:20:36.138328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:50.195252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357575+0000","last_change":"2024-09-12T12:20:35.154274+0000","last_active":"2024-09-12T12:20:36.357575+0000","last_peered":"2024-09-12T12:20:36.357575+0000","last_clean":"2024-09-12T12:20:36.357575+0000","last_became_active":"2024-09-12T12:20:35.153997+0000","last_became_peered":"2024-09-12T12:20:35.153997+0000","last_unstale":"2024-09-12T12:20:36.357575+0000","last_undegraded":"2024-09-12T12:20:36.357575+0000","last_fullsized":"2024-09-12T12:20:36.357575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:17.603565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356966+0000","last_change":"2024-09-12T12:20:35.173071+0000","last_active":"2024-09-12T12:20:36.356966+0000","last_peered":"2024-09-12T12:20:36.356966+0000","last_clean":"2024-09-12T12:20:36.356966+0000","last_became_active":"2024-09-12T12:20:35.172438+0000","last_became_peered":"2024-09-12T12:20:35.172438+0000","last_unstale":"2024-09-12T12:20:36.356966+0000","last_undegraded":"2024-09-12T12:20:36.356966+0000","last_fullsized":"2024-09-12T12:20:36.356966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:26.175293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137526+0000","last_change":"2024-09-12T12:20:35.190298+0000","last_active":"2024-09-12T12:20:36.137526+0000","last_peered":"2024-09-12T12:20:36.137526+0000","last_clean":"2024-09-12T12:20:36.137526+0000","last_became_active":"2024-09-12T12:20:35.189773+0000","last_became_peered":"2024-09-12T12:20:35.189773+0000","last_unstale":"2024-09-12T12:20:36.137526+0000","last_undegraded":"2024-09-12T12:20:36.137526+0000","last_fullsized":"2024-09-12T12:20:36.137526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359568+0000","last_change":"2024-09-12T12:20:35.183641+0000","last_active":"2024-09-12T12:20:36.359568+0000","last_peered":"2024-09-12T12:20:36.359568+0000","last_clean":"2024-09-12T12:20:36.359568+0000","last_became_active":"2024-09-12T12:20:35.182970+0000","last_became_peered":"2024-09-12T12:20:35.182970+0000","last_unstale":"2024-09-12T12:20:36.359568+0000","last_undegraded":"2024-09-12T12:20:36.359568+0000","last_fullsized":"2024-09-12T12:20:36.359568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:50.195252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.360209+0000","last_change":"2024-09-12T12:20:35.197004+0000","last_active":"2024-09-12T12:20:36.360209+0000","last_peered":"2024-09-12T12:20:36.360209+0000","last_clean":"2024-09-12T12:20:36.360209+0000","last_became_active":"2024-09-12T12:20:35.196392+0000","last_became_peered":"2024-09-12T12:20:35.196392+0000","last_unstale":"2024-09-12T12:20:36.360209+0000","last_undegraded":"2024-09-12T12:20:36.360209+0000","last_fullsized":"2024-09-12T12:20:36.360209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:22.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356211+0000","last_change":"2024-09-12T12:20:35.172750+0000","last_active":"2024-09-12T12:20:36.356211+0000","last_peered":"2024-09-12T12:20:36.356211+0000","last_clean":"2024-09-12T12:20:36.356211+0000","last_became_active":"2024-09-12T12:20:35.171985+0000","last_became_peered":"2024-09-12T12:20:35.171985+0000","last_unstale":"2024-09-12T12:20:36.356211+0000","last_undegraded":"2024-09-12T12:20:36.356211+0000","last_fullsized":"2024-09-12T12:20:36.356211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:21:25.885895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355243+0000","last_change":"2024-09-12T12:20:35.172308+0000","last_active":"2024-09-12T12:20:36.355243+0000","last_peered":"2024-09-12T12:20:36.355243+0000","last_clean":"2024-09-12T12:20:36.355243+0000","last_became_active":"2024-09-12T12:20:35.171866+0000","last_became_peered":"2024-09-12T12:20:35.171866+0000","last_unstale":"2024-09-12T12:20:36.355243+0000","last_undegraded":"2024-09-12T12:20:36.355243+0000","last_fullsized":"2024-09-12T12:20:36.355243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:13.240065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357475+0000","last_change":"2024-09-12T12:20:35.155045+0000","last_active":"2024-09-12T12:20:36.357475+0000","last_peered":"2024-09-12T12:20:36.357475+0000","last_clean":"2024-09-12T12:20:36.357475+0000","last_became_active":"2024-09-12T12:20:35.154834+0000","last_became_peered":"2024-09-12T12:20:35.154834+0000","last_unstale":"2024-09-12T12:20:36.357475+0000","last_undegraded":"2024-09-12T12:20:36.357475+0000","last_fullsized":"2024-09-12T12:20:36.357475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:14.773145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140725+0000","last_change":"2024-09-12T12:20:35.156240+0000","last_active":"2024-09-12T12:20:36.140725+0000","last_peered":"2024-09-12T12:20:36.140725+0000","last_clean":"2024-09-12T12:20:36.140725+0000","last_became_active":"2024-09-12T12:20:35.155462+0000","last_became_peered":"2024-09-12T12:20:35.155462+0000","last_unstale":"2024-09-12T12:20:36.140725+0000","last_undegraded":"2024-09-12T12:20:36.140725+0000","last_fullsized":"2024-09-12T12:20:36.140725+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140875+0000","last_change":"2024-09-12T12:20:35.162247+0000","last_active":"2024-09-12T12:20:36.140875+0000","last_peered":"2024-09-12T12:20:36.140875+0000","last_clean":"2024-09-12T12:20:36.140875+0000","last_became_active":"2024-09-12T12:20:35.160236+0000","last_became_peered":"2024-09-12T12:20:35.160236+0000","last_unstale":"2024-09-12T12:20:36.140875+0000","last_undegraded":"2024-09-12T12:20:36.140875+0000","last_fullsized":"2024-09-12T12:20:36.140875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:17:21.014713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355731+0000","last_change":"2024-09-12T12:20:35.148316+0000","last_active":"2024-09-12T12:20:36.355731+0000","last_peered":"2024-09-12T12:20:36.355731+0000","last_clean":"2024-09-12T12:20:36.355731+0000","last_became_active":"2024-09-12T12:20:35.147986+0000","last_became_peered":"2024-09-12T12:20:35.147986+0000","last_unstale":"2024-09-12T12:20:36.355731+0000","last_undegraded":"2024-09-12T12:20:36.355731+0000","last_fullsized":"2024-09-12T12:20:36.355731+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.587701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137141+0000","last_change":"2024-09-12T12:20:35.150689+0000","last_active":"2024-09-12T12:20:36.137141+0000","last_peered":"2024-09-12T12:20:36.137141+0000","last_clean":"2024-09-12T12:20:36.137141+0000","last_became_active":"2024-09-12T12:20:35.149930+0000","last_became_peered":"2024-09-12T12:20:35.149930+0000","last_unstale":"2024-09-12T12:20:36.137141+0000","last_undegraded":"2024-09-12T12:20:36.137141+0000","last_fullsized":"2024-09-12T12:20:36.137141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:42.454081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356283+0000","last_change":"2024-09-12T12:20:35.165003+0000","last_active":"2024-09-12T12:20:36.356283+0000","last_peered":"2024-09-12T12:20:36.356283+0000","last_clean":"2024-09-12T12:20:36.356283+0000","last_became_active":"2024-09-12T12:20:35.164704+0000","last_became_peered":"2024-09-12T12:20:35.164704+0000","last_unstale":"2024-09-12T12:20:36.356283+0000","last_undegraded":"2024-09-12T12:20:36.356283+0000","last_fullsized":"2024-09-12T12:20:36.356283+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:07.194590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356294+0000","last_change":"2024-09-12T12:20:35.176503+0000","last_active":"2024-09-12T12:20:36.356294+0000","last_peered":"2024-09-12T12:20:36.356294+0000","last_clean":"2024-09-12T12:20:36.356294+0000","last_became_active":"2024-09-12T12:20:35.175766+0000","last_became_peered":"2024-09-12T12:20:35.175766+0000","last_unstale":"2024-09-12T12:20:36.356294+0000","last_undegraded":"2024-09-12T12:20:36.356294+0000","last_fullsized":"2024-09-12T12:20:36.356294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:31:39.480802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355992+0000","last_change":"2024-09-12T12:20:35.165263+0000","last_active":"2024-09-12T12:20:36.355992+0000","last_peered":"2024-09-12T12:20:36.355992+0000","last_clean":"2024-09-12T12:20:36.355992+0000","last_became_active":"2024-09-12T12:20:35.164157+0000","last_became_peered":"2024-09-12T12:20:35.164157+0000","last_unstale":"2024-09-12T12:20:36.355992+0000","last_undegraded":"2024-09-12T12:20:36.355992+0000","last_fullsized":"2024-09-12T12:20:36.355992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:57:27.914148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138027+0000","last_change":"2024-09-12T12:20:35.152115+0000","last_active":"2024-09-12T12:20:36.138027+0000","last_peered":"2024-09-12T12:20:36.138027+0000","last_clean":"2024-09-12T12:20:36.138027+0000","last_became_active":"2024-09-12T12:20:35.151461+0000","last_became_peered":"2024-09-12T12:20:35.151461+0000","last_unstale":"2024-09-12T12:20:36.138027+0000","last_undegraded":"2024-09-12T12:20:36.138027+0000","last_fullsized":"2024-09-12T12:20:36.138027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:25:12.696430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140748+0000","last_change":"2024-09-12T12:20:35.166787+0000","last_active":"2024-09-12T12:20:36.140748+0000","last_peered":"2024-09-12T12:20:36.140748+0000","last_clean":"2024-09-12T12:20:36.140748+0000","last_became_active":"2024-09-12T12:20:35.164351+0000","last_became_peered":"2024-09-12T12:20:35.164351+0000","last_unstale":"2024-09-12T12:20:36.140748+0000","last_undegraded":"2024-09-12T12:20:36.140748+0000","last_fullsized":"2024-09-12T12:20:36.140748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:46:59.470812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359410+0000","last_change":"2024-09-12T12:20:35.190696+0000","last_active":"2024-09-12T12:20:36.359410+0000","last_peered":"2024-09-12T12:20:36.359410+0000","last_clean":"2024-09-12T12:20:36.359410+0000","last_became_active":"2024-09-12T12:20:35.190109+0000","last_became_peered":"2024-09-12T12:20:35.190109+0000","last_unstale":"2024-09-12T12:20:36.359410+0000","last_undegraded":"2024-09-12T12:20:36.359410+0000","last_fullsized":"2024-09-12T12:20:36.359410+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:25.941687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357854+0000","last_change":"2024-09-12T12:20:35.186425+0000","last_active":"2024-09-12T12:20:36.357854+0000","last_peered":"2024-09-12T12:20:36.357854+0000","last_clean":"2024-09-12T12:20:36.357854+0000","last_became_active":"2024-09-12T12:20:35.185695+0000","last_became_peered":"2024-09-12T12:20:35.185695+0000","last_unstale":"2024-09-12T12:20:36.357854+0000","last_undegraded":"2024-09-12T12:20:36.357854+0000","last_fullsized":"2024-09-12T12:20:36.357854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:31.416987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358559+0000","last_change":"2024-09-12T12:20:35.179554+0000","last_active":"2024-09-12T12:20:36.358559+0000","last_peered":"2024-09-12T12:20:36.358559+0000","last_clean":"2024-09-12T12:20:36.358559+0000","last_became_active":"2024-09-12T12:20:35.178868+0000","last_became_peered":"2024-09-12T12:20:35.178868+0000","last_unstale":"2024-09-12T12:20:36.358559+0000","last_undegraded":"2024-09-12T12:20:36.358559+0000","last_fullsized":"2024-09-12T12:20:36.358559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:42.454081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356864+0000","last_change":"2024-09-12T12:20:35.164093+0000","last_active":"2024-09-12T12:20:36.356864+0000","last_peered":"2024-09-12T12:20:36.356864+0000","last_clean":"2024-09-12T12:20:36.356864+0000","last_became_active":"2024-09-12T12:20:35.163581+0000","last_became_peered":"2024-09-12T12:20:35.163581+0000","last_unstale":"2024-09-12T12:20:36.356864+0000","last_undegraded":"2024-09-12T12:20:36.356864+0000","last_fullsized":"2024-09-12T12:20:36.356864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:47:45.135412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358445+0000","last_change":"2024-09-12T12:20:34.167240+0000","last_active":"2024-09-12T12:20:36.358445+0000","last_peered":"2024-09-12T12:20:36.358445+0000","last_clean":"2024-09-12T12:20:36.358445+0000","last_became_active":"2024-09-12T12:20:34.166461+0000","last_became_peered":"2024-09-12T12:20:34.166461+0000","last_unstale":"2024-09-12T12:20:36.358445+0000","last_undegraded":"2024-09-12T12:20:36.358445+0000","last_fullsized":"2024-09-12T12:20:36.358445+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:57.357006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358593+0000","last_change":"2024-09-12T12:20:35.197969+0000","last_active":"2024-09-12T12:20:36.358593+0000","last_peered":"2024-09-12T12:20:36.358593+0000","last_clean":"2024-09-12T12:20:36.358593+0000","last_became_active":"2024-09-12T12:20:35.193450+0000","last_became_peered":"2024-09-12T12:20:35.193450+0000","last_unstale":"2024-09-12T12:20:36.358593+0000","last_undegraded":"2024-09-12T12:20:36.358593+0000","last_fullsized":"2024-09-12T12:20:36.358593+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:59:51.081885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356917+0000","last_change":"2024-09-12T12:20:34.162205+0000","last_active":"2024-09-12T12:20:36.356917+0000","last_peered":"2024-09-12T12:20:36.356917+0000","last_clean":"2024-09-12T12:20:36.356917+0000","last_became_active":"2024-09-12T12:20:34.161881+0000","last_became_peered":"2024-09-12T12:20:34.161881+0000","last_unstale":"2024-09-12T12:20:36.356917+0000","last_undegraded":"2024-09-12T12:20:36.356917+0000","last_fullsized":"2024-09-12T12:20:36.356917+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:12:22.305382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141187+0000","last_change":"2024-09-12T12:20:35.179544+0000","last_active":"2024-09-12T12:20:36.141187+0000","last_peered":"2024-09-12T12:20:36.141187+0000","last_clean":"2024-09-12T12:20:36.141187+0000","last_became_active":"2024-09-12T12:20:35.178800+0000","last_became_peered":"2024-09-12T12:20:35.178800+0000","last_unstale":"2024-09-12T12:20:36.141187+0000","last_undegraded":"2024-09-12T12:20:36.141187+0000","last_fullsized":"2024-09-12T12:20:36.141187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:21.050788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355304+0000","last_change":"2024-09-12T12:20:34.162650+0000","last_active":"2024-09-12T12:20:36.355304+0000","last_peered":"2024-09-12T12:20:36.355304+0000","last_clean":"2024-09-12T12:20:36.355304+0000","last_became_active":"2024-09-12T12:20:34.161254+0000","last_became_peered":"2024-09-12T12:20:34.161254+0000","last_unstale":"2024-09-12T12:20:36.355304+0000","last_undegraded":"2024-09-12T12:20:36.355304+0000","last_fullsized":"2024-09-12T12:20:36.355304+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:20.125706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356159+0000","last_change":"2024-09-12T12:20:35.164640+0000","last_active":"2024-09-12T12:20:36.356159+0000","last_peered":"2024-09-12T12:20:36.356159+0000","last_clean":"2024-09-12T12:20:36.356159+0000","last_became_active":"2024-09-12T12:20:35.163578+0000","last_became_peered":"2024-09-12T12:20:35.163578+0000","last_unstale":"2024-09-12T12:20:36.356159+0000","last_undegraded":"2024-09-12T12:20:36.356159+0000","last_fullsized":"2024-09-12T12:20:36.356159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:28.976887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357588+0000","last_change":"2024-09-12T12:20:34.150212+0000","last_active":"2024-09-12T12:20:36.357588+0000","last_peered":"2024-09-12T12:20:36.357588+0000","last_clean":"2024-09-12T12:20:36.357588+0000","last_became_active":"2024-09-12T12:20:34.149775+0000","last_became_peered":"2024-09-12T12:20:34.149775+0000","last_unstale":"2024-09-12T12:20:36.357588+0000","last_undegraded":"2024-09-12T12:20:36.357588+0000","last_fullsized":"2024-09-12T12:20:36.357588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:58.418103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359648+0000","last_change":"2024-09-12T12:20:35.177876+0000","last_active":"2024-09-12T12:20:36.359648+0000","last_peered":"2024-09-12T12:20:36.359648+0000","last_clean":"2024-09-12T12:20:36.359648+0000","last_became_active":"2024-09-12T12:20:35.177096+0000","last_became_peered":"2024-09-12T12:20:35.177096+0000","last_unstale":"2024-09-12T12:20:36.359648+0000","last_undegraded":"2024-09-12T12:20:36.359648+0000","last_fullsized":"2024-09-12T12:20:36.359648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:45:44.391059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141058+0000","last_change":"2024-09-12T12:20:34.155082+0000","last_active":"2024-09-12T12:20:36.141058+0000","last_peered":"2024-09-12T12:20:36.141058+0000","last_clean":"2024-09-12T12:20:36.141058+0000","last_became_active":"2024-09-12T12:20:34.154312+0000","last_became_peered":"2024-09-12T12:20:34.154312+0000","last_unstale":"2024-09-12T12:20:36.141058+0000","last_undegraded":"2024-09-12T12:20:36.141058+0000","last_fullsized":"2024-09-12T12:20:36.141058+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:02.834445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137401+0000","last_change":"2024-09-12T12:20:35.144513+0000","last_active":"2024-09-12T12:20:36.137401+0000","last_peered":"2024-09-12T12:20:36.137401+0000","last_clean":"2024-09-12T12:20:36.137401+0000","last_became_active":"2024-09-12T12:20:35.143849+0000","last_became_peered":"2024-09-12T12:20:35.143849+0000","last_unstale":"2024-09-12T12:20:36.137401+0000","last_undegraded":"2024-09-12T12:20:36.137401+0000","last_fullsized":"2024-09-12T12:20:36.137401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:05:42.965426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140745+0000","last_change":"2024-09-12T12:20:34.162299+0000","last_active":"2024-09-12T12:20:36.140745+0000","last_peered":"2024-09-12T12:20:36.140745+0000","last_clean":"2024-09-12T12:20:36.140745+0000","last_became_active":"2024-09-12T12:20:34.160652+0000","last_became_peered":"2024-09-12T12:20:34.160652+0000","last_unstale":"2024-09-12T12:20:36.140745+0000","last_undegraded":"2024-09-12T12:20:36.140745+0000","last_fullsized":"2024-09-12T12:20:36.140745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:47.749206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.139878+0000","last_change":"2024-09-12T12:20:35.187370+0000","last_active":"2024-09-12T12:20:36.139878+0000","last_peered":"2024-09-12T12:20:36.139878+0000","last_clean":"2024-09-12T12:20:36.139878+0000","last_became_active":"2024-09-12T12:20:35.186743+0000","last_became_peered":"2024-09-12T12:20:35.186743+0000","last_unstale":"2024-09-12T12:20:36.139878+0000","last_undegraded":"2024-09-12T12:20:36.139878+0000","last_fullsized":"2024-09-12T12:20:36.139878+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:16:56.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355406+0000","last_change":"2024-09-12T12:20:34.154781+0000","last_active":"2024-09-12T12:20:36.355406+0000","last_peered":"2024-09-12T12:20:36.355406+0000","last_clean":"2024-09-12T12:20:36.355406+0000","last_became_active":"2024-09-12T12:20:34.154499+0000","last_became_peered":"2024-09-12T12:20:34.154499+0000","last_unstale":"2024-09-12T12:20:36.355406+0000","last_undegraded":"2024-09-12T12:20:36.355406+0000","last_fullsized":"2024-09-12T12:20:36.355406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:07:14.564521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355549+0000","last_change":"2024-09-12T12:20:35.159085+0000","last_active":"2024-09-12T12:20:36.355549+0000","last_peered":"2024-09-12T12:20:36.355549+0000","last_clean":"2024-09-12T12:20:36.355549+0000","last_became_active":"2024-09-12T12:20:35.158722+0000","last_became_peered":"2024-09-12T12:20:35.158722+0000","last_unstale":"2024-09-12T12:20:36.355549+0000","last_undegraded":"2024-09-12T12:20:36.355549+0000","last_fullsized":"2024-09-12T12:20:36.355549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:24.785474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140218+0000","last_change":"2024-09-12T12:20:34.166953+0000","last_active":"2024-09-12T12:20:36.140218+0000","last_peered":"2024-09-12T12:20:36.140218+0000","last_clean":"2024-09-12T12:20:36.140218+0000","last_became_active":"2024-09-12T12:20:34.166192+0000","last_became_peered":"2024-09-12T12:20:34.166192+0000","last_unstale":"2024-09-12T12:20:36.140218+0000","last_undegraded":"2024-09-12T12:20:36.140218+0000","last_fullsized":"2024-09-12T12:20:36.140218+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:31.934869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138049+0000","last_change":"2024-09-12T12:20:35.150750+0000","last_active":"2024-09-12T12:20:36.138049+0000","last_peered":"2024-09-12T12:20:36.138049+0000","last_clean":"2024-09-12T12:20:36.138049+0000","last_became_active":"2024-09-12T12:20:35.150011+0000","last_became_peered":"2024-09-12T12:20:35.150011+0000","last_unstale":"2024-09-12T12:20:36.138049+0000","last_undegraded":"2024-09-12T12:20:36.138049+0000","last_fullsized":"2024-09-12T12:20:36.138049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:54:32.040523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356212+0000","last_change":"2024-09-12T12:20:34.168229+0000","last_active":"2024-09-12T12:20:36.356212+0000","last_peered":"2024-09-12T12:20:36.356212+0000","last_clean":"2024-09-12T12:20:36.356212+0000","last_became_active":"2024-09-12T12:20:34.167930+0000","last_became_peered":"2024-09-12T12:20:34.167930+0000","last_unstale":"2024-09-12T12:20:36.356212+0000","last_undegraded":"2024-09-12T12:20:36.356212+0000","last_fullsized":"2024-09-12T12:20:36.356212+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:11.575182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356095+0000","last_change":"2024-09-12T12:20:35.172164+0000","last_active":"2024-09-12T12:20:36.356095+0000","last_peered":"2024-09-12T12:20:36.356095+0000","last_clean":"2024-09-12T12:20:36.356095+0000","last_became_active":"2024-09-12T12:20:35.171857+0000","last_became_peered":"2024-09-12T12:20:35.171857+0000","last_unstale":"2024-09-12T12:20:36.356095+0000","last_undegraded":"2024-09-12T12:20:36.356095+0000","last_fullsized":"2024-09-12T12:20:36.356095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:55.826703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356964+0000","last_change":"2024-09-12T12:20:34.149095+0000","last_active":"2024-09-12T12:20:36.356964+0000","last_peered":"2024-09-12T12:20:36.356964+0000","last_clean":"2024-09-12T12:20:36.356964+0000","last_became_active":"2024-09-12T12:20:34.148258+0000","last_became_peered":"2024-09-12T12:20:34.148258+0000","last_unstale":"2024-09-12T12:20:36.356964+0000","last_undegraded":"2024-09-12T12:20:36.356964+0000","last_fullsized":"2024-09-12T12:20:36.356964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:32:34.037182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356015+0000","last_change":"2024-09-12T12:20:35.176754+0000","last_active":"2024-09-12T12:20:36.356015+0000","last_peered":"2024-09-12T12:20:36.356015+0000","last_clean":"2024-09-12T12:20:36.356015+0000","last_became_active":"2024-09-12T12:20:35.176424+0000","last_became_peered":"2024-09-12T12:20:35.176424+0000","last_unstale":"2024-09-12T12:20:36.356015+0000","last_undegraded":"2024-09-12T12:20:36.356015+0000","last_fullsized":"2024-09-12T12:20:36.356015+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:21:11.005047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138688+0000","last_change":"2024-09-12T12:20:34.174690+0000","last_active":"2024-09-12T12:20:36.138688+0000","last_peered":"2024-09-12T12:20:36.138688+0000","last_clean":"2024-09-12T12:20:36.138688+0000","last_became_active":"2024-09-12T12:20:34.173961+0000","last_became_peered":"2024-09-12T12:20:34.173961+0000","last_unstale":"2024-09-12T12:20:36.138688+0000","last_undegraded":"2024-09-12T12:20:36.138688+0000","last_fullsized":"2024-09-12T12:20:36.138688+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:20.038270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.139028+0000","last_change":"2024-09-12T12:20:35.172607+0000","last_active":"2024-09-12T12:20:36.139028+0000","last_peered":"2024-09-12T12:20:36.139028+0000","last_clean":"2024-09-12T12:20:36.139028+0000","last_became_active":"2024-09-12T12:20:35.170650+0000","last_became_peered":"2024-09-12T12:20:35.170650+0000","last_unstale":"2024-09-12T12:20:36.139028+0000","last_undegraded":"2024-09-12T12:20:36.139028+0000","last_fullsized":"2024-09-12T12:20:36.139028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:05:22.821230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355386+0000","last_change":"2024-09-12T12:20:34.165218+0000","last_active":"2024-09-12T12:20:36.355386+0000","last_peered":"2024-09-12T12:20:36.355386+0000","last_clean":"2024-09-12T12:20:36.355386+0000","last_became_active":"2024-09-12T12:20:34.161437+0000","last_became_peered":"2024-09-12T12:20:34.161437+0000","last_unstale":"2024-09-12T12:20:36.355386+0000","last_undegraded":"2024-09-12T12:20:36.355386+0000","last_fullsized":"2024-09-12T12:20:36.355386+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:41.453879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355842+0000","last_change":"2024-09-12T12:20:35.158990+0000","last_active":"2024-09-12T12:20:36.355842+0000","last_peered":"2024-09-12T12:20:36.355842+0000","last_clean":"2024-09-12T12:20:36.355842+0000","last_became_active":"2024-09-12T12:20:35.156888+0000","last_became_peered":"2024-09-12T12:20:35.156888+0000","last_unstale":"2024-09-12T12:20:36.355842+0000","last_undegraded":"2024-09-12T12:20:36.355842+0000","last_fullsized":"2024-09-12T12:20:36.355842+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:55.208741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141175+0000","last_change":"2024-09-12T12:20:34.167052+0000","last_active":"2024-09-12T12:20:36.141175+0000","last_peered":"2024-09-12T12:20:36.141175+0000","last_clean":"2024-09-12T12:20:36.141175+0000","last_became_active":"2024-09-12T12:20:34.166290+0000","last_became_peered":"2024-09-12T12:20:34.166290+0000","last_unstale":"2024-09-12T12:20:36.141175+0000","last_undegraded":"2024-09-12T12:20:36.141175+0000","last_fullsized":"2024-09-12T12:20:36.141175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:19.356916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355771+0000","last_change":"2024-09-12T12:20:35.166430+0000","last_active":"2024-09-12T12:20:36.355771+0000","last_peered":"2024-09-12T12:20:36.355771+0000","last_clean":"2024-09-12T12:20:36.355771+0000","last_became_active":"2024-09-12T12:20:35.160670+0000","last_became_peered":"2024-09-12T12:20:35.160670+0000","last_unstale":"2024-09-12T12:20:36.355771+0000","last_undegraded":"2024-09-12T12:20:36.355771+0000","last_fullsized":"2024-09-12T12:20:36.355771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:37:40.029294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355937+0000","last_change":"2024-09-12T12:20:34.164222+0000","last_active":"2024-09-12T12:20:36.355937+0000","last_peered":"2024-09-12T12:20:36.355937+0000","last_clean":"2024-09-12T12:20:36.355937+0000","last_became_active":"2024-09-12T12:20:34.163943+0000","last_became_peered":"2024-09-12T12:20:34.163943+0000","last_unstale":"2024-09-12T12:20:36.355937+0000","last_undegraded":"2024-09-12T12:20:36.355937+0000","last_fullsized":"2024-09-12T12:20:36.355937+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:05.729418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359109+0000","last_change":"2024-09-12T12:20:35.172112+0000","last_active":"2024-09-12T12:20:36.359109+0000","last_peered":"2024-09-12T12:20:36.359109+0000","last_clean":"2024-09-12T12:20:36.359109+0000","last_became_active":"2024-09-12T12:20:35.171294+0000","last_became_peered":"2024-09-12T12:20:35.171294+0000","last_unstale":"2024-09-12T12:20:36.359109+0000","last_undegraded":"2024-09-12T12:20:36.359109+0000","last_fullsized":"2024-09-12T12:20:36.359109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:05:42.965426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355501+0000","last_change":"2024-09-12T12:20:34.163140+0000","last_active":"2024-09-12T12:20:36.355501+0000","last_peered":"2024-09-12T12:20:36.355501+0000","last_clean":"2024-09-12T12:20:36.355501+0000","last_became_active":"2024-09-12T12:20:34.162749+0000","last_became_peered":"2024-09-12T12:20:34.162749+0000","last_unstale":"2024-09-12T12:20:36.355501+0000","last_undegraded":"2024-09-12T12:20:36.355501+0000","last_fullsized":"2024-09-12T12:20:36.355501+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:48.665029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357971+0000","last_change":"2024-09-12T12:20:35.190045+0000","last_active":"2024-09-12T12:20:36.357971+0000","last_peered":"2024-09-12T12:20:36.357971+0000","last_clean":"2024-09-12T12:20:36.357971+0000","last_became_active":"2024-09-12T12:20:35.189475+0000","last_became_peered":"2024-09-12T12:20:35.189475+0000","last_unstale":"2024-09-12T12:20:36.357971+0000","last_undegraded":"2024-09-12T12:20:36.357971+0000","last_fullsized":"2024-09-12T12:20:36.357971+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:18.104578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355594+0000","last_change":"2024-09-12T12:20:34.157183+0000","last_active":"2024-09-12T12:20:36.355594+0000","last_peered":"2024-09-12T12:20:36.355594+0000","last_clean":"2024-09-12T12:20:36.355594+0000","last_became_active":"2024-09-12T12:20:34.156676+0000","last_became_peered":"2024-09-12T12:20:34.156676+0000","last_unstale":"2024-09-12T12:20:36.355594+0000","last_undegraded":"2024-09-12T12:20:36.355594+0000","last_fullsized":"2024-09-12T12:20:36.355594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:17.783533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141400+0000","last_change":"2024-09-12T12:20:35.159266+0000","last_active":"2024-09-12T12:20:36.141400+0000","last_peered":"2024-09-12T12:20:36.141400+0000","last_clean":"2024-09-12T12:20:36.141400+0000","last_became_active":"2024-09-12T12:20:35.158421+0000","last_became_peered":"2024-09-12T12:20:35.158421+0000","last_unstale":"2024-09-12T12:20:36.141400+0000","last_undegraded":"2024-09-12T12:20:36.141400+0000","last_fullsized":"2024-09-12T12:20:36.141400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:24.785474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356354+0000","last_change":"2024-09-12T12:20:34.155817+0000","last_active":"2024-09-12T12:20:36.356354+0000","last_peered":"2024-09-12T12:20:36.356354+0000","last_clean":"2024-09-12T12:20:36.356354+0000","last_became_active":"2024-09-12T12:20:34.154842+0000","last_became_peered":"2024-09-12T12:20:34.154842+0000","last_unstale":"2024-09-12T12:20:36.356354+0000","last_undegraded":"2024-09-12T12:20:36.356354+0000","last_fullsized":"2024-09-12T12:20:36.356354+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:04.798090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138038+0000","last_change":"2024-09-12T12:20:35.169303+0000","last_active":"2024-09-12T12:20:36.138038+0000","last_peered":"2024-09-12T12:20:36.138038+0000","last_clean":"2024-09-12T12:20:36.138038+0000","last_became_active":"2024-09-12T12:20:35.159457+0000","last_became_peered":"2024-09-12T12:20:35.159457+0000","last_unstale":"2024-09-12T12:20:36.138038+0000","last_undegraded":"2024-09-12T12:20:36.138038+0000","last_fullsized":"2024-09-12T12:20:36.138038+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:25.941687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141350+0000","last_change":"2024-09-12T12:20:34.167145+0000","last_active":"2024-09-12T12:20:36.141350+0000","last_peered":"2024-09-12T12:20:36.141350+0000","last_clean":"2024-09-12T12:20:36.141350+0000","last_became_active":"2024-09-12T12:20:34.166392+0000","last_became_peered":"2024-09-12T12:20:34.166392+0000","last_unstale":"2024-09-12T12:20:36.141350+0000","last_undegraded":"2024-09-12T12:20:36.141350+0000","last_fullsized":"2024-09-12T12:20:36.141350+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:28.076097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355743+0000","last_change":"2024-09-12T12:20:35.164408+0000","last_active":"2024-09-12T12:20:36.355743+0000","last_peered":"2024-09-12T12:20:36.355743+0000","last_clean":"2024-09-12T12:20:36.355743+0000","last_became_active":"2024-09-12T12:20:35.163770+0000","last_became_peered":"2024-09-12T12:20:35.163770+0000","last_unstale":"2024-09-12T12:20:36.355743+0000","last_undegraded":"2024-09-12T12:20:36.355743+0000","last_fullsized":"2024-09-12T12:20:36.355743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:38:11.565938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141623+0000","last_change":"2024-09-12T12:20:34.149142+0000","last_active":"2024-09-12T12:20:36.141623+0000","last_peered":"2024-09-12T12:20:36.141623+0000","last_clean":"2024-09-12T12:20:36.141623+0000","last_became_active":"2024-09-12T12:20:34.148431+0000","last_became_peered":"2024-09-12T12:20:34.148431+0000","last_unstale":"2024-09-12T12:20:36.141623+0000","last_undegraded":"2024-09-12T12:20:36.141623+0000","last_fullsized":"2024-09-12T12:20:36.141623+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:13.760627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357818+0000","last_change":"2024-09-12T12:20:35.160542+0000","last_active":"2024-09-12T12:20:36.357818+0000","last_peered":"2024-09-12T12:20:36.357818+0000","last_clean":"2024-09-12T12:20:36.357818+0000","last_became_active":"2024-09-12T12:20:35.160248+0000","last_became_peered":"2024-09-12T12:20:35.160248+0000","last_unstale":"2024-09-12T12:20:36.357818+0000","last_undegraded":"2024-09-12T12:20:36.357818+0000","last_fullsized":"2024-09-12T12:20:36.357818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:20.369434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141131+0000","last_change":"2024-09-12T12:20:34.156849+0000","last_active":"2024-09-12T12:20:36.141131+0000","last_peered":"2024-09-12T12:20:36.141131+0000","last_clean":"2024-09-12T12:20:36.141131+0000","last_became_active":"2024-09-12T12:20:34.153898+0000","last_became_peered":"2024-09-12T12:20:34.153898+0000","last_unstale":"2024-09-12T12:20:36.141131+0000","last_undegraded":"2024-09-12T12:20:36.141131+0000","last_fullsized":"2024-09-12T12:20:36.141131+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:16.591047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357402+0000","last_change":"2024-09-12T12:20:35.162691+0000","last_active":"2024-09-12T12:20:36.357402+0000","last_peered":"2024-09-12T12:20:36.357402+0000","last_clean":"2024-09-12T12:20:36.357402+0000","last_became_active":"2024-09-12T12:20:35.162383+0000","last_became_peered":"2024-09-12T12:20:35.162383+0000","last_unstale":"2024-09-12T12:20:36.357402+0000","last_undegraded":"2024-09-12T12:20:36.357402+0000","last_fullsized":"2024-09-12T12:20:36.357402+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:48.761725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356597+0000","last_change":"2024-09-12T12:20:34.150352+0000","last_active":"2024-09-12T12:20:36.356597+0000","last_peered":"2024-09-12T12:20:36.356597+0000","last_clean":"2024-09-12T12:20:36.356597+0000","last_became_active":"2024-09-12T12:20:34.150067+0000","last_became_peered":"2024-09-12T12:20:34.150067+0000","last_unstale":"2024-09-12T12:20:36.356597+0000","last_undegraded":"2024-09-12T12:20:36.356597+0000","last_fullsized":"2024-09-12T12:20:36.356597+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:58.443790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357745+0000","last_change":"2024-09-12T12:20:35.144745+0000","last_active":"2024-09-12T12:20:36.357745+0000","last_peered":"2024-09-12T12:20:36.357745+0000","last_clean":"2024-09-12T12:20:36.357745+0000","last_became_active":"2024-09-12T12:20:35.144472+0000","last_became_peered":"2024-09-12T12:20:35.144472+0000","last_unstale":"2024-09-12T12:20:36.357745+0000","last_undegraded":"2024-09-12T12:20:36.357745+0000","last_fullsized":"2024-09-12T12:20:36.357745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:03.846963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357940+0000","last_change":"2024-09-12T12:20:34.167240+0000","last_active":"2024-09-12T12:20:36.357940+0000","last_peered":"2024-09-12T12:20:36.357940+0000","last_clean":"2024-09-12T12:20:36.357940+0000","last_became_active":"2024-09-12T12:20:34.163642+0000","last_became_peered":"2024-09-12T12:20:34.163642+0000","last_unstale":"2024-09-12T12:20:36.357940+0000","last_undegraded":"2024-09-12T12:20:36.357940+0000","last_fullsized":"2024-09-12T12:20:36.357940+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:19.356916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357246+0000","last_change":"2024-09-12T12:20:35.144677+0000","last_active":"2024-09-12T12:20:36.357246+0000","last_peered":"2024-09-12T12:20:36.357246+0000","last_clean":"2024-09-12T12:20:36.357246+0000","last_became_active":"2024-09-12T12:20:35.144360+0000","last_became_peered":"2024-09-12T12:20:35.144360+0000","last_unstale":"2024-09-12T12:20:36.357246+0000","last_undegraded":"2024-09-12T12:20:36.357246+0000","last_fullsized":"2024-09-12T12:20:36.357246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:10.456058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140903+0000","last_change":"2024-09-12T12:20:34.147318+0000","last_active":"2024-09-12T12:20:36.140903+0000","last_peered":"2024-09-12T12:20:36.140903+0000","last_clean":"2024-09-12T12:20:36.140903+0000","last_became_active":"2024-09-12T12:20:34.146581+0000","last_became_peered":"2024-09-12T12:20:34.146581+0000","last_unstale":"2024-09-12T12:20:36.140903+0000","last_undegraded":"2024-09-12T12:20:36.140903+0000","last_fullsized":"2024-09-12T12:20:36.140903+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:43:28.217090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359297+0000","last_change":"2024-09-12T12:20:35.192139+0000","last_active":"2024-09-12T12:20:36.359297+0000","last_peered":"2024-09-12T12:20:36.359297+0000","last_clean":"2024-09-12T12:20:36.359297+0000","last_became_active":"2024-09-12T12:20:35.190075+0000","last_became_peered":"2024-09-12T12:20:35.190075+0000","last_unstale":"2024-09-12T12:20:36.359297+0000","last_undegraded":"2024-09-12T12:20:36.359297+0000","last_fullsized":"2024-09-12T12:20:36.359297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:45.787047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.137001+0000","last_change":"2024-09-12T12:20:34.174400+0000","last_active":"2024-09-12T12:20:36.137001+0000","last_peered":"2024-09-12T12:20:36.137001+0000","last_clean":"2024-09-12T12:20:36.137001+0000","last_became_active":"2024-09-12T12:20:34.173969+0000","last_became_peered":"2024-09-12T12:20:34.173969+0000","last_unstale":"2024-09-12T12:20:36.137001+0000","last_undegraded":"2024-09-12T12:20:36.137001+0000","last_fullsized":"2024-09-12T12:20:36.137001+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:17:20.002195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356123+0000","last_change":"2024-09-12T12:20:35.156705+0000","last_active":"2024-09-12T12:20:36.356123+0000","last_peered":"2024-09-12T12:20:36.356123+0000","last_clean":"2024-09-12T12:20:36.356123+0000","last_became_active":"2024-09-12T12:20:35.155895+0000","last_became_peered":"2024-09-12T12:20:35.155895+0000","last_unstale":"2024-09-12T12:20:36.356123+0000","last_undegraded":"2024-09-12T12:20:36.356123+0000","last_fullsized":"2024-09-12T12:20:36.356123+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:43:29.229608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356355+0000","last_change":"2024-09-12T12:20:34.172819+0000","last_active":"2024-09-12T12:20:36.356355+0000","last_peered":"2024-09-12T12:20:36.356355+0000","last_clean":"2024-09-12T12:20:36.356355+0000","last_became_active":"2024-09-12T12:20:34.172454+0000","last_became_peered":"2024-09-12T12:20:34.172454+0000","last_unstale":"2024-09-12T12:20:36.356355+0000","last_undegraded":"2024-09-12T12:20:36.356355+0000","last_fullsized":"2024-09-12T12:20:36.356355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:59:35.934892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.138417+0000","last_change":"2024-09-12T12:20:35.168420+0000","last_active":"2024-09-12T12:20:36.138417+0000","last_peered":"2024-09-12T12:20:36.138417+0000","last_clean":"2024-09-12T12:20:36.138417+0000","last_became_active":"2024-09-12T12:20:35.157401+0000","last_became_peered":"2024-09-12T12:20:35.157401+0000","last_unstale":"2024-09-12T12:20:36.138417+0000","last_undegraded":"2024-09-12T12:20:36.138417+0000","last_fullsized":"2024-09-12T12:20:36.138417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:34.871974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:37.139174+0000","last_change":"2024-09-12T12:20:34.151870+0000","last_active":"2024-09-12T12:20:37.139174+0000","last_peered":"2024-09-12T12:20:37.139174+0000","last_clean":"2024-09-12T12:20:37.139174+0000","last_became_active":"2024-09-12T12:20:34.150672+0000","last_became_peered":"2024-09-12T12:20:34.150672+0000","last_unstale":"2024-09-12T12:20:37.139174+0000","last_undegraded":"2024-09-12T12:20:37.139174+0000","last_fullsized":"2024-09-12T12:20:37.139174+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:33.392858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355328+0000","last_change":"2024-09-12T12:20:35.150674+0000","last_active":"2024-09-12T12:20:36.355328+0000","last_peered":"2024-09-12T12:20:36.355328+0000","last_clean":"2024-09-12T12:20:36.355328+0000","last_became_active":"2024-09-12T12:20:35.150422+0000","last_became_peered":"2024-09-12T12:20:35.150422+0000","last_unstale":"2024-09-12T12:20:36.355328+0000","last_undegraded":"2024-09-12T12:20:36.355328+0000","last_fullsized":"2024-09-12T12:20:36.355328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:21.138224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355943+0000","last_change":"2024-09-12T12:20:34.155182+0000","last_active":"2024-09-12T12:20:36.355943+0000","last_peered":"2024-09-12T12:20:36.355943+0000","last_clean":"2024-09-12T12:20:36.355943+0000","last_became_active":"2024-09-12T12:20:34.154872+0000","last_became_peered":"2024-09-12T12:20:34.154872+0000","last_unstale":"2024-09-12T12:20:36.355943+0000","last_undegraded":"2024-09-12T12:20:36.355943+0000","last_fullsized":"2024-09-12T12:20:36.355943+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:57.357006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.357966+0000","last_change":"2024-09-12T12:20:35.182021+0000","last_active":"2024-09-12T12:20:36.357966+0000","last_peered":"2024-09-12T12:20:36.357966+0000","last_clean":"2024-09-12T12:20:36.357966+0000","last_became_active":"2024-09-12T12:20:35.181397+0000","last_became_peered":"2024-09-12T12:20:35.181397+0000","last_unstale":"2024-09-12T12:20:36.357966+0000","last_undegraded":"2024-09-12T12:20:36.357966+0000","last_fullsized":"2024-09-12T12:20:36.357966+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:16:56.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356682+0000","last_change":"2024-09-12T12:20:34.153660+0000","last_active":"2024-09-12T12:20:36.356682+0000","last_peered":"2024-09-12T12:20:36.356682+0000","last_clean":"2024-09-12T12:20:36.356682+0000","last_became_active":"2024-09-12T12:20:34.152819+0000","last_became_peered":"2024-09-12T12:20:34.152819+0000","last_unstale":"2024-09-12T12:20:36.356682+0000","last_undegraded":"2024-09-12T12:20:36.356682+0000","last_fullsized":"2024-09-12T12:20:36.356682+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:05:27.974012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356543+0000","last_change":"2024-09-12T12:20:35.181331+0000","last_active":"2024-09-12T12:20:36.356543+0000","last_peered":"2024-09-12T12:20:36.356543+0000","last_clean":"2024-09-12T12:20:36.356543+0000","last_became_active":"2024-09-12T12:20:35.181013+0000","last_became_peered":"2024-09-12T12:20:35.181013+0000","last_unstale":"2024-09-12T12:20:36.356543+0000","last_undegraded":"2024-09-12T12:20:36.356543+0000","last_fullsized":"2024-09-12T12:20:36.356543+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:42.466397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.356039+0000","last_change":"2024-09-12T12:20:34.156655+0000","last_active":"2024-09-12T12:20:36.356039+0000","last_peered":"2024-09-12T12:20:36.356039+0000","last_clean":"2024-09-12T12:20:36.356039+0000","last_became_active":"2024-09-12T12:20:34.156384+0000","last_became_peered":"2024-09-12T12:20:34.156384+0000","last_unstale":"2024-09-12T12:20:36.356039+0000","last_undegraded":"2024-09-12T12:20:36.356039+0000","last_fullsized":"2024-09-12T12:20:36.356039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:08.114810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355553+0000","last_change":"2024-09-12T12:20:35.178899+0000","last_active":"2024-09-12T12:20:36.355553+0000","last_peered":"2024-09-12T12:20:36.355553+0000","last_clean":"2024-09-12T12:20:36.355553+0000","last_became_active":"2024-09-12T12:20:35.175704+0000","last_became_peered":"2024-09-12T12:20:35.175704+0000","last_unstale":"2024-09-12T12:20:36.355553+0000","last_undegraded":"2024-09-12T12:20:36.355553+0000","last_fullsized":"2024-09-12T12:20:36.355553+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:04:10.061214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.141383+0000","last_change":"2024-09-12T12:20:34.164506+0000","last_active":"2024-09-12T12:20:36.141383+0000","last_peered":"2024-09-12T12:20:36.141383+0000","last_clean":"2024-09-12T12:20:36.141383+0000","last_became_active":"2024-09-12T12:20:34.163812+0000","last_became_peered":"2024-09-12T12:20:34.163812+0000","last_unstale":"2024-09-12T12:20:36.141383+0000","last_undegraded":"2024-09-12T12:20:36.141383+0000","last_fullsized":"2024-09-12T12:20:36.141383+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:09.443540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.140633+0000","last_change":"2024-09-12T12:20:35.156268+0000","last_active":"2024-09-12T12:20:36.140633+0000","last_peered":"2024-09-12T12:20:36.140633+0000","last_clean":"2024-09-12T12:20:36.140633+0000","last_became_active":"2024-09-12T12:20:35.155510+0000","last_became_peered":"2024-09-12T12:20:35.155510+0000","last_unstale":"2024-09-12T12:20:36.140633+0000","last_undegraded":"2024-09-12T12:20:36.140633+0000","last_fullsized":"2024-09-12T12:20:36.140633+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:31:12.496155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.355620+0000","last_change":"2024-09-12T12:20:34.162403+0000","last_active":"2024-09-12T12:20:36.355620+0000","last_peered":"2024-09-12T12:20:36.355620+0000","last_clean":"2024-09-12T12:20:36.355620+0000","last_became_active":"2024-09-12T12:20:34.161974+0000","last_became_peered":"2024-09-12T12:20:34.161974+0000","last_unstale":"2024-09-12T12:20:36.355620+0000","last_undegraded":"2024-09-12T12:20:36.355620+0000","last_fullsized":"2024-09-12T12:20:36.355620+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:22.796229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359546+0000","last_change":"2024-09-12T12:20:35.164445+0000","last_active":"2024-09-12T12:20:36.359546+0000","last_peered":"2024-09-12T12:20:36.359546+0000","last_clean":"2024-09-12T12:20:36.359546+0000","last_became_active":"2024-09-12T12:20:35.163705+0000","last_became_peered":"2024-09-12T12:20:35.163705+0000","last_unstale":"2024-09-12T12:20:36.359546+0000","last_undegraded":"2024-09-12T12:20:36.359546+0000","last_fullsized":"2024-09-12T12:20:36.359546+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:34.115959+0000","last_clean_scrub_stamp":"2024-09-12T12:20:34.115959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:54:32.040523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.358134+0000","last_change":"2024-09-12T12:20:34.164793+0000","last_active":"2024-09-12T12:20:36.358134+0000","last_peered":"2024-09-12T12:20:36.358134+0000","last_clean":"2024-09-12T12:20:36.358134+0000","last_became_active":"2024-09-12T12:20:34.164086+0000","last_became_peered":"2024-09-12T12:20:34.164086+0000","last_unstale":"2024-09-12T12:20:36.358134+0000","last_undegraded":"2024-09-12T12:20:36.358134+0000","last_fullsized":"2024-09-12T12:20:36.358134+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:20:33.103441+0000","last_clean_scrub_stamp":"2024-09-12T12:20:33.103441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:02.834445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-12T12:20:36.359716+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:20:36.359716+0000","last_peered":"2024-09-12T12:20:36.359716+0000","last_clean":"2024-09-12T12:20:36.359716+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:20:36.359716+0000","last_undegraded":"2024-09-12T12:20:36.359716+0000","last_fullsized":"2024-09-12T12:20:36.359716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":372736,"data_stored":138709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":954368,"data_stored":727533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":364544,"data_stored":137179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":137145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542159,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":372736,"data_stored":138709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":727443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:20:43.080 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:20:43.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T12:20:43.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.246+0000 7fa100a6f640 1 Processor -- start 2024-09-12T12:20:43.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa100a6f640 1 -- start start 2024-09-12T12:20:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa100a6f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa0fc1530a0 0x7fa0fc15c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa100a6f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0fc0d1cf0 0x7fa0fc0d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa100a6f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc15d0b0 con 0x7fa0fc153480 2024-09-12T12:20:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa100a6f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa0fc0cec50 con 0x7fa0fc0d1cf0 2024-09-12T12:20:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa100a6f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa0fc15d390 con 0x7fa0fc1530a0 2024-09-12T12:20:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0f9d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0fc0d1cf0 0x7fa0fc0d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0f9d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0fc0d1cf0 0x7fa0fc0d2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53766/0 (socket says 172.21.15.3:53766) 2024-09-12T12:20:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0f9d9b640 1 -- 172.21.15.3:0/1402349590 learned_addr learned my addr 172.21.15.3:0/1402349590 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0fa59c640 1 --2- 172.21.15.3:0/1402349590 >> v2:172.21.15.142:3300/0 conn(0x7fa0fc1530a0 0x7fa0fc15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0f9d9b640 1 -- 172.21.15.3:0/1402349590 >> v2:172.21.15.142:3300/0 conn(0x7fa0fc1530a0 msgr2=0x7fa0fc15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0f9d9b640 1 --2- 172.21.15.3:0/1402349590 >> v2:172.21.15.142:3300/0 conn(0x7fa0fc1530a0 0x7fa0fc15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0f9d9b640 1 -- 172.21.15.3:0/1402349590 >> v1:172.21.15.142:6789/0 conn(0x7fa0fc153480 legacy=0x7fa0fc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.250+0000 7fa0f9d9b640 1 -- 172.21.15.3:0/1402349590 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc15d670 con 0x7fa0fc0d1cf0 2024-09-12T12:20:43.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0f9d9b640 1 --2- 172.21.15.3:0/1402349590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc0d1cf0 0x7fa0fc0d2210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e6b525ad642eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0f959a640 1 -- 172.21.15.3:0/1402349590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00b1070 con 0x7fa0fc0d1cf0 2024-09-12T12:20:43.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0f959a640 1 -- 172.21.15.3:0/1402349590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f00ac410 con 0x7fa0fc0d1cf0 2024-09-12T12:20:43.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0f959a640 1 -- 172.21.15.3:0/1402349590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00ac7e0 con 0x7fa0fc0d1cf0 2024-09-12T12:20:43.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- 172.21.15.3:0/1402349590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc0d1cf0 msgr2=0x7fa0fc0d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 --2- 172.21.15.3:0/1402349590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc0d1cf0 0x7fa0fc0d2210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f001c4e0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- 172.21.15.3:0/1402349590 shutdown_connections 2024-09-12T12:20:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 --2- 172.21.15.3:0/1402349590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc0d1cf0 0x7fa0fc0d2210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 --2- 172.21.15.3:0/1402349590 >> v2:172.21.15.142:3300/0 conn(0x7fa0fc1530a0 0x7fa0fc15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- 172.21.15.3:0/1402349590 >> 172.21.15.3:0/1402349590 conn(0x7fa0fc070060 msgr2=0x7fa0fc070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- 172.21.15.3:0/1402349590 shutdown_connections 2024-09-12T12:20:43.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- 172.21.15.3:0/1402349590 wait complete. 2024-09-12T12:20:43.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 Processor -- start 2024-09-12T12:20:43.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- start start 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc0d1cf0 0x7fa0fc0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0fc1530a0 0x7fa0fc1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc153480 0x7fa0fc15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0d0660 con 0x7fa0fc153480 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0d2b60 con 0x7fa0fc0d1cf0 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa100a6f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc0d07e0 con 0x7fa0fc1530a0 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0fad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc153480 0x7fa0fc15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0fad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc153480 0x7fa0fc15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53782/0 (socket says 172.21.15.3:53782) 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0fad9d640 1 -- 172.21.15.3:0/474631489 learned_addr learned my addr 172.21.15.3:0/474631489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.254+0000 7fa0fa59c640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc0d1cf0 0x7fa0fc0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0f9d9b640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0fc1530a0 0x7fa0fc1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fad9d640 1 -- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0fc1530a0 msgr2=0x7fa0fc1560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fad9d640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0fc1530a0 0x7fa0fc1560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fad9d640 1 -- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc0d1cf0 msgr2=0x7fa0fc0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fad9d640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc0d1cf0 0x7fa0fc0c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fad9d640 1 -- 172.21.15.3:0/474631489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc153a50 con 0x7fa0fc153480 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fad9d640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc153480 0x7fa0fc15d890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b9a0 tx=0x7fa0f400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0eaffd640 1 -- 172.21.15.3:0/474631489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4019070 con 0x7fa0fc153480 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0eaffd640 1 -- 172.21.15.3:0/474631489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f400cba0 con 0x7fa0fc153480 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc153d30 con 0x7fa0fc153480 2024-09-12T12:20:43.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0eaffd640 1 -- 172.21.15.3:0/474631489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4014440 con 0x7fa0fc153480 2024-09-12T12:20:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1542f0 con 0x7fa0fc153480 2024-09-12T12:20:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc0d2210 con 0x7fa0fc153480 2024-09-12T12:20:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0eaffd640 1 -- 172.21.15.3:0/474631489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0f401c070 con 0x7fa0fc153480 2024-09-12T12:20:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0eaffd640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0dc0678c0 0x7fa0dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0eaffd640 1 -- 172.21.15.3:0/474631489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa0f4085900 con 0x7fa0fc153480 2024-09-12T12:20:43.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fa59c640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0dc0678c0 0x7fa0dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.258+0000 7fa0fa59c640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0dc0678c0 0x7fa0dc069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b9d0 tx=0x7fa0ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.262+0000 7fa0eaffd640 1 -- 172.21.15.3:0/474631489 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa0f4056440 con 0x7fa0fc153480 2024-09-12T12:20:43.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.490+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa0fc072350 con 0x7fa0fc153480 2024-09-12T12:20:43.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa0eaffd640 1 -- 172.21.15.3:0/474631489 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fa0f4055d40 con 0x7fa0fc153480 2024-09-12T12:20:43.498 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T12:20:43.498 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:20:43.498 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-12T12:20:43.498 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-12T12:20:43.498 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T12:20:43.499 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-12T12:20:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0dc0678c0 msgr2=0x7fa0dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0dc0678c0 0x7fa0dc069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b9d0 tx=0x7fa0ec008040 comp rx=0 tx=0).stop 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc153480 msgr2=0x7fa0fc15d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc153480 0x7fa0fc15d890 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400b9a0 tx=0x7fa0f400be60 comp rx=0 tx=0).stop 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 shutdown_connections 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0dc0678c0 0x7fa0dc069d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0fc153480 0x7fa0fc15d890 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0fc1530a0 0x7fa0fc1560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 --2- 172.21.15.3:0/474631489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa0fc0d1cf0 0x7fa0fc0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 >> 172.21.15.3:0/474631489 conn(0x7fa0fc070060 msgr2=0x7fa0fc0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 shutdown_connections 2024-09-12T12:20:43.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.494+0000 7fa100a6f640 1 -- 172.21.15.3:0/474631489 wait complete. 2024-09-12T12:20:43.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:20:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.638+0000 7fed23cef640 1 Processor -- start 2024-09-12T12:20:43.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.638+0000 7fed23cef640 1 -- start start 2024-09-12T12:20:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed23cef640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 0x7fed1c15da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed23cef640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1c15c570 con 0x7fed1c0b47f0 2024-09-12T12:20:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed23cef640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed1c15c850 con 0x7fed1c15bcf0 2024-09-12T12:20:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed23cef640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c15cb30 con 0x7fed1c15c0d0 2024-09-12T12:20:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fed1c0b47f0 0x7fed1c15d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53676/0 (socket says 172.21.15.3:53676) 2024-09-12T12:20:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 -- 172.21.15.3:0/1127550594 learned_addr learned my addr 172.21.15.3:0/1127550594 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:43.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2228c640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 0x7fed1c15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683735271 0 0) 0x7fed1c15c570 con 0x7fed1c0b47f0 2024-09-12T12:20:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed08003530 con 0x7fed1c0b47f0 2024-09-12T12:20:43.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2394420104 0 0) 0x7fed1c15c850 con 0x7fed1c15bcf0 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed1c15c570 con 0x7fed1c15bcf0 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 424625383 0 0) 0x7fed08003530 con 0x7fed1c0b47f0 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 msgr2=0x7fed1c15da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 0x7fed1c15da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 >> v1:172.21.15.142:6789/0 conn(0x7fed1c15bcf0 legacy=0x7fed1c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c15ce10 con 0x7fed1c0b47f0 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fed0c0016a0 con 0x7fed1c0b47f0 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 >> v1:172.21.15.3:6789/0 conn(0x7fed1c0b47f0 legacy=0x7fed1c15d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed08003b80 0x7fed08003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed08004810 0x7fed08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 0x7fed080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c15cb30 con 0x7fed08004810 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed0800bab0 con 0x7fed08003b80 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fed0800bcb0 con 0x7fed1c15c0d0 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed08004810 0x7fed08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed21a8b640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed08003b80 0x7fed08003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2228c640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 0x7fed080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 -- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 msgr2=0x7fed080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 0x7fed080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 -- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed08003b80 msgr2=0x7fed08003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed08003b80 0x7fed08003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2128a640 1 -- 172.21.15.3:0/1127550594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed08003530 con 0x7fed08004810 2024-09-12T12:20:43.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.642+0000 7fed2228c640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed1c15c0d0 0x7fed080051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2128a640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed08004810 0x7fed08004c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00c160 tx=0x7fed0c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e15230b8345dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c007c50 con 0x7fed08004810 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c0a9070 con 0x7fed08004810 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed20a89640 1 -- 172.21.15.3:0/1127550594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c005d20 con 0x7fed08004810 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- 172.21.15.3:0/1127550594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed08004810 msgr2=0x7fed08004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 --2- 172.21.15.3:0/1127550594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed08004810 0x7fed08004c90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00c160 tx=0x7fed0c01cba0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 -- 172.21.15.3:0/1127550594 reap_dead start 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- 172.21.15.3:0/1127550594 shutdown_connections 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- 172.21.15.3:0/1127550594 >> 172.21.15.3:0/1127550594 conn(0x7fed1c073500 msgr2=0x7fed1c0b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- 172.21.15.3:0/1127550594 shutdown_connections 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- 172.21.15.3:0/1127550594 wait complete. 2024-09-12T12:20:43.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 Processor -- start 2024-09-12T12:20:43.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- start start 2024-09-12T12:20:43.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed08004810 0x7fed1c09df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed1c0b0e80 0x7fed1c0b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed1c09c960 0x7fed1c09cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c09aa70 con 0x7fed1c09c960 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c0b55d0 con 0x7fed1c0b0e80 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed23cef640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fed1c09abf0 con 0x7fed08004810 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed1c09c960 0x7fed1c09cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed1c09c960 0x7fed1c09cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53800/0 (socket says 172.21.15.3:53800) 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 -- 172.21.15.3:0/2822905657 learned_addr learned my addr 172.21.15.3:0/2822905657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed21a8b640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed08004810 0x7fed1c09df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2128a640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed1c0b0e80 0x7fed1c0b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 -- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed08004810 msgr2=0x7fed1c09df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed08004810 0x7fed1c09df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 -- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed1c0b0e80 msgr2=0x7fed1c0b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed1c0b0e80 0x7fed1c0b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 -- 172.21.15.3:0/2822905657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c1eff40 con 0x7fed1c09c960 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed2228c640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed1c09c960 0x7fed1c09cdc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0b39d0 tx=0x7fed14010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed127fc640 1 -- 172.21.15.3:0/2822905657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1400f040 con 0x7fed1c09c960 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed127fc640 1 -- 172.21.15.3:0/2822905657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1401c070 con 0x7fed1c09c960 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.646+0000 7fed127fc640 1 -- 172.21.15.3:0/2822905657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1400eac0 con 0x7fed1c09c960 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c0af8f0 con 0x7fed1c09c960 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c0afeb0 con 0x7fed1c09c960 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed1c1f0110 con 0x7fed1c09c960 2024-09-12T12:20:43.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed127fc640 1 -- 172.21.15.3:0/2822905657 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fed1400ec60 con 0x7fed1c09c960 2024-09-12T12:20:43.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed127fc640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed000678c0 0x7fed00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed127fc640 1 -- 172.21.15.3:0/2822905657 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fed14085920 con 0x7fed1c09c960 2024-09-12T12:20:43.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed21a8b640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed000678c0 0x7fed00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.650+0000 7fed21a8b640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed000678c0 0x7fed00069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fed180059b0 tx=0x7fed18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.654+0000 7fed127fc640 1 -- 172.21.15.3:0/2822905657 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fed14026020 con 0x7fed1c09c960 2024-09-12T12:20:43.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.854+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed1c0cd310 con 0x7fed1c09c960 2024-09-12T12:20:43.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed127fc640 1 -- 172.21.15.3:0/2822905657 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5220 (secure 0 0 0) 0x7fed140563c0 con 0x7fed1c09c960 2024-09-12T12:20:43.862 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:20:43.862 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:20:34.509216+0000","modified":"2024-09-12T12:20:36.529296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4812":{"gid":4812,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4232231567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4232231567},{"type":"v1","addr":"172.21.15.3:6835","nonce":4232231567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4812,"qdb_cluster":[4812]},"id":2}]} 2024-09-12T12:20:43.862 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 14 2024-09-12T12:20:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed000678c0 msgr2=0x7fed00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed000678c0 0x7fed00069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fed180059b0 tx=0x7fed18009040 comp rx=0 tx=0).stop 2024-09-12T12:20:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed1c09c960 msgr2=0x7fed1c09cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed1c09c960 0x7fed1c09cdc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0b39d0 tx=0x7fed14010c20 comp rx=0 tx=0).stop 2024-09-12T12:20:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 shutdown_connections 2024-09-12T12:20:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fed000678c0 0x7fed00069d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed1c09c960 0x7fed1c09cdc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed1c0b0e80 0x7fed1c0b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 --2- 172.21.15.3:0/2822905657 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fed08004810 0x7fed1c09df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 >> 172.21.15.3:0/2822905657 conn(0x7fed1c073500 msgr2=0x7fed1c15ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 shutdown_connections 2024-09-12T12:20:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:43.858+0000 7fed23cef640 1 -- 172.21.15.3:0/2822905657 wait complete. 2024-09-12T12:20:44.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:20:34.509216+0000', 'modified': '2024-09-12T12:20:36.529296+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4812}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4812': {'gid': 4812, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/4232231567', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 4232231567}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 4232231567}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4812, 'qdb_cluster': [4812]} 2024-09-12T12:20:44.284 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:20:44.284 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T12:20:44.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccfc61640 1 Processor -- start 2024-09-12T12:20:44.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccfc61640 1 -- start start 2024-09-12T12:20:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccfc61640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0cc80b38b0 0x7f0cc80d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccfc61640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cc806bcc0 0x7f0cc806c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccfc61640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cc80cd6e0 con 0x7f0cc80b1980 2024-09-12T12:20:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccfc61640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80d0fb0 con 0x7f0cc806bcc0 2024-09-12T12:20:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccfc61640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc806c640 con 0x7f0cc80b38b0 2024-09-12T12:20:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccd9fd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0cc80b1980 0x7f0cc80ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53692/0 (socket says 172.21.15.3:53692) 2024-09-12T12:20:44.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccd9fd640 1 -- 172.21.15.3:0/1226819363 learned_addr learned my addr 172.21.15.3:0/1226819363 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:44.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/1226819363 >> v2:172.21.15.3:3300/0 conn(0x7f0cc806bcc0 0x7f0cc806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/1226819363 >> v2:172.21.15.142:3300/0 conn(0x7f0cc80b38b0 0x7f0cc80d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0cce1fe640 1 -- 172.21.15.3:0/1226819363 >> v2:172.21.15.142:3300/0 conn(0x7f0cc80b38b0 msgr2=0x7f0cc80d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/1226819363 >> v2:172.21.15.142:3300/0 conn(0x7f0cc80b38b0 0x7f0cc80d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0cce1fe640 1 -- 172.21.15.3:0/1226819363 >> v1:172.21.15.3:6789/0 conn(0x7f0cc80b1980 legacy=0x7f0cc80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0cce1fe640 1 -- 172.21.15.3:0/1226819363 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc80cd9c0 con 0x7f0cc806bcc0 2024-09-12T12:20:44.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccc9fb640 1 -- 172.21.15.3:0/1226819363 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18278203 0 0) 0x7f0cc80cd6e0 con 0x7f0cc80b1980 2024-09-12T12:20:44.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.418+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/1226819363 >> v2:172.21.15.142:3300/0 conn(0x7f0cc80b38b0 0x7f0cc80d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:44.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/1226819363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc806bcc0 0x7f0cc806c100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80718a0 tx=0x7f0cc001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306a7c6d7c74b51d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccc9fb640 1 -- 172.21.15.3:0/1226819363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00b5070 con 0x7f0cc806bcc0 2024-09-12T12:20:44.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccc9fb640 1 -- 172.21.15.3:0/1226819363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc00a8970 con 0x7f0cc806bcc0 2024-09-12T12:20:44.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccc9fb640 1 -- 172.21.15.3:0/1226819363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00a8d40 con 0x7f0cc806bcc0 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1226819363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc806bcc0 msgr2=0x7f0cc806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1226819363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc806bcc0 0x7f0cc806c100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80718a0 tx=0x7f0cc001ec70 comp rx=0 tx=0).stop 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1226819363 shutdown_connections 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1226819363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc806bcc0 0x7f0cc806c100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1226819363 >> v2:172.21.15.142:3300/0 conn(0x7f0cc80b38b0 0x7f0cc80d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1226819363 >> 172.21.15.3:0/1226819363 conn(0x7f0cc80b2d90 msgr2=0x7f0cc80b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1226819363 shutdown_connections 2024-09-12T12:20:44.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1226819363 wait complete. 2024-09-12T12:20:44.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 Processor -- start 2024-09-12T12:20:44.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- start start 2024-09-12T12:20:44.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cc806bcc0 0x7f0cc80b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80b1980 0x7f0cc8148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc80b38b0 0x7f0cc80ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80b1d60 con 0x7f0cc80b38b0 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80cdf80 con 0x7f0cc80b1980 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0ccfc61640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc80b1ee0 con 0x7f0cc806bcc0 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc80b38b0 0x7f0cc80ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc80b38b0 0x7f0cc80ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53822/0 (socket says 172.21.15.3:53822) 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 -- 172.21.15.3:0/1643862605 learned_addr learned my addr 172.21.15.3:0/1643862605 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 -- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cc806bcc0 msgr2=0x7f0cc80b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cc806bcc0 0x7f0cc80b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 -- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80b1980 msgr2=0x7f0cc8148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80b1980 0x7f0cc8148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 -- 172.21.15.3:0/1643862605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc80aea50 con 0x7f0cc80b38b0 2024-09-12T12:20:44.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.422+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc80b38b0 0x7f0cc80ae150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0cc001b880 tx=0x7f0cc001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cbe7fc640 1 -- 172.21.15.3:0/1643862605 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00be070 con 0x7f0cc80b38b0 2024-09-12T12:20:44.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cbe7fc640 1 -- 172.21.15.3:0/1643862605 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc00b07c0 con 0x7f0cc80b38b0 2024-09-12T12:20:44.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc81499d0 con 0x7f0cc80b38b0 2024-09-12T12:20:44.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cbe7fc640 1 -- 172.21.15.3:0/1643862605 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00b5040 con 0x7f0cc80b38b0 2024-09-12T12:20:44.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc8149f90 con 0x7f0cc80b38b0 2024-09-12T12:20:44.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cbe7fc640 1 -- 172.21.15.3:0/1643862605 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0cc00c1070 con 0x7f0cc80b38b0 2024-09-12T12:20:44.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cbe7fc640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0cac0678c0 0x7f0cac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cbe7fc640 1 -- 172.21.15.3:0/1643862605 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0cc0122600 con 0x7f0cc80b38b0 2024-09-12T12:20:44.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0cac0678c0 0x7f0cac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cd2d1d000 1 Processor -- start 2024-09-12T12:20:44.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0cd2d1d000 1 -- start start 2024-09-12T12:20:44.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.426+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0cac0678c0 0x7f0cac069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4001570 tx=0x7f0cc4007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba07b20 0x55a16ba07f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9be290 0x55a16b9be670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a16b8a5b60 con 0x55a16ba08440 2024-09-12T12:20:44.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a16b80cc70 con 0x55a16b9be290 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a16b7b2ed0 con 0x55a16ba07b20 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9bdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53826/0 (socket says 172.21.15.3:53826) 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3743135192 learned_addr learned my addr 172.21.15.3:0/3743135192 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9be290 0x55a16b9be670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba07b20 0x55a16ba07f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba07b20 msgr2=0x55a16ba07f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba07b20 0x55a16ba07f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9be290 msgr2=0x55a16b9be670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9be290 0x55a16b9be670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3743135192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16b7c4e50 con 0x55a16ba08440 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9bdd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800a930 tx=0x7f0cb800adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6070da5da05ea2b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cab7fe640 1 -- 172.21.15.3:0/3743135192 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8015070 con 0x55a16ba08440 2024-09-12T12:20:44.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cab7fe640 1 -- 172.21.15.3:0/3743135192 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb800cc00 con 0x55a16ba08440 2024-09-12T12:20:44.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cab7fe640 1 -- 172.21.15.3:0/3743135192 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb80113f0 con 0x55a16ba08440 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3743135192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 msgr2=0x55a16b9bdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9bdd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800a930 tx=0x7f0cb800adf0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3743135192 shutdown_connections 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9be290 0x55a16b9be670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9bdd50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3743135192 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba07b20 0x55a16ba07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3743135192 >> 172.21.15.3:0/3743135192 conn(0x55a16b9c4600 msgr2=0x55a16b9bc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3743135192 shutdown_connections 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3743135192 wait complete. 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- start start 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 Processor -- start 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16ba07b20 0x55a16b9c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16b9c3330 0x55a16b9c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a16b86eb00 con 0x55a16ba08440 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a16b7e7360 con 0x55a16ba07b20 2024-09-12T12:20:44.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a16b7e42d0 con 0x55a16b9c3330 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cd2d1d000 10 client.0 _ll_register_callbacks cb 0x55a16b9a9820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53828/0 (socket says 172.21.15.3:53828) 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3706309784 learned_addr learned my addr 172.21.15.3:0/3706309784 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16ba07b20 0x55a16b9c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16b9c3330 0x55a16b9c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16b9c3330 msgr2=0x55a16b9c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16b9c3330 0x55a16b9c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16ba07b20 msgr2=0x55a16b9c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16ba07b20 0x55a16b9c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/3706309784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16b7b3120 con 0x55a16ba08440 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16ba07b20 0x55a16b9c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.430+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16b9c3330 0x55a16b9c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9c8cf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8008010 tx=0x7f0cb8006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6f667c8de9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/3706309784 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb802a060 con 0x55a16ba08440 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/3706309784 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0cb800cdb0 con 0x55a16ba08440 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb800cc00 con 0x55a16ba08440 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/3706309784 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f0cb8030050 con 0x55a16ba08440 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 10 client.5010 fetch_fsmap learned FSMap version 14 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55a16b7ac450 con 0x55a16ba08440 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/3706309784 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb800cc00 con 0x55a16ba08440 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/3706309784 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f0cb8033020 con 0x55a16ba08440 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 10 client.5010 fetch_fsmap finished waiting for FSMap version 14 2024-09-12T12:20:44.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 10 client.5010 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 20 client.5010 populate_metadata read hostname 'smithi003' 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba0df50 0x55a16ba0e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 4 client.5010 mds_command: new command op to 4812 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 --> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a16b7a6c00 con 0x55a16ba0df50 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba0df50 0x55a16ba0e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba0df50 0x55a16ba0e330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb08d42a15290f0 server_cookie=e931226f0cd6826b in_seq=0 out_seq=0 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 10 client.5010 ms_handle_connect on v2:172.21.15.3:6834/4232231567 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/3706309784 <== mds.0 v2:172.21.15.3:6834/4232231567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a16b7a6c00 con 0x55a16ba0df50 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 10 client.5010 handle_command_reply: tid=0 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 client.5010 shutdown 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 msgr2=0x55a16b9c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9c8cf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8008010 tx=0x7f0cb8006be0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 shutdown_connections 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba0df50 0x55a16ba0e330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16b9c3330 0x55a16b9c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba08440 0x55a16b9c8cf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0ca8ff9640 0 client.5010 ms_handle_reset on v2:172.21.15.3:6834/4232231567 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/3706309784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16ba07b20 0x55a16b9c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 >> 172.21.15.3:0/3706309784 conn(0x55a16b9be290 msgr2=0x55a16b9bcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 shutdown_connections 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/3706309784 wait complete. 2024-09-12T12:20:44.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.434+0000 7f0cd2d1d000 20 client.5010 trim_cache size 0 max 16384 2024-09-12T12:20:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 Processor -- start 2024-09-12T12:20:44.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- start start 2024-09-12T12:20:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 0x55a16ba1e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba06cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 0x55a16ba07230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81ff6e0 con 0x55a16ba07b20 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc814c4d0 con 0x55a16b9c3330 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc81ff090 con 0x55a16ba08440 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba06cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba06cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53844/0 (socket says 172.21.15.3:53844) 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/439561464 learned_addr learned my addr 172.21.15.3:0/439561464 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/439561464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 msgr2=0x55a16ba07230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/439561464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 0x55a16ba07230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/439561464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 msgr2=0x55a16ba1e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/439561464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 0x55a16ba1e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/439561464 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16b793fd0 con 0x55a16ba07b20 2024-09-12T12:20:44.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/439561464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba06cf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000c10 tx=0x7f0cb801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52467473d5f66086 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0caaffd640 1 -- 172.21.15.3:0/439561464 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8038070 con 0x55a16ba07b20 2024-09-12T12:20:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0caaffd640 1 -- 172.21.15.3:0/439561464 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb800cc00 con 0x55a16ba07b20 2024-09-12T12:20:44.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0caaffd640 1 -- 172.21.15.3:0/439561464 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8015040 con 0x55a16ba07b20 2024-09-12T12:20:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/439561464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 msgr2=0x55a16ba06cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/439561464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba06cf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000c10 tx=0x7f0cb801d040 comp rx=0 tx=0).stop 2024-09-12T12:20:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/439561464 shutdown_connections 2024-09-12T12:20:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/439561464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 0x55a16ba07230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/439561464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba06cf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/439561464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 0x55a16ba1e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/439561464 >> 172.21.15.3:0/439561464 conn(0x55a16b9be290 msgr2=0x55a16b9c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.454+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/439561464 shutdown_connections 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/439561464 wait complete. 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- start start 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 Processor -- start 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 0x55a16ba140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 0x55a16ba11390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a16b80bf80 con 0x55a16ba07b20 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a16b7c4ca0 con 0x55a16b9c3330 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55a16b8112a0 con 0x55a16ba08440 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 10 client.0 _ll_register_callbacks cb 0x7f0cb800a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba14630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53854/0 (socket says 172.21.15.3:53854) 2024-09-12T12:20:44.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/2258650271 learned_addr learned my addr 172.21.15.3:0/2258650271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 msgr2=0x55a16ba11390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cce1fe640 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 0x55a16ba11390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 0x55a16ba11390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 0x55a16ba140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 msgr2=0x55a16ba140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 0x55a16ba140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 -- 172.21.15.3:0/2258650271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16b88f750 con 0x55a16ba07b20 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd1fc640 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba14630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000c10 tx=0x7f0cb8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656841c929ce8f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/2258650271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8015040 con 0x55a16ba07b20 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/2258650271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0cb800cdb0 con 0x55a16ba07b20 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb800cc00 con 0x55a16ba07b20 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/2258650271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f0cb804b050 con 0x55a16ba07b20 2024-09-12T12:20:44.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 10 client.5016 fetch_fsmap learned FSMap version 14 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55a16b79aa30 con 0x55a16ba07b20 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/2258650271 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb800cc00 con 0x55a16ba07b20 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/2258650271 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f0cb804e020 con 0x55a16ba07b20 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 10 client.5016 fetch_fsmap finished waiting for FSMap version 14 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 10 client.5016 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 20 client.5016 populate_metadata read hostname 'smithi003' 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba11d50 0x55a16ba7a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 4 client.5016 mds_command: new command op to 4812 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 --> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] -- command(tid 0: {"prefix": "status"}) -- 0x55a16b81a3a0 con 0x55a16ba11d50 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba11d50 0x55a16ba7a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ccd9fd640 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba11d50 0x55a16ba7a9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf6cdc234a4d1c5 server_cookie=af4773b8bae19dd0 in_seq=0 out_seq=0 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 10 client.5016 ms_handle_connect on v2:172.21.15.3:6834/4232231567 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 1 -- 172.21.15.3:0/2258650271 <== mds.0 v2:172.21.15.3:6834/4232231567 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a16b81a3a0 con 0x55a16ba11d50 2024-09-12T12:20:44.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 10 client.5016 handle_command_reply: tid=0 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 client.5016 shutdown 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 msgr2=0x55a16ba14630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba14630 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000c10 tx=0x7f0cb8008040 comp rx=0 tx=0).stop 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 shutdown_connections 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:6834/4232231567,v1:172.21.15.3:6835/4232231567] conn(0x55a16ba11d50 0x55a16ba7a9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55a16ba08440 0x55a16ba11390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a16ba07b20 0x55a16ba14630 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 --2- 172.21.15.3:0/2258650271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a16b9c3330 0x55a16ba140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0ca8ff9640 0 client.5016 ms_handle_reset on v2:172.21.15.3:6834/4232231567 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 >> 172.21.15.3:0/2258650271 conn(0x55a16b9be290 msgr2=0x55a16ba05ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 shutdown_connections 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.458+0000 7f0cd2d1d000 1 -- 172.21.15.3:0/2258650271 wait complete. 2024-09-12T12:20:44.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0cd2d1d000 20 client.5016 trim_cache size 0 max 16384 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4812, 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 9.3339953920000003, 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 14, 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 32, 2024-09-12T12:20:44.471 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 32, 2024-09-12T12:20:44.472 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.237983404000001 2024-09-12T12:20:44.472 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:20:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0cac0678c0 msgr2=0x7f0cac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0cac0678c0 0x7f0cac069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4001570 tx=0x7f0cc4007cb0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc80b38b0 msgr2=0x7f0cc80ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc80b38b0 0x7f0cc80ae150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0cc001b880 tx=0x7f0cc001ba10 comp rx=0 tx=0).stop 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 shutdown_connections 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0cac0678c0 0x7f0cac069d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cc80b38b0 0x7f0cc80ae150 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cc80b1980 0x7f0cc8148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 --2- 172.21.15.3:0/1643862605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cc806bcc0 0x7f0cc80b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 >> 172.21.15.3:0/1643862605 conn(0x7f0cc80b2d90 msgr2=0x7f0cc80705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 shutdown_connections 2024-09-12T12:20:44.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.462+0000 7f0ccfc61640 1 -- 172.21.15.3:0/1643862605 wait complete. 2024-09-12T12:20:44.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:20:44.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a428fa640 1 Processor -- start 2024-09-12T12:20:44.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a428fa640 1 -- start start 2024-09-12T12:20:44.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a428fa640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a3c15e4b0 0x7f5a3c0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a428fa640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3c0ad5a0 con 0x7f5a3c0b42b0 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a428fa640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3c06dee0 con 0x7f5a3c0724b0 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a428fa640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5a3c06e810 con 0x7f5a3c15e4b0 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a3c15e4b0 0x7f5a3c0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a3c15e4b0 0x7f5a3c0ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53856/0 (socket says 172.21.15.3:53856) 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3bfff640 1 -- 172.21.15.3:0/3502687687 learned_addr learned my addr 172.21.15.3:0/3502687687 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5a3c0b42b0 0x7f5a3c071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53696/0 (socket says 172.21.15.3:53696) 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3bfff640 1 -- 172.21.15.3:0/3502687687 >> v1:172.21.15.142:6789/0 conn(0x7f5a3c0724b0 legacy=0x7f5a3c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3bfff640 1 -- 172.21.15.3:0/3502687687 >> v1:172.21.15.3:6789/0 conn(0x7f5a3c0b42b0 legacy=0x7f5a3c071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3bfff640 1 -- 172.21.15.3:0/3502687687 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c06e1c0 con 0x7f5a3c15e4b0 2024-09-12T12:20:44.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.594+0000 7f5a3bfff640 1 --2- 172.21.15.3:0/3502687687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c15e4b0 0x7f5a3c0ad060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009870 tx=0x7f5a2c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aca6a4772e704f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a3affd640 1 -- 172.21.15.3:0/3502687687 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0b2070 con 0x7f5a3c15e4b0 2024-09-12T12:20:44.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a3affd640 1 -- 172.21.15.3:0/3502687687 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a2c0a9070 con 0x7f5a3c15e4b0 2024-09-12T12:20:44.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a3affd640 1 -- 172.21.15.3:0/3502687687 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0ad880 con 0x7f5a3c15e4b0 2024-09-12T12:20:44.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- 172.21.15.3:0/3502687687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c15e4b0 msgr2=0x7f5a3c0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3502687687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c15e4b0 0x7f5a3c0ad060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009870 tx=0x7f5a2c01c490 comp rx=0 tx=0).stop 2024-09-12T12:20:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- 172.21.15.3:0/3502687687 shutdown_connections 2024-09-12T12:20:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3502687687 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c15e4b0 0x7f5a3c0ad060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- 172.21.15.3:0/3502687687 >> 172.21.15.3:0/3502687687 conn(0x7f5a3c0b1d80 msgr2=0x7f5a3c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- 172.21.15.3:0/3502687687 shutdown_connections 2024-09-12T12:20:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- 172.21.15.3:0/3502687687 wait complete. 2024-09-12T12:20:44.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 Processor -- start 2024-09-12T12:20:44.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- start start 2024-09-12T12:20:44.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c0724b0 0x7f5a3c150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a3c15e4b0 0x7f5a3c14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c14f120 0x7f5a3c1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c0b3390 con 0x7f5a3c14f120 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c06f7c0 con 0x7f5a3c0724b0 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a428fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3c16ca10 con 0x7f5a3c15e4b0 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c14f120 0x7f5a3c1543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c14f120 0x7f5a3c1543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53866/0 (socket says 172.21.15.3:53866) 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 -- 172.21.15.3:0/3759339272 learned_addr learned my addr 172.21.15.3:0/3759339272 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a3bfff640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c0724b0 0x7f5a3c150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 -- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a3c15e4b0 msgr2=0x7f5a3c14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a3b7fe640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a3c15e4b0 0x7f5a3c14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a3c15e4b0 0x7f5a3c14fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 -- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c0724b0 msgr2=0x7f5a3c150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c0724b0 0x7f5a3c150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 -- 172.21.15.3:0/3759339272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c0afc30 con 0x7f5a3c14f120 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a3bfff640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c0724b0 0x7f5a3c150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a3b7fe640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a3c15e4b0 0x7f5a3c14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a40e97640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c14f120 0x7f5a3c1543b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400c7b0 tx=0x7f5a3400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.598+0000 7f5a38ff9640 1 -- 172.21.15.3:0/3759339272 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34016020 con 0x7f5a3c14f120 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a38ff9640 1 -- 172.21.15.3:0/3759339272 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3400a9f0 con 0x7f5a3c14f120 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c151470 con 0x7f5a3c14f120 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a38ff9640 1 -- 172.21.15.3:0/3759339272 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400ade0 con 0x7f5a3c14f120 2024-09-12T12:20:44.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c0afed0 con 0x7f5a3c14f120 2024-09-12T12:20:44.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3c0ad060 con 0x7f5a3c14f120 2024-09-12T12:20:44.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a38ff9640 1 -- 172.21.15.3:0/3759339272 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a34019020 con 0x7f5a3c14f120 2024-09-12T12:20:44.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a38ff9640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a1c0678c0 0x7f5a1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:44.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a38ff9640 1 -- 172.21.15.3:0/3759339272 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5a34085ba0 con 0x7f5a3c14f120 2024-09-12T12:20:44.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a3bfff640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a1c0678c0 0x7f5a1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:44.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.602+0000 7f5a3bfff640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a1c0678c0 0x7f5a1c069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0af000 tx=0x7f5a2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:44.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.606+0000 7f5a38ff9640 1 -- 172.21.15.3:0/3759339272 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5a34020030 con 0x7f5a3c14f120 2024-09-12T12:20:44.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a3c151af0 con 0x7f5a3c14f120 2024-09-12T12:20:44.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a38ff9640 1 -- 172.21.15.3:0/3759339272 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f5a34056120 con 0x7f5a3c14f120 2024-09-12T12:20:44.807 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a1c0678c0 msgr2=0x7f5a1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a1c0678c0 0x7f5a1c069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0af000 tx=0x7f5a2c008040 comp rx=0 tx=0).stop 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c14f120 msgr2=0x7f5a3c1543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c14f120 0x7f5a3c1543b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400c7b0 tx=0x7f5a3400cc70 comp rx=0 tx=0).stop 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 shutdown_connections 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a1c0678c0 0x7f5a1c069d80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a3c14f120 0x7f5a3c1543b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a3c15e4b0 0x7f5a3c14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 --2- 172.21.15.3:0/3759339272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a3c0724b0 0x7f5a3c150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:44.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 >> 172.21.15.3:0/3759339272 conn(0x7f5a3c0b1d80 msgr2=0x7f5a3c15f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:44.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 shutdown_connections 2024-09-12T12:20:44.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:44.802+0000 7f5a428fa640 1 -- 172.21.15.3:0/3759339272 wait complete. 2024-09-12T12:20:44.824 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:20:44.824 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:20:44.824 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:20:44.824 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:20:44.824 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:44.825 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:20:44.825 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:20:44.825 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:20:44.825 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:20:44.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:44.825 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:20:44.831 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:44.831 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:44.831 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:44.831 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:20:44.832 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42442sec preferred_lft 42442sec 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.833 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:20:44.834 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:44.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:44.835 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:20:44.884 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:20:44.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:44.886 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:20:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:20:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:20:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:20:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:20:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:20:44.936 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:20:44.936 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:44.936 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:20:44.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:20:44.936 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:20:44.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:20:44.937 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:20:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:45.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:45.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:45.058 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:20:45.064 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:20:45.064 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:20:45.064 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:20:45.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:45.065 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:20:45.118 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:20:45.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:45.119 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:45.166 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:20:45.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:45.167 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:45.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:45.272 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:45.319 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:45.319 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:45.319 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:45.319 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:45.319 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:45.320 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:45.320 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:45.320 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:45.320 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:45.321 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:45.321 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:45.321 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:45.321 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:45.322 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:45.323 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:45.323 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:45.323 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:45.323 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:45.323 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:45.323 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:45.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:45.324 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:45.371 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:20:45.372 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:20:45.415 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:45.431 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:45.478 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:45.478 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:45.478 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:45.479 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:45.480 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:45.481 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:45.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:45.482 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:45.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:20:45.494+0000 7f43c7913580 -1 init, newargv = 0x55c6ea2162f0 newargc=15 2024-09-12T12:20:45.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[36777]: starting ceph client 2024-09-12T12:20:45.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[36777]: starting fuse 2024-09-12T12:20:46.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:20:46.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:20:46.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:20:46.623 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:46.672 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:20:46.673 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:20:46.674 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:20:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:20:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:20:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:20:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:20:46.675 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:46.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:46.676 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:20:46.722 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:20:46.722 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:20:46.722 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:20:46.723 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:20:46.724 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:20:46.850 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.36777.asok 2024-09-12T12:20:46.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:46.862 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36777.asok 2024-09-12T12:20:46.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:46.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36777.asok status 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "36777", 2024-09-12T12:20:47.019 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5034, 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:20:47.020 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5034 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 220260417 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:20:47.021 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 220260417 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5034 192.168.0.1:0/220260417", 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/220260417", 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 14, 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 32, 2024-09-12T12:20:47.022 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:20:47.023 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:20:47.023 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:20:47.023 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:20:47.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:47.033 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:20:47.033 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:47.039 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:20:47.039 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:47.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:47.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:47.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:20:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.266+0000 7f2ccd486640 1 Processor -- start 2024-09-12T12:20:47.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.266+0000 7f2ccd486640 1 -- start start 2024-09-12T12:20:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2ccd486640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2cc8159bd0 0x7f2cc806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2ccd486640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:47.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2ccd486640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2cc806f4a0 0x7f2cc80ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2ccd486640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc806f010 con 0x7f2cc806f4a0 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2ccd486640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc806e400 con 0x7f2cc8159bd0 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2ccd486640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc806e6e0 con 0x7f2cc8159fb0 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2cc806f4a0 0x7f2cc80ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2cc806f4a0 0x7f2cc80ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53882/0 (socket says 172.21.15.3:53882) 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 -- 172.21.15.3:0/2637950366 learned_addr learned my addr 172.21.15.3:0/2637950366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc67fc640 1 --2- 172.21.15.3:0/2637950366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc6ffd640 1 --2- 172.21.15.3:0/2637950366 >> v2:172.21.15.142:3300/0 conn(0x7f2cc8159bd0 0x7f2cc806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 -- 172.21.15.3:0/2637950366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 msgr2=0x7f2cc815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 --2- 172.21.15.3:0/2637950366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 -- 172.21.15.3:0/2637950366 >> v2:172.21.15.142:3300/0 conn(0x7f2cc8159bd0 msgr2=0x7f2cc806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 --2- 172.21.15.3:0/2637950366 >> v2:172.21.15.142:3300/0 conn(0x7f2cc8159bd0 0x7f2cc806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 -- 172.21.15.3:0/2637950366 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc806e9c0 con 0x7f2cc806f4a0 2024-09-12T12:20:47.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc77fe640 1 --2- 172.21.15.3:0/2637950366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ad060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8071320 tx=0x7f2cc001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695fd363fee0fb3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:47.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc5ffb640 1 -- 172.21.15.3:0/2637950366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00a5780 con 0x7f2cc806f4a0 2024-09-12T12:20:47.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc5ffb640 1 -- 172.21.15.3:0/2637950366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc00a5920 con 0x7f2cc806f4a0 2024-09-12T12:20:47.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.270+0000 7f2cc5ffb640 1 -- 172.21.15.3:0/2637950366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00a5cf0 con 0x7f2cc806f4a0 2024-09-12T12:20:47.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- 172.21.15.3:0/2637950366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 msgr2=0x7f2cc80ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 --2- 172.21.15.3:0/2637950366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ad060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8071320 tx=0x7f2cc001c9f0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- 172.21.15.3:0/2637950366 shutdown_connections 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 --2- 172.21.15.3:0/2637950366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ad060 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 --2- 172.21.15.3:0/2637950366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 --2- 172.21.15.3:0/2637950366 >> v2:172.21.15.142:3300/0 conn(0x7f2cc8159bd0 0x7f2cc806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- 172.21.15.3:0/2637950366 >> 172.21.15.3:0/2637950366 conn(0x7f2cc80b1d80 msgr2=0x7f2cc80b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- 172.21.15.3:0/2637950366 shutdown_connections 2024-09-12T12:20:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- 172.21.15.3:0/2637950366 wait complete. 2024-09-12T12:20:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 Processor -- start 2024-09-12T12:20:47.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- start start 2024-09-12T12:20:47.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc8159bd0 0x7f2cc809dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc8145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8160210 con 0x7f2cc806f4a0 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc8160090 con 0x7f2cc8159bd0 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc8160390 con 0x7f2cc8159fb0 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53886/0 (socket says 172.21.15.3:53886) 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 -- 172.21.15.3:0/3559371079 learned_addr learned my addr 172.21.15.3:0/3559371079 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc67fc640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc8159bd0 0x7f2cc809dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc77fe640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc8145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 -- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 msgr2=0x7f2cc8145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc8145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 -- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc8159bd0 msgr2=0x7f2cc809dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc8159bd0 0x7f2cc809dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 -- 172.21.15.3:0/3559371079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc80cd520 con 0x7f2cc806f4a0 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cc6ffd640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ce8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d800 tx=0x7f2cb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cbf7fe640 1 -- 172.21.15.3:0/3559371079 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb800b6c0 con 0x7f2cc806f4a0 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cbf7fe640 1 -- 172.21.15.3:0/3559371079 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb8019070 con 0x7f2cc806f4a0 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc80cd800 con 0x7f2cc806f4a0 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2cbf7fe640 1 -- 172.21.15.3:0/3559371079 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb800ba60 con 0x7f2cc806f4a0 2024-09-12T12:20:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.274+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc80cddc0 con 0x7f2cc806f4a0 2024-09-12T12:20:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.278+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc80ad060 con 0x7f2cc806f4a0 2024-09-12T12:20:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.278+0000 7f2cbf7fe640 1 -- 172.21.15.3:0/3559371079 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2cb800bc00 con 0x7f2cc806f4a0 2024-09-12T12:20:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.278+0000 7f2cbf7fe640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ca80678c0 0x7f2ca8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.278+0000 7f2cbf7fe640 1 -- 172.21.15.3:0/3559371079 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2cb8086160 con 0x7f2cc806f4a0 2024-09-12T12:20:47.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.278+0000 7f2cc67fc640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ca80678c0 0x7f2ca8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:47.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.278+0000 7f2cc67fc640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ca80678c0 0x7f2ca8069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80b1590 tx=0x7f2cb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:47.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.282+0000 7f2cbf7fe640 1 -- 172.21.15.3:0/3559371079 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2cb8056c90 con 0x7f2cc806f4a0 2024-09-12T12:20:47.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2cc809e140 con 0x7f2cc806f4a0 2024-09-12T12:20:47.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2cbf7fe640 1 -- 172.21.15.3:0/3559371079 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f2cb8056590 con 0x7f2cc806f4a0 2024-09-12T12:20:47.487 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:20:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ca80678c0 msgr2=0x7f2ca8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:47.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ca80678c0 0x7f2ca8069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80b1590 tx=0x7f2cb0008040 comp rx=0 tx=0).stop 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 msgr2=0x7f2cc80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ce8a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800d800 tx=0x7f2cb800dcc0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 shutdown_connections 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ca80678c0 0x7f2ca8069d80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2cc8159fb0 0x7f2cc8145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cc8159bd0 0x7f2cc809dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 --2- 172.21.15.3:0/3559371079 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cc806f4a0 0x7f2cc80ce8a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 >> 172.21.15.3:0/3559371079 conn(0x7f2cc80b1d80 msgr2=0x7f2cc815d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 shutdown_connections 2024-09-12T12:20:47.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:47.482+0000 7f2ccd486640 1 -- 172.21.15.3:0/3559371079 wait complete. 2024-09-12T12:20:47.506 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:20:47.506 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:20:47.506 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:20:47.506 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:20:47.506 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:47.507 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:20:47.507 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:20:47.507 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:20:47.507 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:20:47.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:47.508 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:20:47.515 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42437sec preferred_lft 42437sec 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:20:47.516 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:47.517 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:20:47.518 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:20:47.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:47.518 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:20:47.570 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:20:47.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:47.572 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:20:47.626 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:20:47.626 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:20:47.626 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:20:47.626 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:20:47.626 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:20:47.626 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:20:47.627 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:20:47.627 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:47.627 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:47.627 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:20:47.627 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:20:47.627 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:20:47.627 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:20:47.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:20:47.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:20:47.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:20:47.628 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:20:47.715 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:20:47.744 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:47.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:47.753 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:20:47.758 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:20:47.758 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:20:47.758 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:20:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:47.758 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:20:47.813 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:20:47.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:47.813 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:47.861 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:20:47.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:20:47.861 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:47.908 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:47.967 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:48.015 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:48.015 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:48.016 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:48.017 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:48.018 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:48.018 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:48.018 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:48.018 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:48.018 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:48.018 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:48.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:48.019 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:48.068 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:20:48.068 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:20:48.113 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:48.127 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:48.174 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:48.175 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:48.176 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:48.177 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:48.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:48.177 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:48.194 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:20:48.190+0000 7f9ee984d580 -1 init, newargv = 0x5591a91e9370 newargc=15 2024-09-12T12:20:48.194 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35511]: starting ceph client 2024-09-12T12:20:48.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35511]: starting fuse 2024-09-12T12:20:49.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:20:49.315 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:20:49.362 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:20:49.362 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:20:49.362 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:20:49.362 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:20:49.362 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:20:49.362 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:20:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:20:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:49.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:49.365 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:20:49.414 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:20:49.414 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:20:49.414 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:20:49.414 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:49.415 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:20:49.416 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:20:49.537 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.35511.asok 2024-09-12T12:20:49.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:20:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:20:49.549 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35511.asok 2024-09-12T12:20:49.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:49.549 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35511.asok status 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "35511", 2024-09-12T12:20:49.704 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 4565, 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 4565 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:49.705 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2170143064 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2170143064 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.4565 192.168.0.1:0/2170143064", 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/2170143064", 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 14, 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 32, 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:20:49.706 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:20:49.707 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:20:49.707 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:20:49.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:49.718 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:20:49.718 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:49.724 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:20:49.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:49.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:49.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:49.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-09-12T12:20:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d107c9640 1 Processor -- start 2024-09-12T12:20:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d107c9640 1 -- start start 2024-09-12T12:20:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d107c9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d080b5f70 0x7f1d0806e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d107c9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d080b6b10 0x7f1d080d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d107c9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d080d3480 con 0x7f1d080b6350 2024-09-12T12:20:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d107c9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1d080d1690 con 0x7f1d080b5f70 2024-09-12T12:20:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d107c9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1d080d1970 con 0x7f1d080b6b10 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0e565640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d080b5f70 0x7f1d0806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0e565640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d080b5f70 0x7f1d0806e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59404/0 (socket says 172.21.15.3:59404) 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0e565640 1 -- 172.21.15.3:0/844527579 learned_addr learned my addr 172.21.15.3:0/844527579 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0dd64640 1 --2- 172.21.15.3:0/844527579 >> v2:172.21.15.142:3300/0 conn(0x7f1d080b6b10 0x7f1d080d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0e565640 1 -- 172.21.15.3:0/844527579 >> v2:172.21.15.142:3300/0 conn(0x7f1d080b6b10 msgr2=0x7f1d080d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0e565640 1 --2- 172.21.15.3:0/844527579 >> v2:172.21.15.142:3300/0 conn(0x7f1d080b6b10 0x7f1d080d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0e565640 1 -- 172.21.15.3:0/844527579 >> v1:172.21.15.142:6789/0 conn(0x7f1d080b6350 legacy=0x7f1d080cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:49.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.918+0000 7f1d0e565640 1 -- 172.21.15.3:0/844527579 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d080d23f0 con 0x7f1d080b5f70 2024-09-12T12:20:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d0e565640 1 --2- 172.21.15.3:0/844527579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b5f70 0x7f1d0806e780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d0808dd60 tx=0x7f1d0401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7b5331373a7ace server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d0d563640 1 -- 172.21.15.3:0/844527579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d040b0070 con 0x7f1d080b5f70 2024-09-12T12:20:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d0d563640 1 -- 172.21.15.3:0/844527579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d04004230 con 0x7f1d080b5f70 2024-09-12T12:20:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d0d563640 1 -- 172.21.15.3:0/844527579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04003620 con 0x7f1d080b5f70 2024-09-12T12:20:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- 172.21.15.3:0/844527579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b5f70 msgr2=0x7f1d0806e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 --2- 172.21.15.3:0/844527579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b5f70 0x7f1d0806e780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d0808dd60 tx=0x7f1d0401c4c0 comp rx=0 tx=0).stop 2024-09-12T12:20:49.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- 172.21.15.3:0/844527579 shutdown_connections 2024-09-12T12:20:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 --2- 172.21.15.3:0/844527579 >> v2:172.21.15.142:3300/0 conn(0x7f1d080b6b10 0x7f1d080d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 --2- 172.21.15.3:0/844527579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b5f70 0x7f1d0806e780 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- 172.21.15.3:0/844527579 >> 172.21.15.3:0/844527579 conn(0x7f1d0809d880 msgr2=0x7f1d0808c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- 172.21.15.3:0/844527579 shutdown_connections 2024-09-12T12:20:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- 172.21.15.3:0/844527579 wait complete. 2024-09-12T12:20:49.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 Processor -- start 2024-09-12T12:20:49.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- start start 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d080b5f70 0x7f1d0815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b6350 0x7f1d08157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d080b6b10 0x7f1d08152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0816b5a0 con 0x7f1d080b6350 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0815ea90 con 0x7f1d080b5f70 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d107c9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d0816b720 con 0x7f1d080b6b10 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d0dd64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b6350 0x7f1d08157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.922+0000 7f1d0dd64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b6350 0x7f1d08157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59412/0 (socket says 172.21.15.3:59412) 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0e565640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d080b5f70 0x7f1d0815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0dd64640 1 -- 172.21.15.3:0/4126767215 learned_addr learned my addr 172.21.15.3:0/4126767215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:49.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0ed66640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d080b6b10 0x7f1d08152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0dd64640 1 -- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d080b6b10 msgr2=0x7f1d08152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0dd64640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d080b6b10 0x7f1d08152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0dd64640 1 -- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d080b5f70 msgr2=0x7f1d0815c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0dd64640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d080b5f70 0x7f1d0815c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0dd64640 1 -- 172.21.15.3:0/4126767215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d081541e0 con 0x7f1d080b6350 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0dd64640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b6350 0x7f1d08157d80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800d830 tx=0x7f1cf800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1cfeffd640 1 -- 172.21.15.3:0/4126767215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800b6c0 con 0x7f1d080b6350 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1cfeffd640 1 -- 172.21.15.3:0/4126767215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf8019070 con 0x7f1d080b6350 2024-09-12T12:20:49.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d081582c0 con 0x7f1d080b6350 2024-09-12T12:20:49.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1cfeffd640 1 -- 172.21.15.3:0/4126767215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800bae0 con 0x7f1d080b6350 2024-09-12T12:20:49.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d080b8630 con 0x7f1d080b6350 2024-09-12T12:20:49.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d080d1c50 con 0x7f1d080b6350 2024-09-12T12:20:49.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1cfeffd640 1 -- 172.21.15.3:0/4126767215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1cf801e460 con 0x7f1d080b6350 2024-09-12T12:20:49.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1cfeffd640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cec0678c0 0x7f1cec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:49.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1cfeffd640 1 -- 172.21.15.3:0/4126767215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1cf80862e0 con 0x7f1d080b6350 2024-09-12T12:20:49.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.926+0000 7f1d0e565640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cec0678c0 0x7f1cec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:49.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.930+0000 7f1d0e565640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cec0678c0 0x7f1cec069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d040ad000 tx=0x7f1d040a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:49.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:49.930+0000 7f1cfeffd640 1 -- 172.21.15.3:0/4126767215 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1cf8056e20 con 0x7f1d080b6350 2024-09-12T12:20:50.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.098+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7f1d080b1280 con 0x7f1d080b6350 2024-09-12T12:20:50.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.102+0000 7f1cfeffd640 1 -- 172.21.15.3:0/4126767215 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7f1cf8056720 con 0x7f1d080b6350 2024-09-12T12:20:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cec0678c0 msgr2=0x7f1cec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cec0678c0 0x7f1cec069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1d040ad000 tx=0x7f1d040a6000 comp rx=0 tx=0).stop 2024-09-12T12:20:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b6350 msgr2=0x7f1d08157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b6350 0x7f1d08157d80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800d830 tx=0x7f1cf800dcf0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 shutdown_connections 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cec0678c0 0x7f1cec069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d080b6b10 0x7f1d08152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080b6350 0x7f1d08157d80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 --2- 172.21.15.3:0/4126767215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d080b5f70 0x7f1d0815c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 >> 172.21.15.3:0/4126767215 conn(0x7f1d0809d880 msgr2=0x7f1d080b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 shutdown_connections 2024-09-12T12:20:50.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.106+0000 7f1d107c9640 1 -- 172.21.15.3:0/4126767215 wait complete. 2024-09-12T12:20:50.128 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:50.128 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:50.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7b54f5640 1 Processor -- start 2024-09-12T12:20:50.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7b54f5640 1 -- start start 2024-09-12T12:20:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7b54f5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b00aedf0 0x7ff7b00b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7b54f5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b00afb00 con 0x7ff7b00d0380 2024-09-12T12:20:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7b54f5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7b00b31b0 con 0x7ff7b01568f0 2024-09-12T12:20:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7b54f5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b00b4ce0 con 0x7ff7b00aedf0 2024-09-12T12:20:50.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7af7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff7b00d0380 0x7ff7b015c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45210/0 (socket says 172.21.15.3:45210) 2024-09-12T12:20:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7af7fe640 1 -- 172.21.15.3:0/3040945985 learned_addr learned my addr 172.21.15.3:0/3040945985 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7ae7fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b00aedf0 0x7ff7b00b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7ae7fc640 1 -- 172.21.15.3:0/3040945985 >> v1:172.21.15.142:6789/0 conn(0x7ff7b01568f0 legacy=0x7ff7b00b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7ae7fc640 1 -- 172.21.15.3:0/3040945985 >> v1:172.21.15.3:6789/0 conn(0x7ff7b00d0380 legacy=0x7ff7b015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7ae7fc640 1 -- 172.21.15.3:0/3040945985 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b00af540 con 0x7ff7b00aedf0 2024-09-12T12:20:50.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.302+0000 7ff7adffb640 1 -- 172.21.15.3:0/3040945985 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909511258 0 0) 0x7ff7b00afb00 con 0x7ff7b00d0380 2024-09-12T12:20:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7ae7fc640 1 --2- 172.21.15.3:0/3040945985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00aedf0 0x7ff7b00b47a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4019447df55b4d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:50.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7adffb640 1 -- 172.21.15.3:0/3040945985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b2070 con 0x7ff7b00aedf0 2024-09-12T12:20:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7adffb640 1 -- 172.21.15.3:0/3040945985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980a57a0 con 0x7ff7b00aedf0 2024-09-12T12:20:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7adffb640 1 -- 172.21.15.3:0/3040945985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980a5b70 con 0x7ff7b00aedf0 2024-09-12T12:20:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- 172.21.15.3:0/3040945985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00aedf0 msgr2=0x7ff7b00b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/3040945985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00aedf0 0x7ff7b00b47a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff798009d90 tx=0x7ff79801c8e0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- 172.21.15.3:0/3040945985 shutdown_connections 2024-09-12T12:20:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/3040945985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00aedf0 0x7ff7b00b47a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- 172.21.15.3:0/3040945985 >> 172.21.15.3:0/3040945985 conn(0x7ff7b006be90 msgr2=0x7ff7b006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- 172.21.15.3:0/3040945985 shutdown_connections 2024-09-12T12:20:50.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- 172.21.15.3:0/3040945985 wait complete. 2024-09-12T12:20:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 Processor -- start 2024-09-12T12:20:50.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- start start 2024-09-12T12:20:50.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b00aedf0 0x7ff7b0155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00d0380 0x7ff7b0071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b01568f0 0x7ff7b0073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b00d0ba0 con 0x7ff7b00d0380 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b015d3b0 con 0x7ff7b00aedf0 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.306+0000 7ff7b54f5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b00730b0 con 0x7ff7b01568f0 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00d0380 0x7ff7b0071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00d0380 0x7ff7b0071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59430/0 (socket says 172.21.15.3:59430) 2024-09-12T12:20:50.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 -- 172.21.15.3:0/1468733914 learned_addr learned my addr 172.21.15.3:0/1468733914 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7aeffd640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b00aedf0 0x7ff7b0155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7af7fe640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b01568f0 0x7ff7b0073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 -- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b01568f0 msgr2=0x7ff7b0073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b01568f0 0x7ff7b0073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 -- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b00aedf0 msgr2=0x7ff7b0155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b00aedf0 0x7ff7b0155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 -- 172.21.15.3:0/1468733914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b00b0590 con 0x7ff7b00d0380 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7af7fe640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b01568f0 0x7ff7b0073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7ae7fc640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00d0380 0x7ff7b0071da0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b0040 tx=0x7ff798019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7a77fe640 1 -- 172.21.15.3:0/1468733914 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980b2050 con 0x7ff7b00d0380 2024-09-12T12:20:50.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7a77fe640 1 -- 172.21.15.3:0/1468733914 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7980be070 con 0x7ff7b00d0380 2024-09-12T12:20:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b0073370 con 0x7ff7b00d0380 2024-09-12T12:20:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7a77fe640 1 -- 172.21.15.3:0/1468733914 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7980ada90 con 0x7ff7b00d0380 2024-09-12T12:20:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b0167aa0 con 0x7ff7b00d0380 2024-09-12T12:20:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b00b47a0 con 0x7ff7b00d0380 2024-09-12T12:20:50.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7a77fe640 1 -- 172.21.15.3:0/1468733914 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff7980adc30 con 0x7ff7b00d0380 2024-09-12T12:20:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7a77fe640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7900678c0 0x7ff790069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7a77fe640 1 -- 172.21.15.3:0/1468733914 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff798120dc0 con 0x7ff7b00d0380 2024-09-12T12:20:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7aeffd640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7900678c0 0x7ff790069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.310+0000 7ff7aeffd640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7900678c0 0x7ff790069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b0af0 tx=0x7ff7a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:50.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.314+0000 7ff7a77fe640 1 -- 172.21.15.3:0/1468733914 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff798016560 con 0x7ff7b00d0380 2024-09-12T12:20:50.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.490+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff7b00b3e30 con 0x7ff7b00d0380 2024-09-12T12:20:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.490+0000 7ff7a77fe640 1 -- 172.21.15.3:0/1468733914 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7ff7980f1990 con 0x7ff7b00d0380 2024-09-12T12:20:50.497 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:50.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7900678c0 msgr2=0x7ff790069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7900678c0 0x7ff790069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b0af0 tx=0x7ff7a0009040 comp rx=0 tx=0).stop 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00d0380 msgr2=0x7ff7b0071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00d0380 0x7ff7b0071da0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b0040 tx=0x7ff798019680 comp rx=0 tx=0).stop 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 shutdown_connections 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7900678c0 0x7ff790069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b01568f0 0x7ff7b0073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b00d0380 0x7ff7b0071da0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 --2- 172.21.15.3:0/1468733914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b00aedf0 0x7ff7b0155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 >> 172.21.15.3:0/1468733914 conn(0x7ff7b006be90 msgr2=0x7ff7b006c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 shutdown_connections 2024-09-12T12:20:50.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.494+0000 7ff7b54f5640 1 -- 172.21.15.3:0/1468733914 wait complete. 2024-09-12T12:20:50.517 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:20:50.517 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:50.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 Processor -- start 2024-09-12T12:20:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 -- start start 2024-09-12T12:20:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcb3c159ea0 0x7fcb3c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb3c06d210 0x7fcb3c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3c06d680 con 0x7fcb3c06d210 2024-09-12T12:20:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcb3c15a280 con 0x7fcb3c159ea0 2024-09-12T12:20:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb42d0d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3c0b5280 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb3c06d210 0x7fcb3c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcb3c06d210 0x7fcb3c0b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59440/0 (socket says 172.21.15.3:59440) 2024-09-12T12:20:50.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 -- 172.21.15.3:0/4000999889 learned_addr learned my addr 172.21.15.3:0/4000999889 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:50.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb40aa9640 1 --2- 172.21.15.3:0/4000999889 >> v2:172.21.15.142:3300/0 conn(0x7fcb3c159ea0 0x7fcb3c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb412aa640 1 --2- 172.21.15.3:0/4000999889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 -- 172.21.15.3:0/4000999889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c0b4ad0 msgr2=0x7fcb3c15fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:50.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 --2- 172.21.15.3:0/4000999889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 -- 172.21.15.3:0/4000999889 >> v2:172.21.15.142:3300/0 conn(0x7fcb3c159ea0 msgr2=0x7fcb3c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:50.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 --2- 172.21.15.3:0/4000999889 >> v2:172.21.15.142:3300/0 conn(0x7fcb3c159ea0 0x7fcb3c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb3bfff640 1 -- 172.21.15.3:0/4000999889 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c0b5560 con 0x7fcb3c06d210 2024-09-12T12:20:50.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.682+0000 7fcb412aa640 1 --2- 172.21.15.3:0/4000999889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb3bfff640 1 --2- 172.21.15.3:0/4000999889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c06d210 0x7fcb3c0b4590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c067b20 tx=0x7fcb2401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01e4505461433e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb3b7fe640 1 -- 172.21.15.3:0/4000999889 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240a64b0 con 0x7fcb3c06d210 2024-09-12T12:20:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb3b7fe640 1 -- 172.21.15.3:0/4000999889 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb240a6650 con 0x7fcb3c06d210 2024-09-12T12:20:50.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb3b7fe640 1 -- 172.21.15.3:0/4000999889 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240a6a20 con 0x7fcb3c06d210 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 -- 172.21.15.3:0/4000999889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c06d210 msgr2=0x7fcb3c0b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/4000999889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c06d210 0x7fcb3c0b4590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c067b20 tx=0x7fcb2401c730 comp rx=0 tx=0).stop 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 -- 172.21.15.3:0/4000999889 shutdown_connections 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/4000999889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/4000999889 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c06d210 0x7fcb3c0b4590 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/4000999889 >> v2:172.21.15.142:3300/0 conn(0x7fcb3c159ea0 0x7fcb3c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 -- 172.21.15.3:0/4000999889 >> 172.21.15.3:0/4000999889 conn(0x7fcb3c072720 msgr2=0x7fcb3c072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:50.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 -- 172.21.15.3:0/4000999889 shutdown_connections 2024-09-12T12:20:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 -- 172.21.15.3:0/4000999889 wait complete. 2024-09-12T12:20:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 Processor -- start 2024-09-12T12:20:50.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.686+0000 7fcb42d0d640 1 -- start start 2024-09-12T12:20:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c06d210 0x7fcb3c144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3c159ea0 0x7fcb3c148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c0cf460 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c0cf2e0 con 0x7fcb3c159ea0 2024-09-12T12:20:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcb3c0cf5e0 con 0x7fcb3c06d210 2024-09-12T12:20:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59448/0 (socket says 172.21.15.3:59448) 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 -- 172.21.15.3:0/2068716552 learned_addr learned my addr 172.21.15.3:0/2068716552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb412aa640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3c159ea0 0x7fcb3c148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb40aa9640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c06d210 0x7fcb3c144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 -- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c06d210 msgr2=0x7fcb3c144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c06d210 0x7fcb3c144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 -- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3c159ea0 msgr2=0x7fcb3c148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3c159ea0 0x7fcb3c148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 -- 172.21.15.3:0/2068716552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c15b730 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb3bfff640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15b1f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401cfd0 tx=0x7fcb240194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb397fa640 1 -- 172.21.15.3:0/2068716552 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240bf070 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb397fa640 1 -- 172.21.15.3:0/2068716552 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb24009190 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c159130 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb397fa640 1 -- 172.21.15.3:0/2068716552 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb240a6a20 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c14bcc0 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c15fee0 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb397fa640 1 -- 172.21.15.3:0/2068716552 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb240c2070 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb397fa640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcb1c0678c0 0x7fcb1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb397fa640 1 -- 172.21.15.3:0/2068716552 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fcb2411f8a0 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.690+0000 7fcb40aa9640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcb1c0678c0 0x7fcb1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.694+0000 7fcb40aa9640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcb1c0678c0 0x7fcb1c069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009a50 tx=0x7fcb34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.694+0000 7fcb397fa640 1 -- 172.21.15.3:0/2068716552 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcb240165e0 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.878+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcb3c14b7b0 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.878+0000 7fcb397fa640 1 -- 172.21.15.3:0/2068716552 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7fcb240f0350 con 0x7fcb3c0b4ad0 2024-09-12T12:20:50.884 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:50.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcb1c0678c0 msgr2=0x7fcb1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcb1c0678c0 0x7fcb1c069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009a50 tx=0x7fcb34008040 comp rx=0 tx=0).stop 2024-09-12T12:20:50.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c0b4ad0 msgr2=0x7fcb3c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15b1f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401cfd0 tx=0x7fcb240194d0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 shutdown_connections 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcb1c0678c0 0x7fcb1c069d80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb3c159ea0 0x7fcb3c148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb3c0b4ad0 0x7fcb3c15b1f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 --2- 172.21.15.3:0/2068716552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcb3c06d210 0x7fcb3c144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 >> 172.21.15.3:0/2068716552 conn(0x7fcb3c072720 msgr2=0x7fcb3c0b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 shutdown_connections 2024-09-12T12:20:50.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:50.882+0000 7fcb42d0d640 1 -- 172.21.15.3:0/2068716552 wait complete. 2024-09-12T12:20:50.902 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:20:50.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:50.902 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:20:50.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[36777]: fuse finished with error 0 and tester_r 0 2024-09-12T12:20:50.915 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:20:57.016 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:20:57.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:57.017 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:20:57.025 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:57.025 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:57.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.198+0000 7f49c1106640 1 Processor -- start 2024-09-12T12:20:57.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.198+0000 7f49c1106640 1 -- start start 2024-09-12T12:20:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.198+0000 7f49c1106640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f49bc0d0a90 0x7f49bc072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.198+0000 7f49c1106640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc073320 0x7f49bc15b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.198+0000 7f49c1106640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49bc06f010 con 0x7f49bc0af550 2024-09-12T12:20:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.198+0000 7f49c1106640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f49bc073ad0 con 0x7f49bc0d0a90 2024-09-12T12:20:57.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.198+0000 7f49c1106640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49bc06f2f0 con 0x7f49bc073320 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f49bc0d0a90 0x7f49bc072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49bb59e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc073320 0x7f49bc15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f49bc0d0a90 0x7f49bc072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:36196/0 (socket says 172.21.15.3:36196) 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 -- 172.21.15.3:0/1786472026 learned_addr learned my addr 172.21.15.3:0/1786472026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 -- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc073320 msgr2=0x7f49bc15b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 --2- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc073320 0x7f49bc15b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 -- 172.21.15.3:0/1786472026 >> v1:172.21.15.142:6789/0 conn(0x7f49bc0af550 legacy=0x7f49bc0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 -- 172.21.15.3:0/1786472026 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc06f5d0 con 0x7f49bc0d0a90 2024-09-12T12:20:57.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49bb59e640 1 --2- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc073320 0x7f49bc15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:57.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49ba59c640 1 --2- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0d0a90 0x7f49bc072de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009e50 tx=0x7f49a401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637988ee72354eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49b9d9b640 1 -- 172.21.15.3:0/1786472026 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40a5880 con 0x7f49bc0d0a90 2024-09-12T12:20:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49b9d9b640 1 -- 172.21.15.3:0/1786472026 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a40a5a20 con 0x7f49bc0d0a90 2024-09-12T12:20:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49b9d9b640 1 -- 172.21.15.3:0/1786472026 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a40a5df0 con 0x7f49bc0d0a90 2024-09-12T12:20:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 -- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0d0a90 msgr2=0x7f49bc072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 --2- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0d0a90 0x7f49bc072de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009e50 tx=0x7f49a401c9a0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 -- 172.21.15.3:0/1786472026 shutdown_connections 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 --2- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc073320 0x7f49bc15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 --2- 172.21.15.3:0/1786472026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0d0a90 0x7f49bc072de0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 -- 172.21.15.3:0/1786472026 >> 172.21.15.3:0/1786472026 conn(0x7f49bc06be90 msgr2=0x7f49bc06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 -- 172.21.15.3:0/1786472026 shutdown_connections 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.202+0000 7f49c1106640 1 -- 172.21.15.3:0/1786472026 wait complete. 2024-09-12T12:20:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 Processor -- start 2024-09-12T12:20:57.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 -- start start 2024-09-12T12:20:57.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc073320 0x7f49bc09e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0af550 0x7f49bc152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc0d0a90 0x7f49bc0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc15eda0 con 0x7f49bc073320 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc15ec20 con 0x7f49bc0af550 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49bc167d70 con 0x7f49bc0d0a90 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc073320 0x7f49bc09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc073320 0x7f49bc09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59456/0 (socket says 172.21.15.3:59456) 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 -- 172.21.15.3:0/1141017789 learned_addr learned my addr 172.21.15.3:0/1141017789 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49ba59c640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0af550 0x7f49bc152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bb59e640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc0d0a90 0x7f49bc0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 -- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc0d0a90 msgr2=0x7f49bc0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc0d0a90 0x7f49bc0ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 -- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0af550 msgr2=0x7f49bc152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0af550 0x7f49bc152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 -- 172.21.15.3:0/1141017789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc0cdeb0 con 0x7f49bc073320 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49bad9d640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc073320 0x7f49bc09e080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00c420 tx=0x7f49ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49b37fe640 1 -- 172.21.15.3:0/1141017789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0052d0 con 0x7f49bc073320 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49b37fe640 1 -- 172.21.15.3:0/1141017789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac005470 con 0x7f49bc073320 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc0ce190 con 0x7f49bc073320 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49b37fe640 1 -- 172.21.15.3:0/1141017789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac007260 con 0x7f49bc073320 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.206+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc0cc820 con 0x7f49bc073320 2024-09-12T12:20:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.210+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49bc15b630 con 0x7f49bc073320 2024-09-12T12:20:57.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.210+0000 7f49b37fe640 1 -- 172.21.15.3:0/1141017789 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f49ac007400 con 0x7f49bc073320 2024-09-12T12:20:57.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.210+0000 7f49b37fe640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f499c0678c0 0x7f499c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.210+0000 7f49b37fe640 1 -- 172.21.15.3:0/1141017789 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f49ac085110 con 0x7f49bc073320 2024-09-12T12:20:57.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.210+0000 7f49ba59c640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f499c0678c0 0x7f499c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.210+0000 7f49ba59c640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f499c0678c0 0x7f499c069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ae000 tx=0x7f49a40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:57.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.214+0000 7f49b37fe640 1 -- 172.21.15.3:0/1141017789 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f49ac055600 con 0x7f49bc073320 2024-09-12T12:20:57.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f49bc15e530 con 0x7f49bc073320 2024-09-12T12:20:57.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49b37fe640 1 -- 172.21.15.3:0/1141017789 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f49ac059900 con 0x7f49bc073320 2024-09-12T12:20:57.399 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:57.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f499c0678c0 msgr2=0x7f499c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f499c0678c0 0x7f499c069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ae000 tx=0x7f49a40a7070 comp rx=0 tx=0).stop 2024-09-12T12:20:57.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc073320 msgr2=0x7f49bc09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc073320 0x7f49bc09e080 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00c420 tx=0x7f49ac00c8e0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 shutdown_connections 2024-09-12T12:20:57.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f499c0678c0 0x7f499c069d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49bc0d0a90 0x7f49bc0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49bc0af550 0x7f49bc152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 --2- 172.21.15.3:0/1141017789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49bc073320 0x7f49bc09e080 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 >> 172.21.15.3:0/1141017789 conn(0x7f49bc06be90 msgr2=0x7f49bc0cf490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:57.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 shutdown_connections 2024-09-12T12:20:57.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.394+0000 7f49c1106640 1 -- 172.21.15.3:0/1141017789 wait complete. 2024-09-12T12:20:57.419 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:20:57.419 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:20:57.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:20:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.542+0000 7f136ba13640 1 Processor -- start 2024-09-12T12:20:57.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.542+0000 7f136ba13640 1 -- start start 2024-09-12T12:20:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f136ba13640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13640b5fc0 0x7f13640d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f136ba13640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f13640ced00 0x7f13640cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f136ba13640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13640d2d90 con 0x7f136415f620 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f136ba13640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f13640d3070 con 0x7f13640b5fc0 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f136ba13640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f13640d3350 con 0x7f13640ced00 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f13697af640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f13640b5fc0 0x7f13640d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f1368fae640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f136415f620 0x7f13640b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45212/0 (socket says 172.21.15.3:45212) 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f1368fae640 1 -- 172.21.15.3:0/2705906463 learned_addr learned my addr 172.21.15.3:0/2705906463 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f1369fb0640 1 --2- 172.21.15.3:0/2705906463 >> v2:172.21.15.142:3300/0 conn(0x7f13640ced00 0x7f13640cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f13697af640 1 -- 172.21.15.3:0/2705906463 >> v2:172.21.15.142:3300/0 conn(0x7f13640ced00 msgr2=0x7f13640cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f13697af640 1 --2- 172.21.15.3:0/2705906463 >> v2:172.21.15.142:3300/0 conn(0x7f13640ced00 0x7f13640cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f13697af640 1 -- 172.21.15.3:0/2705906463 >> v1:172.21.15.3:6789/0 conn(0x7f136415f620 legacy=0x7f13640b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f13697af640 1 -- 172.21.15.3:0/2705906463 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640d3630 con 0x7f13640b5fc0 2024-09-12T12:20:57.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f1369fb0640 1 --2- 172.21.15.3:0/2705906463 >> v2:172.21.15.142:3300/0 conn(0x7f13640ced00 0x7f13640cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f13697af640 1 --2- 172.21.15.3:0/2705906463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13640b5fc0 0x7f13640d26f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f13640ba5c0 tx=0x7f135c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8042841c9cd714d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f135bfff640 1 -- 172.21.15.3:0/2705906463 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0b2070 con 0x7f13640b5fc0 2024-09-12T12:20:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f135bfff640 1 -- 172.21.15.3:0/2705906463 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c0a5920 con 0x7f13640b5fc0 2024-09-12T12:20:57.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.546+0000 7f135bfff640 1 -- 172.21.15.3:0/2705906463 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0a5cf0 con 0x7f13640b5fc0 2024-09-12T12:20:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- 172.21.15.3:0/2705906463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13640b5fc0 msgr2=0x7f13640d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 --2- 172.21.15.3:0/2705906463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13640b5fc0 0x7f13640d26f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f13640ba5c0 tx=0x7f135c01c900 comp rx=0 tx=0).stop 2024-09-12T12:20:57.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- 172.21.15.3:0/2705906463 shutdown_connections 2024-09-12T12:20:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 --2- 172.21.15.3:0/2705906463 >> v2:172.21.15.142:3300/0 conn(0x7f13640ced00 0x7f13640cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 --2- 172.21.15.3:0/2705906463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13640b5fc0 0x7f13640d26f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- 172.21.15.3:0/2705906463 >> 172.21.15.3:0/2705906463 conn(0x7f136408c000 msgr2=0x7f13640d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- 172.21.15.3:0/2705906463 shutdown_connections 2024-09-12T12:20:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- 172.21.15.3:0/2705906463 wait complete. 2024-09-12T12:20:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 Processor -- start 2024-09-12T12:20:57.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- start start 2024-09-12T12:20:57.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13640b5fc0 0x7f136415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13640ced00 0x7f136414fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136415f620 0x7f1364157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f136416b5d0 con 0x7f136415f620 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f136416b450 con 0x7f13640ced00 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f136416b750 con 0x7f13640b5fc0 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136415f620 0x7f1364157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136415f620 0x7f1364157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59474/0 (socket says 172.21.15.3:59474) 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 -- 172.21.15.3:0/3683047429 learned_addr learned my addr 172.21.15.3:0/3683047429 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f13697af640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13640b5fc0 0x7f136415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1368fae640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13640ced00 0x7f136414fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 -- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13640b5fc0 msgr2=0x7f136415ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13640b5fc0 0x7f136415ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 -- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13640ced00 msgr2=0x7f136414fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13640ced00 0x7f136414fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 -- 172.21.15.3:0/3683047429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640cf470 con 0x7f136415f620 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1369fb0640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136415f620 0x7f1364157f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f136000c820 tx=0x7f136000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1359ffb640 1 -- 172.21.15.3:0/3683047429 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360015070 con 0x7f136415f620 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1359ffb640 1 -- 172.21.15.3:0/3683047429 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1360007a70 con 0x7f136415f620 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13640b8ae0 con 0x7f136415f620 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f1359ffb640 1 -- 172.21.15.3:0/3683047429 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360007e60 con 0x7f136415f620 2024-09-12T12:20:57.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.550+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1364157720 con 0x7f136415f620 2024-09-12T12:20:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.554+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13640d26f0 con 0x7f136415f620 2024-09-12T12:20:57.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.554+0000 7f1359ffb640 1 -- 172.21.15.3:0/3683047429 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1360018070 con 0x7f136415f620 2024-09-12T12:20:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.554+0000 7f1359ffb640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13480678c0 0x7f1348069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.554+0000 7f1359ffb640 1 -- 172.21.15.3:0/3683047429 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1360085c70 con 0x7f136415f620 2024-09-12T12:20:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.554+0000 7f13697af640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13480678c0 0x7f1348069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.554+0000 7f13697af640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13480678c0 0x7f1348069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f135c0af000 tx=0x7f135c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.558+0000 7f1359ffb640 1 -- 172.21.15.3:0/3683047429 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1360056720 con 0x7f136415f620 2024-09-12T12:20:57.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f136406edc0 con 0x7f136415f620 2024-09-12T12:20:57.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f1359ffb640 1 -- 172.21.15.3:0/3683047429 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f1360056020 con 0x7f136415f620 2024-09-12T12:20:57.743 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:20:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13480678c0 msgr2=0x7f1348069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13480678c0 0x7f1348069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f135c0af000 tx=0x7f135c0a7070 comp rx=0 tx=0).stop 2024-09-12T12:20:57.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136415f620 msgr2=0x7f1364157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136415f620 0x7f1364157f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f136000c820 tx=0x7f136000cce0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 shutdown_connections 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13480678c0 0x7f1348069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136415f620 0x7f1364157f20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13640ced00 0x7f136414fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 --2- 172.21.15.3:0/3683047429 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13640b5fc0 0x7f136415ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 >> 172.21.15.3:0/3683047429 conn(0x7f136408c000 msgr2=0x7f13640750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 shutdown_connections 2024-09-12T12:20:57.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:20:57.738+0000 7f136ba13640 1 -- 172.21.15.3:0/3683047429 wait complete. 2024-09-12T12:20:57.763 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:20:57.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:20:57.763 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:20:57.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35511]: fuse finished with error 0 and tester_r 0 2024-09-12T12:20:57.778 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:21:03.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:21:03.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:03.881 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:21:03.888 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:04.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.017+0000 7f79b359a640 1 Processor -- start 2024-09-12T12:21:04.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.017+0000 7f79b359a640 1 -- start start 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b359a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159bd0 0x7f79ac06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b359a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79ac15d2d0 0x7f79ac0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b359a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79ac06fbb0 con 0x7f79ac159fb0 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b359a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f79ac06f010 con 0x7f79ac15d2d0 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b359a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79ac06e400 con 0x7f79ac159bd0 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1b37640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79ac15d2d0 0x7f79ac0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1b37640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f79ac15d2d0 0x7f79ac0ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:40206/0 (socket says 172.21.15.3:40206) 2024-09-12T12:21:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1b37640 1 -- 172.21.15.3:0/729971668 learned_addr learned my addr 172.21.15.3:0/729971668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:04.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1336640 1 --2- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159bd0 0x7f79ac06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1b37640 1 -- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159bd0 msgr2=0x7f79ac06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:04.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1b37640 1 --2- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159bd0 0x7f79ac06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1b37640 1 -- 172.21.15.3:0/729971668 >> v1:172.21.15.142:6789/0 conn(0x7f79ac159fb0 legacy=0x7f79ac06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.021+0000 7f79b1b37640 1 -- 172.21.15.3:0/729971668 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac06e6e0 con 0x7f79ac15d2d0 2024-09-12T12:21:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1b37640 1 --2- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 0x7f79ac0ad060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009e50 tx=0x7f79a401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ef76a430ee6aa06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79a3fff640 1 -- 172.21.15.3:0/729971668 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a40a57e0 con 0x7f79ac15d2d0 2024-09-12T12:21:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79a3fff640 1 -- 172.21.15.3:0/729971668 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a40a5980 con 0x7f79ac15d2d0 2024-09-12T12:21:04.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79a3fff640 1 -- 172.21.15.3:0/729971668 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a40a5d50 con 0x7f79ac15d2d0 2024-09-12T12:21:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 msgr2=0x7f79ac0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 --2- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 0x7f79ac0ad060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009e50 tx=0x7f79a401ca00 comp rx=0 tx=0).stop 2024-09-12T12:21:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- 172.21.15.3:0/729971668 shutdown_connections 2024-09-12T12:21:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 --2- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 0x7f79ac0ad060 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 --2- 172.21.15.3:0/729971668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159bd0 0x7f79ac06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- 172.21.15.3:0/729971668 >> 172.21.15.3:0/729971668 conn(0x7f79ac0b1d80 msgr2=0x7f79ac0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- 172.21.15.3:0/729971668 shutdown_connections 2024-09-12T12:21:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- 172.21.15.3:0/729971668 wait complete. 2024-09-12T12:21:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 Processor -- start 2024-09-12T12:21:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- start start 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79ac159bd0 0x7f79ac0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159fb0 0x7f79ac15f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 0x7f79ac157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac169360 con 0x7f79ac159bd0 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac1691e0 con 0x7f79ac15d2d0 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b359a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79ac1694e0 con 0x7f79ac159fb0 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79ac159bd0 0x7f79ac0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79ac159bd0 0x7f79ac0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47140/0 (socket says 172.21.15.3:47140) 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 -- 172.21.15.3:0/2368993527 learned_addr learned my addr 172.21.15.3:0/2368993527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1b37640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 0x7f79ac157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b0b35640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159fb0 0x7f79ac15f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 -- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159fb0 msgr2=0x7f79ac15f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159fb0 0x7f79ac15f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 -- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 msgr2=0x7f79ac157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 0x7f79ac157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.025+0000 7f79b1336640 1 -- 172.21.15.3:0/2368993527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ac0ce0e0 con 0x7f79ac159bd0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79b1336640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79ac159bd0 0x7f79ac0cdae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f799c00e960 tx=0x7f799c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79a1ffb640 1 -- 172.21.15.3:0/2368993527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c017070 con 0x7f79ac159bd0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79a1ffb640 1 -- 172.21.15.3:0/2368993527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799c00ca00 con 0x7f79ac159bd0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ac0ce3c0 con 0x7f79ac159bd0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79a1ffb640 1 -- 172.21.15.3:0/2368993527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00cdb0 con 0x7f79ac159bd0 2024-09-12T12:21:04.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ac0ce980 con 0x7f79ac159bd0 2024-09-12T12:21:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ac0ad060 con 0x7f79ac159bd0 2024-09-12T12:21:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79a1ffb640 1 -- 172.21.15.3:0/2368993527 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f799c01a070 con 0x7f79ac159bd0 2024-09-12T12:21:04.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79a1ffb640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79900678c0 0x7f7990069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79a1ffb640 1 -- 172.21.15.3:0/2368993527 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f799c0860b0 con 0x7f79ac159bd0 2024-09-12T12:21:04.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79b0b35640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79900678c0 0x7f7990069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.029+0000 7f79b0b35640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79900678c0 0x7f7990069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0cf4f0 tx=0x7f79a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:04.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.033+0000 7f79a1ffb640 1 -- 172.21.15.3:0/2368993527 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f799c056bf0 con 0x7f79ac159bd0 2024-09-12T12:21:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.229+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79ac0b2b00 con 0x7f79ac159bd0 2024-09-12T12:21:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79a1ffb640 1 -- 172.21.15.3:0/2368993527 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5220 (secure 0 0 0) 0x7f799c0564f0 con 0x7f79ac159bd0 2024-09-12T12:21:04.238 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:04.238 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:20:34.509216+0000","modified":"2024-09-12T12:20:36.529296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4812":{"gid":4812,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4232231567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4232231567},{"type":"v1","addr":"172.21.15.3:6835","nonce":4232231567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4812,"qdb_cluster":[4812]},"id":2}]} 2024-09-12T12:21:04.239 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 14 2024-09-12T12:21:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79900678c0 msgr2=0x7f7990069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79900678c0 0x7f7990069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0cf4f0 tx=0x7f79a8009040 comp rx=0 tx=0).stop 2024-09-12T12:21:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79ac159bd0 msgr2=0x7f79ac0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79ac159bd0 0x7f79ac0cdae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f799c00e960 tx=0x7f799c00ee20 comp rx=0 tx=0).stop 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 shutdown_connections 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79900678c0 0x7f7990069d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79ac15d2d0 0x7f79ac157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79ac159fb0 0x7f79ac15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 --2- 172.21.15.3:0/2368993527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79ac159bd0 0x7f79ac0cdae0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 >> 172.21.15.3:0/2368993527 conn(0x7f79ac0b1d80 msgr2=0x7f79ac158750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 shutdown_connections 2024-09-12T12:21:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.233+0000 7f79b359a640 1 -- 172.21.15.3:0/2368993527 wait complete. 2024-09-12T12:21:04.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-12T12:21:04.397 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":68,"msgr_send_messages":63,"msgr_recv_bytes":112026,"msgr_send_bytes":17290,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.011218300,"msgr_running_send_time":0.002204502,"msgr_running_recv_time":0.018315657,"msgr_running_fast_dispatch_time":0.000284775,"msgr_send_messages_queue_lat":{"avgcount":63,"sum":0.038832099,"avgtime":0.000616382},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":110064,"msgr_send_encrypted_bytes":14288},"AsyncMessenger::Worker-1":{"msgr_recv_messages":23,"msgr_send_messages":27,"msgr_recv_bytes":6878,"msgr_send_bytes":130514,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.010489592,"msgr_running_send_time":0.001847574,"msgr_running_recv_time":0.007972274,"msgr_running_fast_dispatch_time":0.000538273,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.020212123,"avgtime":0.000777389},"msgr_handle_ack_lat":{"avgcount":19,"sum":0.000004190,"avgtime":0.000000220},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":101584},"AsyncMessenger::Worker-2":{"msgr_recv_messages":46,"msgr_send_messages":39,"msgr_recv_bytes":13422,"msgr_send_bytes":24870,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.007169192,"msgr_running_send_time":0.001256474,"msgr_running_recv_time":0.004055961,"msgr_running_fast_dispatch_time":0.001646335,"msgr_send_messages_queue_lat":{"avgcount":38,"sum":0.341871615,"avgtime":0.008996621},"msgr_handle_ack_lat":{"avgcount":18,"sum":0.000005431,"avgtime":0.000000301},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":50,"sum":0.004265791,"avgtime":0.000085315}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000018123,"avgtime":0.000009061}},"mds":{"request":3,"reply":3,"reply_latency":{"avgcount":3,"sum":0.001142557,"avgtime":0.000380852},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":0,"caps":0,"subtrees":2,"traverse":5,"traverse_hit":5,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":7,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197227,"rdpos":0,"jlat":{"avgcount":5,"sum":0.015609768,"avgtime":0.003121953},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":38200,"heap":231732},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":8,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":2,"sum":0.000587174,"avgtime":0.000293587},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000555383,"avgtime":0.000555383},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":2,"average_load":1,"avg_session_uptime":5,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":221411,"buffer_anon_items":62,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52352,"mds_co_items":211,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":14133,"op_resend":37,"op_reply":44,"op_latency":{"avgcount":44,"sum":1.235948282,"avgtime":0.028089733},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":32,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":132,"get_sum":115916,"get_or_fail_fail":0,"get_or_fail_success":132,"take":0,"take_sum":0,"put":132,"put_sum":115916,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":14133,"put":44,"put_sum":14133,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5614565886e0":{"val":0,"max":3758096384,"get_started":0,"get":7,"get_sum":2923,"get_or_fail_fail":0,"get_or_fail_success":7,"take":0,"take_sum":0,"put":5,"put_sum":2923,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-12T12:21:04.399 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 112026, "msgr_recv_encrypted_bytes": 110064, "msgr_recv_messages": 68, "msgr_running_fast_dispatch_time": 0.000284775, "msgr_running_recv_time": 0.018315657, "msgr_running_send_time": 0.002204502, "msgr_running_total_time": 0.0112183, "msgr_send_bytes": 17290, "msgr_send_encrypted_bytes": 14288, "msgr_send_messages": 63, "msgr_send_messages_queue_lat": { "avgcount": 63, "avgtime": 0.000616382, "sum": 0.038832099 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 19, "avgtime": 2.2e-07, "sum": 4.19e-06 }, "msgr_recv_bytes": 6878, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 23, "msgr_running_fast_dispatch_time": 0.000538273, "msgr_running_recv_time": 0.007972274, "msgr_running_send_time": 0.001847574, "msgr_running_total_time": 0.010489592, "msgr_send_bytes": 130514, "msgr_send_encrypted_bytes": 101584, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000777389, "sum": 0.020212123 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 18, "avgtime": 3.01e-07, "sum": 5.431e-06 }, "msgr_recv_bytes": 13422, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 46, "msgr_running_fast_dispatch_time": 0.001646335, "msgr_running_recv_time": 0.004055961, "msgr_running_send_time": 0.001256474, "msgr_running_total_time": 0.007169192, "msgr_send_bytes": 24870, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 39, "msgr_send_messages_queue_lat": { "avgcount": 38, "avgtime": 0.008996621, "sum": 0.341871615 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 50, "avgtime": 8.5315e-05, "sum": 0.004265791 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.061e-06, "sum": 1.8123e-05 }, "queue_len": 0 }, "mds": { "caps": 0, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 0, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 3, "reply_latency": { "avgcount": 3, "avgtime": 0.000380852, "sum": 0.001142557 }, "request": 3, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 5, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 5, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 7, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.003121953, "sum": 0.015609768 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197227 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 38200 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 8, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 2, "avgtime": 0.000293587, "sum": 0.000587174 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000555383, "sum": 0.000555383 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 5, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 2 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 221411, "buffer_anon_items": 62, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52352, "mds_co_items": 211, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 32, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.028089733, "sum": 1.235948282 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 37, "op_rmw": 0, "op_send": 44, "op_send_bytes": 14133, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 132, "get_or_fail_fail": 0, "get_or_fail_success": 132, "get_started": 0, "get_sum": 115916, "max": 104857600, "put": 132, "put_sum": 115916, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 14133, "take": 44, "take_sum": 14133, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5614565886e0": { "get": 7, "get_or_fail_fail": 0, "get_or_fail_success": 7, "get_started": 0, "get_sum": 2923, "max": 3758096384, "put": 5, "put_sum": 2923, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-12T12:21:04.399 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:21:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff716762640 1 Processor -- start 2024-09-12T12:21:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff716762640 1 -- start start 2024-09-12T12:21:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff716762640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff71015e5d0 0x7ff7100ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff716762640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff7100cfbe0 0x7ff7100b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff716762640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7100d31a0 con 0x7ff7100d5050 2024-09-12T12:21:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff716762640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff7100b1060 con 0x7ff71015e5d0 2024-09-12T12:21:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff716762640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff7100d3480 con 0x7ff7100cfbe0 2024-09-12T12:21:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff70ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff71015e5d0 0x7ff7100ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff70ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff71015e5d0 0x7ff7100ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47150/0 (socket says 172.21.15.3:47150) 2024-09-12T12:21:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff70ffff640 1 -- 172.21.15.3:0/4235829299 learned_addr learned my addr 172.21.15.3:0/4235829299 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff714cff640 1 --2- 172.21.15.3:0/4235829299 >> v2:172.21.15.142:3300/0 conn(0x7ff7100cfbe0 0x7ff7100b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff70ffff640 1 -- 172.21.15.3:0/4235829299 >> v2:172.21.15.142:3300/0 conn(0x7ff7100cfbe0 msgr2=0x7ff7100b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff70ffff640 1 --2- 172.21.15.3:0/4235829299 >> v2:172.21.15.142:3300/0 conn(0x7ff7100cfbe0 0x7ff7100b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff70ffff640 1 -- 172.21.15.3:0/4235829299 >> v1:172.21.15.142:6789/0 conn(0x7ff7100d5050 legacy=0x7ff7100cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.513+0000 7ff70ffff640 1 -- 172.21.15.3:0/4235829299 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7100d3760 con 0x7ff71015e5d0 2024-09-12T12:21:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 --2- 172.21.15.3:0/4235829299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71015e5d0 0x7ff7100ceb60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff7100cb570 tx=0x7ff70401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c873986d1bda3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70effd640 1 -- 172.21.15.3:0/4235829299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040a7070 con 0x7ff71015e5d0 2024-09-12T12:21:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70effd640 1 -- 172.21.15.3:0/4235829299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7040b0070 con 0x7ff71015e5d0 2024-09-12T12:21:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70effd640 1 -- 172.21.15.3:0/4235829299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040ab890 con 0x7ff71015e5d0 2024-09-12T12:21:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- 172.21.15.3:0/4235829299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71015e5d0 msgr2=0x7ff7100ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 --2- 172.21.15.3:0/4235829299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71015e5d0 0x7ff7100ceb60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff7100cb570 tx=0x7ff70401c450 comp rx=0 tx=0).stop 2024-09-12T12:21:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- 172.21.15.3:0/4235829299 shutdown_connections 2024-09-12T12:21:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 --2- 172.21.15.3:0/4235829299 >> v2:172.21.15.142:3300/0 conn(0x7ff7100cfbe0 0x7ff7100b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 --2- 172.21.15.3:0/4235829299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71015e5d0 0x7ff7100ceb60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- 172.21.15.3:0/4235829299 >> 172.21.15.3:0/4235829299 conn(0x7ff71008c000 msgr2=0x7ff7100d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- 172.21.15.3:0/4235829299 shutdown_connections 2024-09-12T12:21:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- 172.21.15.3:0/4235829299 wait complete. 2024-09-12T12:21:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 Processor -- start 2024-09-12T12:21:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- start start 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7100cfbe0 0x7ff7100b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7100d5050 0x7ff7100b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff71015e5d0 0x7ff71015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7100b25d0 con 0x7ff7100cfbe0 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7100b2450 con 0x7ff7100d5050 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff716762640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7100b2750 con 0x7ff71015e5d0 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7100cfbe0 0x7ff7100b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7100cfbe0 0x7ff7100b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47162/0 (socket says 172.21.15.3:47162) 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 -- 172.21.15.3:0/2457519268 learned_addr learned my addr 172.21.15.3:0/2457519268 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff714cff640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff71015e5d0 0x7ff71015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 -- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff71015e5d0 msgr2=0x7ff71015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff71015e5d0 0x7ff71015ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 -- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7100d5050 msgr2=0x7ff7100b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7100d5050 0x7ff7100b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.517+0000 7ff70ffff640 1 -- 172.21.15.3:0/2457519268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71015a190 con 0x7ff7100cfbe0 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70ffff640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7100cfbe0 0x7ff7100b3a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7040ad040 tx=0x7ff70401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70cff9640 1 -- 172.21.15.3:0/2457519268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b4800 con 0x7ff7100cfbe0 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70cff9640 1 -- 172.21.15.3:0/2457519268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7040b49a0 con 0x7ff7100cfbe0 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71015a470 con 0x7ff7100cfbe0 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70cff9640 1 -- 172.21.15.3:0/2457519268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040b4d90 con 0x7ff7100cfbe0 2024-09-12T12:21:04.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7100b3fb0 con 0x7ff7100cfbe0 2024-09-12T12:21:04.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7100ba490 con 0x7ff7100cfbe0 2024-09-12T12:21:04.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70cff9640 1 -- 172.21.15.3:0/2457519268 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff7040c7440 con 0x7ff7100cfbe0 2024-09-12T12:21:04.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70cff9640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff6fc0678c0 0x7ff6fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:04.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70cff9640 1 -- 172.21.15.3:0/2457519268 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff704120490 con 0x7ff7100cfbe0 2024-09-12T12:21:04.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70f7fe640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff6fc0678c0 0x7ff6fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:04.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.521+0000 7ff70f7fe640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff6fc0678c0 0x7ff6fc069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff700009670 tx=0x7ff700006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:04.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.525+0000 7ff70cff9640 1 -- 172.21.15.3:0/2457519268 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff7040160d0 con 0x7ff7100cfbe0 2024-09-12T12:21:04.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.725+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff71006bcc0 con 0x7ff7100cfbe0 2024-09-12T12:21:04.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff70cff9640 1 -- 172.21.15.3:0/2457519268 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7ff7040f1180 con 0x7ff7100cfbe0 2024-09-12T12:21:04.734 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:21:04.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff6fc0678c0 msgr2=0x7ff6fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff6fc0678c0 0x7ff6fc069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff700009670 tx=0x7ff700006bd0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7100cfbe0 msgr2=0x7ff7100b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7100cfbe0 0x7ff7100b3a70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff7040ad040 tx=0x7ff70401ac90 comp rx=0 tx=0).stop 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 shutdown_connections 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff6fc0678c0 0x7ff6fc069d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff71015e5d0 0x7ff71015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7100d5050 0x7ff7100b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 --2- 172.21.15.3:0/2457519268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7100cfbe0 0x7ff7100b3a70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 >> 172.21.15.3:0/2457519268 conn(0x7ff71008c000 msgr2=0x7ff7100b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 shutdown_connections 2024-09-12T12:21:04.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04.729+0000 7ff716762640 1 -- 172.21.15.3:0/2457519268 wait complete. 2024-09-12T12:21:04.752 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:21:04.752 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:21:04.752 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:21:04.752 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:21:04.753 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:04.753 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:21:04.753 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:21:04.753 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:21:04.753 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:21:04.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:04.753 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:21:04.759 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:21:04.759 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:21:04.759 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:21:04.759 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:21:04.760 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42422sec preferred_lft 42422sec 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:21:04.761 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:04.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:04.763 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:21:04.812 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:21:04.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:04.813 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:21:04.864 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:21:04.865 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:21:04.865 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:21:04.865 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:21:04.865 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:21:04.865 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:21:04.865 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:21:04.865 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:21:04.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:21:04.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:21:04.865 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:21:04.866 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:21:04.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:21:04.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:21:04.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:04.994 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:21:05.000 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:21:05.000 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:21:05.000 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:21:05.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:05.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:21:05.055 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:21:05.055 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:21:05.055 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:05.107 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:21:05.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:21:05.107 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:05.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:21:05.212 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:21:05.261 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:21:05.262 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:05.263 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:05.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:05.264 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:21:05.311 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:21:05.311 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:21:05.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:21:05.370 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:05.418 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:21:05.418 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:21:05.419 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:21:05.420 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:21:05.421 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:05.421 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:05.421 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:05.421 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:05.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:05.422 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:21:05.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:21:05.433+0000 7f9d2b110580 -1 init, newargv = 0x5625fb127870 newargc=15 2024-09-12T12:21:05.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[37082]: starting ceph client 2024-09-12T12:21:05.446 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[37082]: starting fuse 2024-09-12T12:21:06.547 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:21:06.564 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:06.615 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:06.616 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:21:06.617 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:21:06.618 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:21:06.618 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:06.618 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:06.618 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:06.618 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:06.618 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:06.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:06.619 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:21:06.666 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:21:06.667 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:21:06.667 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:21:06.668 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:21:06.669 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:21:06.669 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:21:06.669 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:06.669 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:21:06.669 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:21:06.794 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.37082.asok 2024-09-12T12:21:06.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:21:06.807 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37082.asok 2024-09-12T12:21:06.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:06.807 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37082.asok status 2024-09-12T12:21:06.961 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:21:06.961 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:21:06.961 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:21:06.961 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:21:06.961 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "37082", 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5088, 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5088 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:06.962 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3981484462 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3981484462 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5088 192.168.0.1:0/3981484462", 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3981484462", 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 14, 2024-09-12T12:21:06.963 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 32, 2024-09-12T12:21:06.964 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:21:06.964 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:21:06.964 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:21:06.964 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:21:06.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:06.976 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:21:06.977 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:06.982 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:21:06.982 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:06.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:06.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:07.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-12T12:21:07.239 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":69,"msgr_send_messages":64,"msgr_recv_bytes":112298,"msgr_send_bytes":17930,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.011415128,"msgr_running_send_time":0.002287814,"msgr_running_recv_time":0.018403252,"msgr_running_fast_dispatch_time":0.000284775,"msgr_send_messages_queue_lat":{"avgcount":64,"sum":0.038878248,"avgtime":0.000607472},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":110336,"msgr_send_encrypted_bytes":14928},"AsyncMessenger::Worker-1":{"msgr_recv_messages":28,"msgr_send_messages":31,"msgr_recv_bytes":8244,"msgr_send_bytes":131474,"msgr_created_connections":9,"msgr_active_connections":4,"msgr_running_total_time":0.011202283,"msgr_running_send_time":0.002038139,"msgr_running_recv_time":0.008487046,"msgr_running_fast_dispatch_time":0.000592611,"msgr_send_messages_queue_lat":{"avgcount":30,"sum":0.020276345,"avgtime":0.000675878},"msgr_handle_ack_lat":{"avgcount":28,"sum":0.000006303,"avgtime":0.000000225},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":101584},"AsyncMessenger::Worker-2":{"msgr_recv_messages":47,"msgr_send_messages":40,"msgr_recv_bytes":13668,"msgr_send_bytes":25631,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.007243037,"msgr_running_send_time":0.001284366,"msgr_running_recv_time":0.004071600,"msgr_running_fast_dispatch_time":0.001669498,"msgr_send_messages_queue_lat":{"avgcount":39,"sum":0.341892103,"avgtime":0.008766464},"msgr_handle_ack_lat":{"avgcount":18,"sum":0.000005431,"avgtime":0.000000301},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":52,"sum":0.004318767,"avgtime":0.000083053}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000018123,"avgtime":0.000009061}},"mds":{"request":4,"reply":4,"reply_latency":{"avgcount":4,"sum":0.001351905,"avgtime":0.000337976},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":7,"traverse_hit":7,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":8,"evex":0,"evtrm":0,"ev":8,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197670,"rdpos":0,"jlat":{"avgcount":6,"sum":0.017306053,"avgtime":0.002884342},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":3,"cap-":2,"rss":38464,"heap":231732},"mds_server":{"dispatch_client_request":4,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":11,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":3,"sum":0.000796522,"avgtime":0.000265507},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000555383,"avgtime":0.000555383},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":3,"session_remove":2,"sessions_open":1,"sessions_stale":0,"total_load":3,"average_load":3,"avg_session_uptime":5,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":221411,"buffer_anon_items":62,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52352,"mds_co_items":211,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":45,"op_send_bytes":14576,"op_resend":37,"op_reply":45,"op_latency":{"avgcount":45,"sum":1.235948282,"avgtime":0.027465517},"op_inflight":0,"oplen_avg":{"avgcount":45,"sum":69},"op":45,"op_r":0,"op_w":45,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":6,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":32,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":139,"get_sum":117118,"get_or_fail_fail":0,"get_or_fail_success":139,"take":0,"take_sum":0,"put":139,"put_sum":117118,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":14576,"put":45,"put_sum":14576,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":45,"take_sum":45,"put":45,"put_sum":45,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5614565886e0":{"val":0,"max":3758096384,"get_started":0,"get":8,"get_sum":3366,"get_or_fail_fail":0,"get_or_fail_success":8,"take":0,"take_sum":0,"put":6,"put_sum":3366,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-12T12:21:07.240 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 112298, "msgr_recv_encrypted_bytes": 110336, "msgr_recv_messages": 69, "msgr_running_fast_dispatch_time": 0.000284775, "msgr_running_recv_time": 0.018403252, "msgr_running_send_time": 0.002287814, "msgr_running_total_time": 0.011415128, "msgr_send_bytes": 17930, "msgr_send_encrypted_bytes": 14928, "msgr_send_messages": 64, "msgr_send_messages_queue_lat": { "avgcount": 64, "avgtime": 0.000607472, "sum": 0.038878248 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 4, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 28, "avgtime": 2.25e-07, "sum": 6.303e-06 }, "msgr_recv_bytes": 8244, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 28, "msgr_running_fast_dispatch_time": 0.000592611, "msgr_running_recv_time": 0.008487046, "msgr_running_send_time": 0.002038139, "msgr_running_total_time": 0.011202283, "msgr_send_bytes": 131474, "msgr_send_encrypted_bytes": 101584, "msgr_send_messages": 31, "msgr_send_messages_queue_lat": { "avgcount": 30, "avgtime": 0.000675878, "sum": 0.020276345 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 18, "avgtime": 3.01e-07, "sum": 5.431e-06 }, "msgr_recv_bytes": 13668, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 47, "msgr_running_fast_dispatch_time": 0.001669498, "msgr_running_recv_time": 0.0040716, "msgr_running_send_time": 0.001284366, "msgr_running_total_time": 0.007243037, "msgr_send_bytes": 25631, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 40, "msgr_send_messages_queue_lat": { "avgcount": 39, "avgtime": 0.008766464, "sum": 0.341892103 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 52, "avgtime": 8.3053e-05, "sum": 0.004318767 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.061e-06, "sum": 1.8123e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 4, "reply_latency": { "avgcount": 4, "avgtime": 0.000337976, "sum": 0.001351905 }, "request": 4, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 7, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 7, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 8, "evadd": 8, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 6, "avgtime": 0.002884342, "sum": 0.017306053 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197670 }, "mds_mem": { "cap": 1, "cap+": 3, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 38464 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 4, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 11, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 3, "avgtime": 0.000265507, "sum": 0.000796522 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000555383, "sum": 0.000555383 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 3, "avg_session_uptime": 5, "mdthresh_evicted": 0, "session_add": 3, "session_count": 1, "session_remove": 2, "sessions_open": 1, "sessions_stale": 0, "total_load": 3 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 221411, "buffer_anon_items": 62, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52352, "mds_co_items": 211, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 32, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 45, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 45, "avgtime": 0.027465517, "sum": 1.235948282 }, "op_pg": 0, "op_r": 0, "op_reply": 45, "op_resend": 37, "op_rmw": 0, "op_send": 45, "op_send_bytes": 14576, "op_w": 45, "oplen_avg": { "avgcount": 45, "sum": 69 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 6, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 139, "get_or_fail_fail": 0, "get_or_fail_success": 139, "get_started": 0, "get_sum": 117118, "max": 104857600, "put": 139, "put_sum": 117118, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 45, "put_sum": 14576, "take": 45, "take_sum": 14576, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 45, "put_sum": 45, "take": 45, "take_sum": 45, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5614565886e0": { "get": 8, "get_or_fail_fail": 0, "get_or_fail_success": 8, "get_started": 0, "get_sum": 3366, "max": 3758096384, "put": 6, "put_sum": 3366, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-12T12:21:07.241 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:21:07.241 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:07.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:21:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bee7e640 1 Processor -- start 2024-09-12T12:21:07.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bee7e640 1 -- start start 2024-09-12T12:21:07.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bee7e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28b815e7e0 0x7f28b80cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bee7e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bee7e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b80b37b0 con 0x7f28b80cfa90 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bee7e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f28b80b6ac0 con 0x7f28b815e7e0 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bee7e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f28b80b1a70 con 0x7f28b80d4c00 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bd41b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f28b80cfa90 0x7f28b80d24f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45820/0 (socket says 172.21.15.3:45820) 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bd41b640 1 -- 172.21.15.3:0/1257746739 learned_addr learned my addr 172.21.15.3:0/1257746739 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bcc1a640 1 --2- 172.21.15.3:0/1257746739 >> v2:172.21.15.3:3300/0 conn(0x7f28b815e7e0 0x7f28b80cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28b7fff640 1 --2- 172.21.15.3:0/1257746739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28b77fe640 1 -- 172.21.15.3:0/1257746739 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266146539 0 0) 0x7f28b80b37b0 con 0x7f28b80cfa90 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28b77fe640 1 -- 172.21.15.3:0/1257746739 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28a4003530 con 0x7f28b80cfa90 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bcc1a640 1 -- 172.21.15.3:0/1257746739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 msgr2=0x7f28b80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bcc1a640 1 --2- 172.21.15.3:0/1257746739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bcc1a640 1 -- 172.21.15.3:0/1257746739 >> v1:172.21.15.3:6789/0 conn(0x7f28b80cfa90 legacy=0x7f28b80d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.417+0000 7f28bcc1a640 1 -- 172.21.15.3:0/1257746739 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b80d5170 con 0x7f28b815e7e0 2024-09-12T12:21:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bcc1a640 1 --2- 172.21.15.3:0/1257746739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b80cec20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f28a00097c0 tx=0x7f28a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4396ab4523331cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:07.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28b77fe640 1 -- 172.21.15.3:0/1257746739 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a00b0070 con 0x7f28b815e7e0 2024-09-12T12:21:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28b77fe640 1 -- 172.21.15.3:0/1257746739 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a00ab600 con 0x7f28b815e7e0 2024-09-12T12:21:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28b77fe640 1 -- 172.21.15.3:0/1257746739 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a00ab9b0 con 0x7f28b815e7e0 2024-09-12T12:21:07.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 -- 172.21.15.3:0/1257746739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 msgr2=0x7f28b80cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 --2- 172.21.15.3:0/1257746739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b80cec20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f28a00097c0 tx=0x7f28a001c4a0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 -- 172.21.15.3:0/1257746739 shutdown_connections 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 --2- 172.21.15.3:0/1257746739 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b80cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 --2- 172.21.15.3:0/1257746739 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b80cec20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 -- 172.21.15.3:0/1257746739 >> 172.21.15.3:0/1257746739 conn(0x7f28b808c000 msgr2=0x7f28b80d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 -- 172.21.15.3:0/1257746739 shutdown_connections 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.421+0000 7f28bee7e640 1 -- 172.21.15.3:0/1257746739 wait complete. 2024-09-12T12:21:07.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 Processor -- start 2024-09-12T12:21:07.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 -- start start 2024-09-12T12:21:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80cfa90 0x7f28b814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80cf160 con 0x7f28b815e7e0 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80b3a20 con 0x7f28b80cfa90 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f28b816b240 con 0x7f28b80d4c00 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b8157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47194/0 (socket says 172.21.15.3:47194) 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 -- 172.21.15.3:0/2970770582 learned_addr learned my addr 172.21.15.3:0/2970770582 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28b7fff640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bcc1a640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80cfa90 0x7f28b814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 -- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 msgr2=0x7f28b815dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b815dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 -- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80cfa90 msgr2=0x7f28b814e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80cfa90 0x7f28b814e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 -- 172.21.15.3:0/2970770582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b814e640 con 0x7f28b815e7e0 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bd41b640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b8157380 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28b000ca20 tx=0x7f28b000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28b57fa640 1 -- 172.21.15.3:0/2970770582 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0017070 con 0x7f28b815e7e0 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28b57fa640 1 -- 172.21.15.3:0/2970770582 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b000abd0 con 0x7f28b815e7e0 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b815ccb0 con 0x7f28b815e7e0 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28b57fa640 1 -- 172.21.15.3:0/2970770582 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0012400 con 0x7f28b815e7e0 2024-09-12T12:21:07.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b815d270 con 0x7f28b815e7e0 2024-09-12T12:21:07.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.425+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b806ebc0 con 0x7f28b815e7e0 2024-09-12T12:21:07.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.429+0000 7f28b57fa640 1 -- 172.21.15.3:0/2970770582 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f28b001a070 con 0x7f28b815e7e0 2024-09-12T12:21:07.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.429+0000 7f28b57fa640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.429+0000 7f28b57fa640 1 -- 172.21.15.3:0/2970770582 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f28b0085960 con 0x7f28b815e7e0 2024-09-12T12:21:07.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.429+0000 7f28bcc1a640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.429+0000 7f28bcc1a640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28a00ad000 tx=0x7f28a00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:07.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.433+0000 7f28b57fa640 1 -- 172.21.15.3:0/2970770582 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f28b0056490 con 0x7f28b815e7e0 2024-09-12T12:21:07.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28b815b480 con 0x7f28b815e7e0 2024-09-12T12:21:07.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28b57fa640 1 -- 172.21.15.3:0/2970770582 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f28b0055d90 con 0x7f28b815e7e0 2024-09-12T12:21:07.615 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 msgr2=0x7f2898069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f28a00ad000 tx=0x7f28a00a6000 comp rx=0 tx=0).stop 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 msgr2=0x7f28b8157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b8157380 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28b000ca20 tx=0x7f28b000cee0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 shutdown_connections 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b815e7e0 0x7f28b8157380 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80d4c00 0x7f28b815dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 --2- 172.21.15.3:0/2970770582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80cfa90 0x7f28b814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.609+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 >> 172.21.15.3:0/2970770582 conn(0x7f28b808c000 msgr2=0x7f28b80b54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.613+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 shutdown_connections 2024-09-12T12:21:07.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.613+0000 7f28bee7e640 1 -- 172.21.15.3:0/2970770582 wait complete. 2024-09-12T12:21:07.635 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:21:07.635 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:07.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:21:07.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.805+0000 7f41ba2e1640 1 Processor -- start 2024-09-12T12:21:07.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.805+0000 7f41ba2e1640 1 -- start start 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.805+0000 7f41ba2e1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41b40af550 0x7f41b40d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b406f010 con 0x7f41b40734f0 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b406f2f0 con 0x7f41b40d08c0 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f41b40d0d10 con 0x7f41b40af550 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f41b40af550 0x7f41b40d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b887e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f41b40734f0 0x7f41b415b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45836/0 (socket says 172.21.15.3:45836) 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b887e640 1 -- 172.21.15.3:0/3184291776 learned_addr learned my addr 172.21.15.3:0/3184291776 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b3fff640 1 -- 172.21.15.3:0/3184291776 >> v1:172.21.15.142:6789/0 conn(0x7f41b40d08c0 legacy=0x7f41b4072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b3fff640 1 -- 172.21.15.3:0/3184291776 >> v1:172.21.15.3:6789/0 conn(0x7f41b40734f0 legacy=0x7f41b415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b3fff640 1 -- 172.21.15.3:0/3184291776 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b406f5d0 con 0x7f41b40af550 2024-09-12T12:21:07.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b2ffd640 1 -- 172.21.15.3:0/3184291776 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581656339 0 0) 0x7f41b406f010 con 0x7f41b40734f0 2024-09-12T12:21:07.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b3fff640 1 --2- 172.21.15.3:0/3184291776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40af550 0x7f41b40d0380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d9be13b335e74b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:07.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b2ffd640 1 -- 172.21.15.3:0/3184291776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40b0070 con 0x7f41b40af550 2024-09-12T12:21:07.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b2ffd640 1 -- 172.21.15.3:0/3184291776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a40ab600 con 0x7f41b40af550 2024-09-12T12:21:07.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41b2ffd640 1 -- 172.21.15.3:0/3184291776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40ab9d0 con 0x7f41b40af550 2024-09-12T12:21:07.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 -- 172.21.15.3:0/3184291776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40af550 msgr2=0x7f41b40d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/3184291776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40af550 0x7f41b40d0380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401c410 comp rx=0 tx=0).stop 2024-09-12T12:21:07.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 -- 172.21.15.3:0/3184291776 shutdown_connections 2024-09-12T12:21:07.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/3184291776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40af550 0x7f41b40d0380 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 -- 172.21.15.3:0/3184291776 >> 172.21.15.3:0/3184291776 conn(0x7f41b406be90 msgr2=0x7f41b406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:07.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.809+0000 7f41ba2e1640 1 -- 172.21.15.3:0/3184291776 shutdown_connections 2024-09-12T12:21:07.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 -- 172.21.15.3:0/3184291776 wait complete. 2024-09-12T12:21:07.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 Processor -- start 2024-09-12T12:21:07.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 -- start start 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41b40734f0 0x7f41b4151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41b40af550 0x7f41b415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40d08c0 0x7f41b415a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4168f00 con 0x7f41b40d08c0 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4168d80 con 0x7f41b40af550 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41ba2e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41b4169080 con 0x7f41b40734f0 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40d08c0 0x7f41b415a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40d08c0 0x7f41b415a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47220/0 (socket says 172.21.15.3:47220) 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 -- 172.21.15.3:0/1742028390 learned_addr learned my addr 172.21.15.3:0/1742028390 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b37fe640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41b40af550 0x7f41b415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b3fff640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41b40734f0 0x7f41b4151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 -- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41b40734f0 msgr2=0x7f41b4151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41b40734f0 0x7f41b4151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 -- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41b40af550 msgr2=0x7f41b415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41b40af550 0x7f41b415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:07.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 -- 172.21.15.3:0/1742028390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b415c8e0 con 0x7f41b40d08c0 2024-09-12T12:21:07.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.813+0000 7f41b887e640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40d08c0 0x7f41b415a470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00c480 tx=0x7f41ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:07.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41b0ff9640 1 -- 172.21.15.3:0/1742028390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac018070 con 0x7f41b40d08c0 2024-09-12T12:21:07.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41b0ff9640 1 -- 172.21.15.3:0/1742028390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac0074e0 con 0x7f41b40d08c0 2024-09-12T12:21:07.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41b0ff9640 1 -- 172.21.15.3:0/1742028390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac004a30 con 0x7f41b40d08c0 2024-09-12T12:21:07.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b415cba0 con 0x7f41b40d08c0 2024-09-12T12:21:07.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b4156d80 con 0x7f41b40d08c0 2024-09-12T12:21:07.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b415cd70 con 0x7f41b40d08c0 2024-09-12T12:21:07.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41b0ff9640 1 -- 172.21.15.3:0/1742028390 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f41ac004bd0 con 0x7f41b40d08c0 2024-09-12T12:21:07.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41b0ff9640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f41a00678f0 0x7f41a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:07.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41b0ff9640 1 -- 172.21.15.3:0/1742028390 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f41ac084eb0 con 0x7f41b40d08c0 2024-09-12T12:21:07.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.817+0000 7f41b3fff640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f41a00678f0 0x7f41a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:07.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.821+0000 7f41b3fff640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f41a00678f0 0x7f41a0069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ad000 tx=0x7f41a40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:07.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.821+0000 7f41b0ff9640 1 -- 172.21.15.3:0/1742028390 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f41ac0559e0 con 0x7f41b40d08c0 2024-09-12T12:21:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.997+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41b4156f80 con 0x7f41b40d08c0 2024-09-12T12:21:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:07.997+0000 7f41b0ff9640 1 -- 172.21.15.3:0/1742028390 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f41b4156f80 con 0x7f41b40d08c0 2024-09-12T12:21:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:21:08.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f41a00678f0 msgr2=0x7f41a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f41a00678f0 0x7f41a0069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ad000 tx=0x7f41a40a6000 comp rx=0 tx=0).stop 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40d08c0 msgr2=0x7f41b415a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40d08c0 0x7f41b415a470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00c480 tx=0x7f41ac00c940 comp rx=0 tx=0).stop 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 shutdown_connections 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f41a00678f0 0x7f41a0069db0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41b40d08c0 0x7f41b415a470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41b40af550 0x7f41b415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 --2- 172.21.15.3:0/1742028390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41b40734f0 0x7f41b4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 >> 172.21.15.3:0/1742028390 conn(0x7f41b406be90 msgr2=0x7f41b40750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 shutdown_connections 2024-09-12T12:21:08.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:08.001+0000 7f41ba2e1640 1 -- 172.21.15.3:0/1742028390 wait complete. 2024-09-12T12:21:08.025 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:21:08.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:08.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:08.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[37082]: fuse finished with error 0 and tester_r 0 2024-09-12T12:21:08.037 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:21:14.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:21:14.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:14.139 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:21:14.147 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-12T12:21:14.343 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":70,"msgr_send_messages":65,"msgr_recv_bytes":112570,"msgr_send_bytes":18570,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.011682586,"msgr_running_send_time":0.002415751,"msgr_running_recv_time":0.018523154,"msgr_running_fast_dispatch_time":0.000284775,"msgr_send_messages_queue_lat":{"avgcount":65,"sum":0.038960034,"avgtime":0.000599385},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":110608,"msgr_send_encrypted_bytes":15568},"AsyncMessenger::Worker-1":{"msgr_recv_messages":30,"msgr_send_messages":34,"msgr_recv_bytes":8780,"msgr_send_bytes":135208,"msgr_created_connections":9,"msgr_active_connections":3,"msgr_running_total_time":0.011746103,"msgr_running_send_time":0.002297648,"msgr_running_recv_time":0.008659109,"msgr_running_fast_dispatch_time":0.000631376,"msgr_send_messages_queue_lat":{"avgcount":33,"sum":0.020463979,"avgtime":0.000620120},"msgr_handle_ack_lat":{"avgcount":30,"sum":0.000006632,"avgtime":0.000000221},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":105200},"AsyncMessenger::Worker-2":{"msgr_recv_messages":49,"msgr_send_messages":42,"msgr_recv_bytes":14160,"msgr_send_bytes":26498,"msgr_created_connections":6,"msgr_active_connections":3,"msgr_running_total_time":0.007453115,"msgr_running_send_time":0.001380173,"msgr_running_recv_time":0.004114740,"msgr_running_fast_dispatch_time":0.001726947,"msgr_send_messages_queue_lat":{"avgcount":41,"sum":0.341951776,"avgtime":0.008340287},"msgr_handle_ack_lat":{"avgcount":18,"sum":0.000005431,"avgtime":0.000000301},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":55,"sum":0.004379929,"avgtime":0.000079635}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000018123,"avgtime":0.000009061}},"mds":{"request":4,"reply":4,"reply_latency":{"avgcount":4,"sum":0.001351905,"avgtime":0.000337976},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":0,"caps":0,"subtrees":2,"traverse":7,"traverse_hit":7,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":9,"evex":0,"evtrm":0,"ev":9,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197811,"rdpos":0,"jlat":{"avgcount":7,"sum":0.019510262,"avgtime":0.002787180},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":3,"cap-":3,"rss":38728,"heap":231732},"mds_server":{"dispatch_client_request":4,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":12,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":3,"sum":0.000796522,"avgtime":0.000265507},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000555383,"avgtime":0.000555383},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":3,"session_remove":3,"sessions_open":0,"sessions_stale":0,"total_load":3,"average_load":3,"avg_session_uptime":5,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":221723,"buffer_anon_items":65,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5312,"osdmap_items":75,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52352,"mds_co_items":211,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":47,"op_send_bytes":14807,"op_resend":37,"op_reply":47,"op_latency":{"avgcount":47,"sum":1.243948116,"avgtime":0.026466981},"op_inflight":0,"oplen_avg":{"avgcount":47,"sum":71},"op":47,"op_r":0,"op_w":47,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":7,"osdop_writefull":9,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":32,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":144,"get_sum":117916,"get_or_fail_fail":0,"get_or_fail_success":144,"take":0,"take_sum":0,"put":144,"put_sum":117916,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":14807,"put":47,"put_sum":14807,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":47,"put":47,"put_sum":47,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5614565886e0":{"val":0,"max":3758096384,"get_started":0,"get":9,"get_sum":3507,"get_or_fail_fail":0,"get_or_fail_success":9,"take":0,"take_sum":0,"put":7,"put_sum":3507,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-12T12:21:14.345 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 112570, "msgr_recv_encrypted_bytes": 110608, "msgr_recv_messages": 70, "msgr_running_fast_dispatch_time": 0.000284775, "msgr_running_recv_time": 0.018523154, "msgr_running_send_time": 0.002415751, "msgr_running_total_time": 0.011682586, "msgr_send_bytes": 18570, "msgr_send_encrypted_bytes": 15568, "msgr_send_messages": 65, "msgr_send_messages_queue_lat": { "avgcount": 65, "avgtime": 0.000599385, "sum": 0.038960034 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 9, "msgr_handle_ack_lat": { "avgcount": 30, "avgtime": 2.21e-07, "sum": 6.632e-06 }, "msgr_recv_bytes": 8780, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 30, "msgr_running_fast_dispatch_time": 0.000631376, "msgr_running_recv_time": 0.008659109, "msgr_running_send_time": 0.002297648, "msgr_running_total_time": 0.011746103, "msgr_send_bytes": 135208, "msgr_send_encrypted_bytes": 105200, "msgr_send_messages": 34, "msgr_send_messages_queue_lat": { "avgcount": 33, "avgtime": 0.00062012, "sum": 0.020463979 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 18, "avgtime": 3.01e-07, "sum": 5.431e-06 }, "msgr_recv_bytes": 14160, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 49, "msgr_running_fast_dispatch_time": 0.001726947, "msgr_running_recv_time": 0.00411474, "msgr_running_send_time": 0.001380173, "msgr_running_total_time": 0.007453115, "msgr_send_bytes": 26498, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 42, "msgr_send_messages_queue_lat": { "avgcount": 41, "avgtime": 0.008340287, "sum": 0.341951776 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 55, "avgtime": 7.9635e-05, "sum": 0.004379929 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 9.061e-06, "sum": 1.8123e-05 }, "queue_len": 0 }, "mds": { "caps": 0, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 0, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 4, "reply_latency": { "avgcount": 4, "avgtime": 0.000337976, "sum": 0.001351905 }, "request": 4, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 7, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 7, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 9, "evadd": 9, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 7, "avgtime": 0.00278718, "sum": 0.019510262 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197811 }, "mds_mem": { "cap": 0, "cap+": 3, "cap-": 3, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 38728 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 4, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 12, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 3, "avgtime": 0.000265507, "sum": 0.000796522 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000555383, "sum": 0.000555383 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 3, "avg_session_uptime": 5, "mdthresh_evicted": 0, "session_add": 3, "session_count": 0, "session_remove": 3, "sessions_open": 0, "sessions_stale": 0, "total_load": 3 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 221723, "buffer_anon_items": 65, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52352, "mds_co_items": 211, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5312, "osdmap_items": 75, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 32, "map_full": 0, "map_inc": 1, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 47, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 47, "avgtime": 0.026466981, "sum": 1.243948116 }, "op_pg": 0, "op_r": 0, "op_reply": 47, "op_resend": 37, "op_rmw": 0, "op_send": 47, "op_send_bytes": 14807, "op_w": 47, "oplen_avg": { "avgcount": 47, "sum": 71 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 7, "osdop_writefull": 9, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 144, "get_or_fail_fail": 0, "get_or_fail_success": 144, "get_started": 0, "get_sum": 117916, "max": 104857600, "put": 144, "put_sum": 117916, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 47, "put_sum": 14807, "take": 47, "take_sum": 14807, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 47, "put_sum": 47, "take": 47, "take_sum": 47, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5614565886e0": { "get": 9, "get_or_fail_fail": 0, "get_or_fail_success": 9, "get_started": 0, "get_sum": 3507, "max": 3758096384, "put": 7, "put_sum": 3507, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-12T12:21:14.345 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-12T12:21:14.345 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:21:14.345 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:21:14.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:21:14.454 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:21:14.454 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:21:14.455 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:21:14.455 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:14.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:21:14.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:21:14.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:14.461 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:21:14.506 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-12T12:21:14.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:21:14.507 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:21:14.507 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:21:14.507 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:14.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:21:14.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:21:14.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:14.514 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:21:14.560 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-12T12:21:14.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:21:14.561 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:21:14.562 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:21:14.562 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:14.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:21:14.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:21:14.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:14.568 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T12:21:14.615 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-12T12:21:14.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:21:14.616 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:21:14.616 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:21:14.616 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:14.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:21:14.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:21:14.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:14.622 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-12T12:21:14.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:21:14.674 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:14.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd187515640 1 Processor -- start 2024-09-12T12:21:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd187515640 1 -- start start 2024-09-12T12:21:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd187515640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd1800ad770 0x7fd18006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd187515640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd18006fbb0 con 0x7fd180159bd0 2024-09-12T12:21:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd187515640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd18006e400 con 0x7fd180159fb0 2024-09-12T12:21:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd187515640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd18006f010 con 0x7fd1800ad770 2024-09-12T12:21:14.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd180159bd0 0x7fd18006f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57116/0 (socket says 172.21.15.3:57116) 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 -- 172.21.15.3:0/46642591 learned_addr learned my addr 172.21.15.3:0/46642591 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd185ab2640 1 --2- 172.21.15.3:0/46642591 >> v2:172.21.15.142:3300/0 conn(0x7fd1800ad770 0x7fd18006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147059782 0 0) 0x7fd18006fbb0 con 0x7fd180159bd0 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd16c003530 con 0x7fd180159bd0 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 754496118 0 0) 0x7fd16c003530 con 0x7fd180159bd0 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 >> v2:172.21.15.142:3300/0 conn(0x7fd1800ad770 msgr2=0x7fd18006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 --2- 172.21.15.3:0/46642591 >> v2:172.21.15.142:3300/0 conn(0x7fd1800ad770 0x7fd18006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 >> v1:172.21.15.142:6789/0 conn(0x7fd180159fb0 legacy=0x7fd1800ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18006e6e0 con 0x7fd180159bd0 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fd17c001680 con 0x7fd180159bd0 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 >> v1:172.21.15.3:6789/0 conn(0x7fd180159bd0 legacy=0x7fd18006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd16c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd16c004810 0x7fd16c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16c0051d0 0x7fd16c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd18006f010 con 0x7fd16c0039c0 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c00c070 con 0x7fd16c0051d0 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c00c270 con 0x7fd16c004810 2024-09-12T12:21:14.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd16c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd184ab0640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd16c004810 0x7fd16c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd185ab2640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16c0051d0 0x7fd16c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 -- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd16c004810 msgr2=0x7fd16c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd16c004810 0x7fd16c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 -- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16c0051d0 msgr2=0x7fd16c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16c0051d0 0x7fd16c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd1852b1640 1 -- 172.21.15.3:0/46642591 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c003530 con 0x7fd16c0039c0 2024-09-12T12:21:14.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.801+0000 7fd185ab2640 1 -- 172.21.15.3:0/46642591 reap_dead start 2024-09-12T12:21:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd1852b1640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd16c003de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c240 tx=0x7fd17c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ea9a372abc6dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c007d20 con 0x7fd16c0039c0 2024-09-12T12:21:14.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17c0a9070 con 0x7fd16c0039c0 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd173fff640 1 -- 172.21.15.3:0/46642591 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c005d80 con 0x7fd16c0039c0 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 -- 172.21.15.3:0/46642591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 msgr2=0x7fd16c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd16c003de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00c240 tx=0x7fd17c01cc10 comp rx=0 tx=0).stop 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 -- 172.21.15.3:0/46642591 shutdown_connections 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 --2- 172.21.15.3:0/46642591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd16c003de0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 -- 172.21.15.3:0/46642591 >> 172.21.15.3:0/46642591 conn(0x7fd1800b1d80 msgr2=0x7fd1800b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 -- 172.21.15.3:0/46642591 shutdown_connections 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 -- 172.21.15.3:0/46642591 wait complete. 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 Processor -- start 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.805+0000 7fd187515640 1 -- start start 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd1800cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1800cce70 0x7fd1800cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1801450c0 0x7fd180145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd18015d6e0 con 0x7fd16c0039c0 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd180073db0 con 0x7fd1800cce70 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1800b3890 con 0x7fd1801450c0 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd1800cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd1800cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50520/0 (socket says 172.21.15.3:50520) 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 -- 172.21.15.3:0/2529084987 learned_addr learned my addr 172.21.15.3:0/2529084987 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd184ab0640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1800cce70 0x7fd1800cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd185ab2640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1801450c0 0x7fd180145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 -- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1801450c0 msgr2=0x7fd180145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1801450c0 0x7fd180145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 -- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1800cce70 msgr2=0x7fd1800cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1800cce70 0x7fd1800cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 -- 172.21.15.3:0/2529084987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1800cd970 con 0x7fd16c0039c0 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd1852b1640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd1800cf700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01ca40 tx=0x7fd16c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd171ffb640 1 -- 172.21.15.3:0/2529084987 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0bc070 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd171ffb640 1 -- 172.21.15.3:0/2529084987 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17c001480 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd171ffb640 1 -- 172.21.15.3:0/2529084987 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c007d20 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1800cdc50 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1800afd80 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1800ad060 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd171ffb640 1 -- 172.21.15.3:0/2529084987 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd17c007360 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd171ffb640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1640678f0 0x7fd164069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd171ffb640 1 -- 172.21.15.3:0/2529084987 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd17c11f0e0 con 0x7fd16c0039c0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.809+0000 7fd184ab0640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1640678f0 0x7fd164069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.813+0000 7fd184ab0640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1640678f0 0x7fd164069db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd180152450 tx=0x7fd174009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:14.813+0000 7fd171ffb640 1 -- 172.21.15.3:0/2529084987 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd17c016950 con 0x7fd16c0039c0 2024-09-12T12:21:15.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.017+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1800b2b00 con 0x7fd16c0039c0 2024-09-12T12:21:15.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.017+0000 7fd171ffb640 1 -- 172.21.15.3:0/2529084987 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5220 (secure 0 0 0) 0x7fd17c0efc20 con 0x7fd16c0039c0 2024-09-12T12:21:15.026 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:15.026 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:20:34.509216+0000","modified":"2024-09-12T12:20:36.529296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4812":{"gid":4812,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4232231567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4232231567},{"type":"v1","addr":"172.21.15.3:6835","nonce":4232231567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4812,"qdb_cluster":[4812]},"id":2}]} 2024-09-12T12:21:15.026 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 14 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1640678f0 msgr2=0x7fd164069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1640678f0 0x7fd164069db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd180152450 tx=0x7fd174009040 comp rx=0 tx=0).stop 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 msgr2=0x7fd1800cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd1800cf700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01ca40 tx=0x7fd16c005840 comp rx=0 tx=0).stop 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 shutdown_connections 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1640678f0 0x7fd164069db0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1801450c0 0x7fd180145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1800cce70 0x7fd1800cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 --2- 172.21.15.3:0/2529084987 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd16c0039c0 0x7fd1800cf700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 >> 172.21.15.3:0/2529084987 conn(0x7fd1800b1d80 msgr2=0x7fd18015e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 shutdown_connections 2024-09-12T12:21:15.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.021+0000 7fd187515640 1 -- 172.21.15.3:0/2529084987 wait complete. 2024-09-12T12:21:15.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:15.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faeedccc640 1 Processor -- start 2024-09-12T12:21:15.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faeedccc640 1 -- start start 2024-09-12T12:21:15.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faeedccc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faeedccc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee815ed40 con 0x7faee815e890 2024-09-12T12:21:15.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faeedccc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee80b5940 con 0x7faee80cd6b0 2024-09-12T12:21:15.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faeedccc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faee80b5c20 con 0x7faee815b5c0 2024-09-12T12:21:15.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee7fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7faee815e890 0x7faee80b4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57124/0 (socket says 172.21.15.3:57124) 2024-09-12T12:21:15.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee7fff640 1 -- 172.21.15.3:0/734347091 learned_addr learned my addr 172.21.15.3:0/734347091 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:15.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee77fe640 1 --2- 172.21.15.3:0/734347091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee67fc640 1 -- 172.21.15.3:0/734347091 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318741835 0 0) 0x7faee815ed40 con 0x7faee815e890 2024-09-12T12:21:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee67fc640 1 -- 172.21.15.3:0/734347091 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faed0003530 con 0x7faee815e890 2024-09-12T12:21:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee67fc640 1 -- 172.21.15.3:0/734347091 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2917879433 0 0) 0x7faed0003530 con 0x7faee815e890 2024-09-12T12:21:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee77fe640 1 -- 172.21.15.3:0/734347091 >> v1:172.21.15.142:6789/0 conn(0x7faee80cd6b0 legacy=0x7faee815e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee77fe640 1 -- 172.21.15.3:0/734347091 >> v1:172.21.15.3:6789/0 conn(0x7faee815e890 legacy=0x7faee80b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.169+0000 7faee77fe640 1 -- 172.21.15.3:0/734347091 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee815d6a0 con 0x7faee815b5c0 2024-09-12T12:21:15.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faee77fe640 1 --2- 172.21.15.3:0/734347091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee809aa70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faed8009820 tx=0x7faed801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8248b1565bc4960a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faee67fc640 1 -- 172.21.15.3:0/734347091 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80b0070 con 0x7faee815b5c0 2024-09-12T12:21:15.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faee67fc640 1 -- 172.21.15.3:0/734347091 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed80ab400 con 0x7faee815b5c0 2024-09-12T12:21:15.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faee67fc640 1 -- 172.21.15.3:0/734347091 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0001a00 con 0x7faee815b5c0 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 -- 172.21.15.3:0/734347091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 msgr2=0x7faee809aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 --2- 172.21.15.3:0/734347091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee809aa70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faed8009820 tx=0x7faed801c350 comp rx=0 tx=0).stop 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 -- 172.21.15.3:0/734347091 shutdown_connections 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 --2- 172.21.15.3:0/734347091 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee809aa70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 -- 172.21.15.3:0/734347091 >> 172.21.15.3:0/734347091 conn(0x7faee80734b0 msgr2=0x7faee80b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 -- 172.21.15.3:0/734347091 shutdown_connections 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 -- 172.21.15.3:0/734347091 wait complete. 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 Processor -- start 2024-09-12T12:21:15.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.173+0000 7faeedccc640 1 -- start start 2024-09-12T12:21:15.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee80cd6b0 0x7faee814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee8149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee815e890 0x7faee8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faee80cdc10 con 0x7faee80cd6b0 2024-09-12T12:21:15.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faee80cda90 con 0x7faee815e890 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faee8169040 con 0x7faee815b5c0 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee80cd6b0 0x7faee814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee80cd6b0 0x7faee814b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50526/0 (socket says 172.21.15.3:50526) 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 -- 172.21.15.3:0/1545255955 learned_addr learned my addr 172.21.15.3:0/1545255955 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee6ffd640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee8149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee7fff640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee815e890 0x7faee8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 -- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 msgr2=0x7faee8149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee8149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 -- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee815e890 msgr2=0x7faee8154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee815e890 0x7faee8154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 -- 172.21.15.3:0/1545255955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee8155450 con 0x7faee80cd6b0 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee77fe640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee80cd6b0 0x7faee814b950 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c10 tx=0x7faed801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faecffff640 1 -- 172.21.15.3:0/1545255955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80b0050 con 0x7faee80cd6b0 2024-09-12T12:21:15.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faecffff640 1 -- 172.21.15.3:0/1545255955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed80b1040 con 0x7faee80cd6b0 2024-09-12T12:21:15.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faecffff640 1 -- 172.21.15.3:0/1545255955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80b7650 con 0x7faee80cd6b0 2024-09-12T12:21:15.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee814ab90 con 0x7faee80cd6b0 2024-09-12T12:21:15.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee814b150 con 0x7faee80cd6b0 2024-09-12T12:21:15.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee809aa70 con 0x7faee80cd6b0 2024-09-12T12:21:15.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faecffff640 1 -- 172.21.15.3:0/1545255955 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faed80be030 con 0x7faee80cd6b0 2024-09-12T12:21:15.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faecffff640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faed40678c0 0x7faed4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faecffff640 1 -- 172.21.15.3:0/1545255955 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7faed811fba0 con 0x7faee80cd6b0 2024-09-12T12:21:15.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.177+0000 7faee6ffd640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faed40678c0 0x7faed4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.181+0000 7faee6ffd640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faed40678c0 0x7faed4069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faee815a990 tx=0x7faedc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.181+0000 7faecffff640 1 -- 172.21.15.3:0/1545255955 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7faed80c5020 con 0x7faee80cd6b0 2024-09-12T12:21:15.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.381+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faee806f260 con 0x7faee80cd6b0 2024-09-12T12:21:15.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.381+0000 7faecffff640 1 -- 172.21.15.3:0/1545255955 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5220 (secure 0 0 0) 0x7faed8016010 con 0x7faee80cd6b0 2024-09-12T12:21:15.388 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:15.388 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:20:34.509216+0000","modified":"2024-09-12T12:20:36.529296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4812":{"gid":4812,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4232231567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4232231567},{"type":"v1","addr":"172.21.15.3:6835","nonce":4232231567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4812,"qdb_cluster":[4812]},"id":2}]} 2024-09-12T12:21:15.388 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 14 2024-09-12T12:21:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faed40678c0 msgr2=0x7faed4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faed40678c0 0x7faed4069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faee815a990 tx=0x7faedc009040 comp rx=0 tx=0).stop 2024-09-12T12:21:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee80cd6b0 msgr2=0x7faee814b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee80cd6b0 0x7faee814b950 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c10 tx=0x7faed801ab90 comp rx=0 tx=0).stop 2024-09-12T12:21:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 shutdown_connections 2024-09-12T12:21:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faed40678c0 0x7faed4069d80 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faee815a990 tx=0x7faedc009040 comp rx=0 tx=0).stop 2024-09-12T12:21:15.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faee815e890 0x7faee8154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faee815b5c0 0x7faee8149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 --2- 172.21.15.3:0/1545255955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faee80cd6b0 0x7faee814b950 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 >> 172.21.15.3:0/1545255955 conn(0x7faee80734b0 msgr2=0x7faee806fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 shutdown_connections 2024-09-12T12:21:15.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.385+0000 7faeedccc640 1 -- 172.21.15.3:0/1545255955 wait complete. 2024-09-12T12:21:15.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:21:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0e460640 1 Processor -- start 2024-09-12T12:21:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0e460640 1 -- start start 2024-09-12T12:21:15.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0e460640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b080cfbe0 0x7f1b081695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0e460640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b080d31a0 con 0x7f1b0815e5d0 2024-09-12T12:21:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0e460640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b080d3480 con 0x7f1b080d5050 2024-09-12T12:21:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0e460640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b080b1060 con 0x7f1b080cfbe0 2024-09-12T12:21:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b07fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1b0815e5d0 0x7f1b080ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57138/0 (socket says 172.21.15.3:57138) 2024-09-12T12:21:15.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b07fff640 1 -- 172.21.15.3:0/2380417261 learned_addr learned my addr 172.21.15.3:0/2380417261 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0c9fd640 1 --2- 172.21.15.3:0/2380417261 >> v2:172.21.15.142:3300/0 conn(0x7f1b080cfbe0 0x7f1b081695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118014977 0 0) 0x7f1b080d31a0 con 0x7f1b0815e5d0 2024-09-12T12:21:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1af4003530 con 0x7f1b0815e5d0 2024-09-12T12:21:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2636711083 0 0) 0x7f1af4003530 con 0x7f1b0815e5d0 2024-09-12T12:21:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 >> v2:172.21.15.142:3300/0 conn(0x7f1b080cfbe0 msgr2=0x7f1b081695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 --2- 172.21.15.3:0/2380417261 >> v2:172.21.15.142:3300/0 conn(0x7f1b080cfbe0 0x7f1b081695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 >> v1:172.21.15.142:6789/0 conn(0x7f1b080d5050 legacy=0x7f1b080cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b080d3760 con 0x7f1b0815e5d0 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1af8001680 con 0x7f1b0815e5d0 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 >> v1:172.21.15.3:6789/0 conn(0x7f1b0815e5d0 legacy=0x7f1b080ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1af4004810 0x7f1af4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1af400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080b1060 con 0x7f1af40051d0 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1af400c170 con 0x7f1af40039c0 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1af400c370 con 0x7f1af4004810 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0c9fd640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1af400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b07fff640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b077fe640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1af4004810 0x7f1af4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0c9fd640 1 -- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1af4004810 msgr2=0x7f1af4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0c9fd640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1af4004810 0x7f1af4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0c9fd640 1 -- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1af40039c0 msgr2=0x7f1af4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0c9fd640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b0c9fd640 1 -- 172.21.15.3:0/2380417261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af4003530 con 0x7f1af40051d0 2024-09-12T12:21:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b07fff640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1af40039c0 0x7f1af4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:15.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.525+0000 7f1b077fe640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1af4004810 0x7f1af4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0c9fd640 1 -- 172.21.15.3:0/2380417261 reap_dead start 2024-09-12T12:21:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0c9fd640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1af400b9d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009fc0 tx=0x7f1b0001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cba470f2036deb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00009c10 con 0x7f1af40051d0 2024-09-12T12:21:15.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b000a5d10 con 0x7f1af40051d0 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b06ffd640 1 -- 172.21.15.3:0/2380417261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00003be0 con 0x7f1af40051d0 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 -- 172.21.15.3:0/2380417261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 msgr2=0x7f1af400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1af400b9d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009fc0 tx=0x7f1b0001ccc0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 -- 172.21.15.3:0/2380417261 shutdown_connections 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2380417261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1af400b9d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 -- 172.21.15.3:0/2380417261 >> 172.21.15.3:0/2380417261 conn(0x7f1b0808c000 msgr2=0x7f1b080d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 -- 172.21.15.3:0/2380417261 shutdown_connections 2024-09-12T12:21:15.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 -- 172.21.15.3:0/2380417261 wait complete. 2024-09-12T12:21:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 Processor -- start 2024-09-12T12:21:15.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.529+0000 7f1b0e460640 1 -- start start 2024-09-12T12:21:15.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1b08159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b080ce5c0 0x7f1b0815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b080b8d30 0x7f1b080b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080cf4d0 con 0x7f1af40051d0 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0815ebc0 con 0x7f1b080ce5c0 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b080cf650 con 0x7f1b080b8d30 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1b08159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1b08159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50548/0 (socket says 172.21.15.3:50548) 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 -- 172.21.15.3:0/2030043906 learned_addr learned my addr 172.21.15.3:0/2030043906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b077fe640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b080ce5c0 0x7f1b0815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0c9fd640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b080b8d30 0x7f1b080b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 -- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b080b8d30 msgr2=0x7f1b080b91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b080b8d30 0x7f1b080b91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 -- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b080ce5c0 msgr2=0x7f1b0815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b080ce5c0 0x7f1b0815ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 -- 172.21.15.3:0/2030043906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b081f5280 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b07fff640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1b08159d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1af80115c0 tx=0x7f1af8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b04ff9640 1 -- 172.21.15.3:0/2030043906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8010040 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b04ff9640 1 -- 172.21.15.3:0/2030043906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af801a070 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b04ff9640 1 -- 172.21.15.3:0/2030043906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af800fa80 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0815f6b0 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0815fc70 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b080ceb60 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b04ff9640 1 -- 172.21.15.3:0/2030043906 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1af800fc20 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b04ff9640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1ae80678f0 0x7f1ae8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b04ff9640 1 -- 172.21.15.3:0/2030043906 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1af8088bc0 con 0x7f1af40051d0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.533+0000 7f1b077fe640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1ae80678f0 0x7f1ae8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.537+0000 7f1b077fe640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1ae80678f0 0x7f1ae8069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b080ce9c0 tx=0x7f1af0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.537+0000 7f1b04ff9640 1 -- 172.21.15.3:0/2030043906 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1af8059790 con 0x7f1af40051d0 2024-09-12T12:21:15.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.705+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b080ba490 con 0x7f1af40051d0 2024-09-12T12:21:15.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b04ff9640 1 -- 172.21.15.3:0/2030043906 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7f1af8059090 con 0x7f1af40051d0 2024-09-12T12:21:15.714 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:15.715 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:20:36.129741+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:20:32.417571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T12:20:33.506170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1ae80678f0 msgr2=0x7f1ae8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1ae80678f0 0x7f1ae8069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b080ce9c0 tx=0x7f1af0009040 comp rx=0 tx=0).stop 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 msgr2=0x7f1b08159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1b08159d00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1af80115c0 tx=0x7f1af8011a80 comp rx=0 tx=0).stop 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 shutdown_connections 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1ae80678f0 0x7f1ae8069db0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b080b8d30 0x7f1b080b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b080ce5c0 0x7f1b0815ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 --2- 172.21.15.3:0/2030043906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1af40051d0 0x7f1b08159d00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 >> 172.21.15.3:0/2030043906 conn(0x7f1b0808c000 msgr2=0x7f1b0815ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:15.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 shutdown_connections 2024-09-12T12:21:15.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.709+0000 7f1b0e460640 1 -- 172.21.15.3:0/2030043906 wait complete. 2024-09-12T12:21:15.734 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:21:15.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d2cc5640 1 Processor -- start 2024-09-12T12:21:15.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d2cc5640 1 -- start start 2024-09-12T12:21:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d2cc5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc8cc161ed0 0x7fc8cc0c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d2cc5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8cc0ce9b0 con 0x7fc8cc0d2920 2024-09-12T12:21:15.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d2cc5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8cc163020 con 0x7fc8cc08c410 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d2cc5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc8cc0d3000 con 0x7fc8cc161ed0 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d1262640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc8cc0d2920 0x7fc8cc0cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57140/0 (socket says 172.21.15.3:57140) 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d1262640 1 -- 172.21.15.3:0/3373690675 learned_addr learned my addr 172.21.15.3:0/3373690675 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d0a61640 1 --2- 172.21.15.3:0/3373690675 >> v2:172.21.15.142:3300/0 conn(0x7fc8cc161ed0 0x7fc8cc0c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230172236 0 0) 0x7fc8cc0ce9b0 con 0x7fc8cc0d2920 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8b4003530 con 0x7fc8cc0d2920 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4141778138 0 0) 0x7fc8b4003530 con 0x7fc8cc0d2920 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 >> v2:172.21.15.142:3300/0 conn(0x7fc8cc161ed0 msgr2=0x7fc8cc0c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 --2- 172.21.15.3:0/3373690675 >> v2:172.21.15.142:3300/0 conn(0x7fc8cc161ed0 0x7fc8cc0c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 >> v1:172.21.15.142:6789/0 conn(0x7fc8cc08c410 legacy=0x7fc8cc162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc0d32e0 con 0x7fc8cc0d2920 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fc8c4001a00 con 0x7fc8cc0d2920 2024-09-12T12:21:15.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 >> v1:172.21.15.3:6789/0 conn(0x7fc8cc0d2920 legacy=0x7fc8cc0cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8b40039c0 0x7fc8b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8b40051d0 0x7fc8b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc0d3000 con 0x7fc8b4004810 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b400c080 con 0x7fc8b40039c0 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b400c280 con 0x7fc8b40051d0 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cbfff640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d0a61640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8b40039c0 0x7fc8b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d1262640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8b40051d0 0x7fc8b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cbfff640 1 -- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8b40051d0 msgr2=0x7fc8b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:15.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cbfff640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8b40051d0 0x7fc8b400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cbfff640 1 -- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8b40039c0 msgr2=0x7fc8b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:15.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cbfff640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8b40039c0 0x7fc8b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8cbfff640 1 -- 172.21.15.3:0/3373690675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b4003530 con 0x7fc8b4004810 2024-09-12T12:21:15.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.845+0000 7fc8d1262640 1 -- 172.21.15.3:0/3373690675 reap_dead start 2024-09-12T12:21:15.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8cbfff640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8b4004c90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0001780 tx=0x7fc8c001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb3e919e64d85e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00013a0 con 0x7fc8b4004810 2024-09-12T12:21:15.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c00052f0 con 0x7fc8b4004810 2024-09-12T12:21:15.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8cb7fe640 1 -- 172.21.15.3:0/3373690675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0003840 con 0x7fc8b4004810 2024-09-12T12:21:15.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/3373690675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 msgr2=0x7fc8b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8b4004c90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0001780 tx=0x7fc8c001c400 comp rx=0 tx=0).stop 2024-09-12T12:21:15.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/3373690675 shutdown_connections 2024-09-12T12:21:15.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/3373690675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8b4004c90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/3373690675 >> 172.21.15.3:0/3373690675 conn(0x7fc8cc08c000 msgr2=0x7fc8cc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:15.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/3373690675 shutdown_connections 2024-09-12T12:21:15.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/3373690675 wait complete. 2024-09-12T12:21:15.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.849+0000 7fc8d2cc5640 1 Processor -- start 2024-09-12T12:21:15.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 -- start start 2024-09-12T12:21:15.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8cc147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8cc1f4230 0x7fc8cc1e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8cc1494a0 0x7fc8cc0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc0c8430 con 0x7fc8b4004810 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc0c82b0 con 0x7fc8cc1494a0 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8cc162350 con 0x7fc8cc1f4230 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8cc147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8cc147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50570/0 (socket says 172.21.15.3:50570) 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 -- 172.21.15.3:0/339507691 learned_addr learned my addr 172.21.15.3:0/339507691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:15.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d1262640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8cc1494a0 0x7fc8cc0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8cbfff640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8cc1f4230 0x7fc8cc1e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 -- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8cc1f4230 msgr2=0x7fc8cc1e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8cc1f4230 0x7fc8cc1e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 -- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8cc1494a0 msgr2=0x7fc8cc0cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8cc1494a0 0x7fc8cc0cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 -- 172.21.15.3:0/339507691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc1e8550 con 0x7fc8b4004810 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d1262640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8cc1494a0 0x7fc8cc0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d0a61640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8cc147d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc011690 tx=0x7fc8bc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8c97fa640 1 -- 172.21.15.3:0/339507691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00f9c0 con 0x7fc8b4004810 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8c97fa640 1 -- 172.21.15.3:0/339507691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8bc00fb60 con 0x7fc8b4004810 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8cc148400 con 0x7fc8b4004810 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8c97fa640 1 -- 172.21.15.3:0/339507691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc015460 con 0x7fc8b4004810 2024-09-12T12:21:15.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8cc1489c0 con 0x7fc8b4004810 2024-09-12T12:21:15.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8cc162680 con 0x7fc8b4004810 2024-09-12T12:21:15.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8c97fa640 1 -- 172.21.15.3:0/339507691 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8bc015600 con 0x7fc8b4004810 2024-09-12T12:21:15.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8c97fa640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8b80678c0 0x7fc8b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:15.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.853+0000 7fc8c97fa640 1 -- 172.21.15.3:0/339507691 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc8bc0866a0 con 0x7fc8b4004810 2024-09-12T12:21:15.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.857+0000 7fc8cbfff640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8b80678c0 0x7fc8b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:15.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.857+0000 7fc8cbfff640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8b80678c0 0x7fc8b8069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0004000 tx=0x7fc8c00011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:15.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:15.861+0000 7fc8c97fa640 1 -- 172.21.15.3:0/339507691 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc8bc019030 con 0x7fc8b4004810 2024-09-12T12:21:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.053+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8cc0c8620 con 0x7fc8b4004810 2024-09-12T12:21:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.053+0000 7fc8c97fa640 1 -- 172.21.15.3:0/339507691 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5220 (secure 0 0 0) 0x7fc8bc0571d0 con 0x7fc8b4004810 2024-09-12T12:21:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:16.062 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:20:34.509216+0000","modified":"2024-09-12T12:20:36.529296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4812":{"gid":4812,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4232231567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4232231567},{"type":"v1","addr":"172.21.15.3:6835","nonce":4232231567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4812,"qdb_cluster":[4812]},"id":2}]} 2024-09-12T12:21:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 14 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8b80678c0 msgr2=0x7fc8b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8b80678c0 0x7fc8b8069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0004000 tx=0x7fc8c00011a0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 msgr2=0x7fc8cc147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8cc147d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc011690 tx=0x7fc8bc011b50 comp rx=0 tx=0).stop 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 shutdown_connections 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8b80678c0 0x7fc8b8069d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8cc1494a0 0x7fc8cc0cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8cc1f4230 0x7fc8cc1e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 --2- 172.21.15.3:0/339507691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8b4004810 0x7fc8cc147d70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 >> 172.21.15.3:0/339507691 conn(0x7fc8cc08c000 msgr2=0x7fc8cc06e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 shutdown_connections 2024-09-12T12:21:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.057+0000 7fc8d2cc5640 1 -- 172.21.15.3:0/339507691 wait complete. 2024-09-12T12:21:16.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:16.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.181+0000 7fdc51449640 1 Processor -- start 2024-09-12T12:21:16.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc51449640 1 -- start start 2024-09-12T12:21:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc51449640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc4c15b2d0 0x7fdc4c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc51449640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c073640 0x7fdc4c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc51449640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc4c15f150 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc51449640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdc4c0ae920 con 0x7fdc4c15b2d0 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc51449640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c15b6b0 con 0x7fdc4c073640 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc4c15b2d0 0x7fdc4c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4a7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdc4c06f6e0 0x7fdc4c072f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57154/0 (socket says 172.21.15.3:57154) 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdc4c15b2d0 0x7fdc4c0b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50576/0 (socket says 172.21.15.3:50576) 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4a7fc640 1 -- 172.21.15.3:0/1291621507 learned_addr learned my addr 172.21.15.3:0/1291621507 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4b7fe640 1 --2- 172.21.15.3:0/1291621507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c073640 0x7fdc4c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4affd640 1 -- 172.21.15.3:0/1291621507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c073640 msgr2=0x7fdc4c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4affd640 1 --2- 172.21.15.3:0/1291621507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c073640 0x7fdc4c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4affd640 1 -- 172.21.15.3:0/1291621507 >> v1:172.21.15.3:6789/0 conn(0x7fdc4c06f6e0 legacy=0x7fdc4c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc49ffb640 1 -- 172.21.15.3:0/1291621507 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3382569691 0 0) 0x7fdc4c15f150 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4affd640 1 -- 172.21.15.3:0/1291621507 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c15ee70 con 0x7fdc4c15b2d0 2024-09-12T12:21:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4b7fe640 1 --2- 172.21.15.3:0/1291621507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c073640 0x7fdc4c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc4affd640 1 --2- 172.21.15.3:0/1291621507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c15b2d0 0x7fdc4c0b3080 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ba60 tx=0x7fdc3c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669fc039e98cf047 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc49ffb640 1 -- 172.21.15.3:0/1291621507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0b4070 con 0x7fdc4c15b2d0 2024-09-12T12:21:16.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc49ffb640 1 -- 172.21.15.3:0/1291621507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3c0a7b10 con 0x7fdc4c15b2d0 2024-09-12T12:21:16.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.185+0000 7fdc49ffb640 1 -- 172.21.15.3:0/1291621507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0af440 con 0x7fdc4c15b2d0 2024-09-12T12:21:16.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1291621507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c15b2d0 msgr2=0x7fdc4c0b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 --2- 172.21.15.3:0/1291621507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c15b2d0 0x7fdc4c0b3080 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ba60 tx=0x7fdc3c01eb40 comp rx=0 tx=0).stop 2024-09-12T12:21:16.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1291621507 shutdown_connections 2024-09-12T12:21:16.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 --2- 172.21.15.3:0/1291621507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c073640 0x7fdc4c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 --2- 172.21.15.3:0/1291621507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c15b2d0 0x7fdc4c0b3080 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1291621507 >> 172.21.15.3:0/1291621507 conn(0x7fdc4c0b1d80 msgr2=0x7fdc4c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:16.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1291621507 shutdown_connections 2024-09-12T12:21:16.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1291621507 wait complete. 2024-09-12T12:21:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 Processor -- start 2024-09-12T12:21:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- start start 2024-09-12T12:21:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c06f6e0 0x7fdc4c14f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4c073640 0x7fdc4c151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c15b2d0 0x7fdc4c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c0af700 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4c06d8d0 con 0x7fdc4c073640 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4c0af880 con 0x7fdc4c15b2d0 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c06f6e0 0x7fdc4c14f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c06f6e0 0x7fdc4c14f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50582/0 (socket says 172.21.15.3:50582) 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 -- 172.21.15.3:0/1468778744 learned_addr learned my addr 172.21.15.3:0/1468778744 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 -- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c15b2d0 msgr2=0x7fdc4c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c15b2d0 0x7fdc4c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 -- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4c073640 msgr2=0x7fdc4c151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4c073640 0x7fdc4c151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 -- 172.21.15.3:0/1468778744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c153f70 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc4affd640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c06f6e0 0x7fdc4c14f8c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c01c770 tx=0x7fdc3c01b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc437fe640 1 -- 172.21.15.3:0/1468778744 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0bf020 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc437fe640 1 -- 172.21.15.3:0/1468778744 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3c0afcc0 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4c158c90 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc437fe640 1 -- 172.21.15.3:0/1468778744 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0b4040 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4c159250 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.189+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4c159450 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.193+0000 7fdc437fe640 1 -- 172.21.15.3:0/1468778744 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc3c0c2020 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.193+0000 7fdc437fe640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc300678f0 0x7fdc30069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.193+0000 7fdc437fe640 1 -- 172.21.15.3:0/1468778744 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdc3c123130 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.193+0000 7fdc4a7fc640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc300678f0 0x7fdc30069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.193+0000 7fdc4a7fc640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc300678f0 0x7fdc30069db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c15a940 tx=0x7fdc34007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.197+0000 7fdc437fe640 1 -- 172.21.15.3:0/1468778744 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdc4c159450 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.385+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc4c0aedb0 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.385+0000 7fdc437fe640 1 -- 172.21.15.3:0/1468778744 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5220 (secure 0 0 0) 0x7fdc3c0c9030 con 0x7fdc4c06f6e0 2024-09-12T12:21:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:16.394 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":14,"btime":"2024-09-12T12:20:37:172299+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:20:34.509216+0000","modified":"2024-09-12T12:20:36.529296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4812},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4812":{"gid":4812,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4232231567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4232231567},{"type":"v1","addr":"172.21.15.3:6835","nonce":4232231567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4812,"qdb_cluster":[4812]},"id":2}]} 2024-09-12T12:21:16.394 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 14 2024-09-12T12:21:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc300678f0 msgr2=0x7fdc30069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc300678f0 0x7fdc30069db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c15a940 tx=0x7fdc34007cf0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c06f6e0 msgr2=0x7fdc4c14f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c06f6e0 0x7fdc4c14f8c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c01c770 tx=0x7fdc3c01b8e0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 shutdown_connections 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc300678f0 0x7fdc30069db0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc4c15b2d0 0x7fdc4c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc4c073640 0x7fdc4c151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 --2- 172.21.15.3:0/1468778744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc4c06f6e0 0x7fdc4c14f8c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 >> 172.21.15.3:0/1468778744 conn(0x7fdc4c0b1d80 msgr2=0x7fdc4c0b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 shutdown_connections 2024-09-12T12:21:16.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.389+0000 7fdc51449640 1 -- 172.21.15.3:0/1468778744 wait complete. 2024-09-12T12:21:16.411 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:21:16.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:21:16.492 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:21:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.513+0000 7fdfa2e56640 1 Processor -- start 2024-09-12T12:21:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.513+0000 7fdfa2e56640 1 -- start start 2024-09-12T12:21:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.513+0000 7fdfa2e56640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf9c15e630 0x7fdf9c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.513+0000 7fdfa2e56640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0c77f0 0x7fdf9c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf9c15ea10 con 0x7fdf9c0cf3f0 2024-09-12T12:21:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9c0c6180 con 0x7fdf9c15e630 2024-09-12T12:21:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c06c780 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf9c15e630 0x7fdf9c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf9c15e630 0x7fdf9c0ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50584/0 (socket says 172.21.15.3:50584) 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdf9bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdf9c0cf3f0 0x7fdf9c06bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57162/0 (socket says 172.21.15.3:57162) 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/2806474406 learned_addr learned my addr 172.21.15.3:0/2806474406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa13f3640 1 --2- 172.21.15.3:0/2806474406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0c77f0 0x7fdf9c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/2806474406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0c77f0 msgr2=0x7fdf9c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 --2- 172.21.15.3:0/2806474406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0c77f0 0x7fdf9c0c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/2806474406 >> v1:172.21.15.3:6789/0 conn(0x7fdf9c0cf3f0 legacy=0x7fdf9c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/2806474406 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c06ca60 con 0x7fdf9c15e630 2024-09-12T12:21:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa0bf2640 1 --2- 172.21.15.3:0/2806474406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c15e630 0x7fdf9c0ceeb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009870 tx=0x7fdf9001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3d1882c136ef32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdf9b7fe640 1 -- 172.21.15.3:0/2806474406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900b1070 con 0x7fdf9c15e630 2024-09-12T12:21:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdf9b7fe640 1 -- 172.21.15.3:0/2806474406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf900ac620 con 0x7fdf9c15e630 2024-09-12T12:21:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdf9b7fe640 1 -- 172.21.15.3:0/2806474406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900ac9f0 con 0x7fdf9c15e630 2024-09-12T12:21:16.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- 172.21.15.3:0/2806474406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c15e630 msgr2=0x7fdf9c0ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/2806474406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c15e630 0x7fdf9c0ceeb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009870 tx=0x7fdf9001c4d0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- 172.21.15.3:0/2806474406 shutdown_connections 2024-09-12T12:21:16.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/2806474406 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0c77f0 0x7fdf9c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/2806474406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c15e630 0x7fdf9c0ceeb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- 172.21.15.3:0/2806474406 >> 172.21.15.3:0/2806474406 conn(0x7fdf9c09aa70 msgr2=0x7fdf9c0b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:16.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- 172.21.15.3:0/2806474406 shutdown_connections 2024-09-12T12:21:16.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.517+0000 7fdfa2e56640 1 -- 172.21.15.3:0/2806474406 wait complete. 2024-09-12T12:21:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 Processor -- start 2024-09-12T12:21:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 -- start start 2024-09-12T12:21:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c0c77f0 0x7fdf9c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf9c15e630 0x7fdf9c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0af760 0x7fdf9c0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c06c3d0 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c16b150 con 0x7fdf9c15e630 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c0ae7d0 con 0x7fdf9c0af760 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c0c77f0 0x7fdf9c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c0c77f0 0x7fdf9c0ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50598/0 (socket says 172.21.15.3:50598) 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/1573894888 learned_addr learned my addr 172.21.15.3:0/1573894888 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0af760 msgr2=0x7fdf9c0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0af760 0x7fdf9c0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf9c15e630 msgr2=0x7fdf9c14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf9c15e630 0x7fdf9c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 -- 172.21.15.3:0/1573894888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c0aeab0 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa0bf2640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c0c77f0 0x7fdf9c0ae1f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001a100 tx=0x7fdf90019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdf997fa640 1 -- 172.21.15.3:0/1573894888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900b1040 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdf997fa640 1 -- 172.21.15.3:0/1573894888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf900ba070 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdf997fa640 1 -- 172.21.15.3:0/1573894888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900b5880 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c0aed90 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c0b02a0 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c0c7ca0 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdf997fa640 1 -- 172.21.15.3:0/1573894888 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf900ac4e0 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdf997fa640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf880678c0 0x7fdf88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.521+0000 7fdf997fa640 1 -- 172.21.15.3:0/1573894888 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdf9011fb40 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.525+0000 7fdf9bfff640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf880678c0 0x7fdf88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.525+0000 7fdf9bfff640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf880678c0 0x7fdf88069d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c000f70 tx=0x7fdf8c007a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.525+0000 7fdf997fa640 1 -- 172.21.15.3:0/1573894888 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdf90016150 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.685+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf9c1559e0 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.685+0000 7fdf997fa640 1 -- 172.21.15.3:0/1573894888 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7fdf900f0680 con 0x7fdf9c0c77f0 2024-09-12T12:21:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:16.692 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":32,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:20:36.129741+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:20:32.417571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-12T12:20:33.506170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf880678c0 msgr2=0x7fdf88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf880678c0 0x7fdf88069d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c000f70 tx=0x7fdf8c007a10 comp rx=0 tx=0).stop 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c0c77f0 msgr2=0x7fdf9c0ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c0c77f0 0x7fdf9c0ae1f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001a100 tx=0x7fdf90019270 comp rx=0 tx=0).stop 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 shutdown_connections 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf880678c0 0x7fdf88069d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf9c0af760 0x7fdf9c0b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf9c15e630 0x7fdf9c14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 --2- 172.21.15.3:0/1573894888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf9c0c77f0 0x7fdf9c0ae1f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 >> 172.21.15.3:0/1573894888 conn(0x7fdf9c09aa70 msgr2=0x7fdf9c0b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:16.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 shutdown_connections 2024-09-12T12:21:16.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.689+0000 7fdfa2e56640 1 -- 172.21.15.3:0/1573894888 wait complete. 2024-09-12T12:21:16.710 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T12:21:16.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69ce41640 1 Processor -- start 2024-09-12T12:21:16.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69ce41640 1 -- start start 2024-09-12T12:21:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69ce41640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6980d4c00 0x7fd6980cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69ce41640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6980cfa90 0x7fd6980b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69ce41640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6980d5170 con 0x7fd69815e7e0 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69ce41640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd6980cff00 con 0x7fd6980d4c00 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69ce41640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6980b6ac0 con 0x7fd6980cfa90 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd68dd9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd6980d4c00 0x7fd6980cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69659c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd69815e7e0 0x7fd6980cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:57174/0 (socket says 172.21.15.3:57174) 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd69659c640 1 -- 172.21.15.3:0/3364256116 learned_addr learned my addr 172.21.15.3:0/3364256116 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd68dd9b640 1 -- 172.21.15.3:0/3364256116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6980cfa90 msgr2=0x7fd6980b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd696d9d640 1 --2- 172.21.15.3:0/3364256116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6980cfa90 0x7fd6980b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd68dd9b640 1 --2- 172.21.15.3:0/3364256116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6980cfa90 0x7fd6980b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd68dd9b640 1 -- 172.21.15.3:0/3364256116 >> v1:172.21.15.3:6789/0 conn(0x7fd69815e7e0 legacy=0x7fd6980cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd695d9b640 1 -- 172.21.15.3:0/3364256116 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2315067969 0 0) 0x7fd6980d5170 con 0x7fd69815e7e0 2024-09-12T12:21:16.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.869+0000 7fd68dd9b640 1 -- 172.21.15.3:0/3364256116 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6980b1cc0 con 0x7fd6980d4c00 2024-09-12T12:21:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 --2- 172.21.15.3:0/3364256116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6980cf550 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd6800097f0 tx=0x7fd68001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a5b62ce623178b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd695d9b640 1 -- 172.21.15.3:0/3364256116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800b0070 con 0x7fd6980d4c00 2024-09-12T12:21:16.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd695d9b640 1 -- 172.21.15.3:0/3364256116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6800ab600 con 0x7fd6980d4c00 2024-09-12T12:21:16.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd695d9b640 1 -- 172.21.15.3:0/3364256116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800ab9d0 con 0x7fd6980d4c00 2024-09-12T12:21:16.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- 172.21.15.3:0/3364256116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 msgr2=0x7fd6980cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 --2- 172.21.15.3:0/3364256116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6980cf550 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd6800097f0 tx=0x7fd68001c4a0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- 172.21.15.3:0/3364256116 shutdown_connections 2024-09-12T12:21:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 --2- 172.21.15.3:0/3364256116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6980cfa90 0x7fd6980b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 --2- 172.21.15.3:0/3364256116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6980cf550 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- 172.21.15.3:0/3364256116 >> 172.21.15.3:0/3364256116 conn(0x7fd69808c000 msgr2=0x7fd6980d44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- 172.21.15.3:0/3364256116 shutdown_connections 2024-09-12T12:21:16.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- 172.21.15.3:0/3364256116 wait complete. 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 Processor -- start 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- start start 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6980cfa90 0x7fd6980b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6981618d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd69815e7e0 0x7fd6981569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd6980b3670 con 0x7fd6980d4c00 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd6980b34f0 con 0x7fd6980cfa90 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd69ce41640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6980b37f0 con 0x7fd69815e7e0 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6981618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6981618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50608/0 (socket says 172.21.15.3:50608) 2024-09-12T12:21:16.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 -- 172.21.15.3:0/354619432 learned_addr learned my addr 172.21.15.3:0/354619432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 -- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd69815e7e0 msgr2=0x7fd6981569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd69815e7e0 0x7fd6981569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 -- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6980cfa90 msgr2=0x7fd6980b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6980cfa90 0x7fd6980b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 -- 172.21.15.3:0/354619432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6980c8c90 con 0x7fd6980d4c00 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.873+0000 7fd68dd9b640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6981618d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd68001a0d0 tx=0x7fd680019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd68f7fe640 1 -- 172.21.15.3:0/354619432 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800b0040 con 0x7fd6980d4c00 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd68f7fe640 1 -- 172.21.15.3:0/354619432 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6800ba070 con 0x7fd6980d4c00 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6980b9250 con 0x7fd6980d4c00 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd68f7fe640 1 -- 172.21.15.3:0/354619432 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800b4830 con 0x7fd6980d4c00 2024-09-12T12:21:16.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6980b9810 con 0x7fd6980d4c00 2024-09-12T12:21:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6980b9040 con 0x7fd6980d4c00 2024-09-12T12:21:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd68f7fe640 1 -- 172.21.15.3:0/354619432 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd6800ab4c0 con 0x7fd6980d4c00 2024-09-12T12:21:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd68f7fe640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6780678c0 0x7fd678069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd68f7fe640 1 -- 172.21.15.3:0/354619432 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd68011f9a0 con 0x7fd6980d4c00 2024-09-12T12:21:16.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd69659c640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6780678c0 0x7fd678069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.877+0000 7fd69659c640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6780678c0 0x7fd678069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6880011f0 tx=0x7fd688007930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:16.881+0000 7fd68f7fe640 1 -- 172.21.15.3:0/354619432 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd680016090 con 0x7fd6980d4c00 2024-09-12T12:21:17.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.081+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd6980b27a0 con 0x7fd6980d4c00 2024-09-12T12:21:17.088 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:21:17.081+0000 7f0a5d435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:21:17.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd68f7fe640 1 -- 172.21.15.3:0/354619432 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7fd6800f04e0 con 0x7fd6980d4c00 2024-09-12T12:21:17.091 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:21:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6780678c0 msgr2=0x7fd678069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6780678c0 0x7fd678069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6880011f0 tx=0x7fd688007930 comp rx=0 tx=0).stop 2024-09-12T12:21:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 msgr2=0x7fd6981618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6981618d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd68001a0d0 tx=0x7fd680019240 comp rx=0 tx=0).stop 2024-09-12T12:21:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 shutdown_connections 2024-09-12T12:21:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6780678c0 0x7fd678069d80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd69815e7e0 0x7fd6981569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6980d4c00 0x7fd6981618d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 --2- 172.21.15.3:0/354619432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6980cfa90 0x7fd6980b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.085+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 >> 172.21.15.3:0/354619432 conn(0x7fd69808c000 msgr2=0x7fd6980b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:17.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.089+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 shutdown_connections 2024-09-12T12:21:17.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.089+0000 7fd69ce41640 1 -- 172.21.15.3:0/354619432 wait complete. 2024-09-12T12:21:17.095 INFO:tasks.ceph.mds.c.smithi003.stderr: -1934> 2024-09-12T12:20:39.086+0000 7feebac6b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:21:17.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:21:17.131 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:21:17.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59f09e640 1 Processor -- start 2024-09-12T12:21:17.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59f09e640 1 -- start start 2024-09-12T12:21:17.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59f09e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd5980b6b10 0x7fd59806e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59f09e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5980b6350 0x7fd5980cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59f09e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5980d1690 con 0x7fd5980b5f70 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59f09e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd5980d1970 con 0x7fd5980b6350 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59f09e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd5980d23f0 con 0x7fd5980b6b10 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59d63b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5980b6350 0x7fd5980cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59d63b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5980b6350 0x7fd5980cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50618/0 (socket says 172.21.15.3:50618) 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59d63b640 1 -- 172.21.15.3:0/1667278402 learned_addr learned my addr 172.21.15.3:0/1667278402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd58ffff640 1 --2- 172.21.15.3:0/1667278402 >> v2:172.21.15.142:3300/0 conn(0x7fd5980b6b10 0x7fd59806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59d63b640 1 -- 172.21.15.3:0/1667278402 >> v2:172.21.15.142:3300/0 conn(0x7fd5980b6b10 msgr2=0x7fd59806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59d63b640 1 --2- 172.21.15.3:0/1667278402 >> v2:172.21.15.142:3300/0 conn(0x7fd5980b6b10 0x7fd59806e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59d63b640 1 -- 172.21.15.3:0/1667278402 >> v1:172.21.15.142:6789/0 conn(0x7fd5980b5f70 legacy=0x7fd5980d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:17.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.217+0000 7fd59d63b640 1 -- 172.21.15.3:0/1667278402 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5980d26d0 con 0x7fd5980b6350 2024-09-12T12:21:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59d63b640 1 --2- 172.21.15.3:0/1667278402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980cf000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd59808dd60 tx=0x7fd59001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98109ba4f474d438 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd58f7fe640 1 -- 172.21.15.3:0/1667278402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900a5780 con 0x7fd5980b6350 2024-09-12T12:21:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd58f7fe640 1 -- 172.21.15.3:0/1667278402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5900a5920 con 0x7fd5980b6350 2024-09-12T12:21:17.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd58f7fe640 1 -- 172.21.15.3:0/1667278402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900a5cf0 con 0x7fd5980b6350 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 -- 172.21.15.3:0/1667278402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 msgr2=0x7fd5980cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 --2- 172.21.15.3:0/1667278402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980cf000 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd59808dd60 tx=0x7fd59001c9b0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 -- 172.21.15.3:0/1667278402 shutdown_connections 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 --2- 172.21.15.3:0/1667278402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980cf000 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 --2- 172.21.15.3:0/1667278402 >> v2:172.21.15.142:3300/0 conn(0x7fd5980b6b10 0x7fd59806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 -- 172.21.15.3:0/1667278402 >> 172.21.15.3:0/1667278402 conn(0x7fd59809d880 msgr2=0x7fd59808c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 -- 172.21.15.3:0/1667278402 shutdown_connections 2024-09-12T12:21:17.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 -- 172.21.15.3:0/1667278402 wait complete. 2024-09-12T12:21:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 Processor -- start 2024-09-12T12:21:17.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.221+0000 7fd59f09e640 1 -- start start 2024-09-12T12:21:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5980b5f70 0x7fd598152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5980b6b10 0x7fd59815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:17.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd59816b720 con 0x7fd5980b6350 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd59816b5a0 con 0x7fd5980b6b10 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd59816b8a0 con 0x7fd5980b5f70 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50624/0 (socket says 172.21.15.3:50624) 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 -- 172.21.15.3:0/2220310594 learned_addr learned my addr 172.21.15.3:0/2220310594 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59ce3a640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5980b5f70 0x7fd598152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 -- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5980b5f70 msgr2=0x7fd598152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5980b5f70 0x7fd598152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 -- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5980b6b10 msgr2=0x7fd59815d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59d63b640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5980b6b10 0x7fd59815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5980b6b10 0x7fd59815d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 -- 172.21.15.3:0/2220310594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd598153360 con 0x7fd5980b6350 2024-09-12T12:21:17.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58ffff640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980c9e10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd58800c820 tx=0x7fd58800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58d7fa640 1 -- 172.21.15.3:0/2220310594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588015070 con 0x7fd5980b6350 2024-09-12T12:21:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58d7fa640 1 -- 172.21.15.3:0/2220310594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd588004240 con 0x7fd5980b6350 2024-09-12T12:21:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58d7fa640 1 -- 172.21.15.3:0/2220310594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58800ce10 con 0x7fd5980b6350 2024-09-12T12:21:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5980b7880 con 0x7fd5980b6350 2024-09-12T12:21:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5980b7e40 con 0x7fd5980b6350 2024-09-12T12:21:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5980cf000 con 0x7fd5980b6350 2024-09-12T12:21:17.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58d7fa640 1 -- 172.21.15.3:0/2220310594 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd588007830 con 0x7fd5980b6350 2024-09-12T12:21:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58d7fa640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5840678c0 0x7fd584069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd58d7fa640 1 -- 172.21.15.3:0/2220310594 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7fd588085b70 con 0x7fd5980b6350 2024-09-12T12:21:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.225+0000 7fd59ce3a640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5840678c0 0x7fd584069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:17.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.229+0000 7fd59ce3a640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5840678c0 0x7fd584069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd59400b810 tx=0x7fd594008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:17.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.229+0000 7fd58d7fa640 1 -- 172.21.15.3:0/2220310594 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd598153360 con 0x7fd5980b6350 2024-09-12T12:21:17.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:17.425+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd59806e560 con 0x7fd5980b6350 2024-09-12T12:21:18.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd58d7fa640 1 -- 172.21.15.3:0/2220310594 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7fd588056650 con 0x7fd5980b6350 2024-09-12T12:21:18.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5840678c0 msgr2=0x7fd584069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5840678c0 0x7fd584069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd59400b810 tx=0x7fd594008040 comp rx=0 tx=0).stop 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 msgr2=0x7fd5980c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980c9e10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd58800c820 tx=0x7fd58800cce0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 shutdown_connections 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5840678c0 0x7fd584069d80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5980b6b10 0x7fd59815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5980b6350 0x7fd5980c9e10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 --2- 172.21.15.3:0/2220310594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5980b5f70 0x7fd598152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 >> 172.21.15.3:0/2220310594 conn(0x7fd59809d880 msgr2=0x7fd59806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 shutdown_connections 2024-09-12T12:21:18.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.089+0000 7fd59f09e640 1 -- 172.21.15.3:0/2220310594 wait complete. 2024-09-12T12:21:18.121 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:21:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f1500d96640 1 Processor -- start 2024-09-12T12:21:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f1500d96640 1 -- start start 2024-09-12T12:21:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f1500d96640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14fc06f150 0x7f14fc0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f1500d96640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f14fc06f5b0 0x7f14fc0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f1500d96640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14fc06e0b0 con 0x7f14fc159dd0 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f1500d96640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f14fc06e390 con 0x7f14fc06f150 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f1500d96640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f14fc06e670 con 0x7f14fc06f5b0 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14f9d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14fc06f150 0x7f14fc0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14fa59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14fc159dd0 0x7f14fc15d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60376/0 (socket says 172.21.15.3:60376) 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14fa59c640 1 -- 172.21.15.3:0/2471769081 learned_addr learned my addr 172.21.15.3:0/2471769081 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14f9d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14fc06f150 0x7f14fc0ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40560/0 (socket says 172.21.15.3:40560) 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14fad9d640 1 --2- 172.21.15.3:0/2471769081 >> v2:172.21.15.142:3300/0 conn(0x7f14fc06f5b0 0x7f14fc0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14f9d9b640 1 -- 172.21.15.3:0/2471769081 >> v2:172.21.15.142:3300/0 conn(0x7f14fc06f5b0 msgr2=0x7f14fc0ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14f9d9b640 1 --2- 172.21.15.3:0/2471769081 >> v2:172.21.15.142:3300/0 conn(0x7f14fc06f5b0 0x7f14fc0ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14f9d9b640 1 -- 172.21.15.3:0/2471769081 >> v1:172.21.15.3:6789/0 conn(0x7f14fc159dd0 legacy=0x7f14fc15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14f959a640 1 -- 172.21.15.3:0/2471769081 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1649278484 0 0) 0x7f14fc06e0b0 con 0x7f14fc159dd0 2024-09-12T12:21:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.297+0000 7f14f9d9b640 1 -- 172.21.15.3:0/2471769081 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc06e950 con 0x7f14fc06f150 2024-09-12T12:21:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f14f9d9b640 1 --2- 172.21.15.3:0/2471769081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc06f150 0x7f14fc0ad060 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0701d0 tx=0x7f14f001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420dfa9abf48ca83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f14f959a640 1 -- 172.21.15.3:0/2471769081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f00a7070 con 0x7f14fc06f150 2024-09-12T12:21:18.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f14f959a640 1 -- 172.21.15.3:0/2471769081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f00b0070 con 0x7f14fc06f150 2024-09-12T12:21:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f14f959a640 1 -- 172.21.15.3:0/2471769081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f00ab890 con 0x7f14fc06f150 2024-09-12T12:21:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 -- 172.21.15.3:0/2471769081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc06f150 msgr2=0x7f14fc0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 --2- 172.21.15.3:0/2471769081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc06f150 0x7f14fc0ad060 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0701d0 tx=0x7f14f001c450 comp rx=0 tx=0).stop 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 -- 172.21.15.3:0/2471769081 shutdown_connections 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 --2- 172.21.15.3:0/2471769081 >> v2:172.21.15.142:3300/0 conn(0x7f14fc06f5b0 0x7f14fc0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 --2- 172.21.15.3:0/2471769081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc06f150 0x7f14fc0ad060 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 -- 172.21.15.3:0/2471769081 >> 172.21.15.3:0/2471769081 conn(0x7f14fc0b1d80 msgr2=0x7f14fc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 -- 172.21.15.3:0/2471769081 shutdown_connections 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 -- 172.21.15.3:0/2471769081 wait complete. 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 Processor -- start 2024-09-12T12:21:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.301+0000 7f1500d96640 1 -- start start 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14fc06f150 0x7f14fc145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14fc06f5b0 0x7f14fc154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc159dd0 0x7f14fc151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0b2bc0 con 0x7f14fc159dd0 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0b2a40 con 0x7f14fc06f150 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc169200 con 0x7f14fc06f5b0 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc159dd0 0x7f14fc151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc159dd0 0x7f14fc151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40564/0 (socket says 172.21.15.3:40564) 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 -- 172.21.15.3:0/4176311793 learned_addr learned my addr 172.21.15.3:0/4176311793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fa59c640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14fc06f150 0x7f14fc145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14f9d9b640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14fc06f5b0 0x7f14fc154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 -- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14fc06f5b0 msgr2=0x7f14fc154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14fc06f5b0 0x7f14fc154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 -- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14fc06f150 msgr2=0x7f14fc145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14fc06f150 0x7f14fc145660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 -- 172.21.15.3:0/4176311793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc152710 con 0x7f14fc159dd0 2024-09-12T12:21:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14fad9d640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc159dd0 0x7f14fc151930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f14f4007e40 tx=0x7f14f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14eaffd640 1 -- 172.21.15.3:0/4176311793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4014620 con 0x7f14fc159dd0 2024-09-12T12:21:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14eaffd640 1 -- 172.21.15.3:0/4176311793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f4019070 con 0x7f14fc159dd0 2024-09-12T12:21:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14eaffd640 1 -- 172.21.15.3:0/4176311793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4014a20 con 0x7f14fc159dd0 2024-09-12T12:21:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc1667d0 con 0x7f14fc159dd0 2024-09-12T12:21:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc166d90 con 0x7f14fc159dd0 2024-09-12T12:21:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc1665d0 con 0x7f14fc159dd0 2024-09-12T12:21:18.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14eaffd640 1 -- 172.21.15.3:0/4176311793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f14f4014bc0 con 0x7f14fc159dd0 2024-09-12T12:21:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.305+0000 7f14eaffd640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14dc0678c0 0x7f14dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.309+0000 7f14eaffd640 1 -- 172.21.15.3:0/4176311793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7f14f40865c0 con 0x7f14fc159dd0 2024-09-12T12:21:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.309+0000 7f14fa59c640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14dc0678c0 0x7f14dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.309+0000 7f14fa59c640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14dc0678c0 0x7f14dc069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f14ec000f70 tx=0x7f14ec007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.317+0000 7f14eaffd640 1 -- 172.21.15.3:0/4176311793 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f14f40570a0 con 0x7f14fc159dd0 2024-09-12T12:21:18.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:18.481+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f14fc15e470 con 0x7f14fc159dd0 2024-09-12T12:21:19.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.101+0000 7f14eaffd640 1 -- 172.21.15.3:0/4176311793 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7f14f40569a0 con 0x7f14fc159dd0 2024-09-12T12:21:19.104 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14dc0678c0 msgr2=0x7f14dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14dc0678c0 0x7f14dc069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f14ec000f70 tx=0x7f14ec007c10 comp rx=0 tx=0).stop 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc159dd0 msgr2=0x7f14fc151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc159dd0 0x7f14fc151930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f14f4007e40 tx=0x7f14f400e4b0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 shutdown_connections 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14dc0678c0 0x7f14dc069d80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14fc159dd0 0x7f14fc151930 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14fc06f5b0 0x7f14fc154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 --2- 172.21.15.3:0/4176311793 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14fc06f150 0x7f14fc145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 >> 172.21.15.3:0/4176311793 conn(0x7f14fc0b1d80 msgr2=0x7f14fc0750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:19.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 shutdown_connections 2024-09-12T12:21:19.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.105+0000 7f1500d96640 1 -- 172.21.15.3:0/4176311793 wait complete. 2024-09-12T12:21:19.132 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:21:19.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.261+0000 7f223083c640 1 Processor -- start 2024-09-12T12:21:19.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.261+0000 7f223083c640 1 -- start start 2024-09-12T12:21:19.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f223083c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f222c0ad770 0x7f222c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f223083c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222c06e0b0 con 0x7f222c159dd0 2024-09-12T12:21:19.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f223083c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f222c06e390 con 0x7f222c06f150 2024-09-12T12:21:19.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f223083c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f222c06e670 con 0x7f222c0ad770 2024-09-12T12:21:19.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222a59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f222c159dd0 0x7f222c15d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60384/0 (socket says 172.21.15.3:60384) 2024-09-12T12:21:19.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222a59c640 1 -- 172.21.15.3:0/2870142978 learned_addr learned my addr 172.21.15.3:0/2870142978 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:19.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222ad9d640 1 --2- 172.21.15.3:0/2870142978 >> v2:172.21.15.142:3300/0 conn(0x7f222c0ad770 0x7f222c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607644887 0 0) 0x7f222c06e0b0 con 0x7f222c159dd0 2024-09-12T12:21:19.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2214003530 con 0x7f222c159dd0 2024-09-12T12:21:19.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3970250326 0 0) 0x7f2214003530 con 0x7f222c159dd0 2024-09-12T12:21:19.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 >> v2:172.21.15.142:3300/0 conn(0x7f222c0ad770 msgr2=0x7f222c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:19.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 --2- 172.21.15.3:0/2870142978 >> v2:172.21.15.142:3300/0 conn(0x7f222c0ad770 0x7f222c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 >> v1:172.21.15.142:6789/0 conn(0x7f222c06f150 legacy=0x7f222c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c06e950 con 0x7f222c159dd0 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f221c001680 con 0x7f222c159dd0 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 >> v1:172.21.15.3:6789/0 conn(0x7f222c159dd0 legacy=0x7f222c15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2214003a10 0x7f2214003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f2214004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22140051d0 0x7f221400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f222c06e670 con 0x7f2214004810 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f221400c0d0 con 0x7f2214003a10 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f221400c2d0 con 0x7f22140051d0 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f2229d9b640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f2214004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222a59c640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2214003a10 0x7f2214003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222ad9d640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22140051d0 0x7f221400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f2229d9b640 1 -- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22140051d0 msgr2=0x7f221400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:19.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f2229d9b640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22140051d0 0x7f221400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f2229d9b640 1 -- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2214003a10 msgr2=0x7f2214003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:19.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f2229d9b640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2214003a10 0x7f2214003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f2229d9b640 1 -- 172.21.15.3:0/2870142978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2214003530 con 0x7f2214004810 2024-09-12T12:21:19.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222ad9d640 1 -- 172.21.15.3:0/2870142978 reap_dead start 2024-09-12T12:21:19.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f2229d9b640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f2214004c90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2220001e70 tx=0x7f222001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923abeb05491d20f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:19.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220001a90 con 0x7f2214004810 2024-09-12T12:21:19.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2220003b80 con 0x7f2214004810 2024-09-12T12:21:19.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.265+0000 7f222959a640 1 -- 172.21.15.3:0/2870142978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220005080 con 0x7f2214004810 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- 172.21.15.3:0/2870142978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 msgr2=0x7f2214004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f2214004c90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2220001e70 tx=0x7f222001c730 comp rx=0 tx=0).stop 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- 172.21.15.3:0/2870142978 shutdown_connections 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 --2- 172.21.15.3:0/2870142978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f2214004c90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- 172.21.15.3:0/2870142978 >> 172.21.15.3:0/2870142978 conn(0x7f222c0b1d80 msgr2=0x7f222c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- 172.21.15.3:0/2870142978 shutdown_connections 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- 172.21.15.3:0/2870142978 wait complete. 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 Processor -- start 2024-09-12T12:21:19.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- start start 2024-09-12T12:21:19.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f222c1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222c157e60 0x7f222c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f222c1562b0 0x7f222c156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f222c15a1b0 con 0x7f2214004810 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f222c06dd40 con 0x7f222c157e60 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f222c0b2a40 con 0x7f222c1562b0 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f222c1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f222c1f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40592/0 (socket says 172.21.15.3:40592) 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 -- 172.21.15.3:0/2274244576 learned_addr learned my addr 172.21.15.3:0/2274244576 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f2229d9b640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222c157e60 0x7f222c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222ad9d640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f222c1562b0 0x7f222c156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 -- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f222c1562b0 msgr2=0x7f222c156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f222c1562b0 0x7f222c156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 -- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222c157e60 msgr2=0x7f222c1582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222c157e60 0x7f222c1582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 -- 172.21.15.3:0/2274244576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c1f5630 con 0x7f2214004810 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f222a59c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f222c1f50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f221c011980 tx=0x7f221c011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f221affd640 1 -- 172.21.15.3:0/2274244576 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c010040 con 0x7f2214004810 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f221affd640 1 -- 172.21.15.3:0/2274244576 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c01a070 con 0x7f2214004810 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f221affd640 1 -- 172.21.15.3:0/2274244576 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c015410 con 0x7f2214004810 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c143d70 con 0x7f2214004810 2024-09-12T12:21:19.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.269+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c142fb0 con 0x7f2214004810 2024-09-12T12:21:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.273+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c06f5b0 con 0x7f2214004810 2024-09-12T12:21:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.273+0000 7f221affd640 1 -- 172.21.15.3:0/2274244576 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f221c0155b0 con 0x7f2214004810 2024-09-12T12:21:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.273+0000 7f221affd640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f220c0678f0 0x7f220c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.273+0000 7f221affd640 1 -- 172.21.15.3:0/2274244576 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5357+0+0 (secure 0 0 0) 0x7f221c088e50 con 0x7f2214004810 2024-09-12T12:21:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.273+0000 7f2229d9b640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f220c0678f0 0x7f220c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:19.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.273+0000 7f2229d9b640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f220c0678f0 0x7f220c069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2220005000 tx=0x7f22200a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:19.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.277+0000 7f221affd640 1 -- 172.21.15.3:0/2274244576 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f221c059b80 con 0x7f2214004810 2024-09-12T12:21:19.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:19.445+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f222c06fae0 con 0x7f2214004810 2024-09-12T12:21:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.109+0000 7f221affd640 1 -- 172.21.15.3:0/2274244576 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7f221c059480 con 0x7f2214004810 2024-09-12T12:21:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T12:21:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f220c0678f0 msgr2=0x7f220c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f220c0678f0 0x7f220c069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2220005000 tx=0x7f22200a9040 comp rx=0 tx=0).stop 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 msgr2=0x7f222c1f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f222c1f50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f221c011980 tx=0x7f221c011e40 comp rx=0 tx=0).stop 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 shutdown_connections 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f220c0678f0 0x7f220c069db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f222c1562b0 0x7f222c156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222c157e60 0x7f222c1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 --2- 172.21.15.3:0/2274244576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2214004810 0x7f222c1f50f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.113+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 >> 172.21.15.3:0/2274244576 conn(0x7f222c0b1d80 msgr2=0x7f222c15a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:20.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.117+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 shutdown_connections 2024-09-12T12:21:20.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.117+0000 7f223083c640 1 -- 172.21.15.3:0/2274244576 wait complete. 2024-09-12T12:21:20.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-09-12T12:21:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.265+0000 7f212a930640 1 Processor -- start 2024-09-12T12:21:20.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.265+0000 7f212a930640 1 -- start start 2024-09-12T12:21:20.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f212a930640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f212406f150 0x7f21240ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f212a930640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212406f5b0 0x7f21240ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f212a930640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f212406e0b0 con 0x7f2124159dd0 2024-09-12T12:21:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f212a930640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f212406e390 con 0x7f212406f150 2024-09-12T12:21:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f212a930640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f212406e670 con 0x7f212406f5b0 2024-09-12T12:21:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f212406f150 0x7f21240ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f212406f150 0x7f21240ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40600/0 (socket says 172.21.15.3:40600) 2024-09-12T12:21:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 -- 172.21.15.3:0/4192476459 learned_addr learned my addr 172.21.15.3:0/4192476459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f2128ecd640 1 --2- 172.21.15.3:0/4192476459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212406f5b0 0x7f21240ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 -- 172.21.15.3:0/4192476459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212406f5b0 msgr2=0x7f21240ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 --2- 172.21.15.3:0/4192476459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212406f5b0 0x7f21240ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f2122ffd640 1 -- 172.21.15.3:0/4192476459 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 240329769 0 0) 0x7f212406e0b0 con 0x7f2124159dd0 2024-09-12T12:21:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 -- 172.21.15.3:0/4192476459 >> v1:172.21.15.3:6789/0 conn(0x7f2124159dd0 legacy=0x7f212415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 -- 172.21.15.3:0/4192476459 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212406e950 con 0x7f212406f150 2024-09-12T12:21:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f2128ecd640 1 --2- 172.21.15.3:0/4192476459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212406f5b0 0x7f21240ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:20.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f21237fe640 1 --2- 172.21.15.3:0/4192476459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f150 0x7f21240ad060 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2118009870 tx=0x7f211801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eac69cb77fc1537 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f2122ffd640 1 -- 172.21.15.3:0/4192476459 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180ab640 con 0x7f212406f150 2024-09-12T12:21:20.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f2122ffd640 1 -- 172.21.15.3:0/4192476459 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21180b0070 con 0x7f212406f150 2024-09-12T12:21:20.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.269+0000 7f2122ffd640 1 -- 172.21.15.3:0/4192476459 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180aba00 con 0x7f212406f150 2024-09-12T12:21:20.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- 172.21.15.3:0/4192476459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f150 msgr2=0x7f21240ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 --2- 172.21.15.3:0/4192476459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f150 0x7f21240ad060 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2118009870 tx=0x7f211801c530 comp rx=0 tx=0).stop 2024-09-12T12:21:20.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- 172.21.15.3:0/4192476459 shutdown_connections 2024-09-12T12:21:20.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 --2- 172.21.15.3:0/4192476459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f212406f5b0 0x7f21240ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 --2- 172.21.15.3:0/4192476459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f150 0x7f21240ad060 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- 172.21.15.3:0/4192476459 >> 172.21.15.3:0/4192476459 conn(0x7f21240b1d80 msgr2=0x7f21240b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- 172.21.15.3:0/4192476459 shutdown_connections 2024-09-12T12:21:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- 172.21.15.3:0/4192476459 wait complete. 2024-09-12T12:21:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 Processor -- start 2024-09-12T12:21:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- start start 2024-09-12T12:21:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f212406f150 0x7f21240727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f5b0 0x7f21241455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2124159dd0 0x7f21241665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21240b2bc0 con 0x7f212406f5b0 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21240b2a40 con 0x7f212406f150 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2124169200 con 0x7f2124159dd0 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f5b0 0x7f21241455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f5b0 0x7f21241455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40606/0 (socket says 172.21.15.3:40606) 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 -- 172.21.15.3:0/1187439136 learned_addr learned my addr 172.21.15.3:0/1187439136 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f2128ecd640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2124159dd0 0x7f21241665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f2123fff640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f212406f150 0x7f21240727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 -- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2124159dd0 msgr2=0x7f21241665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2124159dd0 0x7f21241665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 -- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f212406f150 msgr2=0x7f21240727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f212406f150 0x7f21240727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 -- 172.21.15.3:0/1187439136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2124155290 con 0x7f212406f5b0 2024-09-12T12:21:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f2128ecd640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2124159dd0 0x7f21241665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f2123fff640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f212406f150 0x7f21240727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f21237fe640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f5b0 0x7f21241455d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f211801a160 tx=0x7f211801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f2120ff9640 1 -- 172.21.15.3:0/1187439136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180be5f0 con 0x7f212406f5b0 2024-09-12T12:21:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f2120ff9640 1 -- 172.21.15.3:0/1187439136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21180be7b0 con 0x7f212406f5b0 2024-09-12T12:21:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f2120ff9640 1 -- 172.21.15.3:0/1187439136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21180beba0 con 0x7f212406f5b0 2024-09-12T12:21:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2124155530 con 0x7f212406f5b0 2024-09-12T12:21:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.273+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2124155af0 con 0x7f212406f5b0 2024-09-12T12:21:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.277+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212415d2f0 con 0x7f212406f5b0 2024-09-12T12:21:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.277+0000 7f2120ff9640 1 -- 172.21.15.3:0/1187439136 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f21180b43f0 con 0x7f212406f5b0 2024-09-12T12:21:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.277+0000 7f2120ff9640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21100678c0 0x7f2110069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.277+0000 7f2120ff9640 1 -- 172.21.15.3:0/1187439136 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f2118120530 con 0x7f212406f5b0 2024-09-12T12:21:20.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.277+0000 7f2123fff640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21100678c0 0x7f2110069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.277+0000 7f2123fff640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21100678c0 0x7f2110069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f211c007fd0 tx=0x7f211c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.281+0000 7f2120ff9640 1 -- 172.21.15.3:0/1187439136 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f21180c4060 con 0x7f212406f5b0 2024-09-12T12:21:20.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.445+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) -- 0x7f2124156760 con 0x7f212406f5b0 2024-09-12T12:21:20.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.449+0000 7f2120ff9640 1 -- 172.21.15.3:0/1187439136 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v12) ==== 106+0+0 (secure 0 0 0) 0x7f2118016150 con 0x7f212406f5b0 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21100678c0 msgr2=0x7f2110069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21100678c0 0x7f2110069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f211c007fd0 tx=0x7f211c009040 comp rx=0 tx=0).stop 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f5b0 msgr2=0x7f21241455d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f5b0 0x7f21241455d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f211801a160 tx=0x7f211801ad70 comp rx=0 tx=0).stop 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 shutdown_connections 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21100678c0 0x7f2110069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2124159dd0 0x7f21241665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f212406f5b0 0x7f21241455d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 --2- 172.21.15.3:0/1187439136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f212406f150 0x7f21240727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 >> 172.21.15.3:0/1187439136 conn(0x7f21240b1d80 msgr2=0x7f21240750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 shutdown_connections 2024-09-12T12:21:20.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.453+0000 7f212a930640 1 -- 172.21.15.3:0/1187439136 wait complete. 2024-09-12T12:21:20.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T12:21:20.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.593+0000 7f2b936ce640 1 Processor -- start 2024-09-12T12:21:20.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.593+0000 7f2b936ce640 1 -- start start 2024-09-12T12:21:20.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.593+0000 7f2b936ce640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2b8c15b130 0x7f2b8c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.593+0000 7f2b936ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c073640 0x7f2b8c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.593+0000 7f2b936ce640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8c15f0b0 con 0x7f2b8c06f6e0 2024-09-12T12:21:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.593+0000 7f2b936ce640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2b8c0ae920 con 0x7f2b8c15b130 2024-09-12T12:21:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.593+0000 7f2b936ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c15edd0 con 0x7f2b8c073640 2024-09-12T12:21:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2b8c06f6e0 0x7f2b8c072f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60406/0 (socket says 172.21.15.3:60406) 2024-09-12T12:21:20.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 -- 172.21.15.3:0/1991352383 learned_addr learned my addr 172.21.15.3:0/1991352383 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52042073 0 0) 0x7f2b8c15f0b0 con 0x7f2b8c06f6e0 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b78005740 con 0x7f2b8c06f6e0 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b91c6b640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c073640 0x7f2b8c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b9146a640 1 --2- 172.21.15.3:0/1991352383 >> v2:172.21.15.142:3300/0 conn(0x7f2b8c15b130 0x7f2b8c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1641312205 0 0) 0x7f2b78005740 con 0x7f2b8c06f6e0 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c073640 msgr2=0x7f2b8c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c073640 0x7f2b8c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 >> v2:172.21.15.142:3300/0 conn(0x7f2b8c15b130 msgr2=0x7f2b8c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 --2- 172.21.15.3:0/1991352383 >> v2:172.21.15.142:3300/0 conn(0x7f2b8c15b130 0x7f2b8c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c0adff0 con 0x7f2b8c06f6e0 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f2b7c001680 con 0x7f2b8c06f6e0 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b9146a640 1 --2- 172.21.15.3:0/1991352383 >> v2:172.21.15.142:3300/0 conn(0x7f2b8c15b130 0x7f2b8c0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 >> v1:172.21.15.3:6789/0 conn(0x7f2b8c06f6e0 legacy=0x7f2b8c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b78005bd0 0x7f2b78005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b78006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b780073e0 0x7f2b7800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c0ae920 con 0x7f2b78006a20 2024-09-12T12:21:20.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c15edd0 con 0x7f2b78005bd0 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7800e220 con 0x7f2b780073e0 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b91c6b640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c073640 0x7f2b8c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b78006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b9146a640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b78005bd0 0x7f2b78005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b91c6b640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b780073e0 0x7f2b7800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 -- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b780073e0 msgr2=0x7f2b7800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b780073e0 0x7f2b7800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 -- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b78005bd0 msgr2=0x7f2b78005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b78005bd0 0x7f2b78005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 -- 172.21.15.3:0/1991352383 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b78005740 con 0x7f2b78006a20 2024-09-12T12:21:20.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b91c6b640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b780073e0 0x7f2b7800dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:20.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b90c69640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b78006ea0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c240 tx=0x7f2b7c01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a277c9844366a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c007d60 con 0x7f2b78006a20 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7c005b00 con 0x7f2b78006a20 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.597+0000 7f2b83fff640 1 -- 172.21.15.3:0/1991352383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c007070 con 0x7f2b78006a20 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- 172.21.15.3:0/1991352383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 msgr2=0x7f2b78006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 --2- 172.21.15.3:0/1991352383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b78006ea0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c240 tx=0x7f2b7c01f3f0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b91c6b640 1 -- 172.21.15.3:0/1991352383 reap_dead start 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- 172.21.15.3:0/1991352383 shutdown_connections 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- 172.21.15.3:0/1991352383 >> 172.21.15.3:0/1991352383 conn(0x7f2b8c0b1d80 msgr2=0x7f2b8c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- 172.21.15.3:0/1991352383 shutdown_connections 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- 172.21.15.3:0/1991352383 wait complete. 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 Processor -- start 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- start start 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b8c14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c13ef50 0x7f2b8c1f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c1f4a80 0x7f2b8c14e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c06d860 con 0x7f2b78006a20 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c0ae5b0 con 0x7f2b8c13ef50 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c15b510 con 0x7f2b8c1f4a80 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b8c14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b8c14c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40628/0 (socket says 172.21.15.3:40628) 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 -- 172.21.15.3:0/3016160095 learned_addr learned my addr 172.21.15.3:0/3016160095 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b91c6b640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c1f4a80 0x7f2b8c14e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b90c69640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c13ef50 0x7f2b8c1f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 -- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c1f4a80 msgr2=0x7f2b8c14e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c1f4a80 0x7f2b8c14e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 -- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c13ef50 msgr2=0x7f2b8c1f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c13ef50 0x7f2b8c1f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 -- 172.21.15.3:0/3016160095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c14f670 con 0x7f2b78006a20 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b90c69640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c13ef50 0x7f2b8c1f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b9146a640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b8c14c960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b88010890 tx=0x7f2b88010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b81ffb640 1 -- 172.21.15.3:0/3016160095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8801a020 con 0x7f2b78006a20 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b81ffb640 1 -- 172.21.15.3:0/3016160095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b8800eb60 con 0x7f2b78006a20 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c14f950 con 0x7f2b78006a20 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b81ffb640 1 -- 172.21.15.3:0/3016160095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800f040 con 0x7f2b78006a20 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.601+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c151160 con 0x7f2b78006a20 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.605+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c072940 con 0x7f2b78006a20 2024-09-12T12:21:20.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.605+0000 7f2b81ffb640 1 -- 172.21.15.3:0/3016160095 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2b8801d020 con 0x7f2b78006a20 2024-09-12T12:21:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.605+0000 7f2b81ffb640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2b700678f0 0x7f2b70069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.605+0000 7f2b81ffb640 1 -- 172.21.15.3:0/3016160095 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f2b88086250 con 0x7f2b78006a20 2024-09-12T12:21:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.605+0000 7f2b90c69640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2b700678f0 0x7f2b70069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.605+0000 7f2b90c69640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2b700678f0 0x7f2b70069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0ae000 tx=0x7f2b7c001ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.609+0000 7f2b81ffb640 1 -- 172.21.15.3:0/3016160095 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2b88024020 con 0x7f2b78006a20 2024-09-12T12:21:20.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.773+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2b8c06cd80 con 0x7f2b78006a20 2024-09-12T12:21:20.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b81ffb640 1 -- 172.21.15.3:0/3016160095 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f2b88057090 con 0x7f2b78006a20 2024-09-12T12:21:20.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2b700678f0 msgr2=0x7f2b70069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2b700678f0 0x7f2b70069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0ae000 tx=0x7f2b7c001ac0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 msgr2=0x7f2b8c14c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b8c14c960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2b88010890 tx=0x7f2b88010d50 comp rx=0 tx=0).stop 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 shutdown_connections 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2b700678f0 0x7f2b70069db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2b8c1f4a80 0x7f2b8c14e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b8c13ef50 0x7f2b8c1f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 --2- 172.21.15.3:0/3016160095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b78006a20 0x7f2b8c14c960 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 >> 172.21.15.3:0/3016160095 conn(0x7f2b8c0b1d80 msgr2=0x7f2b8c0b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 shutdown_connections 2024-09-12T12:21:20.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.781+0000 7f2b936ce640 1 -- 172.21.15.3:0/3016160095 wait complete. 2024-09-12T12:21:20.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close' 2024-09-12T12:21:20.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.925+0000 7fe33f486640 1 Processor -- start 2024-09-12T12:21:20.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.925+0000 7fe33f486640 1 -- start start 2024-09-12T12:21:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33f486640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe33806f6e0 0x7fe338072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33f486640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe33815f150 con 0x7fe33815b2d0 2024-09-12T12:21:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33f486640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe33815b6b0 con 0x7fe3380cc300 2024-09-12T12:21:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33f486640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe3380b3790 con 0x7fe33806f6e0 2024-09-12T12:21:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33ca21640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe33806f6e0 0x7fe338072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33ca21640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe33806f6e0 0x7fe338072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40632/0 (socket says 172.21.15.3:40632) 2024-09-12T12:21:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33ca21640 1 -- 172.21.15.3:0/3304487823 learned_addr learned my addr 172.21.15.3:0/3304487823 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33d222640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe33815b2d0 0x7fe3380b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60420/0 (socket says 172.21.15.3:60420) 2024-09-12T12:21:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33ca21640 1 -- 172.21.15.3:0/3304487823 >> v1:172.21.15.142:6789/0 conn(0x7fe3380cc300 legacy=0x7fe3380aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33ca21640 1 -- 172.21.15.3:0/3304487823 >> v1:172.21.15.3:6789/0 conn(0x7fe33815b2d0 legacy=0x7fe3380b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe337fff640 1 -- 172.21.15.3:0/3304487823 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732767042 0 0) 0x7fe33815f150 con 0x7fe33815b2d0 2024-09-12T12:21:20.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33ca21640 1 -- 172.21.15.3:0/3304487823 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33815ee70 con 0x7fe33806f6e0 2024-09-12T12:21:20.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.929+0000 7fe33ca21640 1 --2- 172.21.15.3:0/3304487823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33806f6e0 0x7fe338072f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb278fff382bd9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe337fff640 1 -- 172.21.15.3:0/3304487823 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0b0070 con 0x7fe33806f6e0 2024-09-12T12:21:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe337fff640 1 -- 172.21.15.3:0/3304487823 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c0ab600 con 0x7fe33806f6e0 2024-09-12T12:21:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe337fff640 1 -- 172.21.15.3:0/3304487823 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0ab9d0 con 0x7fe33806f6e0 2024-09-12T12:21:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- 172.21.15.3:0/3304487823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33806f6e0 msgr2=0x7fe338072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 --2- 172.21.15.3:0/3304487823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33806f6e0 0x7fe338072f30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009870 tx=0x7fe32c01c560 comp rx=0 tx=0).stop 2024-09-12T12:21:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- 172.21.15.3:0/3304487823 shutdown_connections 2024-09-12T12:21:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 --2- 172.21.15.3:0/3304487823 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33806f6e0 0x7fe338072f30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- 172.21.15.3:0/3304487823 >> 172.21.15.3:0/3304487823 conn(0x7fe3380b1d80 msgr2=0x7fe3380b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:20.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- 172.21.15.3:0/3304487823 shutdown_connections 2024-09-12T12:21:20.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- 172.21.15.3:0/3304487823 wait complete. 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 Processor -- start 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- start start 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe33806f6e0 0x7fe33815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3380cc300 0x7fe338153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33815b2d0 0x7fe3381542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe33806d8d0 con 0x7fe33815b2d0 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe338073880 con 0x7fe33806f6e0 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe33806ede0 con 0x7fe3380cc300 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33815b2d0 0x7fe3381542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33815b2d0 0x7fe3381542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40644/0 (socket says 172.21.15.3:40644) 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 -- 172.21.15.3:0/1452275094 learned_addr learned my addr 172.21.15.3:0/1452275094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33d222640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe33806f6e0 0x7fe33815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33ca21640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3380cc300 0x7fe338153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 -- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3380cc300 msgr2=0x7fe338153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3380cc300 0x7fe338153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 -- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe33806f6e0 msgr2=0x7fe33815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe33806f6e0 0x7fe33815cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 -- 172.21.15.3:0/1452275094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33815d870 con 0x7fe33815b2d0 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33d222640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe33806f6e0 0x7fe33815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33da23640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33815b2d0 0x7fe3381542d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c430 tx=0x7fe32800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe335ffb640 1 -- 172.21.15.3:0/1452275094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328005220 con 0x7fe33815b2d0 2024-09-12T12:21:20.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe335ffb640 1 -- 172.21.15.3:0/1452275094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3280053c0 con 0x7fe33815b2d0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33815db50 con 0x7fe33815b2d0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.933+0000 7fe335ffb640 1 -- 172.21.15.3:0/1452275094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328007270 con 0x7fe33815b2d0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.937+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33815e110 con 0x7fe33815b2d0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.937+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33806f170 con 0x7fe33815b2d0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.937+0000 7fe335ffb640 1 -- 172.21.15.3:0/1452275094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe328019020 con 0x7fe33815b2d0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.937+0000 7fe335ffb640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe3240678c0 0x7fe324069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.937+0000 7fe335ffb640 1 -- 172.21.15.3:0/1452275094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fe328084df0 con 0x7fe33815b2d0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.937+0000 7fe33d222640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe3240678c0 0x7fe324069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.937+0000 7fe33d222640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe3240678c0 0x7fe324069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3300012f0 tx=0x7fe3300019e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:20.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:20.941+0000 7fe335ffb640 1 -- 172.21.15.3:0/1452275094 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe328055b90 con 0x7fe33815b2d0 2024-09-12T12:21:21.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:21.169+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]} v 0) -- 0x7fe33806cd80 con 0x7fe33815b2d0 2024-09-12T12:21:22.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.121+0000 7fe335ffb640 1 -- 172.21.15.3:0/1452275094 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_mount_conn_close"]}]=0 v98) ==== 144+0+0 (secure 0 0 0) 0x7fe328055490 con 0x7fe33815b2d0 2024-09-12T12:21:22.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe3240678c0 msgr2=0x7fe324069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe3240678c0 0x7fe324069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe3300012f0 tx=0x7fe3300019e0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33815b2d0 msgr2=0x7fe3381542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33815b2d0 0x7fe3381542d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c430 tx=0x7fe32800c8f0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 shutdown_connections 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe3240678c0 0x7fe324069d80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe33815b2d0 0x7fe3381542d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3380cc300 0x7fe338153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 --2- 172.21.15.3:0/1452275094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe33806f6e0 0x7fe33815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 >> 172.21.15.3:0/1452275094 conn(0x7fe3380b1d80 msgr2=0x7fe3380b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 shutdown_connections 2024-09-12T12:21:22.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.125+0000 7fe33f486640 1 -- 172.21.15.3:0/1452275094 wait complete. 2024-09-12T12:21:22.151 INFO:tasks.cephfs_test_runner:That when a client unmounts, the thread count on the MDS goes back ... ok 2024-09-12T12:21:22.151 INFO:tasks.cephfs_test_runner:Starting test: test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) Check that mds evicts blocklisted client 2024-09-12T12:21:22.151 INFO:tasks.cephfs_test_runner:test_session_evict_blocklisted (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-12T12:21:22.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:21:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 Processor -- start 2024-09-12T12:21:22.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 -- start start 2024-09-12T12:21:22.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf7815a980 0x7fcf780706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf780cf430 0x7fcf7815ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7806ccd0 0x7fcf7806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcf78070d30 con 0x7fcf780cf430 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcf78071010 con 0x7fcf7815a980 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7ca71640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf780b57f0 con 0x7fcf7806ccd0 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf780cf430 0x7fcf7815ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcf780cf430 0x7fcf7815ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40648/0 (socket says 172.21.15.3:40648) 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 -- 172.21.15.3:0/866593488 learned_addr learned my addr 172.21.15.3:0/866593488 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf7659c640 1 --2- 172.21.15.3:0/866593488 >> v2:172.21.15.142:3300/0 conn(0x7fcf7815a980 0x7fcf780706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf76d9d640 1 --2- 172.21.15.3:0/866593488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7806ccd0 0x7fcf7806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 -- 172.21.15.3:0/866593488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7806ccd0 msgr2=0x7fcf7806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 --2- 172.21.15.3:0/866593488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7806ccd0 0x7fcf7806d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 -- 172.21.15.3:0/866593488 >> v2:172.21.15.142:3300/0 conn(0x7fcf7815a980 msgr2=0x7fcf780706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 --2- 172.21.15.3:0/866593488 >> v2:172.21.15.142:3300/0 conn(0x7fcf7815a980 0x7fcf780706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.277+0000 7fcf75d9b640 1 -- 172.21.15.3:0/866593488 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf780b5ad0 con 0x7fcf780cf430 2024-09-12T12:21:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf75d9b640 1 --2- 172.21.15.3:0/866593488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf780cf430 0x7fcf7815ad60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf780d0000 tx=0x7fcf6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc568e5c3be25136 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7559a640 1 -- 172.21.15.3:0/866593488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600b0070 con 0x7fcf780cf430 2024-09-12T12:21:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7559a640 1 -- 172.21.15.3:0/866593488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf600ab600 con 0x7fcf780cf430 2024-09-12T12:21:22.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7559a640 1 -- 172.21.15.3:0/866593488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600ab9d0 con 0x7fcf780cf430 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- 172.21.15.3:0/866593488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf780cf430 msgr2=0x7fcf7815ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/866593488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf780cf430 0x7fcf7815ad60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf780d0000 tx=0x7fcf6001c520 comp rx=0 tx=0).stop 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- 172.21.15.3:0/866593488 shutdown_connections 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/866593488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf7806ccd0 0x7fcf7806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/866593488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf780cf430 0x7fcf7815ad60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/866593488 >> v2:172.21.15.142:3300/0 conn(0x7fcf7815a980 0x7fcf780706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- 172.21.15.3:0/866593488 >> 172.21.15.3:0/866593488 conn(0x7fcf78072720 msgr2=0x7fcf78072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- 172.21.15.3:0/866593488 shutdown_connections 2024-09-12T12:21:22.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- 172.21.15.3:0/866593488 wait complete. 2024-09-12T12:21:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 Processor -- start 2024-09-12T12:21:22.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- start start 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7806ccd0 0x7fcf78167930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf780cf430 0x7fcf78156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf7815a980 0x7fcf78138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcf780b6170 con 0x7fcf7806ccd0 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf780b5ff0 con 0x7fcf7815a980 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7ca71640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf780b62f0 con 0x7fcf780cf430 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7806ccd0 0x7fcf78167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7806ccd0 0x7fcf78167930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40654/0 (socket says 172.21.15.3:40654) 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 -- 172.21.15.3:0/287511876 learned_addr learned my addr 172.21.15.3:0/287511876 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 -- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf780cf430 msgr2=0x7fcf78156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf780cf430 0x7fcf78156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 -- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf7815a980 msgr2=0x7fcf78138260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf7815a980 0x7fcf78138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 -- 172.21.15.3:0/287511876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf78157720 con 0x7fcf7806ccd0 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf7659c640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7806ccd0 0x7fcf78167930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf70007e40 tx=0x7fcf7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf6effd640 1 -- 172.21.15.3:0/287511876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70014620 con 0x7fcf7806ccd0 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf6effd640 1 -- 172.21.15.3:0/287511876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf70019070 con 0x7fcf7806ccd0 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.281+0000 7fcf6effd640 1 -- 172.21.15.3:0/287511876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70014a00 con 0x7fcf7806ccd0 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf78157a00 con 0x7fcf7806ccd0 2024-09-12T12:21:22.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf78157fc0 con 0x7fcf7806ccd0 2024-09-12T12:21:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7806d160 con 0x7fcf7806ccd0 2024-09-12T12:21:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf6effd640 1 -- 172.21.15.3:0/287511876 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcf70014ba0 con 0x7fcf7806ccd0 2024-09-12T12:21:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf6effd640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf580678f0 0x7fcf58069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf6effd640 1 -- 172.21.15.3:0/287511876 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fcf700861b0 con 0x7fcf7806ccd0 2024-09-12T12:21:22.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf75d9b640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf580678f0 0x7fcf58069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.285+0000 7fcf75d9b640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf580678f0 0x7fcf58069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf600ad000 tx=0x7fcf60004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.289+0000 7fcf6effd640 1 -- 172.21.15.3:0/287511876 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcf70056ff0 con 0x7fcf7806ccd0 2024-09-12T12:21:22.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.453+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf780b49f0 con 0x7fcf7806ccd0 2024-09-12T12:21:22.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf6effd640 1 -- 172.21.15.3:0/287511876 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+12970 (secure 0 0 0) 0x7fcf700568f0 con 0x7fcf7806ccd0 2024-09-12T12:21:22.459 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:22.459 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":35,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:21:20.104194+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/4232231567":"2024-09-13T12:21:17.087310+0000","172.21.15.3:6834/4232231567":"2024-09-13T12:21:17.087310+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:21:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf580678f0 msgr2=0x7fcf58069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf580678f0 0x7fcf58069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf600ad000 tx=0x7fcf60004740 comp rx=0 tx=0).stop 2024-09-12T12:21:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7806ccd0 msgr2=0x7fcf78167930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7806ccd0 0x7fcf78167930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf70007e40 tx=0x7fcf7000e4b0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 shutdown_connections 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcf580678f0 0x7fcf58069db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf7815a980 0x7fcf78138260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf780cf430 0x7fcf78156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 --2- 172.21.15.3:0/287511876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf7806ccd0 0x7fcf78167930 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 >> 172.21.15.3:0/287511876 conn(0x7fcf78072720 msgr2=0x7fcf780750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 shutdown_connections 2024-09-12T12:21:22.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.457+0000 7fcf7ca71640 1 -- 172.21.15.3:0/287511876 wait complete. 2024-09-12T12:21:22.478 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:19:37.735209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:21:22.478 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T12:21:22.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.593+0000 7fedd69f4640 1 Processor -- start 2024-09-12T12:21:22.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.593+0000 7fedd69f4640 1 -- start start 2024-09-12T12:21:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedd69f4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fedd0159dd0 0x7fedd015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedd69f4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd00ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedd69f4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd015d830 con 0x7fedd006f5b0 2024-09-12T12:21:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedd69f4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fedd006e0b0 con 0x7fedd0159dd0 2024-09-12T12:21:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedd69f4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fedd006e390 con 0x7fedd006f150 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fedd0159dd0 0x7fedd015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fedd0159dd0 0x7fedd015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40662/0 (socket says 172.21.15.3:40662) 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 -- 172.21.15.3:0/3949815232 learned_addr learned my addr 172.21.15.3:0/3949815232 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 -- 172.21.15.3:0/3949815232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 msgr2=0x7fedd00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcf7fe640 1 --2- 172.21.15.3:0/3949815232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd00ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 --2- 172.21.15.3:0/3949815232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd00ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 -- 172.21.15.3:0/3949815232 >> v1:172.21.15.142:6789/0 conn(0x7fedd006f5b0 legacy=0x7fedd00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 -- 172.21.15.3:0/3949815232 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd006e670 con 0x7fedd0159dd0 2024-09-12T12:21:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedcffff640 1 --2- 172.21.15.3:0/3949815232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015d2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67fa2db713f2485 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedceffd640 1 -- 172.21.15.3:0/3949815232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00b0070 con 0x7fedd0159dd0 2024-09-12T12:21:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedceffd640 1 -- 172.21.15.3:0/3949815232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc00ab600 con 0x7fedd0159dd0 2024-09-12T12:21:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.597+0000 7fedceffd640 1 -- 172.21.15.3:0/3949815232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc00ab9d0 con 0x7fedd0159dd0 2024-09-12T12:21:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- 172.21.15.3:0/3949815232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 msgr2=0x7fedd015d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 --2- 172.21.15.3:0/3949815232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015d2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009870 tx=0x7fedc001c310 comp rx=0 tx=0).stop 2024-09-12T12:21:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- 172.21.15.3:0/3949815232 shutdown_connections 2024-09-12T12:21:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 --2- 172.21.15.3:0/3949815232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 --2- 172.21.15.3:0/3949815232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015d2f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- 172.21.15.3:0/3949815232 >> 172.21.15.3:0/3949815232 conn(0x7fedd00b1d80 msgr2=0x7fedd00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- 172.21.15.3:0/3949815232 shutdown_connections 2024-09-12T12:21:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- 172.21.15.3:0/3949815232 wait complete. 2024-09-12T12:21:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 Processor -- start 2024-09-12T12:21:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- start start 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd0099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd006f5b0 0x7fedd015b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fedd015a1b0 con 0x7fedd0159dd0 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00ad9b0 con 0x7fedd006f5b0 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fedd00b2a40 con 0x7fedd006f150 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40668/0 (socket says 172.21.15.3:40668) 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 -- 172.21.15.3:0/665885945 learned_addr learned my addr 172.21.15.3:0/665885945 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedcf7fe640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd006f5b0 0x7fedd015b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedcffff640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd0099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 -- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 msgr2=0x7fedd0099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd0099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 -- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd006f5b0 msgr2=0x7fedd015b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd006f5b0 0x7fedd015b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 -- 172.21.15.3:0/665885945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd00b0b70 con 0x7fedd0159dd0 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd4f91640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015ba00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fedc800b970 tx=0x7fedc800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedccff9640 1 -- 172.21.15.3:0/665885945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8019070 con 0x7fedd0159dd0 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedccff9640 1 -- 172.21.15.3:0/665885945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc800cb40 con 0x7fedd0159dd0 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd009de90 con 0x7fedd0159dd0 2024-09-12T12:21:22.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedccff9640 1 -- 172.21.15.3:0/665885945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8014440 con 0x7fedd0159dd0 2024-09-12T12:21:22.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.601+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd00d0570 con 0x7fedd0159dd0 2024-09-12T12:21:22.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.605+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd00ad060 con 0x7fedd0159dd0 2024-09-12T12:21:22.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.605+0000 7fedccff9640 1 -- 172.21.15.3:0/665885945 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fedc801c070 con 0x7fedd0159dd0 2024-09-12T12:21:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.605+0000 7fedccff9640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedbc0678c0 0x7fedbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.605+0000 7fedccff9640 1 -- 172.21.15.3:0/665885945 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fedc8085630 con 0x7fedd0159dd0 2024-09-12T12:21:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.605+0000 7fedcffff640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedbc0678c0 0x7fedbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.605+0000 7fedcffff640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedbc0678c0 0x7fedbc069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedc00ad000 tx=0x7fedc00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.609+0000 7fedccff9640 1 -- 172.21.15.3:0/665885945 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fedc8056500 con 0x7fedd0159dd0 2024-09-12T12:21:22.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.769+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fedd006fae0 con 0x7fedd0159dd0 2024-09-12T12:21:22.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedccff9640 1 -- 172.21.15.3:0/665885945 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fedc8055e00 con 0x7fedd0159dd0 2024-09-12T12:21:22.774 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T12:21:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedbc0678c0 msgr2=0x7fedbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedbc0678c0 0x7fedbc069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedc00ad000 tx=0x7fedc00a6000 comp rx=0 tx=0).stop 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 msgr2=0x7fedd015ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015ba00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fedc800b970 tx=0x7fedc800be30 comp rx=0 tx=0).stop 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 shutdown_connections 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedbc0678c0 0x7fedbc069d80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedd0159dd0 0x7fedd015ba00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd006f5b0 0x7fedd015b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 --2- 172.21.15.3:0/665885945 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedd006f150 0x7fedd0099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 >> 172.21.15.3:0/665885945 conn(0x7fedd00b1d80 msgr2=0x7fedd00750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 shutdown_connections 2024-09-12T12:21:22.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.773+0000 7fedd69f4640 1 -- 172.21.15.3:0/665885945 wait complete. 2024-09-12T12:21:22.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-09-12T12:21:22.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 Processor -- start 2024-09-12T12:21:22.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 -- start start 2024-09-12T12:21:22.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07cc159bd0 0x7f07cc06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07cc159fb0 0x7f07cc15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc06f4a0 0x7f07cc0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f07cc06f010 con 0x7f07cc159fb0 2024-09-12T12:21:22.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f07cc06e400 con 0x7f07cc159bd0 2024-09-12T12:21:22.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07d122f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc06e6e0 con 0x7f07cc06f4a0 2024-09-12T12:21:22.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07cc159fb0 0x7f07cc15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f07cc159fb0 0x7f07cc15d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40684/0 (socket says 172.21.15.3:40684) 2024-09-12T12:21:22.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 -- 172.21.15.3:0/2630580320 learned_addr learned my addr 172.21.15.3:0/2630580320 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:22.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07cad9d640 1 --2- 172.21.15.3:0/2630580320 >> v2:172.21.15.142:3300/0 conn(0x7f07cc159bd0 0x7f07cc06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07cb59e640 1 --2- 172.21.15.3:0/2630580320 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc06f4a0 0x7f07cc0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 -- 172.21.15.3:0/2630580320 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc06f4a0 msgr2=0x7f07cc0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 --2- 172.21.15.3:0/2630580320 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc06f4a0 0x7f07cc0ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 -- 172.21.15.3:0/2630580320 >> v2:172.21.15.142:3300/0 conn(0x7f07cc159bd0 msgr2=0x7f07cc06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 --2- 172.21.15.3:0/2630580320 >> v2:172.21.15.142:3300/0 conn(0x7f07cc159bd0 0x7f07cc06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.909+0000 7f07ca59c640 1 -- 172.21.15.3:0/2630580320 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc06e9c0 con 0x7f07cc159fb0 2024-09-12T12:21:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07ca59c640 1 --2- 172.21.15.3:0/2630580320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc15d2d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f07cc071320 tx=0x7f07c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ee24dd9fb76a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07c9d9b640 1 -- 172.21.15.3:0/2630580320 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00b0070 con 0x7f07cc159fb0 2024-09-12T12:21:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07c9d9b640 1 -- 172.21.15.3:0/2630580320 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c00ab600 con 0x7f07cc159fb0 2024-09-12T12:21:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07c9d9b640 1 -- 172.21.15.3:0/2630580320 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00ab9d0 con 0x7f07cc159fb0 2024-09-12T12:21:22.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- 172.21.15.3:0/2630580320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 msgr2=0x7f07cc15d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 --2- 172.21.15.3:0/2630580320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc15d2d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f07cc071320 tx=0x7f07c001c520 comp rx=0 tx=0).stop 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- 172.21.15.3:0/2630580320 shutdown_connections 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 --2- 172.21.15.3:0/2630580320 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc06f4a0 0x7f07cc0ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 --2- 172.21.15.3:0/2630580320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc15d2d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 --2- 172.21.15.3:0/2630580320 >> v2:172.21.15.142:3300/0 conn(0x7f07cc159bd0 0x7f07cc06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- 172.21.15.3:0/2630580320 >> 172.21.15.3:0/2630580320 conn(0x7f07cc0b1d80 msgr2=0x7f07cc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- 172.21.15.3:0/2630580320 shutdown_connections 2024-09-12T12:21:22.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- 172.21.15.3:0/2630580320 wait complete. 2024-09-12T12:21:22.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 Processor -- start 2024-09-12T12:21:22.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- start start 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07cc06f4a0 0x7f07cc1455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc159bd0 0x7f07cc155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc15fe40 con 0x7f07cc159fb0 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc0b3aa0 con 0x7f07cc06f4a0 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07d122f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc15ffc0 con 0x7f07cc159bd0 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07cb59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07cb59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40696/0 (socket says 172.21.15.3:40696) 2024-09-12T12:21:22.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.913+0000 7f07cb59e640 1 -- 172.21.15.3:0/4266400715 learned_addr learned my addr 172.21.15.3:0/4266400715 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cad9d640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07cc06f4a0 0x7f07cc1455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07ca59c640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc159bd0 0x7f07cc155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cb59e640 1 -- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc159bd0 msgr2=0x7f07cc155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cb59e640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc159bd0 0x7f07cc155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cb59e640 1 -- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07cc06f4a0 msgr2=0x7f07cc1455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cb59e640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07cc06f4a0 0x7f07cc1455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cb59e640 1 -- 172.21.15.3:0/4266400715 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc159020 con 0x7f07cc159fb0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07ca59c640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc159bd0 0x7f07cc155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cb59e640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc155f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007e40 tx=0x7f07c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07b37fe640 1 -- 172.21.15.3:0/4266400715 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4014620 con 0x7f07cc159fb0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07b37fe640 1 -- 172.21.15.3:0/4266400715 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c4019070 con 0x7f07cc159fb0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07cc159300 con 0x7f07cc159fb0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07b37fe640 1 -- 172.21.15.3:0/4266400715 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4014a20 con 0x7f07cc159fb0 2024-09-12T12:21:22.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07cc0737f0 con 0x7f07cc159fb0 2024-09-12T12:21:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07cc0ad060 con 0x7f07cc159fb0 2024-09-12T12:21:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07b37fe640 1 -- 172.21.15.3:0/4266400715 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f07c4014bc0 con 0x7f07cc159fb0 2024-09-12T12:21:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07b37fe640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f07b80678f0 0x7f07b8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:22.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07b37fe640 1 -- 172.21.15.3:0/4266400715 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f07c40861c0 con 0x7f07cc159fb0 2024-09-12T12:21:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cad9d640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f07b80678f0 0x7f07b8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:22.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.917+0000 7f07cad9d640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f07b80678f0 0x7f07b8069db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00b950 tx=0x7f07bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:22.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:22.921+0000 7f07b37fe640 1 -- 172.21.15.3:0/4266400715 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f07c4057090 con 0x7f07cc159fb0 2024-09-12T12:21:23.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:23.137+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7f07cc0b2b00 con 0x7f07cc159fb0 2024-09-12T12:21:24.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.133+0000 7f07b37fe640 1 -- 172.21.15.3:0/4266400715 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v100) ==== 156+0+0 (secure 0 0 0) 0x7f07c4056990 con 0x7f07cc159fb0 2024-09-12T12:21:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f07b80678f0 msgr2=0x7f07b8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f07b80678f0 0x7f07b8069db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00b950 tx=0x7f07bc008040 comp rx=0 tx=0).stop 2024-09-12T12:21:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 msgr2=0x7f07cc155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc155f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007e40 tx=0x7f07c400e4b0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 shutdown_connections 2024-09-12T12:21:24.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f07b80678f0 0x7f07b8069db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f07cc159fb0 0x7f07cc155f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07cc159bd0 0x7f07cc155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 --2- 172.21.15.3:0/4266400715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07cc06f4a0 0x7f07cc1455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 >> 172.21.15.3:0/4266400715 conn(0x7f07cc0b1d80 msgr2=0x7f07cc15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 shutdown_connections 2024-09-12T12:21:24.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.137+0000 7f07d122f640 1 -- 172.21.15.3:0/4266400715 wait complete. 2024-09-12T12:21:24.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T12:21:24.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faceea65640 1 Processor -- start 2024-09-12T12:21:24.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faceea65640 1 -- start start 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faceea65640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7face8159dd0 0x7face815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faceea65640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face815d830 con 0x7face80ad770 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faceea65640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face806e0b0 con 0x7face806f150 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faceea65640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7face806e390 con 0x7face8159dd0 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7facec801640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7face8159dd0 0x7face815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faced002640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7face80ad770 0x7face80b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60430/0 (socket says 172.21.15.3:60430) 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7faced002640 1 -- 172.21.15.3:0/1168339162 learned_addr learned my addr 172.21.15.3:0/1168339162 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7face77fe640 1 -- 172.21.15.3:0/1168339162 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3281013479 0 0) 0x7face00014a0 con 0x7face80ad770 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7face77fe640 1 -- 172.21.15.3:0/1168339162 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facd8003530 con 0x7face80ad770 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7facec801640 1 -- 172.21.15.3:0/1168339162 >> v1:172.21.15.142:6789/0 conn(0x7face806f150 legacy=0x7face80ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7facec801640 1 -- 172.21.15.3:0/1168339162 >> v1:172.21.15.3:6789/0 conn(0x7face80ad770 legacy=0x7face80b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.293+0000 7facec801640 1 -- 172.21.15.3:0/1168339162 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face806e670 con 0x7face8159dd0 2024-09-12T12:21:24.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7facec801640 1 --2- 172.21.15.3:0/1168339162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8159dd0 0x7face815d2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097c0 tx=0x7facdc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f396819c6debd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:24.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face77fe640 1 -- 172.21.15.3:0/1168339162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0a7070 con 0x7face8159dd0 2024-09-12T12:21:24.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face77fe640 1 -- 172.21.15.3:0/1168339162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc0b0070 con 0x7face8159dd0 2024-09-12T12:21:24.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face77fe640 1 -- 172.21.15.3:0/1168339162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0ab890 con 0x7face8159dd0 2024-09-12T12:21:24.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- 172.21.15.3:0/1168339162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8159dd0 msgr2=0x7face815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 --2- 172.21.15.3:0/1168339162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8159dd0 0x7face815d2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097c0 tx=0x7facdc01c470 comp rx=0 tx=0).stop 2024-09-12T12:21:24.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- 172.21.15.3:0/1168339162 shutdown_connections 2024-09-12T12:21:24.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 --2- 172.21.15.3:0/1168339162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8159dd0 0x7face815d2f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- 172.21.15.3:0/1168339162 >> 172.21.15.3:0/1168339162 conn(0x7face80b1d80 msgr2=0x7face80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:24.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- 172.21.15.3:0/1168339162 shutdown_connections 2024-09-12T12:21:24.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- 172.21.15.3:0/1168339162 wait complete. 2024-09-12T12:21:24.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 Processor -- start 2024-09-12T12:21:24.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- start start 2024-09-12T12:21:24.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7face806f150 0x7face8165770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face80ad770 0x7face8166f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7face8159dd0 0x7face80aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7face815a1b0 con 0x7face80ad770 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7face806f940 con 0x7face8159dd0 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faceea65640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7face80b2a40 con 0x7face806f150 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face80ad770 0x7face8166f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face80ad770 0x7face8166f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40722/0 (socket says 172.21.15.3:40722) 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 -- 172.21.15.3:0/1292796521 learned_addr learned my addr 172.21.15.3:0/1292796521 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7facec801640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7face806f150 0x7face8165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faced002640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7face8159dd0 0x7face80aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 -- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7face806f150 msgr2=0x7face8165770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7face806f150 0x7face8165770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 -- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7face8159dd0 msgr2=0x7face80aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7face8159dd0 0x7face80aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 -- 172.21.15.3:0/1292796521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face8165e70 con 0x7face80ad770 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7faced002640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7face8159dd0 0x7face80aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:24.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7facec801640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7face806f150 0x7face8165770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:24.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.297+0000 7face7fff640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face80ad770 0x7face8166f10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7facd400c420 tx=0x7facd400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:24.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7face57fa640 1 -- 172.21.15.3:0/1292796521 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40052d0 con 0x7face80ad770 2024-09-12T12:21:24.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7face57fa640 1 -- 172.21.15.3:0/1292796521 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4005470 con 0x7face80ad770 2024-09-12T12:21:24.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7face57fa640 1 -- 172.21.15.3:0/1292796521 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4007260 con 0x7face80ad770 2024-09-12T12:21:24.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face8166150 con 0x7face80ad770 2024-09-12T12:21:24.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face8166710 con 0x7face80ad770 2024-09-12T12:21:24.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face815d2f0 con 0x7face80ad770 2024-09-12T12:21:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7face57fa640 1 -- 172.21.15.3:0/1292796521 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7facd4007400 con 0x7face80ad770 2024-09-12T12:21:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7face57fa640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faccc0678c0 0x7faccc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7face57fa640 1 -- 172.21.15.3:0/1292796521 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7facd4084e40 con 0x7face80ad770 2024-09-12T12:21:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7facec801640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faccc0678c0 0x7faccc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.301+0000 7facec801640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faccc0678c0 0x7faccc069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7facdc0af000 tx=0x7facdc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:24.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.305+0000 7face57fa640 1 -- 172.21.15.3:0/1292796521 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7facd4055d10 con 0x7face80ad770 2024-09-12T12:21:24.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.469+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7face8160660 con 0x7face80ad770 2024-09-12T12:21:24.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7face57fa640 1 -- 172.21.15.3:0/1292796521 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7facd4055610 con 0x7face80ad770 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faccc0678c0 msgr2=0x7faccc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faccc0678c0 0x7faccc069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7facdc0af000 tx=0x7facdc008040 comp rx=0 tx=0).stop 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face80ad770 msgr2=0x7face8166f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face80ad770 0x7face8166f10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7facd400c420 tx=0x7facd400c8e0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 shutdown_connections 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faccc0678c0 0x7faccc069d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7face8159dd0 0x7face80aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face80ad770 0x7face8166f10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 --2- 172.21.15.3:0/1292796521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7face806f150 0x7face8165770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.477+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 >> 172.21.15.3:0/1292796521 conn(0x7face80b1d80 msgr2=0x7face80ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.481+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 shutdown_connections 2024-09-12T12:21:24.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.481+0000 7faceea65640 1 -- 172.21.15.3:0/1292796521 wait complete. 2024-09-12T12:21:24.499 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:21:24.500 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:21:24.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:21:24.606 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:21:24.607 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:21:24.607 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:24.612 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:21:24.612 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:21:24.612 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:24.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:21:24.619 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:21:24.619 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:24.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:21:24.660 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:21:24.660 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:21:24.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:21:24.666 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T12:21:24.667 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea384a6640 1 Processor -- start 2024-09-12T12:21:24.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea384a6640 1 -- start start 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea384a6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 0x7fea300ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea384a6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fea300d5050 0x7fea300cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea384a6640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea300d31a0 con 0x7fea300cfa10 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea384a6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fea300b1060 con 0x7fea300d5050 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea384a6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea300cfe40 con 0x7fea3015e5d0 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea36a43640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea300cfa10 0x7fea30162280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60440/0 (socket says 172.21.15.3:60440) 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea36a43640 1 -- 172.21.15.3:0/3823866369 learned_addr learned my addr 172.21.15.3:0/3823866369 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35a41640 1 --2- 172.21.15.3:0/3823866369 >> v2:172.21.15.142:3300/0 conn(0x7fea300d5050 0x7fea300cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea36242640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 0x7fea300ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 706381697 0 0) 0x7fea300d31a0 con 0x7fea300cfa10 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea1c003530 con 0x7fea300cfa10 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3295471113 0 0) 0x7fea1c003530 con 0x7fea300cfa10 2024-09-12T12:21:24.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 msgr2=0x7fea300ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 0x7fea300ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 >> v2:172.21.15.142:3300/0 conn(0x7fea300d5050 msgr2=0x7fea300cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 --2- 172.21.15.3:0/3823866369 >> v2:172.21.15.142:3300/0 conn(0x7fea300d5050 0x7fea300cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300d3480 con 0x7fea300cfa10 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fea28001a00 con 0x7fea300cfa10 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 >> v1:172.21.15.3:6789/0 conn(0x7fea300cfa10 legacy=0x7fea30162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 0x7fea1c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c003f40 0x7fea1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea1c004d50 0x7fea1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea300b1060 con 0x7fea1c003f40 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea300cfe40 con 0x7fea1c004d50 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea1c00b9d0 con 0x7fea3015e5d0 2024-09-12T12:21:24.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35a41640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c003f40 0x7fea1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea36242640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 0x7fea1c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea36a43640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea1c004d50 0x7fea1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35a41640 1 -- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 msgr2=0x7fea1c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35a41640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea3015e5d0 0x7fea1c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35a41640 1 -- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea1c004d50 msgr2=0x7fea1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35a41640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea1c004d50 0x7fea1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea35a41640 1 -- 172.21.15.3:0/3823866369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1c003530 con 0x7fea1c003f40 2024-09-12T12:21:24.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.833+0000 7fea36a43640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea1c004d50 0x7fea1c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea35a41640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c003f40 0x7fea1c004810 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fea20009ae0 tx=0x7fea2001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d37159b01fbe59b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200b2070 con 0x7fea1c003f40 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea200a56a0 con 0x7fea1c003f40 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea35240640 1 -- 172.21.15.3:0/3823866369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200096b0 con 0x7fea1c003f40 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 -- 172.21.15.3:0/3823866369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c003f40 msgr2=0x7fea1c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 --2- 172.21.15.3:0/3823866369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea1c003f40 0x7fea1c004810 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fea20009ae0 tx=0x7fea2001ca50 comp rx=0 tx=0).stop 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea36a43640 1 -- 172.21.15.3:0/3823866369 reap_dead start 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 -- 172.21.15.3:0/3823866369 shutdown_connections 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 -- 172.21.15.3:0/3823866369 >> 172.21.15.3:0/3823866369 conn(0x7fea3008c000 msgr2=0x7fea300d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 -- 172.21.15.3:0/3823866369 shutdown_connections 2024-09-12T12:21:24.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 -- 172.21.15.3:0/3823866369 wait complete. 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 Processor -- start 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.837+0000 7fea384a6640 1 -- start start 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea1c003f40 0x7fea300cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea301e90d0 0x7fea30157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea300d1060 0x7fea300d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015eb30 con 0x7fea300d1060 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea3015e9b0 con 0x7fea301e90d0 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea300b2500 con 0x7fea1c003f40 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea300d1060 0x7fea300d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea300d1060 0x7fea300d14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40754/0 (socket says 172.21.15.3:40754) 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 -- 172.21.15.3:0/978765667 learned_addr learned my addr 172.21.15.3:0/978765667 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea35a41640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea301e90d0 0x7fea30157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36242640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea1c003f40 0x7fea300cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 -- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea1c003f40 msgr2=0x7fea300cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea1c003f40 0x7fea300cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 -- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea301e90d0 msgr2=0x7fea30157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea301e90d0 0x7fea30157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 -- 172.21.15.3:0/978765667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300cdaa0 con 0x7fea300d1060 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea35a41640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea301e90d0 0x7fea30157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36a43640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea300d1060 0x7fea300d14e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fea280013e0 tx=0x7fea28010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea26ffd640 1 -- 172.21.15.3:0/978765667 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2801a070 con 0x7fea300d1060 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea26ffd640 1 -- 172.21.15.3:0/978765667 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea28011040 con 0x7fea300d1060 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea26ffd640 1 -- 172.21.15.3:0/978765667 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2800ed50 con 0x7fea300d1060 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea300ccc60 con 0x7fea300d1060 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea300cd200 con 0x7fea300d1060 2024-09-12T12:21:24.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea300cf4d0 con 0x7fea300d1060 2024-09-12T12:21:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea26ffd640 1 -- 172.21.15.3:0/978765667 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fea280143a0 con 0x7fea300d1060 2024-09-12T12:21:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea26ffd640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea140678f0 0x7fea14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea26ffd640 1 -- 172.21.15.3:0/978765667 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fea28087da0 con 0x7fea300d1060 2024-09-12T12:21:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.841+0000 7fea36242640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea140678f0 0x7fea14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.845+0000 7fea36242640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea140678f0 0x7fea14069db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009950 tx=0x7fea2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:24.845+0000 7fea26ffd640 1 -- 172.21.15.3:0/978765667 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fea28010e60 con 0x7fea300d1060 2024-09-12T12:21:25.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.045+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea300ba490 con 0x7fea300d1060 2024-09-12T12:21:25.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.045+0000 7fea26ffd640 1 -- 172.21.15.3:0/978765667 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3908 (secure 0 0 0) 0x7fea28058be0 con 0x7fea300d1060 2024-09-12T12:21:25.049 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:25.049 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"btime":"2024-09-12T12:21:18:090022+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4484,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/925159726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":925159726},{"type":"v1","addr":"172.21.15.142:6836","nonce":925159726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2814906328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2814906328},{"type":"v1","addr":"172.21.15.142:6837","nonce":2814906328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4809,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4018428152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4018428152},{"type":"v1","addr":"172.21.15.3:6833","nonce":4018428152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5160,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2500194448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2500194448},{"type":"v1","addr":"172.21.15.3:6835","nonce":2500194448}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[]} 2024-09-12T12:21:25.049 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 16 2024-09-12T12:21:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea140678f0 msgr2=0x7fea14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea140678f0 0x7fea14069db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009950 tx=0x7fea2c008040 comp rx=0 tx=0).stop 2024-09-12T12:21:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea300d1060 msgr2=0x7fea300d14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea300d1060 0x7fea300d14e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fea280013e0 tx=0x7fea28010c10 comp rx=0 tx=0).stop 2024-09-12T12:21:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 shutdown_connections 2024-09-12T12:21:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea140678f0 0x7fea14069db0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:25.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea300d1060 0x7fea300d14e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea301e90d0 0x7fea30157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 --2- 172.21.15.3:0/978765667 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea1c003f40 0x7fea300cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 >> 172.21.15.3:0/978765667 conn(0x7fea3008c000 msgr2=0x7fea300b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 shutdown_connections 2024-09-12T12:21:25.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.049+0000 7fea384a6640 1 -- 172.21.15.3:0/978765667 wait complete. 2024-09-12T12:21:25.067 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:21:25.067 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T12:21:25.068 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:21:25.068 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:21:25.068 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:21:25.068 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T12:21:25.068 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:21:25.068 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:21:25.069 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:21:25.069 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T12:21:25.069 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:21:25.069 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:21:25.069 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:21:25.069 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T12:21:25.070 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:21:25.070 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:21:25.078 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:21:25.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:21:25.079 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:21:25.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:21:25.083 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:21:25.083 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:21:25.084 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:21:25.084 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:21:25.122 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:21:25.123 DEBUG:teuthology.parallel:result is None 2024-09-12T12:21:25.124 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:21:25.124 DEBUG:teuthology.parallel:result is None 2024-09-12T12:21:25.126 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:21:25.126 DEBUG:teuthology.parallel:result is None 2024-09-12T12:21:25.126 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:21:25.127 DEBUG:teuthology.parallel:result is None 2024-09-12T12:21:25.127 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T12:21:25.187 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:21:25.195 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:21:25.198 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:21:25.199 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:21:25.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.245+0000 7f0701cfc640 1 Processor -- start 2024-09-12T12:21:25.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.245+0000 7f0701cfc640 1 -- start start 2024-09-12T12:21:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.245+0000 7f0701cfc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b3a30 0x7f06fc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:25.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.245+0000 7f0701cfc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06fc06be60 con 0x7f06fc0b0de0 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.245+0000 7f0701cfc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06fc06c140 con 0x7f06fc0b2010 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.245+0000 7f0701cfc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06fc0b2d60 con 0x7f06fc0b3a30 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06fbfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f06fc0b0de0 0x7f06fc0b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60450/0 (socket says 172.21.15.3:60450) 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06fbfff640 1 -- 172.21.15.3:0/2802035155 learned_addr learned my addr 172.21.15.3:0/2802035155 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06f3fff640 1 --2- 172.21.15.3:0/2802035155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b3a30 0x7f06fc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1613080029 0 0) 0x7f06fc06be60 con 0x7f06fc0b0de0 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06e8003530 con 0x7f06fc0b0de0 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801406489 0 0) 0x7f06fc06c140 con 0x7f06fc0b2010 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06fc06be60 con 0x7f06fc0b2010 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1871617743 0 0) 0x7f06e8003530 con 0x7f06fc0b0de0 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06f3fff640 1 -- 172.21.15.3:0/2802035155 >> v1:172.21.15.142:6789/0 conn(0x7f06fc0b2010 legacy=0x7f06fc0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06f3fff640 1 -- 172.21.15.3:0/2802035155 >> v1:172.21.15.3:6789/0 conn(0x7f06fc0b0de0 legacy=0x7f06fc0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:25.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06f3fff640 1 -- 172.21.15.3:0/2802035155 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc06c420 con 0x7f06fc0b3a30 2024-09-12T12:21:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06f3fff640 1 --2- 172.21.15.3:0/2802035155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b3a30 0x7f06fc0cd6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06e40097f0 tx=0x7f06e401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9ebfbea205ad6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40b0070 con 0x7f06fc0b3a30 2024-09-12T12:21:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e40ab400 con 0x7f06fc0b3a30 2024-09-12T12:21:25.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f06faffd640 1 -- 172.21.15.3:0/2802035155 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4001a00 con 0x7f06fc0b3a30 2024-09-12T12:21:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f0701cfc640 1 -- 172.21.15.3:0/2802035155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b3a30 msgr2=0x7f06fc0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f0701cfc640 1 --2- 172.21.15.3:0/2802035155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b3a30 0x7f06fc0cd6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06e40097f0 tx=0x7f06e401c370 comp rx=0 tx=0).stop 2024-09-12T12:21:25.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f0701cfc640 1 -- 172.21.15.3:0/2802035155 shutdown_connections 2024-09-12T12:21:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f0701cfc640 1 --2- 172.21.15.3:0/2802035155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b3a30 0x7f06fc0cd6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f0701cfc640 1 -- 172.21.15.3:0/2802035155 >> 172.21.15.3:0/2802035155 conn(0x7f06fc0b2f10 msgr2=0x7f06fc0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f0701cfc640 1 -- 172.21.15.3:0/2802035155 shutdown_connections 2024-09-12T12:21:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.249+0000 7f0701cfc640 1 -- 172.21.15.3:0/2802035155 wait complete. 2024-09-12T12:21:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 Processor -- start 2024-09-12T12:21:25.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 -- start start 2024-09-12T12:21:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc0b0de0 0x7f06fc13d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b2010 0x7f06fc155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06fc0b3a30 0x7f06fc138e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc0ae370 con 0x7f06fc0b3a30 2024-09-12T12:21:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc0ae1f0 con 0x7f06fc0b0de0 2024-09-12T12:21:25.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06fc16a6d0 con 0x7f06fc0b2010 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06fc0b3a30 0x7f06fc138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06fc0b3a30 0x7f06fc138e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40802/0 (socket says 172.21.15.3:40802) 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 -- 172.21.15.3:0/856827692 learned_addr learned my addr 172.21.15.3:0/856827692 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fb7fe640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc0b0de0 0x7f06fc13d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06f3fff640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b2010 0x7f06fc155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 -- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b2010 msgr2=0x7f06fc155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b2010 0x7f06fc155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 -- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc0b0de0 msgr2=0x7f06fc13d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc0b0de0 0x7f06fc13d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 -- 172.21.15.3:0/856827692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc13d980 con 0x7f06fc0b3a30 2024-09-12T12:21:25.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06fbfff640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06fc0b3a30 0x7f06fc138e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f06f400cc30 tx=0x7f06f400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:25.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06f8ff9640 1 -- 172.21.15.3:0/856827692 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4017070 con 0x7f06fc0b3a30 2024-09-12T12:21:25.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06f8ff9640 1 -- 172.21.15.3:0/856827692 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f400ce40 con 0x7f06fc0b3a30 2024-09-12T12:21:25.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc1561e0 con 0x7f06fc0b3a30 2024-09-12T12:21:25.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06f8ff9640 1 -- 172.21.15.3:0/856827692 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4012640 con 0x7f06fc0b3a30 2024-09-12T12:21:25.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc06f620 con 0x7f06fc0b3a30 2024-09-12T12:21:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc0cd6c0 con 0x7f06fc0b3a30 2024-09-12T12:21:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.253+0000 7f06f8ff9640 1 -- 172.21.15.3:0/856827692 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06f401a070 con 0x7f06fc0b3a30 2024-09-12T12:21:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.257+0000 7f06f8ff9640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06dc0678c0 0x7f06dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.257+0000 7f06f8ff9640 1 -- 172.21.15.3:0/856827692 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f06f4085980 con 0x7f06fc0b3a30 2024-09-12T12:21:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.257+0000 7f06fb7fe640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06dc0678c0 0x7f06dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:25.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.257+0000 7f06fb7fe640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06dc0678c0 0x7f06dc069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0011f0 tx=0x7f06ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:25.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.261+0000 7f06f8ff9640 1 -- 172.21.15.3:0/856827692 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f06f4056720 con 0x7f06fc0b3a30 2024-09-12T12:21:25.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:25.437+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f06fc0b3540 con 0x7f06fc0b3a30 2024-09-12T12:21:26.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.157+0000 7f06f8ff9640 1 -- 172.21.15.3:0/856827692 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) ==== 97+0+0 (secure 0 0 0) 0x7f06f4056020 con 0x7f06fc0b3a30 2024-09-12T12:21:26.160 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-12T12:21:26.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06dc0678c0 msgr2=0x7f06dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06dc0678c0 0x7f06dc069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0011f0 tx=0x7f06ec009040 comp rx=0 tx=0).stop 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06fc0b3a30 msgr2=0x7f06fc138e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06fc0b3a30 0x7f06fc138e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f06f400cc30 tx=0x7f06f400a430 comp rx=0 tx=0).stop 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 shutdown_connections 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06dc0678c0 0x7f06dc069d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06fc0b3a30 0x7f06fc138e70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06fc0b2010 0x7f06fc155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 --2- 172.21.15.3:0/856827692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc0b0de0 0x7f06fc13d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 >> 172.21.15.3:0/856827692 conn(0x7f06fc0b2f10 msgr2=0x7f06fc0750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 shutdown_connections 2024-09-12T12:21:26.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.161+0000 7f0701cfc640 1 -- 172.21.15.3:0/856827692 wait complete. 2024-09-12T12:21:26.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T12:21:26.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9053756640 1 Processor -- start 2024-09-12T12:21:26.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9053756640 1 -- start start 2024-09-12T12:21:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9053756640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 0x7f904c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9053756640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c06fbb0 con 0x7f904c15d2d0 2024-09-12T12:21:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9053756640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c06e400 con 0x7f904c159fb0 2024-09-12T12:21:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9053756640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f904c06f010 con 0x7f904c159bd0 2024-09-12T12:21:26.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9051cf3640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f904c15d2d0 0x7f904c0ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60464/0 (socket says 172.21.15.3:60464) 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9051cf3640 1 -- 172.21.15.3:0/1124639198 learned_addr learned my addr 172.21.15.3:0/1124639198 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f90514f2640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 0x7f904c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782136808 0 0) 0x7f904c06fbb0 con 0x7f904c15d2d0 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9038003530 con 0x7f904c15d2d0 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 37265028 0 0) 0x7f9038003530 con 0x7f904c15d2d0 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 msgr2=0x7f904c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 0x7f904c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 >> v1:172.21.15.142:6789/0 conn(0x7f904c159fb0 legacy=0x7f904c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c06e6e0 con 0x7f904c15d2d0 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f9044001a00 con 0x7f904c15d2d0 2024-09-12T12:21:26.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 >> v1:172.21.15.3:6789/0 conn(0x7f904c15d2d0 legacy=0x7f904c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380039f0 0x7f9038003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 0x7f9038004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9038004d50 0x7f90380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f904c06f010 con 0x7f90380039f0 2024-09-12T12:21:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f903800ba80 con 0x7f9038004d50 2024-09-12T12:21:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f903800bc80 con 0x7f904c159bd0 2024-09-12T12:21:26.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f90514f2640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380039f0 0x7f9038003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f9050cf1640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 0x7f9038004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f90514f2640 1 -- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 msgr2=0x7f9038004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f90514f2640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f904c159bd0 0x7f9038004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f90514f2640 1 -- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9038004d50 msgr2=0x7f90380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f90514f2640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9038004d50 0x7f90380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.305+0000 7f90514f2640 1 -- 172.21.15.3:0/1124639198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9038003530 con 0x7f90380039f0 2024-09-12T12:21:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f90514f2640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380039f0 0x7f9038003e30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9040009a90 tx=0x7f904001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaa67bcf05e2bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400b2070 con 0x7f90380039f0 2024-09-12T12:21:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90400a55d0 con 0x7f90380039f0 2024-09-12T12:21:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f903ffff640 1 -- 172.21.15.3:0/1124639198 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040009690 con 0x7f90380039f0 2024-09-12T12:21:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 -- 172.21.15.3:0/1124639198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380039f0 msgr2=0x7f9038003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 --2- 172.21.15.3:0/1124639198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380039f0 0x7f9038003e30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9040009a90 tx=0x7f904001ca00 comp rx=0 tx=0).stop 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9051cf3640 1 -- 172.21.15.3:0/1124639198 reap_dead start 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 -- 172.21.15.3:0/1124639198 shutdown_connections 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 -- 172.21.15.3:0/1124639198 >> 172.21.15.3:0/1124639198 conn(0x7f904c0b1d80 msgr2=0x7f904c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 -- 172.21.15.3:0/1124639198 shutdown_connections 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 -- 172.21.15.3:0/1124639198 wait complete. 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 Processor -- start 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.309+0000 7f9053756640 1 -- start start 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380039f0 0x7f904c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f904c073310 0x7f904c1f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904c15e5a0 0x7f904c15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0b3a10 con 0x7f904c15e5a0 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0b3890 con 0x7f904c073310 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0ad770 con 0x7f90380039f0 2024-09-12T12:21:26.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904c15e5a0 0x7f904c15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904c15e5a0 0x7f904c15ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40822/0 (socket says 172.21.15.3:40822) 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 -- 172.21.15.3:0/577064319 learned_addr learned my addr 172.21.15.3:0/577064319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f90514f2640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380039f0 0x7f904c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9050cf1640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f904c073310 0x7f904c1f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 -- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380039f0 msgr2=0x7f904c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380039f0 0x7f904c0cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 -- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f904c073310 msgr2=0x7f904c1f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f904c073310 0x7f904c1f9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 -- 172.21.15.3:0/577064319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c071ff0 con 0x7f904c15e5a0 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9051cf3640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904c15e5a0 0x7f904c15ea20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f904c06ef80 tx=0x7f904400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f903dffb640 1 -- 172.21.15.3:0/577064319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904400a670 con 0x7f904c15e5a0 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f903dffb640 1 -- 172.21.15.3:0/577064319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904400a810 con 0x7f904c15e5a0 2024-09-12T12:21:26.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c0722d0 con 0x7f904c15e5a0 2024-09-12T12:21:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f903dffb640 1 -- 172.21.15.3:0/577064319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904400abc0 con 0x7f904c15e5a0 2024-09-12T12:21:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c1fa180 con 0x7f904c15e5a0 2024-09-12T12:21:26.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c06dd40 con 0x7f904c15e5a0 2024-09-12T12:21:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f903dffb640 1 -- 172.21.15.3:0/577064319 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9044007110 con 0x7f904c15e5a0 2024-09-12T12:21:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f903dffb640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90300678c0 0x7f9030069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f903dffb640 1 -- 172.21.15.3:0/577064319 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f90440852c0 con 0x7f904c15e5a0 2024-09-12T12:21:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.313+0000 7f90514f2640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90300678c0 0x7f9030069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.317+0000 7f90514f2640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90300678c0 0x7f9030069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90400af000 tx=0x7f90400046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:26.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.317+0000 7f903dffb640 1 -- 172.21.15.3:0/577064319 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f90440561e0 con 0x7f904c15e5a0 2024-09-12T12:21:26.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.541+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f904c0b2b00 con 0x7f904c15e5a0 2024-09-12T12:21:26.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.541+0000 7f903dffb640 1 -- 172.21.15.3:0/577064319 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f9044055ae0 con 0x7f904c15e5a0 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg==", 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:21:26.545 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA==", 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.546 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg==", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw==", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.547 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w==", 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.548 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA==", 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.549 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw==", 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg==", 2024-09-12T12:21:26.550 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA==", 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ==", 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ==", 2024-09-12T12:21:26.552 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA==", 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.553 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA==", 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw==", 2024-09-12T12:21:26.554 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw==", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:21:26.555 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA==", 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.556 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKNVOAxAAMmejHiBys/ZYsGdB0oiroQ==", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA==", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:21:26.557 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg==", 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:21:26.558 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:21:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90300678c0 msgr2=0x7f9030069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90300678c0 0x7f9030069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90400af000 tx=0x7f90400046d0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904c15e5a0 msgr2=0x7f904c15ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904c15e5a0 0x7f904c15ea20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f904c06ef80 tx=0x7f904400cf00 comp rx=0 tx=0).stop 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 shutdown_connections 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f90300678c0 0x7f9030069d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f904c15e5a0 0x7f904c15ea20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f904c073310 0x7f904c1f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 --2- 172.21.15.3:0/577064319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380039f0 0x7f904c0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 >> 172.21.15.3:0/577064319 conn(0x7f904c0b1d80 msgr2=0x7f904c0750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 shutdown_connections 2024-09-12T12:21:26.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.545+0000 7f9053756640 1 -- 172.21.15.3:0/577064319 wait complete. 2024-09-12T12:21:26.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T12:21:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.685+0000 7f3d1b3fc640 1 Processor -- start 2024-09-12T12:21:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.685+0000 7f3d1b3fc640 1 -- start start 2024-09-12T12:21:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d1b3fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d1415b2d0 0x7f3d140b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d1b3fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d1406f6e0 0x7f3d14072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d1b3fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3d140cc300 0x7f3d14073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d1b3fc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3d140ae920 con 0x7f3d1415b2d0 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d1b3fc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3d1415f150 con 0x7f3d140cc300 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d1b3fc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1415b6b0 con 0x7f3d1406f6e0 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d1415b2d0 0x7f3d140b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3d1415b2d0 0x7f3d140b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40826/0 (socket says 172.21.15.3:40826) 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 -- 172.21.15.3:0/2419218328 learned_addr learned my addr 172.21.15.3:0/2419218328 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19999640 1 --2- 172.21.15.3:0/2419218328 >> v2:172.21.15.142:3300/0 conn(0x7f3d140cc300 0x7f3d14073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d18997640 1 --2- 172.21.15.3:0/2419218328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d1406f6e0 0x7f3d14072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 -- 172.21.15.3:0/2419218328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d1406f6e0 msgr2=0x7f3d14072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 --2- 172.21.15.3:0/2419218328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d1406f6e0 0x7f3d14072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 -- 172.21.15.3:0/2419218328 >> v2:172.21.15.142:3300/0 conn(0x7f3d140cc300 msgr2=0x7f3d14073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 --2- 172.21.15.3:0/2419218328 >> v2:172.21.15.142:3300/0 conn(0x7f3d140cc300 0x7f3d14073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 -- 172.21.15.3:0/2419218328 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1415ee70 con 0x7f3d1415b2d0 2024-09-12T12:21:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d19198640 1 --2- 172.21.15.3:0/2419218328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415b2d0 0x7f3d140b3080 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d14072e40 tx=0x7f3d1001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa85f9c1d98b342f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d0bfff640 1 -- 172.21.15.3:0/2419218328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100aa610 con 0x7f3d1415b2d0 2024-09-12T12:21:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d0bfff640 1 -- 172.21.15.3:0/2419218328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d100af070 con 0x7f3d1415b2d0 2024-09-12T12:21:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.689+0000 7f3d0bfff640 1 -- 172.21.15.3:0/2419218328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100aa9d0 con 0x7f3d1415b2d0 2024-09-12T12:21:26.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/2419218328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415b2d0 msgr2=0x7f3d140b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/2419218328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415b2d0 0x7f3d140b3080 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d14072e40 tx=0x7f3d1001c4f0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/2419218328 shutdown_connections 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/2419218328 >> v2:172.21.15.142:3300/0 conn(0x7f3d140cc300 0x7f3d14073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/2419218328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d1406f6e0 0x7f3d14072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/2419218328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1415b2d0 0x7f3d140b3080 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/2419218328 >> 172.21.15.3:0/2419218328 conn(0x7f3d140b1d80 msgr2=0x7f3d140b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/2419218328 shutdown_connections 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/2419218328 wait complete. 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 Processor -- start 2024-09-12T12:21:26.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- start start 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1406f6e0 0x7f3d1414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d140cc300 0x7f3d14157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d1415b2d0 0x7f3d141575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3d140b35c0 con 0x7f3d1406f6e0 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1406d8d0 con 0x7f3d1415b2d0 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3d140b3740 con 0x7f3d140cc300 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1406f6e0 0x7f3d1414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1406f6e0 0x7f3d1414f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40842/0 (socket says 172.21.15.3:40842) 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 -- 172.21.15.3:0/3602100883 learned_addr learned my addr 172.21.15.3:0/3602100883 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d18997640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d140cc300 0x7f3d14157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19999640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d1415b2d0 0x7f3d141575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 -- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d140cc300 msgr2=0x7f3d14157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d140cc300 0x7f3d14157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 -- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d1415b2d0 msgr2=0x7f3d141575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d1415b2d0 0x7f3d141575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 -- 172.21.15.3:0/3602100883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d14158e80 con 0x7f3d1406f6e0 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d18997640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d140cc300 0x7f3d14157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19999640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d1415b2d0 0x7f3d141575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d19198640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1406f6e0 0x7f3d1414f260 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d1001aba0 tx=0x7f3d1001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d09ffb640 1 -- 172.21.15.3:0/3602100883 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100bd5f0 con 0x7f3d1406f6e0 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d09ffb640 1 -- 172.21.15.3:0/3602100883 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d100bd790 con 0x7f3d1406f6e0 2024-09-12T12:21:26.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d09ffb640 1 -- 172.21.15.3:0/3602100883 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100bdb80 con 0x7f3d1406f6e0 2024-09-12T12:21:26.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d14159160 con 0x7f3d1406f6e0 2024-09-12T12:21:26.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.693+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d14157b00 con 0x7f3d1406f6e0 2024-09-12T12:21:26.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.697+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d140af5c0 con 0x7f3d1406f6e0 2024-09-12T12:21:26.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.697+0000 7f3d09ffb640 1 -- 172.21.15.3:0/3602100883 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d100b33f0 con 0x7f3d1406f6e0 2024-09-12T12:21:26.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.697+0000 7f3d09ffb640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3cf80678c0 0x7f3cf8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:26.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.697+0000 7f3d09ffb640 1 -- 172.21.15.3:0/3602100883 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f3d10120420 con 0x7f3d1406f6e0 2024-09-12T12:21:26.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.697+0000 7f3d18997640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3cf80678c0 0x7f3cf8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:26.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.697+0000 7f3d18997640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3cf80678c0 0x7f3cf8069d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d14165560 tx=0x7f3d04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:26.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.701+0000 7f3d09ffb640 1 -- 172.21.15.3:0/3602100883 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3d100c3060 con 0x7f3d1406f6e0 2024-09-12T12:21:26.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.869+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3d1406cd80 con 0x7f3d1406f6e0 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.869+0000 7f3d09ffb640 1 -- 172.21.15.3:0/3602100883 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) ==== 84+0+65 (secure 0 0 0) 0x7f3d100a7070 con 0x7f3d1406f6e0 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:26.873 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T12:21:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3cf80678c0 msgr2=0x7f3cf8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3cf80678c0 0x7f3cf8069d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d14165560 tx=0x7f3d04008040 comp rx=0 tx=0).stop 2024-09-12T12:21:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1406f6e0 msgr2=0x7f3d1414f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:26.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1406f6e0 0x7f3d1414f260 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d1001aba0 tx=0x7f3d1001ad30 comp rx=0 tx=0).stop 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 shutdown_connections 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3cf80678c0 0x7f3cf8069d80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d1415b2d0 0x7f3d141575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3d140cc300 0x7f3d14157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 --2- 172.21.15.3:0/3602100883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d1406f6e0 0x7f3d1414f260 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 >> 172.21.15.3:0/3602100883 conn(0x7f3d140b1d80 msgr2=0x7f3d1415db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 shutdown_connections 2024-09-12T12:21:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:26.873+0000 7f3d1b3fc640 1 -- 172.21.15.3:0/3602100883 wait complete. 2024-09-12T12:21:26.892 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:21:26.893 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:21:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.013+0000 7f525b07e640 1 Processor -- start 2024-09-12T12:21:27.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.013+0000 7f525b07e640 1 -- start start 2024-09-12T12:21:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f525b07e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f525406f150 0x7f52540ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f525b07e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f525406f5b0 0x7f52540ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f525b07e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f525406e0b0 con 0x7f5254159dd0 2024-09-12T12:21:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f525b07e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f525406e390 con 0x7f525406f5b0 2024-09-12T12:21:27.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f525b07e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f525406e670 con 0x7f525406f150 2024-09-12T12:21:27.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f525961b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f525406f5b0 0x7f52540ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:27.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5253fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f525406f150 0x7f52540ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:27.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5258e1a640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f5254159dd0 0x7f525415d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:53120/0 (socket says 172.21.15.3:53120) 2024-09-12T12:21:27.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5258e1a640 1 -- 172.21.15.3:0/1348492466 learned_addr learned my addr 172.21.15.3:0/1348492466 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:27.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5253fff640 1 -- 172.21.15.3:0/1348492466 >> v2:172.21.15.142:3300/0 conn(0x7f525406f5b0 msgr2=0x7f52540ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:27.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5253fff640 1 --2- 172.21.15.3:0/1348492466 >> v2:172.21.15.142:3300/0 conn(0x7f525406f5b0 0x7f52540ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:27.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5253fff640 1 -- 172.21.15.3:0/1348492466 >> v1:172.21.15.142:6789/0 conn(0x7f5254159dd0 legacy=0x7f525415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:27.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5253fff640 1 -- 172.21.15.3:0/1348492466 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525406e950 con 0x7f525406f150 2024-09-12T12:21:27.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f525961b640 1 --2- 172.21.15.3:0/1348492466 >> v2:172.21.15.142:3300/0 conn(0x7f525406f5b0 0x7f52540ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:27.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.017+0000 7f5253fff640 1 --2- 172.21.15.3:0/1348492466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f525406f150 0x7f52540ad060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097c0 tx=0x7f523c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce92a266bfbb830 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:27.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f52537fe640 1 -- 172.21.15.3:0/1348492466 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c0aa5f0 con 0x7f525406f150 2024-09-12T12:21:27.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f52537fe640 1 -- 172.21.15.3:0/1348492466 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523c0af070 con 0x7f525406f150 2024-09-12T12:21:27.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f52537fe640 1 -- 172.21.15.3:0/1348492466 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c0aa9b0 con 0x7f525406f150 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- 172.21.15.3:0/1348492466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f525406f150 msgr2=0x7f52540ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 --2- 172.21.15.3:0/1348492466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f525406f150 0x7f52540ad060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097c0 tx=0x7f523c01c470 comp rx=0 tx=0).stop 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- 172.21.15.3:0/1348492466 shutdown_connections 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 --2- 172.21.15.3:0/1348492466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f525406f150 0x7f52540ad060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 --2- 172.21.15.3:0/1348492466 >> v2:172.21.15.142:3300/0 conn(0x7f525406f5b0 0x7f52540ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- 172.21.15.3:0/1348492466 >> 172.21.15.3:0/1348492466 conn(0x7f52540b1d80 msgr2=0x7f52540b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- 172.21.15.3:0/1348492466 shutdown_connections 2024-09-12T12:21:27.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- 172.21.15.3:0/1348492466 wait complete. 2024-09-12T12:21:27.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 Processor -- start 2024-09-12T12:21:27.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- start start 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525406f150 0x7f5254144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525406f5b0 0x7f5254155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5254159dd0 0x7f525414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52540b2bc0 con 0x7f525406f5b0 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52540b2a40 con 0x7f525406f150 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525b07e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5254169200 con 0x7f5254159dd0 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525406f5b0 0x7f5254155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525406f5b0 0x7f5254155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40844/0 (socket says 172.21.15.3:40844) 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 -- 172.21.15.3:0/4161493352 learned_addr learned my addr 172.21.15.3:0/4161493352 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f525961b640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5254159dd0 0x7f525414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5258e1a640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525406f150 0x7f5254144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:27.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 -- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5254159dd0 msgr2=0x7f525414ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5254159dd0 0x7f525414ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 -- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525406f150 msgr2=0x7f5254144b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525406f150 0x7f5254144b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 -- 172.21.15.3:0/4161493352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52541451c0 con 0x7f525406f5b0 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.021+0000 7f5253fff640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525406f5b0 0x7f5254155f30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097c0 tx=0x7f523c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f52517fa640 1 -- 172.21.15.3:0/4161493352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c0be610 con 0x7f525406f5b0 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f52517fa640 1 -- 172.21.15.3:0/4161493352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523c0be7b0 con 0x7f525406f5b0 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5254145460 con 0x7f525406f5b0 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f52517fa640 1 -- 172.21.15.3:0/4161493352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c0beba0 con 0x7f525406f5b0 2024-09-12T12:21:27.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5254156470 con 0x7f525406f5b0 2024-09-12T12:21:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5254145910 con 0x7f525406f5b0 2024-09-12T12:21:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f52517fa640 1 -- 172.21.15.3:0/4161493352 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f523c0bc020 con 0x7f525406f5b0 2024-09-12T12:21:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f52517fa640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52340678f0 0x7f5234069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:27.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f52517fa640 1 -- 172.21.15.3:0/4161493352 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f523c120040 con 0x7f525406f5b0 2024-09-12T12:21:27.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f5258e1a640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52340678f0 0x7f5234069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:27.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.025+0000 7f5258e1a640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52340678f0 0x7f5234069db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5244000f70 tx=0x7f5244009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:27.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.029+0000 7f52517fa640 1 -- 172.21.15.3:0/4161493352 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f523c0c5030 con 0x7f525406f5b0 2024-09-12T12:21:27.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:27.193+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f525415d2f0 con 0x7f525406f5b0 2024-09-12T12:21:28.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.161+0000 7f52517fa640 1 -- 172.21.15.3:0/4161493352 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) ==== 138+0+0 (secure 0 0 0) 0x7f523c016090 con 0x7f525406f5b0 2024-09-12T12:21:28.164 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52340678f0 msgr2=0x7f5234069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52340678f0 0x7f5234069db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5244000f70 tx=0x7f5244009040 comp rx=0 tx=0).stop 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525406f5b0 msgr2=0x7f5254155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525406f5b0 0x7f5254155f30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097c0 tx=0x7f523c01acb0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 shutdown_connections 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52340678f0 0x7f5234069db0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5254159dd0 0x7f525414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f525406f5b0 0x7f5254155f30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 --2- 172.21.15.3:0/4161493352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f525406f150 0x7f5254144b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 >> 172.21.15.3:0/4161493352 conn(0x7f52540b1d80 msgr2=0x7f5254075190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:28.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 shutdown_connections 2024-09-12T12:21:28.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.165+0000 7f525b07e640 1 -- 172.21.15.3:0/4161493352 wait complete. 2024-09-12T12:21:28.196 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:21:28.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 Processor -- start 2024-09-12T12:21:28.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 -- start start 2024-09-12T12:21:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffbd0159dd0 0x7ffbd015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:28.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffbd006f150 0x7ffbd00ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd006f5b0 0x7ffbd00ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd015d830 con 0x7ffbd0159dd0 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ffbd006e0b0 con 0x7ffbd006f150 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd6e02640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd006e390 con 0x7ffbd006f5b0 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd4b9e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffbd0159dd0 0x7ffbd015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd4b9e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ffbd0159dd0 0x7ffbd015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47084/0 (socket says 172.21.15.3:47084) 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.329+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/4156253083 learned_addr learned my addr 172.21.15.3:0/4156253083 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd539f640 1 --2- 172.21.15.3:0/4156253083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd006f5b0 0x7ffbd00ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:28.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbc7fff640 1 --2- 172.21.15.3:0/4156253083 >> v2:172.21.15.142:3300/0 conn(0x7ffbd006f150 0x7ffbd00ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/4156253083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd006f5b0 msgr2=0x7ffbd00ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd4b9e640 1 --2- 172.21.15.3:0/4156253083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd006f5b0 0x7ffbd00ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/4156253083 >> v2:172.21.15.142:3300/0 conn(0x7ffbd006f150 msgr2=0x7ffbd00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd4b9e640 1 --2- 172.21.15.3:0/4156253083 >> v2:172.21.15.142:3300/0 conn(0x7ffbd006f150 0x7ffbd00ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/4156253083 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd006e670 con 0x7ffbd0159dd0 2024-09-12T12:21:28.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbc7fff640 1 --2- 172.21.15.3:0/4156253083 >> v2:172.21.15.142:3300/0 conn(0x7ffbd006f150 0x7ffbd00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd4b9e640 1 --2- 172.21.15.3:0/4156253083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd0159dd0 0x7ffbd015d2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0071c10 tx=0x7ffbcc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a38e728b2d8168 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbc77fe640 1 -- 172.21.15.3:0/4156253083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b1070 con 0x7ffbd0159dd0 2024-09-12T12:21:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbc77fe640 1 -- 172.21.15.3:0/4156253083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0ac620 con 0x7ffbd0159dd0 2024-09-12T12:21:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbc77fe640 1 -- 172.21.15.3:0/4156253083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0ac9f0 con 0x7ffbd0159dd0 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd6e02640 1 -- 172.21.15.3:0/4156253083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd0159dd0 msgr2=0x7ffbd015d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/4156253083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd0159dd0 0x7ffbd015d2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0071c10 tx=0x7ffbcc01c4f0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd6e02640 1 -- 172.21.15.3:0/4156253083 shutdown_connections 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/4156253083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd006f5b0 0x7ffbd00ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/4156253083 >> v2:172.21.15.142:3300/0 conn(0x7ffbd006f150 0x7ffbd00ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/4156253083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd0159dd0 0x7ffbd015d2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.333+0000 7ffbd6e02640 1 -- 172.21.15.3:0/4156253083 >> 172.21.15.3:0/4156253083 conn(0x7ffbd00b1d80 msgr2=0x7ffbd00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- 172.21.15.3:0/4156253083 shutdown_connections 2024-09-12T12:21:28.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- 172.21.15.3:0/4156253083 wait complete. 2024-09-12T12:21:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 Processor -- start 2024-09-12T12:21:28.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- start start 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd006f150 0x7ffbd0166330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbd006f5b0 0x7ffbd0165360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd0159dd0 0x7ffbd00750d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd00b2a40 con 0x7ffbd006f150 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd015a1b0 con 0x7ffbd006f5b0 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd00b2bc0 con 0x7ffbd0159dd0 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd006f150 0x7ffbd0166330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd006f150 0x7ffbd0166330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47100/0 (socket says 172.21.15.3:47100) 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/1797554010 learned_addr learned my addr 172.21.15.3:0/1797554010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd539f640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd0159dd0 0x7ffbd00750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbc7fff640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbd006f5b0 0x7ffbd0165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd0159dd0 msgr2=0x7ffbd00750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd0159dd0 0x7ffbd00750d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbd006f5b0 msgr2=0x7ffbd0165360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbd006f5b0 0x7ffbd0165360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 -- 172.21.15.3:0/1797554010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd0160ab0 con 0x7ffbd006f150 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbc7fff640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbd006f5b0 0x7ffbd0165360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd4b9e640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd006f150 0x7ffbd0166330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0af040 tx=0x7ffbcc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbc57fa640 1 -- 172.21.15.3:0/1797554010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b1040 con 0x7ffbd006f150 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbc57fa640 1 -- 172.21.15.3:0/1797554010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0bd070 con 0x7ffbd006f150 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd0160210 con 0x7ffbd006f150 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbc57fa640 1 -- 172.21.15.3:0/1797554010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0b7880 con 0x7ffbd006f150 2024-09-12T12:21:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd01607d0 con 0x7ffbd006f150 2024-09-12T12:21:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.337+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd015d2f0 con 0x7ffbd006f150 2024-09-12T12:21:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.341+0000 7ffbc57fa640 1 -- 172.21.15.3:0/1797554010 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffbcc0ac4a0 con 0x7ffbd006f150 2024-09-12T12:21:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.341+0000 7ffbc57fa640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffbbc0678f0 0x7ffbbc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.341+0000 7ffbc57fa640 1 -- 172.21.15.3:0/1797554010 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5227+0+0 (secure 0 0 0) 0x7ffbcc120110 con 0x7ffbd006f150 2024-09-12T12:21:28.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.341+0000 7ffbc7fff640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffbbc0678f0 0x7ffbbc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.341+0000 7ffbc7fff640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffbbc0678f0 0x7ffbbc069db0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00b0de0 tx=0x7ffbc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.345+0000 7ffbc57fa640 1 -- 172.21.15.3:0/1797554010 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffbcc016110 con 0x7ffbd006f150 2024-09-12T12:21:28.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:28.505+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffbd006fae0 con 0x7ffbd006f150 2024-09-12T12:21:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.173+0000 7ffbc57fa640 1 -- 172.21.15.3:0/1797554010 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) ==== 170+0+0 (secure 0 0 0) 0x7ffbcc0f0d00 con 0x7ffbd006f150 2024-09-12T12:21:29.175 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T12:21:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffbbc0678f0 msgr2=0x7ffbbc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffbbc0678f0 0x7ffbbc069db0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00b0de0 tx=0x7ffbc0008040 comp rx=0 tx=0).stop 2024-09-12T12:21:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd006f150 msgr2=0x7ffbd0166330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd006f150 0x7ffbd0166330 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0af040 tx=0x7ffbcc01ad30 comp rx=0 tx=0).stop 2024-09-12T12:21:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 shutdown_connections 2024-09-12T12:21:29.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffbbc0678f0 0x7ffbbc069db0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffbd0159dd0 0x7ffbd00750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffbd006f5b0 0x7ffbd0165360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 --2- 172.21.15.3:0/1797554010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffbd006f150 0x7ffbd0166330 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 >> 172.21.15.3:0/1797554010 conn(0x7ffbd00b1d80 msgr2=0x7ffbd015a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 shutdown_connections 2024-09-12T12:21:29.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.177+0000 7ffbd6e02640 1 -- 172.21.15.3:0/1797554010 wait complete. 2024-09-12T12:21:29.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:21:29.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1f5509640 1 Processor -- start 2024-09-12T12:21:29.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1f5509640 1 -- start start 2024-09-12T12:21:29.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1f5509640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 0x7fc1f00b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1f5509640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1f006e0b0 con 0x7fc1f0159dd0 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1f5509640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1f006e390 con 0x7fc1f006f150 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1f5509640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f006e670 con 0x7fc1f00ad770 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1eeffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc1f0159dd0 0x7fc1f015d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35456/0 (socket says 172.21.15.3:35456) 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1eeffd640 1 -- 172.21.15.3:0/396351085 learned_addr learned my addr 172.21.15.3:0/396351085 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1399518616 0 0) 0x7fc1f006e0b0 con 0x7fc1f0159dd0 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1d8003530 con 0x7fc1f0159dd0 2024-09-12T12:21:29.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ef7fe640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 0x7fc1f00b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3138230214 0 0) 0x7fc1d8003530 con 0x7fc1f0159dd0 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 msgr2=0x7fc1f00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 0x7fc1f00b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 >> v1:172.21.15.142:6789/0 conn(0x7fc1f006f150 legacy=0x7fc1f00ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f006e950 con 0x7fc1f0159dd0 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fc1e0001680 con 0x7fc1f0159dd0 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 >> v1:172.21.15.3:6789/0 conn(0x7fc1f0159dd0 legacy=0x7fc1f015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d80039f0 0x7fc1d8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 0x7fc1d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1d8004d50 0x7fc1d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f006e670 con 0x7fc1d8004d50 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d800ba80 con 0x7fc1d80039f0 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d800bc80 con 0x7fc1f00ad770 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ef7fe640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1d8004d50 0x7fc1d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1eeffd640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d80039f0 0x7fc1d8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ee7fc640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 0x7fc1d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ef7fe640 1 -- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 msgr2=0x7fc1d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ef7fe640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f00ad770 0x7fc1d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ef7fe640 1 -- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d80039f0 msgr2=0x7fc1d8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ef7fe640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1d80039f0 0x7fc1d8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.341+0000 7fc1ef7fe640 1 -- 172.21.15.3:0/396351085 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d8003530 con 0x7fc1d8004d50 2024-09-12T12:21:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1ef7fe640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1d8004d50 0x7fc1d80051d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0070210 tx=0x7fc1e801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad6610d75a5675d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80b2070 con 0x7fc1d8004d50 2024-09-12T12:21:29.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e80a5c30 con 0x7fc1d8004d50 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1edffb640 1 -- 172.21.15.3:0/396351085 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8009c10 con 0x7fc1d8004d50 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 -- 172.21.15.3:0/396351085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1d8004d50 msgr2=0x7fc1d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 --2- 172.21.15.3:0/396351085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1d8004d50 0x7fc1d80051d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0070210 tx=0x7fc1e801cc20 comp rx=0 tx=0).stop 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1ef7fe640 1 -- 172.21.15.3:0/396351085 reap_dead start 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 -- 172.21.15.3:0/396351085 shutdown_connections 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 -- 172.21.15.3:0/396351085 >> 172.21.15.3:0/396351085 conn(0x7fc1f00b1d80 msgr2=0x7fc1f00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 -- 172.21.15.3:0/396351085 shutdown_connections 2024-09-12T12:21:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 -- 172.21.15.3:0/396351085 wait complete. 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 Processor -- start 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.345+0000 7fc1f5509640 1 -- start start 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f0145540 0x7fc1f009b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f009ddd0 0x7fc1f015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1f00af870 0x7fc1f00afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f015a1b0 con 0x7fc1f0145540 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f006dd40 con 0x7fc1f00af870 2024-09-12T12:21:29.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f00b2a40 con 0x7fc1f009ddd0 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f0145540 0x7fc1f009b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f0145540 0x7fc1f009b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47114/0 (socket says 172.21.15.3:47114) 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 -- 172.21.15.3:0/1134237907 learned_addr learned my addr 172.21.15.3:0/1134237907 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1ef7fe640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1f00af870 0x7fc1f00afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1ee7fc640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f009ddd0 0x7fc1f015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 -- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f009ddd0 msgr2=0x7fc1f015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f009ddd0 0x7fc1f015f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 -- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1f00af870 msgr2=0x7fc1f00afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1f00af870 0x7fc1f00afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 -- 172.21.15.3:0/1134237907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f015cb80 con 0x7fc1f0145540 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1eeffd640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f0145540 0x7fc1f009b230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000eaf0 tx=0x7fc1e00016a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:29.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1d77fe640 1 -- 172.21.15.3:0/1134237907 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000c870 con 0x7fc1f0145540 2024-09-12T12:21:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1d77fe640 1 -- 172.21.15.3:0/1134237907 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e0018070 con 0x7fc1f0145540 2024-09-12T12:21:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f009c1d0 con 0x7fc1f0145540 2024-09-12T12:21:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1d77fe640 1 -- 172.21.15.3:0/1134237907 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000cc70 con 0x7fc1f0145540 2024-09-12T12:21:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f009bc10 con 0x7fc1f0145540 2024-09-12T12:21:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f006f5b0 con 0x7fc1f0145540 2024-09-12T12:21:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1d77fe640 1 -- 172.21.15.3:0/1134237907 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc1e0026430 con 0x7fc1f0145540 2024-09-12T12:21:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1d77fe640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1dc0678f0 0x7fc1dc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.349+0000 7fc1d77fe640 1 -- 172.21.15.3:0/1134237907 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5609+0+0 (secure 0 0 0) 0x7fc1e00862a0 con 0x7fc1f0145540 2024-09-12T12:21:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.353+0000 7fc1ee7fc640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1dc0678f0 0x7fc1dc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:29.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.353+0000 7fc1ee7fc640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1dc0678f0 0x7fc1dc069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0153af0 tx=0x7fc1e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:29.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.353+0000 7fc1d77fe640 1 -- 172.21.15.3:0/1134237907 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1f015cb80 con 0x7fc1f0145540 2024-09-12T12:21:29.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:29.561+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc1f006fae0 con 0x7fc1f0145540 2024-09-12T12:21:30.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.201+0000 7fc1d77fe640 1 -- 172.21.15.3:0/1134237907 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) ==== 172+0+0 (secure 0 0 0) 0x7fc1e0056e20 con 0x7fc1f0145540 2024-09-12T12:21:30.208 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 6 and data pool 7 2024-09-12T12:21:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1dc0678f0 msgr2=0x7fc1dc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1dc0678f0 0x7fc1dc069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0153af0 tx=0x7fc1e4009040 comp rx=0 tx=0).stop 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f0145540 msgr2=0x7fc1f009b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f0145540 0x7fc1f009b230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000eaf0 tx=0x7fc1e00016a0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 shutdown_connections 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1dc0678f0 0x7fc1dc069db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1f00af870 0x7fc1f00afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1f009ddd0 0x7fc1f015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 --2- 172.21.15.3:0/1134237907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1f0145540 0x7fc1f009b230 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000eaf0 tx=0x7fc1e00016a0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.217+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 >> 172.21.15.3:0/1134237907 conn(0x7fc1f00b1d80 msgr2=0x7fc1f00b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:30.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.221+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 shutdown_connections 2024-09-12T12:21:30.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.221+0000 7fc1f5509640 1 -- 172.21.15.3:0/1134237907 wait complete. 2024-09-12T12:21:30.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:21:30.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.373+0000 7f5081c52640 1 Processor -- start 2024-09-12T12:21:30.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.373+0000 7f5081c52640 1 -- start start 2024-09-12T12:21:30.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f5081c52640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f507c159bd0 0x7f507c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f5081c52640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f5081c52640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f507c06f4a0 0x7f507c0ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f5081c52640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f507c06f010 con 0x7f507c06f4a0 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f5081c52640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f507c06e400 con 0x7f507c159bd0 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f5081c52640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f507c06e6e0 con 0x7f507c159fb0 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f507c06f4a0 0x7f507c0ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f507c06f4a0 0x7f507c0ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47120/0 (socket says 172.21.15.3:47120) 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 -- 172.21.15.3:0/1824239544 learned_addr learned my addr 172.21.15.3:0/1824239544 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507b7fe640 1 --2- 172.21.15.3:0/1824239544 >> v2:172.21.15.142:3300/0 conn(0x7f507c159bd0 0x7f507c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507affd640 1 --2- 172.21.15.3:0/1824239544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 -- 172.21.15.3:0/1824239544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 msgr2=0x7f507c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:30.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 --2- 172.21.15.3:0/1824239544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 -- 172.21.15.3:0/1824239544 >> v2:172.21.15.142:3300/0 conn(0x7f507c159bd0 msgr2=0x7f507c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:30.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 --2- 172.21.15.3:0/1824239544 >> v2:172.21.15.142:3300/0 conn(0x7f507c159bd0 0x7f507c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.377+0000 7f507bfff640 1 -- 172.21.15.3:0/1824239544 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c06e9c0 con 0x7f507c06f4a0 2024-09-12T12:21:30.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f507bfff640 1 --2- 172.21.15.3:0/1824239544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0ad060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f507c071320 tx=0x7f507401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d191ea243f16891 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:30.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f507a7fc640 1 -- 172.21.15.3:0/1824239544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50740ab5f0 con 0x7f507c06f4a0 2024-09-12T12:21:30.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f507a7fc640 1 -- 172.21.15.3:0/1824239544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50740b0070 con 0x7f507c06f4a0 2024-09-12T12:21:30.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f507a7fc640 1 -- 172.21.15.3:0/1824239544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50740ab9b0 con 0x7f507c06f4a0 2024-09-12T12:21:30.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- 172.21.15.3:0/1824239544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 msgr2=0x7f507c0ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 --2- 172.21.15.3:0/1824239544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0ad060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f507c071320 tx=0x7f507401c520 comp rx=0 tx=0).stop 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- 172.21.15.3:0/1824239544 shutdown_connections 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 --2- 172.21.15.3:0/1824239544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0ad060 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 --2- 172.21.15.3:0/1824239544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 --2- 172.21.15.3:0/1824239544 >> v2:172.21.15.142:3300/0 conn(0x7f507c159bd0 0x7f507c06dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- 172.21.15.3:0/1824239544 >> 172.21.15.3:0/1824239544 conn(0x7f507c0b1d80 msgr2=0x7f507c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- 172.21.15.3:0/1824239544 shutdown_connections 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- 172.21.15.3:0/1824239544 wait complete. 2024-09-12T12:21:30.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 Processor -- start 2024-09-12T12:21:30.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- start start 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c159bd0 0x7f507c0995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c099af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f507c160210 con 0x7f507c06f4a0 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f507c160090 con 0x7f507c159bd0 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f5081c52640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f507c160390 con 0x7f507c159fb0 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f507b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f507b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47124/0 (socket says 172.21.15.3:47124) 2024-09-12T12:21:30.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.381+0000 7f507b7fe640 1 -- 172.21.15.3:0/2511226251 learned_addr learned my addr 172.21.15.3:0/2511226251 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507affd640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c159bd0 0x7f507c0995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507bfff640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c099af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507b7fe640 1 -- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 msgr2=0x7f507c099af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507b7fe640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c099af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507b7fe640 1 -- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c159bd0 msgr2=0x7f507c0995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507b7fe640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c159bd0 0x7f507c0995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507b7fe640 1 -- 172.21.15.3:0/2511226251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c15c110 con 0x7f507c06f4a0 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507affd640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c159bd0 0x7f507c0995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507b7fe640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0b1090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f506400d800 tx=0x7f506400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5073fff640 1 -- 172.21.15.3:0/2511226251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506400b6c0 con 0x7f507c06f4a0 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5073fff640 1 -- 172.21.15.3:0/2511226251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5064019070 con 0x7f507c06f4a0 2024-09-12T12:21:30.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507c0cfd50 con 0x7f507c06f4a0 2024-09-12T12:21:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5073fff640 1 -- 172.21.15.3:0/2511226251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506400ba60 con 0x7f507c06f4a0 2024-09-12T12:21:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f507c157690 con 0x7f507c06f4a0 2024-09-12T12:21:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507c0ad060 con 0x7f507c06f4a0 2024-09-12T12:21:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5073fff640 1 -- 172.21.15.3:0/2511226251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f506400bc00 con 0x7f507c06f4a0 2024-09-12T12:21:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5073fff640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f505c0678c0 0x7f505c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f5073fff640 1 -- 172.21.15.3:0/2511226251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f5064086090 con 0x7f507c06f4a0 2024-09-12T12:21:30.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.385+0000 7f507affd640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f505c0678c0 0x7f505c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.389+0000 7f507affd640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f505c0678c0 0x7f505c069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f507c0cc740 tx=0x7f506c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:30.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.389+0000 7f5073fff640 1 -- 172.21.15.3:0/2511226251 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5064056c60 con 0x7f507c06f4a0 2024-09-12T12:21:30.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.553+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f507c0b2b00 con 0x7f507c06f4a0 2024-09-12T12:21:30.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.557+0000 7f5073fff640 1 -- 172.21.15.3:0/2511226251 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16626 (secure 0 0 0) 0x7f5064056560 con 0x7f507c06f4a0 2024-09-12T12:21:30.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:30.560 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":42,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:21:29.562503+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:21:27.233763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T12:21:28.546504+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:21:30.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f505c0678c0 msgr2=0x7f505c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f505c0678c0 0x7f505c069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f507c0cc740 tx=0x7f506c00b040 comp rx=0 tx=0).stop 2024-09-12T12:21:30.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 msgr2=0x7f507c0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0b1090 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f506400d800 tx=0x7f506400dcc0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 shutdown_connections 2024-09-12T12:21:30.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f505c0678c0 0x7f505c069d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f507c159fb0 0x7f507c099af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c159bd0 0x7f507c0995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 --2- 172.21.15.3:0/2511226251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f507c06f4a0 0x7f507c0b1090 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 >> 172.21.15.3:0/2511226251 conn(0x7f507c0b1d80 msgr2=0x7f507c15d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:30.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 shutdown_connections 2024-09-12T12:21:30.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.561+0000 7f5081c52640 1 -- 172.21.15.3:0/2511226251 wait complete. 2024-09-12T12:21:30.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:21:30.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa91e57640 1 Processor -- start 2024-09-12T12:21:30.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa91e57640 1 -- start start 2024-09-12T12:21:30.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa91e57640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 0x7ffa8c06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa91e57640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8c06fbb0 con 0x7ffa8c159bd0 2024-09-12T12:21:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa91e57640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8c06e400 con 0x7ffa8c15d2d0 2024-09-12T12:21:30.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa91e57640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8c06f010 con 0x7ffa8c159fb0 2024-09-12T12:21:30.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ffa8c159bd0 0x7ffa8c06f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35462/0 (socket says 172.21.15.3:35462) 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8b7fe640 1 -- 172.21.15.3:0/3187623414 learned_addr learned my addr 172.21.15.3:0/3187623414 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 0x7ffa8c06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 863573430 0 0) 0x7ffa8c06fbb0 con 0x7ffa8c159bd0 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa74003530 con 0x7ffa8c159bd0 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 57659049 0 0) 0x7ffa74003530 con 0x7ffa8c159bd0 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 msgr2=0x7ffa8c06dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 0x7ffa8c06dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 >> v1:172.21.15.142:6789/0 conn(0x7ffa8c15d2d0 legacy=0x7ffa8c0ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c06e6e0 con 0x7ffa8c159bd0 2024-09-12T12:21:30.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7ffa7c001680 con 0x7ffa8c159bd0 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 >> v1:172.21.15.3:6789/0 conn(0x7ffa8c159bd0 legacy=0x7ffa8c06f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740039f0 0x7ffa74003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 0x7ffa740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c06f010 con 0x7ffa74004810 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7400bab0 con 0x7ffa740039f0 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7400bcb0 con 0x7ffa8c159fb0 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8bfff640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 0x7ffa740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8affd640 1 -- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 msgr2=0x7ffa740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c159fb0 0x7ffa740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8affd640 1 -- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740039f0 msgr2=0x7ffa74003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:30.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8b7fe640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740039f0 0x7ffa74003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa740039f0 0x7ffa74003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.697+0000 7ffa8affd640 1 -- 172.21.15.3:0/3187623414 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa74003530 con 0x7ffa74004810 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa74004c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c15a3b0 tx=0x7ffa8001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6a66c33d581fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa800b2070 con 0x7ffa74004810 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa800a55c0 con 0x7ffa74004810 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa8a7fc640 1 -- 172.21.15.3:0/3187623414 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80009690 con 0x7ffa74004810 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa91e57640 1 -- 172.21.15.3:0/3187623414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 msgr2=0x7ffa74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa91e57640 1 --2- 172.21.15.3:0/3187623414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa74004c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c15a3b0 tx=0x7ffa8001c9d0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa8bfff640 1 -- 172.21.15.3:0/3187623414 reap_dead start 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa91e57640 1 -- 172.21.15.3:0/3187623414 shutdown_connections 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa91e57640 1 -- 172.21.15.3:0/3187623414 >> 172.21.15.3:0/3187623414 conn(0x7ffa8c0b1d80 msgr2=0x7ffa8c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa91e57640 1 -- 172.21.15.3:0/3187623414 shutdown_connections 2024-09-12T12:21:30.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.701+0000 7ffa91e57640 1 -- 172.21.15.3:0/3187623414 wait complete. 2024-09-12T12:21:30.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 Processor -- start 2024-09-12T12:21:30.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 -- start start 2024-09-12T12:21:30.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa8c1459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c0b0bd0 0x7ffa8c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8c099390 0x7ffa8c099840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c15d710 con 0x7ffa74004810 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c06eca0 con 0x7ffa8c099390 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8c0ad770 con 0x7ffa8c0b0bd0 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa8c1459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa8c1459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47152/0 (socket says 172.21.15.3:47152) 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 -- 172.21.15.3:0/3411977146 learned_addr learned my addr 172.21.15.3:0/3411977146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8bfff640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8c099390 0x7ffa8c099840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c0b0bd0 0x7ffa8c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 -- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c0b0bd0 msgr2=0x7ffa8c0b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c0b0bd0 0x7ffa8c0b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 -- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8c099390 msgr2=0x7ffa8c099840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8c099390 0x7ffa8c099840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 -- 172.21.15.3:0/3411977146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c154440 con 0x7ffa74004810 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa8b7fe640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa8c1459f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c010cf0 tx=0x7ffa7c00e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa73fff640 1 -- 172.21.15.3:0/3411977146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c01b070 con 0x7ffa74004810 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa73fff640 1 -- 172.21.15.3:0/3411977146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c011040 con 0x7ffa74004810 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa73fff640 1 -- 172.21.15.3:0/3411977146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0145a0 con 0x7ffa74004810 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c0b1640 con 0x7ffa74004810 2024-09-12T12:21:30.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c154920 con 0x7ffa74004810 2024-09-12T12:21:30.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.705+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8c06dd40 con 0x7ffa74004810 2024-09-12T12:21:30.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.709+0000 7ffa73fff640 1 -- 172.21.15.3:0/3411977146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffa7c01e070 con 0x7ffa74004810 2024-09-12T12:21:30.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.709+0000 7ffa73fff640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffa780678c0 0x7ffa78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:30.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.709+0000 7ffa73fff640 1 -- 172.21.15.3:0/3411977146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7ffa7c088960 con 0x7ffa74004810 2024-09-12T12:21:30.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.709+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffa780678c0 0x7ffa78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:30.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.709+0000 7ffa8affd640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffa780678c0 0x7ffa78069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa800af000 tx=0x7ffa800046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:30.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.713+0000 7ffa73fff640 1 -- 172.21.15.3:0/3411977146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ffa7c058e50 con 0x7ffa74004810 2024-09-12T12:21:30.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.873+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa8c0b2b00 con 0x7ffa74004810 2024-09-12T12:21:30.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.873+0000 7ffa73fff640 1 -- 172.21.15.3:0/3411977146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16626 (secure 0 0 0) 0x7ffa7c05d150 con 0x7ffa74004810 2024-09-12T12:21:30.877 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:30.878 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":42,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:21:29.562503+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:21:27.233763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T12:21:28.546504+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:21:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffa780678c0 msgr2=0x7ffa78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffa780678c0 0x7ffa78069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa800af000 tx=0x7ffa800046d0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 msgr2=0x7ffa8c1459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa8c1459f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c010cf0 tx=0x7ffa7c00e4d0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 shutdown_connections 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ffa780678c0 0x7ffa78069d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa8c099390 0x7ffa8c099840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa8c0b0bd0 0x7ffa8c0b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 --2- 172.21.15.3:0/3411977146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa74004810 0x7ffa8c1459f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 >> 172.21.15.3:0/3411977146 conn(0x7ffa8c0b1d80 msgr2=0x7ffa8c15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 shutdown_connections 2024-09-12T12:21:30.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:30.877+0000 7ffa91e57640 1 -- 172.21.15.3:0/3411977146 wait complete. 2024-09-12T12:21:30.896 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:21:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5b56b6640 1 Processor -- start 2024-09-12T12:21:31.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5b56b6640 1 -- start start 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5b56b6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd5b006f6e0 0x7fd5b0072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5b56b6640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b015f0b0 con 0x7fd5b015b130 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5b56b6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b015edd0 con 0x7fd5b00cc270 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5b56b6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b00b3790 con 0x7fd5b006f6e0 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd5b015b130 0x7fd5b00b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35474/0 (socket says 172.21.15.3:35474) 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 -- 172.21.15.3:0/2438169216 learned_addr learned my addr 172.21.15.3:0/2438169216 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025445886 0 0) 0x7fd5b015f0b0 con 0x7fd5b015b130 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd59c003530 con 0x7fd5b015b130 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5a7fff640 1 --2- 172.21.15.3:0/2438169216 >> v2:172.21.15.142:3300/0 conn(0x7fd5b006f6e0 0x7fd5b0072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3092955225 0 0) 0x7fd59c003530 con 0x7fd5b015b130 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 >> v2:172.21.15.142:3300/0 conn(0x7fd5b006f6e0 msgr2=0x7fd5b0072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 --2- 172.21.15.3:0/2438169216 >> v2:172.21.15.142:3300/0 conn(0x7fd5b006f6e0 0x7fd5b0072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 >> v1:172.21.15.142:6789/0 conn(0x7fd5b00cc270 legacy=0x7fd5b00aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b00adff0 con 0x7fd5b015b130 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fd5a0001680 con 0x7fd5b015b130 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 >> v1:172.21.15.3:6789/0 conn(0x7fd5b015b130 legacy=0x7fd5b00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd59c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59c004810 0x7fd59c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59c0051d0 0x7fd59c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00b3790 con 0x7fd59c0039c0 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c00c070 con 0x7fd59c0051d0 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c00c270 con 0x7fd59c004810 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd59c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5af7fe640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59c0051d0 0x7fd59c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5a7fff640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59c004810 0x7fd59c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 -- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59c004810 msgr2=0x7fd59c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd59c004810 0x7fd59c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 -- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59c0051d0 msgr2=0x7fd59c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd59c0051d0 0x7fd59c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5af7fe640 1 -- 172.21.15.3:0/2438169216 reap_dead start 2024-09-12T12:21:31.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.021+0000 7fd5aeffd640 1 -- 172.21.15.3:0/2438169216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c003530 con 0x7fd59c0039c0 2024-09-12T12:21:31.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5aeffd640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd59c003de0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c240 tx=0x7fd5a001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08f65dc2a966c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:31.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0007d20 con 0x7fd59c0039c0 2024-09-12T12:21:31.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a00a9070 con 0x7fd59c0039c0 2024-09-12T12:21:31.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5ae7fc640 1 -- 172.21.15.3:0/2438169216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0005d80 con 0x7fd59c0039c0 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- 172.21.15.3:0/2438169216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 msgr2=0x7fd59c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd59c003de0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c240 tx=0x7fd5a001cc80 comp rx=0 tx=0).stop 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- 172.21.15.3:0/2438169216 shutdown_connections 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/2438169216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd59c003de0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- 172.21.15.3:0/2438169216 >> 172.21.15.3:0/2438169216 conn(0x7fd5b00b1d80 msgr2=0x7fd5b00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- 172.21.15.3:0/2438169216 shutdown_connections 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- 172.21.15.3:0/2438169216 wait complete. 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 Processor -- start 2024-09-12T12:21:31.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- start start 2024-09-12T12:21:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd5b01529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5b015a4c0 0x7fd5b014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5b0151b80 0x7fd5b0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00b3610 con 0x7fd59c0039c0 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b00b3490 con 0x7fd5b015a4c0 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5b56b6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b015b510 con 0x7fd5b0151b80 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5aeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd5b01529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5aeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd5b01529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47166/0 (socket says 172.21.15.3:47166) 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.025+0000 7fd5aeffd640 1 -- 172.21.15.3:0/3768566653 learned_addr learned my addr 172.21.15.3:0/3768566653 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5af7fe640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5b0151b80 0x7fd5b0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a7fff640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5b015a4c0 0x7fd5b014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5aeffd640 1 -- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5b0151b80 msgr2=0x7fd5b0151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5aeffd640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5b0151b80 0x7fd5b0151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5aeffd640 1 -- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5b015a4c0 msgr2=0x7fd5b014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5aeffd640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5b015a4c0 0x7fd5b014d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5aeffd640 1 -- 172.21.15.3:0/3768566653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01530e0 con 0x7fd59c0039c0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5af7fe640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5b0151b80 0x7fd5b0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5aeffd640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd5b01529e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5a001cab0 tx=0x7fd59c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a77fe640 1 -- 172.21.15.3:0/3768566653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0007d20 con 0x7fd59c0039c0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a77fe640 1 -- 172.21.15.3:0/3768566653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a0007180 con 0x7fd59c0039c0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01533c0 con 0x7fd59c0039c0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a77fe640 1 -- 172.21.15.3:0/3768566653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0007530 con 0x7fd59c0039c0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b0151380 con 0x7fd59c0039c0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b00af550 con 0x7fd59c0039c0 2024-09-12T12:21:31.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a77fe640 1 -- 172.21.15.3:0/3768566653 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5a00a5850 con 0x7fd59c0039c0 2024-09-12T12:21:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a77fe640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5900678f0 0x7fd590069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a77fe640 1 -- 172.21.15.3:0/3768566653 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fd5a011f5c0 con 0x7fd59c0039c0 2024-09-12T12:21:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a7fff640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5900678f0 0x7fd590069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:31.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.029+0000 7fd5a7fff640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5900678f0 0x7fd590069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5b01503a0 tx=0x7fd598008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:31.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.033+0000 7fd5a77fe640 1 -- 172.21.15.3:0/3768566653 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd5a0016990 con 0x7fd59c0039c0 2024-09-12T12:21:31.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:31.229+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd5b01ed4e0 con 0x7fd59c0039c0 2024-09-12T12:21:32.166 INFO:tasks.ceph.mds.d.smithi142.stderr:2024-09-12T12:21:32.161+0000 7f44b59e2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:21:32.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.225+0000 7fd5a77fe640 1 -- 172.21.15.3:0/3768566653 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7fd5a00f0190 con 0x7fd59c0039c0 2024-09-12T12:21:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5900678f0 msgr2=0x7fd590069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5900678f0 0x7fd590069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd5b01503a0 tx=0x7fd598008040 comp rx=0 tx=0).stop 2024-09-12T12:21:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 msgr2=0x7fd5b01529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:32.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd5b01529e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5a001cab0 tx=0x7fd59c005840 comp rx=0 tx=0).stop 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 shutdown_connections 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5900678f0 0x7fd590069db0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5b0151b80 0x7fd5b0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5b015a4c0 0x7fd5b014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 --2- 172.21.15.3:0/3768566653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd59c0039c0 0x7fd5b01529e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 >> 172.21.15.3:0/3768566653 conn(0x7fd5b00b1d80 msgr2=0x7fd5b015e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 shutdown_connections 2024-09-12T12:21:32.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.229+0000 7fd5b56b6640 1 -- 172.21.15.3:0/3768566653 wait complete. 2024-09-12T12:21:32.255 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T12:21:32.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T12:21:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686b3ee640 1 Processor -- start 2024-09-12T12:21:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686b3ee640 1 -- start start 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686b3ee640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f68640d2920 0x7f686415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686b3ee640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686406eb00 con 0x7f68640d0510 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686b3ee640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68641620b0 con 0x7f686406d090 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686b3ee640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f68640727e0 con 0x7f68640d2920 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686918a640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f68640d0510 0x7f68640720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35476/0 (socket says 172.21.15.3:35476) 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686918a640 1 -- 172.21.15.3:0/3180984551 learned_addr learned my addr 172.21.15.3:0/3180984551 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686998b640 1 --2- 172.21.15.3:0/3180984551 >> v2:172.21.15.142:3300/0 conn(0x7f68640d2920 0x7f686415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090007049 0 0) 0x7f686406eb00 con 0x7f68640d0510 2024-09-12T12:21:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6850003530 con 0x7f68640d0510 2024-09-12T12:21:32.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 810405547 0 0) 0x7f6850003530 con 0x7f68640d0510 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 >> v2:172.21.15.142:3300/0 conn(0x7f68640d2920 msgr2=0x7f686415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 --2- 172.21.15.3:0/3180984551 >> v2:172.21.15.142:3300/0 conn(0x7f68640d2920 0x7f686415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 >> v1:172.21.15.142:6789/0 conn(0x7f686406d090 legacy=0x7f68640d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6864162390 con 0x7f68640d0510 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f6860001680 con 0x7f68640d0510 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 >> v1:172.21.15.3:6789/0 conn(0x7f68640d0510 legacy=0x7f68640720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68500039c0 0x7f6850003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6850004810 0x7f6850004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500051d0 0x7f685000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68640727e0 con 0x7f68500039c0 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f685000c070 con 0x7f68500051d0 2024-09-12T12:21:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f685000c270 con 0x7f6850004810 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686918a640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68500039c0 0x7f6850003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686998b640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500051d0 0x7f685000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f6868989640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6850004810 0x7f6850004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.385+0000 7f686918a640 1 -- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6850004810 msgr2=0x7f6850004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686918a640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6850004810 0x7f6850004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686918a640 1 -- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500051d0 msgr2=0x7f685000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686918a640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500051d0 0x7f685000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686918a640 1 -- 172.21.15.3:0/3180984551 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6850003530 con 0x7f68500039c0 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686998b640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500051d0 0x7f685000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f6868989640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6850004810 0x7f6850004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:32.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686998b640 1 -- 172.21.15.3:0/3180984551 reap_dead start 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686918a640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68500039c0 0x7f6850003de0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f686000c210 tx=0x7f686001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77ec51ce9859acb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860007d40 con 0x7f68500039c0 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68600a9070 con 0x7f68500039c0 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f685bfff640 1 -- 172.21.15.3:0/3180984551 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860005d80 con 0x7f68500039c0 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 -- 172.21.15.3:0/3180984551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68500039c0 msgr2=0x7f6850003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68500039c0 0x7f6850003de0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f686000c210 tx=0x7f686001cc60 comp rx=0 tx=0).stop 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 -- 172.21.15.3:0/3180984551 shutdown_connections 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 --2- 172.21.15.3:0/3180984551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68500039c0 0x7f6850003de0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 -- 172.21.15.3:0/3180984551 >> 172.21.15.3:0/3180984551 conn(0x7f6864070060 msgr2=0x7f6864070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:32.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 -- 172.21.15.3:0/3180984551 shutdown_connections 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 -- 172.21.15.3:0/3180984551 wait complete. 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.389+0000 7f686b3ee640 1 Processor -- start 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 -- start start 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500039c0 0x7f68640b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68640c6ee0 0x7f68640c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68640c8dd0 0x7f68640adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68640d08f0 con 0x7f68640c8dd0 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6864162fa0 con 0x7f68500039c0 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f68640d2620 con 0x7f68640c6ee0 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68640c8dd0 0x7f68640adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68640c8dd0 0x7f68640adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47188/0 (socket says 172.21.15.3:47188) 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 -- 172.21.15.3:0/524242769 learned_addr learned my addr 172.21.15.3:0/524242769 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686918a640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500039c0 0x7f68640b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f6868989640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68640c6ee0 0x7f68640c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 -- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68640c6ee0 msgr2=0x7f68640c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68640c6ee0 0x7f68640c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 -- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500039c0 msgr2=0x7f68640b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500039c0 0x7f68640b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 -- 172.21.15.3:0/524242769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68641f8c90 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686998b640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68640c8dd0 0x7f68640adcb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6864071c40 tx=0x7f685c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f6859ffb640 1 -- 172.21.15.3:0/524242769 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c01a030 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f6859ffb640 1 -- 172.21.15.3:0/524242769 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c010da0 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f6859ffb640 1 -- 172.21.15.3:0/524242769 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c003be0 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68640c7ac0 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68640c8080 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68640724e0 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f6859ffb640 1 -- 172.21.15.3:0/524242769 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f685c01d070 con 0x7f68640c8dd0 2024-09-12T12:21:32.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f6859ffb640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68480678f0 0x7f6848069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f6859ffb640 1 -- 172.21.15.3:0/524242769 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f685c086670 con 0x7f68640c8dd0 2024-09-12T12:21:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.393+0000 7f686918a640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68480678f0 0x7f6848069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.397+0000 7f686918a640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68480678f0 0x7f6848069db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68600ab000 tx=0x7f68600b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:32.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.401+0000 7f6859ffb640 1 -- 172.21.15.3:0/524242769 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f685c057240 con 0x7f68640c8dd0 2024-09-12T12:21:32.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:32.601+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6864161690 con 0x7f68640c8dd0 2024-09-12T12:21:33.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.225+0000 7f6859ffb640 1 -- 172.21.15.3:0/524242769 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f685c056b40 con 0x7f68640c8dd0 2024-09-12T12:21:33.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68480678f0 msgr2=0x7f6848069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68480678f0 0x7f6848069db0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68600ab000 tx=0x7f68600b0040 comp rx=0 tx=0).stop 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68640c8dd0 msgr2=0x7f68640adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68640c8dd0 0x7f68640adcb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6864071c40 tx=0x7f685c010590 comp rx=0 tx=0).stop 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 shutdown_connections 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68480678f0 0x7f6848069db0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68640c8dd0 0x7f68640adcb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68640c6ee0 0x7f68640c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 --2- 172.21.15.3:0/524242769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68500039c0 0x7f68640b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 >> 172.21.15.3:0/524242769 conn(0x7f6864070060 msgr2=0x7f68640750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 shutdown_connections 2024-09-12T12:21:33.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.229+0000 7f686b3ee640 1 -- 172.21.15.3:0/524242769 wait complete. 2024-09-12T12:21:33.252 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T12:21:33.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2051a71640 1 Processor -- start 2024-09-12T12:21:33.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2051a71640 1 -- start start 2024-09-12T12:21:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2051a71640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f204c15b2d0 0x7f204c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2051a71640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f204c073640 0x7f204c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2051a71640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f204c15f150 con 0x7f204c06f6e0 2024-09-12T12:21:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2051a71640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f204c0ae920 con 0x7f204c073640 2024-09-12T12:21:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2051a71640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f204c15b6b0 con 0x7f204c15b2d0 2024-09-12T12:21:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2042ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f204c06f6e0 0x7f204c072f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35484/0 (socket says 172.21.15.3:35484) 2024-09-12T12:21:33.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f2042ffd640 1 -- 172.21.15.3:0/2189857528 learned_addr learned my addr 172.21.15.3:0/2189857528 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204bfff640 1 --2- 172.21.15.3:0/2189857528 >> v2:172.21.15.3:3300/0 conn(0x7f204c073640 0x7f204c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204b7fe640 1 --2- 172.21.15.3:0/2189857528 >> v2:172.21.15.142:3300/0 conn(0x7f204c15b2d0 0x7f204c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204affd640 1 -- 172.21.15.3:0/2189857528 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4085237070 0 0) 0x7f204c15f150 con 0x7f204c06f6e0 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204affd640 1 -- 172.21.15.3:0/2189857528 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2038003530 con 0x7f204c06f6e0 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204bfff640 1 -- 172.21.15.3:0/2189857528 >> v2:172.21.15.142:3300/0 conn(0x7f204c15b2d0 msgr2=0x7f204c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204bfff640 1 --2- 172.21.15.3:0/2189857528 >> v2:172.21.15.142:3300/0 conn(0x7f204c15b2d0 0x7f204c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204bfff640 1 -- 172.21.15.3:0/2189857528 >> v1:172.21.15.3:6789/0 conn(0x7f204c06f6e0 legacy=0x7f204c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.377+0000 7f204bfff640 1 -- 172.21.15.3:0/2189857528 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c15ee70 con 0x7f204c073640 2024-09-12T12:21:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204bfff640 1 --2- 172.21.15.3:0/2189857528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c073640 0x7f204c0aedb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f204c06fe30 tx=0x7f204401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292c5d669a9f20a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204affd640 1 -- 172.21.15.3:0/2189857528 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20440b2070 con 0x7f204c073640 2024-09-12T12:21:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204affd640 1 -- 172.21.15.3:0/2189857528 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20440a59c0 con 0x7f204c073640 2024-09-12T12:21:33.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204affd640 1 -- 172.21.15.3:0/2189857528 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20440a5d90 con 0x7f204c073640 2024-09-12T12:21:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- 172.21.15.3:0/2189857528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c073640 msgr2=0x7f204c0aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 --2- 172.21.15.3:0/2189857528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c073640 0x7f204c0aedb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f204c06fe30 tx=0x7f204401c900 comp rx=0 tx=0).stop 2024-09-12T12:21:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- 172.21.15.3:0/2189857528 shutdown_connections 2024-09-12T12:21:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 --2- 172.21.15.3:0/2189857528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c073640 0x7f204c0aedb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 --2- 172.21.15.3:0/2189857528 >> v2:172.21.15.142:3300/0 conn(0x7f204c15b2d0 0x7f204c0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- 172.21.15.3:0/2189857528 >> 172.21.15.3:0/2189857528 conn(0x7f204c0b1d80 msgr2=0x7f204c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- 172.21.15.3:0/2189857528 shutdown_connections 2024-09-12T12:21:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- 172.21.15.3:0/2189857528 wait complete. 2024-09-12T12:21:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 Processor -- start 2024-09-12T12:21:33.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- start start 2024-09-12T12:21:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c06f6e0 0x7f204c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f204c073640 0x7f204c14a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f204c15b2d0 0x7f204c152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f204c0af700 con 0x7f204c06f6e0 2024-09-12T12:21:33.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f204c06d8b0 con 0x7f204c15b2d0 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2051a71640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f204c0af880 con 0x7f204c073640 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c06f6e0 0x7f204c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c06f6e0 0x7f204c15dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47196/0 (socket says 172.21.15.3:47196) 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 -- 172.21.15.3:0/3842774560 learned_addr learned my addr 172.21.15.3:0/3842774560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f2042ffd640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f204c073640 0x7f204c14a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 -- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f204c073640 msgr2=0x7f204c14a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f204c073640 0x7f204c14a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 -- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f204c15b2d0 msgr2=0x7f204c152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204bfff640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f204c15b2d0 0x7f204c152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f204c15b2d0 0x7f204c152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 -- 172.21.15.3:0/3842774560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c1537a0 con 0x7f204c06f6e0 2024-09-12T12:21:33.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.381+0000 7f204b7fe640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c06f6e0 0x7f204c15dd80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f203c00d800 tx=0x7f203c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2048ff9640 1 -- 172.21.15.3:0/3842774560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c00b6c0 con 0x7f204c06f6e0 2024-09-12T12:21:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2048ff9640 1 -- 172.21.15.3:0/3842774560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f203c019070 con 0x7f204c06f6e0 2024-09-12T12:21:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c153a80 con 0x7f204c06f6e0 2024-09-12T12:21:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2048ff9640 1 -- 172.21.15.3:0/3842774560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c00ba60 con 0x7f204c06f6e0 2024-09-12T12:21:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1f5080 con 0x7f204c06f6e0 2024-09-12T12:21:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c1490f0 con 0x7f204c06f6e0 2024-09-12T12:21:33.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2048ff9640 1 -- 172.21.15.3:0/3842774560 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f203c00bc00 con 0x7f204c06f6e0 2024-09-12T12:21:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2048ff9640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f202c0678c0 0x7f202c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2048ff9640 1 -- 172.21.15.3:0/3842774560 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f203c086190 con 0x7f204c06f6e0 2024-09-12T12:21:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2042ffd640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f202c0678c0 0x7f202c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.385+0000 7f2042ffd640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f202c0678c0 0x7f202c069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2034007fd0 tx=0x7f2034009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:33.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.389+0000 7f2048ff9640 1 -- 172.21.15.3:0/3842774560 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f203c056cc0 con 0x7f204c06f6e0 2024-09-12T12:21:33.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.585+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f204c15bd40 con 0x7f204c06f6e0 2024-09-12T12:21:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.585+0000 7f2048ff9640 1 -- 172.21.15.3:0/3842774560 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5220 (secure 0 0 0) 0x7f203c0565c0 con 0x7f204c06f6e0 2024-09-12T12:21:33.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:33.589 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"btime":"2024-09-12T12:21:33:226221+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:32.604624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5262]},"id":3}]} 2024-09-12T12:21:33.589 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-09-12T12:21:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f202c0678c0 msgr2=0x7f202c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f202c0678c0 0x7f202c069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2034007fd0 tx=0x7f2034009040 comp rx=0 tx=0).stop 2024-09-12T12:21:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c06f6e0 msgr2=0x7f204c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c06f6e0 0x7f204c15dd80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f203c00d800 tx=0x7f203c00dcc0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 shutdown_connections 2024-09-12T12:21:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f202c0678c0 0x7f202c069d80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f204c15b2d0 0x7f204c152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f204c073640 0x7f204c14a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 --2- 172.21.15.3:0/3842774560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f204c06f6e0 0x7f204c15dd80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 >> 172.21.15.3:0/3842774560 conn(0x7f204c0b1d80 msgr2=0x7f204c06ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 shutdown_connections 2024-09-12T12:21:33.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.589+0000 7f2051a71640 1 -- 172.21.15.3:0/3842774560 wait complete. 2024-09-12T12:21:33.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:21:33.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793f7d2640 1 Processor -- start 2024-09-12T12:21:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793f7d2640 1 -- start start 2024-09-12T12:21:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793f7d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 0x7f79380705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793f7d2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79380cefe0 con 0x7f79380d0a00 2024-09-12T12:21:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793f7d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79380cf2c0 con 0x7f79381629c0 2024-09-12T12:21:33.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793f7d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79380cdf70 con 0x7f79380b0980 2024-09-12T12:21:33.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793cd6d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79380d0a00 0x7f79380cb930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35498/0 (socket says 172.21.15.3:35498) 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.729+0000 7f793cd6d640 1 -- 172.21.15.3:0/2268958882 learned_addr learned my addr 172.21.15.3:0/2268958882 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 0x7f79380705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3262766398 0 0) 0x7f79380cefe0 con 0x7f79380d0a00 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7924003530 con 0x7f79380d0a00 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4196355693 0 0) 0x7f7924003530 con 0x7f79380d0a00 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 msgr2=0x7f79380705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 0x7f79380705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 >> v1:172.21.15.142:6789/0 conn(0x7f79381629c0 legacy=0x7f79380cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79380cf5a0 con 0x7f79380d0a00 2024-09-12T12:21:33.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f7928001680 con 0x7f79380d0a00 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 >> v1:172.21.15.3:6789/0 conn(0x7f79380d0a00 legacy=0x7f79380cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 0x7f79240039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924003f70 0x7f7924004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7924004d50 0x7f79240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cdf70 con 0x7f7924004d50 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f792400bc10 con 0x7f7924003f70 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f792400be10 con 0x7f79380b0980 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7924004d50 0x7f79240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793cd6d640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924003f70 0x7f7924004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793d56e640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 0x7f79240039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 -- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 msgr2=0x7f79240039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b0980 0x7f79240039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 -- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924003f70 msgr2=0x7f7924004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924003f70 0x7f7924004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 -- 172.21.15.3:0/2268958882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7924003530 con 0x7f7924004d50 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7924004d50 0x7f79240051d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7930009fc0 tx=0x7f793001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f415a18e9ff06b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300b2070 con 0x7f7924004d50 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79300a5b80 con 0x7f7924004d50 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f792ffff640 1 -- 172.21.15.3:0/2268958882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930009c10 con 0x7f7924004d50 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793f7d2640 1 -- 172.21.15.3:0/2268958882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7924004d50 msgr2=0x7f79240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793f7d2640 1 --2- 172.21.15.3:0/2268958882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7924004d50 0x7f79240051d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7930009fc0 tx=0x7f793001cc80 comp rx=0 tx=0).stop 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793dd6f640 1 -- 172.21.15.3:0/2268958882 reap_dead start 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793f7d2640 1 -- 172.21.15.3:0/2268958882 shutdown_connections 2024-09-12T12:21:33.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.733+0000 7f793f7d2640 1 -- 172.21.15.3:0/2268958882 >> 172.21.15.3:0/2268958882 conn(0x7f793809aa70 msgr2=0x7f79380b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- 172.21.15.3:0/2268958882 shutdown_connections 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- 172.21.15.3:0/2268958882 wait complete. 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 Processor -- start 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- start start 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924004d50 0x7f79380b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b00b0 0x7f793814c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79380b5d50 0x7f79380b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cc1c0 con 0x7f79380b5d50 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79380cc040 con 0x7f7924004d50 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7938070db0 con 0x7f79380b00b0 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79380b5d50 0x7f79380b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79380b5d50 0x7f79380b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47210/0 (socket says 172.21.15.3:47210) 2024-09-12T12:21:33.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 -- 172.21.15.3:0/1748215789 learned_addr learned my addr 172.21.15.3:0/1748215789 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793d56e640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924004d50 0x7f79380b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793cd6d640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b00b0 0x7f793814c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 -- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b00b0 msgr2=0x7f793814c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b00b0 0x7f793814c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 -- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924004d50 msgr2=0x7f79380b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924004d50 0x7f79380b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 -- 172.21.15.3:0/1748215789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79380b70e0 con 0x7f79380b5d50 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793dd6f640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79380b5d50 0x7f79380b61d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7930002240 tx=0x7f7930009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f792dffb640 1 -- 172.21.15.3:0/1748215789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300bd070 con 0x7f79380b5d50 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f792dffb640 1 -- 172.21.15.3:0/1748215789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79300a5980 con 0x7f79380b5d50 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793814ce10 con 0x7f79380b5d50 2024-09-12T12:21:33.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f792dffb640 1 -- 172.21.15.3:0/1748215789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300b2040 con 0x7f79380b5d50 2024-09-12T12:21:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79380b6e00 con 0x7f79380b5d50 2024-09-12T12:21:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.737+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79380cb930 con 0x7f79380b5d50 2024-09-12T12:21:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.741+0000 7f792dffb640 1 -- 172.21.15.3:0/1748215789 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f79300c0070 con 0x7f79380b5d50 2024-09-12T12:21:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.741+0000 7f792dffb640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f791c0678c0 0x7f791c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:33.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.741+0000 7f792dffb640 1 -- 172.21.15.3:0/1748215789 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f793011fb90 con 0x7f79380b5d50 2024-09-12T12:21:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.741+0000 7f793d56e640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f791c0678c0 0x7f791c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:33.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.741+0000 7f793d56e640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f791c0678c0 0x7f791c069d80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7934000c10 tx=0x7f7934009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:33.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.745+0000 7f792dffb640 1 -- 172.21.15.3:0/1748215789 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7930016990 con 0x7f79380b5d50 2024-09-12T12:21:33.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.905+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79380b3710 con 0x7f79380b5d50 2024-09-12T12:21:33.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.909+0000 7f792dffb640 1 -- 172.21.15.3:0/1748215789 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16613 (secure 0 0 0) 0x7f79300eff50 con 0x7f79380b5d50 2024-09-12T12:21:33.912 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:33.913 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":43,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:21:31.199688+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:21:27.233763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T12:21:28.546504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:21:33.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f791c0678c0 msgr2=0x7f791c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f791c0678c0 0x7f791c069d80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7934000c10 tx=0x7f7934009040 comp rx=0 tx=0).stop 2024-09-12T12:21:33.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79380b5d50 msgr2=0x7f79380b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79380b5d50 0x7f79380b61d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7930002240 tx=0x7f7930009750 comp rx=0 tx=0).stop 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 shutdown_connections 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f791c0678c0 0x7f791c069d80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79380b5d50 0x7f79380b61d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79380b00b0 0x7f793814c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 --2- 172.21.15.3:0/1748215789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7924004d50 0x7f79380b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 >> 172.21.15.3:0/1748215789 conn(0x7f793809aa70 msgr2=0x7f79380b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 shutdown_connections 2024-09-12T12:21:33.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:33.913+0000 7f793f7d2640 1 -- 172.21.15.3:0/1748215789 wait complete. 2024-09-12T12:21:33.930 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:21:33.931 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:21:34.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.045+0000 7f6d3d900640 1 Processor -- start 2024-09-12T12:21:34.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.045+0000 7f6d3d900640 1 -- start start 2024-09-12T12:21:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d3d900640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6d3809ba60 0x7f6d3809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d3d900640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d380721e0 con 0x7f6d380703d0 2024-09-12T12:21:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d3d900640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d380ce7c0 con 0x7f6d380cd4a0 2024-09-12T12:21:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d3d900640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6d380ccd50 con 0x7f6d3809ba60 2024-09-12T12:21:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d36ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d380703d0 0x7f6d3809b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35502/0 (socket says 172.21.15.3:35502) 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d36ffd640 1 -- 172.21.15.3:0/954663742 learned_addr learned my addr 172.21.15.3:0/954663742 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748521817 0 0) 0x7f6d380721e0 con 0x7f6d380703d0 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d20003530 con 0x7f6d380703d0 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d367fc640 1 --2- 172.21.15.3:0/954663742 >> v2:172.21.15.142:3300/0 conn(0x7f6d3809ba60 0x7f6d3809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 626586329 0 0) 0x7f6d20003530 con 0x7f6d380703d0 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 >> v2:172.21.15.142:3300/0 conn(0x7f6d3809ba60 msgr2=0x7f6d3809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 --2- 172.21.15.3:0/954663742 >> v2:172.21.15.142:3300/0 conn(0x7f6d3809ba60 0x7f6d3809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 >> v1:172.21.15.142:6789/0 conn(0x7f6d380cd4a0 legacy=0x7f6d380cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d380ceaa0 con 0x7f6d380703d0 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f6d28001680 con 0x7f6d380703d0 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 >> v1:172.21.15.3:6789/0 conn(0x7f6d380703d0 legacy=0x7f6d3809b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d200039c0 0x7f6d20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d20004810 0x7f6d20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2000c040 con 0x7f6d200051d0 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2000c1f0 con 0x7f6d200039c0 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2000c3f0 con 0x7f6d20004810 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d377fe640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d36ffd640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d200039c0 0x7f6d20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d367fc640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d20004810 0x7f6d20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d377fe640 1 -- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d20004810 msgr2=0x7f6d20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d377fe640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d20004810 0x7f6d20004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d377fe640 1 -- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d200039c0 msgr2=0x7f6d20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d377fe640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d200039c0 0x7f6d20003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.049+0000 7f6d377fe640 1 -- 172.21.15.3:0/954663742 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d20003530 con 0x7f6d200051d0 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d377fe640 1 -- 172.21.15.3:0/954663742 reap_dead start 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d377fe640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d2000b9d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6d30001e70 tx=0x7f6d3001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1223f733a60f320 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30001a90 con 0x7f6d200051d0 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d30003ba0 con 0x7f6d200051d0 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d35ffb640 1 -- 172.21.15.3:0/954663742 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30005020 con 0x7f6d200051d0 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- 172.21.15.3:0/954663742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 msgr2=0x7f6d2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d2000b9d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6d30001e70 tx=0x7f6d3001c780 comp rx=0 tx=0).stop 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- 172.21.15.3:0/954663742 shutdown_connections 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 --2- 172.21.15.3:0/954663742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d2000b9d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- 172.21.15.3:0/954663742 >> 172.21.15.3:0/954663742 conn(0x7f6d380b3e60 msgr2=0x7f6d380b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- 172.21.15.3:0/954663742 shutdown_connections 2024-09-12T12:21:34.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- 172.21.15.3:0/954663742 wait complete. 2024-09-12T12:21:34.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 Processor -- start 2024-09-12T12:21:34.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- start start 2024-09-12T12:21:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d380c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38154920 0x7f6d3809e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d3809e610 0x7f6d38155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d380ccf80 con 0x7f6d200051d0 2024-09-12T12:21:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d38071cb0 con 0x7f6d38154920 2024-09-12T12:21:34.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3809aa70 con 0x7f6d3809e610 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d380c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d380c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47238/0 (socket says 172.21.15.3:47238) 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 -- 172.21.15.3:0/1902303378 learned_addr learned my addr 172.21.15.3:0/1902303378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 -- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d3809e610 msgr2=0x7f6d38155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d3809e610 0x7f6d38155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 -- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38154920 msgr2=0x7f6d3809e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d367fc640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38154920 0x7f6d3809e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38154920 0x7f6d3809e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 -- 172.21.15.3:0/1902303378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d380c9c70 con 0x7f6d200051d0 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d36ffd640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d380c9440 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e780 tx=0x7f6d2800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d1ffff640 1 -- 172.21.15.3:0/1902303378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800c660 con 0x7f6d200051d0 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d1ffff640 1 -- 172.21.15.3:0/1902303378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d28018070 con 0x7f6d200051d0 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d1ffff640 1 -- 172.21.15.3:0/1902303378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800ca40 con 0x7f6d200051d0 2024-09-12T12:21:34.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d380c9f50 con 0x7f6d200051d0 2024-09-12T12:21:34.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.053+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d380ca510 con 0x7f6d200051d0 2024-09-12T12:21:34.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.057+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3809ea30 con 0x7f6d200051d0 2024-09-12T12:21:34.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.057+0000 7f6d1ffff640 1 -- 172.21.15.3:0/1902303378 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d2800cbe0 con 0x7f6d200051d0 2024-09-12T12:21:34.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.057+0000 7f6d1ffff640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d240678c0 0x7f6d24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.057+0000 7f6d1ffff640 1 -- 172.21.15.3:0/1902303378 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f6d280859d0 con 0x7f6d200051d0 2024-09-12T12:21:34.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.057+0000 7f6d367fc640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d240678c0 0x7f6d24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.057+0000 7f6d367fc640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d240678c0 0x7f6d24069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d3815f3f0 tx=0x7f6d2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.061+0000 7f6d1ffff640 1 -- 172.21.15.3:0/1902303378 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6d28056510 con 0x7f6d200051d0 2024-09-12T12:21:34.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.213+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d3809ae00 con 0x7f6d240678c0 2024-09-12T12:21:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.233+0000 7f6d1ffff640 1 -- 172.21.15.3:0/1902303378 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250086 (secure 0 0 0) 0x7f6d3809ae00 con 0x7f6d240678c0 2024-09-12T12:21:34.237 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:34.239 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d240678c0 msgr2=0x7f6d24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d240678c0 0x7f6d24069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6d3815f3f0 tx=0x7f6d2c008040 comp rx=0 tx=0).stop 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 msgr2=0x7f6d380c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d380c9440 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800e780 tx=0x7f6d2800ec40 comp rx=0 tx=0).stop 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 shutdown_connections 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d240678c0 0x7f6d24069d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d3809e610 0x7f6d38155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d38154920 0x7f6d3809e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 --2- 172.21.15.3:0/1902303378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d200051d0 0x7f6d380c9440 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 >> 172.21.15.3:0/1902303378 conn(0x7f6d380b3e60 msgr2=0x7f6d380cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:34.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 shutdown_connections 2024-09-12T12:21:34.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.237+0000 7f6d3d900640 1 -- 172.21.15.3:0/1902303378 wait complete. 2024-09-12T12:21:34.254 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-12T12:21:33.691853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225428,"kb_used_data":5032,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749292,"statfs":{"total":773094113280,"available":772863275008,"internally_reserved":0,"allocated":5152768,"data_stored":2742512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526019"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.208627+0000","last_change":"2024-09-12T12:21:30.200850+0000","last_active":"2024-09-12T12:21:30.208627+0000","last_peered":"2024-09-12T12:21:30.208627+0000","last_clean":"2024-09-12T12:21:30.208627+0000","last_became_active":"2024-09-12T12:21:30.200142+0000","last_became_peered":"2024-09-12T12:21:30.200142+0000","last_unstale":"2024-09-12T12:21:30.208627+0000","last_undegraded":"2024-09-12T12:21:30.208627+0000","last_fullsized":"2024-09-12T12:21:30.208627+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209219+0000","last_change":"2024-09-12T12:21:30.208435+0000","last_active":"2024-09-12T12:21:31.209219+0000","last_peered":"2024-09-12T12:21:31.209219+0000","last_clean":"2024-09-12T12:21:31.209219+0000","last_became_active":"2024-09-12T12:21:30.208134+0000","last_became_peered":"2024-09-12T12:21:30.208134+0000","last_unstale":"2024-09-12T12:21:31.209219+0000","last_undegraded":"2024-09-12T12:21:31.209219+0000","last_fullsized":"2024-09-12T12:21:31.209219+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:33.718518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210339+0000","last_change":"2024-09-12T12:21:30.212483+0000","last_active":"2024-09-12T12:21:31.210339+0000","last_peered":"2024-09-12T12:21:31.210339+0000","last_clean":"2024-09-12T12:21:31.210339+0000","last_became_active":"2024-09-12T12:21:30.211973+0000","last_became_peered":"2024-09-12T12:21:30.211973+0000","last_unstale":"2024-09-12T12:21:31.210339+0000","last_undegraded":"2024-09-12T12:21:31.210339+0000","last_fullsized":"2024-09-12T12:21:31.210339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:30.898304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227851+0000","last_change":"2024-09-12T12:21:30.222772+0000","last_active":"2024-09-12T12:21:30.227851+0000","last_peered":"2024-09-12T12:21:30.227851+0000","last_clean":"2024-09-12T12:21:30.227851+0000","last_became_active":"2024-09-12T12:21:30.222582+0000","last_became_peered":"2024-09-12T12:21:30.222582+0000","last_unstale":"2024-09-12T12:21:30.227851+0000","last_undegraded":"2024-09-12T12:21:30.227851+0000","last_fullsized":"2024-09-12T12:21:30.227851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:53.948158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207968+0000","last_change":"2024-09-12T12:21:30.208596+0000","last_active":"2024-09-12T12:21:31.207968+0000","last_peered":"2024-09-12T12:21:31.207968+0000","last_clean":"2024-09-12T12:21:31.207968+0000","last_became_active":"2024-09-12T12:21:30.208202+0000","last_became_peered":"2024-09-12T12:21:30.208202+0000","last_unstale":"2024-09-12T12:21:31.207968+0000","last_undegraded":"2024-09-12T12:21:31.207968+0000","last_fullsized":"2024-09-12T12:21:31.207968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210712+0000","last_change":"2024-09-12T12:21:30.204887+0000","last_active":"2024-09-12T12:21:30.210712+0000","last_peered":"2024-09-12T12:21:30.210712+0000","last_clean":"2024-09-12T12:21:30.210712+0000","last_became_active":"2024-09-12T12:21:30.204053+0000","last_became_peered":"2024-09-12T12:21:30.204053+0000","last_unstale":"2024-09-12T12:21:30.210712+0000","last_undegraded":"2024-09-12T12:21:30.210712+0000","last_fullsized":"2024-09-12T12:21:30.210712+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225399+0000","last_change":"2024-09-12T12:21:30.220921+0000","last_active":"2024-09-12T12:21:30.225399+0000","last_peered":"2024-09-12T12:21:30.225399+0000","last_clean":"2024-09-12T12:21:30.225399+0000","last_became_active":"2024-09-12T12:21:30.220376+0000","last_became_peered":"2024-09-12T12:21:30.220376+0000","last_unstale":"2024-09-12T12:21:30.225399+0000","last_undegraded":"2024-09-12T12:21:30.225399+0000","last_fullsized":"2024-09-12T12:21:30.225399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:49.354057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233115+0000","last_change":"2024-09-12T12:21:30.229582+0000","last_active":"2024-09-12T12:21:30.233115+0000","last_peered":"2024-09-12T12:21:30.233115+0000","last_clean":"2024-09-12T12:21:30.233115+0000","last_became_active":"2024-09-12T12:21:30.229229+0000","last_became_peered":"2024-09-12T12:21:30.229229+0000","last_unstale":"2024-09-12T12:21:30.233115+0000","last_undegraded":"2024-09-12T12:21:30.233115+0000","last_fullsized":"2024-09-12T12:21:30.233115+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:42:40.496362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227913+0000","last_change":"2024-09-12T12:21:30.222613+0000","last_active":"2024-09-12T12:21:30.227913+0000","last_peered":"2024-09-12T12:21:30.227913+0000","last_clean":"2024-09-12T12:21:30.227913+0000","last_became_active":"2024-09-12T12:21:30.222213+0000","last_became_peered":"2024-09-12T12:21:30.222213+0000","last_unstale":"2024-09-12T12:21:30.227913+0000","last_undegraded":"2024-09-12T12:21:30.227913+0000","last_fullsized":"2024-09-12T12:21:30.227913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:22.314764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233213+0000","last_change":"2024-09-12T12:21:30.229703+0000","last_active":"2024-09-12T12:21:30.233213+0000","last_peered":"2024-09-12T12:21:30.233213+0000","last_clean":"2024-09-12T12:21:30.233213+0000","last_became_active":"2024-09-12T12:21:30.229280+0000","last_became_peered":"2024-09-12T12:21:30.229280+0000","last_unstale":"2024-09-12T12:21:30.233213+0000","last_undegraded":"2024-09-12T12:21:30.233213+0000","last_fullsized":"2024-09-12T12:21:30.233213+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:06:44.348610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208772+0000","last_change":"2024-09-12T12:21:30.208502+0000","last_active":"2024-09-12T12:21:31.208772+0000","last_peered":"2024-09-12T12:21:31.208772+0000","last_clean":"2024-09-12T12:21:31.208772+0000","last_became_active":"2024-09-12T12:21:30.207768+0000","last_became_peered":"2024-09-12T12:21:30.207768+0000","last_unstale":"2024-09-12T12:21:31.208772+0000","last_undegraded":"2024-09-12T12:21:31.208772+0000","last_fullsized":"2024-09-12T12:21:31.208772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896095+0000","last_change":"2024-09-12T12:21:30.227759+0000","last_active":"2024-09-12T12:21:31.896095+0000","last_peered":"2024-09-12T12:21:31.896095+0000","last_clean":"2024-09-12T12:21:31.896095+0000","last_became_active":"2024-09-12T12:21:30.227324+0000","last_became_peered":"2024-09-12T12:21:30.227324+0000","last_unstale":"2024-09-12T12:21:31.896095+0000","last_undegraded":"2024-09-12T12:21:31.896095+0000","last_fullsized":"2024-09-12T12:21:31.896095+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:39.438927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.218203+0000","last_change":"2024-09-12T12:21:30.213039+0000","last_active":"2024-09-12T12:21:30.218203+0000","last_peered":"2024-09-12T12:21:30.218203+0000","last_clean":"2024-09-12T12:21:30.218203+0000","last_became_active":"2024-09-12T12:21:30.212232+0000","last_became_peered":"2024-09-12T12:21:30.212232+0000","last_unstale":"2024-09-12T12:21:30.218203+0000","last_undegraded":"2024-09-12T12:21:30.218203+0000","last_fullsized":"2024-09-12T12:21:30.218203+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:04.883109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919869+0000","last_change":"2024-09-12T12:21:30.216647+0000","last_active":"2024-09-12T12:21:31.919869+0000","last_peered":"2024-09-12T12:21:31.919869+0000","last_clean":"2024-09-12T12:21:31.919869+0000","last_became_active":"2024-09-12T12:21:30.216256+0000","last_became_peered":"2024-09-12T12:21:30.216256+0000","last_unstale":"2024-09-12T12:21:31.919869+0000","last_undegraded":"2024-09-12T12:21:31.919869+0000","last_fullsized":"2024-09-12T12:21:31.919869+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:02.339672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210550+0000","last_change":"2024-09-12T12:21:30.214244+0000","last_active":"2024-09-12T12:21:31.210550+0000","last_peered":"2024-09-12T12:21:31.210550+0000","last_clean":"2024-09-12T12:21:31.210550+0000","last_became_active":"2024-09-12T12:21:30.213464+0000","last_became_peered":"2024-09-12T12:21:30.213464+0000","last_unstale":"2024-09-12T12:21:31.210550+0000","last_undegraded":"2024-09-12T12:21:31.210550+0000","last_fullsized":"2024-09-12T12:21:31.210550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:22.724033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210473+0000","last_change":"2024-09-12T12:21:30.213222+0000","last_active":"2024-09-12T12:21:31.210473+0000","last_peered":"2024-09-12T12:21:31.210473+0000","last_clean":"2024-09-12T12:21:31.210473+0000","last_became_active":"2024-09-12T12:21:30.212819+0000","last_became_peered":"2024-09-12T12:21:30.212819+0000","last_unstale":"2024-09-12T12:21:31.210473+0000","last_undegraded":"2024-09-12T12:21:31.210473+0000","last_fullsized":"2024-09-12T12:21:31.210473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:20.425017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920825+0000","last_change":"2024-09-12T12:21:30.216805+0000","last_active":"2024-09-12T12:21:31.920825+0000","last_peered":"2024-09-12T12:21:31.920825+0000","last_clean":"2024-09-12T12:21:31.920825+0000","last_became_active":"2024-09-12T12:21:30.216287+0000","last_became_peered":"2024-09-12T12:21:30.216287+0000","last_unstale":"2024-09-12T12:21:31.920825+0000","last_undegraded":"2024-09-12T12:21:31.920825+0000","last_fullsized":"2024-09-12T12:21:31.920825+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896373+0000","last_change":"2024-09-12T12:21:30.211007+0000","last_active":"2024-09-12T12:21:31.896373+0000","last_peered":"2024-09-12T12:21:31.896373+0000","last_clean":"2024-09-12T12:21:31.896373+0000","last_became_active":"2024-09-12T12:21:30.207831+0000","last_became_peered":"2024-09-12T12:21:30.207831+0000","last_unstale":"2024-09-12T12:21:31.896373+0000","last_undegraded":"2024-09-12T12:21:31.896373+0000","last_fullsized":"2024-09-12T12:21:31.896373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:49.469026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224229+0000","last_change":"2024-09-12T12:21:30.216387+0000","last_active":"2024-09-12T12:21:30.224229+0000","last_peered":"2024-09-12T12:21:30.224229+0000","last_clean":"2024-09-12T12:21:30.224229+0000","last_became_active":"2024-09-12T12:21:30.215852+0000","last_became_peered":"2024-09-12T12:21:30.215852+0000","last_unstale":"2024-09-12T12:21:30.224229+0000","last_undegraded":"2024-09-12T12:21:30.224229+0000","last_fullsized":"2024-09-12T12:21:30.224229+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:38.107312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920593+0000","last_change":"2024-09-12T12:21:30.225623+0000","last_active":"2024-09-12T12:21:31.920593+0000","last_peered":"2024-09-12T12:21:31.920593+0000","last_clean":"2024-09-12T12:21:31.920593+0000","last_became_active":"2024-09-12T12:21:30.222578+0000","last_became_peered":"2024-09-12T12:21:30.222578+0000","last_unstale":"2024-09-12T12:21:31.920593+0000","last_undegraded":"2024-09-12T12:21:31.920593+0000","last_fullsized":"2024-09-12T12:21:31.920593+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:28.867805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919887+0000","last_change":"2024-09-12T12:21:30.217472+0000","last_active":"2024-09-12T12:21:31.919887+0000","last_peered":"2024-09-12T12:21:31.919887+0000","last_clean":"2024-09-12T12:21:31.919887+0000","last_became_active":"2024-09-12T12:21:30.217177+0000","last_became_peered":"2024-09-12T12:21:30.217177+0000","last_unstale":"2024-09-12T12:21:31.919887+0000","last_undegraded":"2024-09-12T12:21:31.919887+0000","last_fullsized":"2024-09-12T12:21:31.919887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920545+0000","last_change":"2024-09-12T12:21:30.226600+0000","last_active":"2024-09-12T12:21:31.920545+0000","last_peered":"2024-09-12T12:21:31.920545+0000","last_clean":"2024-09-12T12:21:31.920545+0000","last_became_active":"2024-09-12T12:21:30.226195+0000","last_became_peered":"2024-09-12T12:21:30.226195+0000","last_unstale":"2024-09-12T12:21:31.920545+0000","last_undegraded":"2024-09-12T12:21:31.920545+0000","last_fullsized":"2024-09-12T12:21:31.920545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:11.697205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896333+0000","last_change":"2024-09-12T12:21:30.227132+0000","last_active":"2024-09-12T12:21:31.896333+0000","last_peered":"2024-09-12T12:21:31.896333+0000","last_clean":"2024-09-12T12:21:31.896333+0000","last_became_active":"2024-09-12T12:21:30.226820+0000","last_became_peered":"2024-09-12T12:21:30.226820+0000","last_unstale":"2024-09-12T12:21:31.896333+0000","last_undegraded":"2024-09-12T12:21:31.896333+0000","last_fullsized":"2024-09-12T12:21:31.896333+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:14.450296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210369+0000","last_change":"2024-09-12T12:21:30.207017+0000","last_active":"2024-09-12T12:21:31.210369+0000","last_peered":"2024-09-12T12:21:31.210369+0000","last_clean":"2024-09-12T12:21:31.210369+0000","last_became_active":"2024-09-12T12:21:30.206281+0000","last_became_peered":"2024-09-12T12:21:30.206281+0000","last_unstale":"2024-09-12T12:21:31.210369+0000","last_undegraded":"2024-09-12T12:21:31.210369+0000","last_fullsized":"2024-09-12T12:21:31.210369+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:59.576497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209179+0000","last_change":"2024-09-12T12:21:30.208170+0000","last_active":"2024-09-12T12:21:31.209179+0000","last_peered":"2024-09-12T12:21:31.209179+0000","last_clean":"2024-09-12T12:21:31.209179+0000","last_became_active":"2024-09-12T12:21:30.207326+0000","last_became_peered":"2024-09-12T12:21:30.207326+0000","last_unstale":"2024-09-12T12:21:31.209179+0000","last_undegraded":"2024-09-12T12:21:31.209179+0000","last_fullsized":"2024-09-12T12:21:31.209179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920532+0000","last_change":"2024-09-12T12:21:30.226334+0000","last_active":"2024-09-12T12:21:31.920532+0000","last_peered":"2024-09-12T12:21:31.920532+0000","last_clean":"2024-09-12T12:21:31.920532+0000","last_became_active":"2024-09-12T12:21:30.225985+0000","last_became_peered":"2024-09-12T12:21:30.225985+0000","last_unstale":"2024-09-12T12:21:31.920532+0000","last_undegraded":"2024-09-12T12:21:31.920532+0000","last_fullsized":"2024-09-12T12:21:31.920532+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:49.456119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208773+0000","last_change":"2024-09-12T12:21:30.209539+0000","last_active":"2024-09-12T12:21:31.208773+0000","last_peered":"2024-09-12T12:21:31.208773+0000","last_clean":"2024-09-12T12:21:31.208773+0000","last_became_active":"2024-09-12T12:21:30.209224+0000","last_became_peered":"2024-09-12T12:21:30.209224+0000","last_unstale":"2024-09-12T12:21:31.208773+0000","last_undegraded":"2024-09-12T12:21:31.208773+0000","last_fullsized":"2024-09-12T12:21:31.208773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896155+0000","last_change":"2024-09-12T12:21:30.225947+0000","last_active":"2024-09-12T12:21:31.896155+0000","last_peered":"2024-09-12T12:21:31.896155+0000","last_clean":"2024-09-12T12:21:31.896155+0000","last_became_active":"2024-09-12T12:21:30.225590+0000","last_became_peered":"2024-09-12T12:21:30.225590+0000","last_unstale":"2024-09-12T12:21:31.896155+0000","last_undegraded":"2024-09-12T12:21:31.896155+0000","last_fullsized":"2024-09-12T12:21:31.896155+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.919842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232731+0000","last_change":"2024-09-12T12:21:30.224922+0000","last_active":"2024-09-12T12:21:30.232731+0000","last_peered":"2024-09-12T12:21:30.232731+0000","last_clean":"2024-09-12T12:21:30.232731+0000","last_became_active":"2024-09-12T12:21:30.224326+0000","last_became_peered":"2024-09-12T12:21:30.224326+0000","last_unstale":"2024-09-12T12:21:30.232731+0000","last_undegraded":"2024-09-12T12:21:30.232731+0000","last_fullsized":"2024-09-12T12:21:30.232731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174745+0000","last_change":"2024-09-12T12:21:28.174745+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174745+0000","last_undegraded":"2024-09-12T12:21:28.174745+0000","last_fullsized":"2024-09-12T12:21:28.174745+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920715+0000","last_change":"2024-09-12T12:21:30.222316+0000","last_active":"2024-09-12T12:21:31.920715+0000","last_peered":"2024-09-12T12:21:31.920715+0000","last_clean":"2024-09-12T12:21:31.920715+0000","last_became_active":"2024-09-12T12:21:30.221990+0000","last_became_peered":"2024-09-12T12:21:30.221990+0000","last_unstale":"2024-09-12T12:21:31.920715+0000","last_undegraded":"2024-09-12T12:21:31.920715+0000","last_fullsized":"2024-09-12T12:21:31.920715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:52.055517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210844+0000","last_change":"2024-09-12T12:21:29.219231+0000","last_active":"2024-09-12T12:21:31.210844+0000","last_peered":"2024-09-12T12:21:31.210844+0000","last_clean":"2024-09-12T12:21:31.210844+0000","last_became_active":"2024-09-12T12:21:29.218147+0000","last_became_peered":"2024-09-12T12:21:29.218147+0000","last_unstale":"2024-09-12T12:21:31.210844+0000","last_undegraded":"2024-09-12T12:21:31.210844+0000","last_fullsized":"2024-09-12T12:21:31.210844+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209841+0000","last_change":"2024-09-12T12:21:30.227814+0000","last_active":"2024-09-12T12:21:31.209841+0000","last_peered":"2024-09-12T12:21:31.209841+0000","last_clean":"2024-09-12T12:21:31.209841+0000","last_became_active":"2024-09-12T12:21:30.227419+0000","last_became_peered":"2024-09-12T12:21:30.227419+0000","last_unstale":"2024-09-12T12:21:31.209841+0000","last_undegraded":"2024-09-12T12:21:31.209841+0000","last_fullsized":"2024-09-12T12:21:31.209841+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:06:39.410795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233639+0000","last_change":"2024-09-12T12:21:29.188168+0000","last_active":"2024-09-12T12:21:30.233639+0000","last_peered":"2024-09-12T12:21:30.233639+0000","last_clean":"2024-09-12T12:21:30.233639+0000","last_became_active":"2024-09-12T12:21:29.187846+0000","last_became_peered":"2024-09-12T12:21:29.187846+0000","last_unstale":"2024-09-12T12:21:30.233639+0000","last_undegraded":"2024-09-12T12:21:30.233639+0000","last_fullsized":"2024-09-12T12:21:30.233639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224153+0000","last_change":"2024-09-12T12:21:30.217678+0000","last_active":"2024-09-12T12:21:30.224153+0000","last_peered":"2024-09-12T12:21:30.224153+0000","last_clean":"2024-09-12T12:21:30.224153+0000","last_became_active":"2024-09-12T12:21:30.216952+0000","last_became_peered":"2024-09-12T12:21:30.216952+0000","last_unstale":"2024-09-12T12:21:30.224153+0000","last_undegraded":"2024-09-12T12:21:30.224153+0000","last_fullsized":"2024-09-12T12:21:30.224153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.229864+0000","last_change":"2024-09-12T12:21:29.225871+0000","last_active":"2024-09-12T12:21:30.229864+0000","last_peered":"2024-09-12T12:21:30.229864+0000","last_clean":"2024-09-12T12:21:30.229864+0000","last_became_active":"2024-09-12T12:21:29.225182+0000","last_became_peered":"2024-09-12T12:21:29.225182+0000","last_unstale":"2024-09-12T12:21:30.229864+0000","last_undegraded":"2024-09-12T12:21:30.229864+0000","last_fullsized":"2024-09-12T12:21:30.229864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:58.566585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209173+0000","last_change":"2024-09-12T12:21:30.217628+0000","last_active":"2024-09-12T12:21:31.209173+0000","last_peered":"2024-09-12T12:21:31.209173+0000","last_clean":"2024-09-12T12:21:31.209173+0000","last_became_active":"2024-09-12T12:21:30.217335+0000","last_became_peered":"2024-09-12T12:21:30.217335+0000","last_unstale":"2024-09-12T12:21:31.209173+0000","last_undegraded":"2024-09-12T12:21:31.209173+0000","last_fullsized":"2024-09-12T12:21:31.209173+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:08.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.176529+0000","last_change":"2024-09-12T12:21:28.176529+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.176529+0000","last_undegraded":"2024-09-12T12:21:28.176529+0000","last_fullsized":"2024-09-12T12:21:28.176529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210164+0000","last_change":"2024-09-12T12:21:30.206432+0000","last_active":"2024-09-12T12:21:30.210164+0000","last_peered":"2024-09-12T12:21:30.210164+0000","last_clean":"2024-09-12T12:21:30.210164+0000","last_became_active":"2024-09-12T12:21:30.205726+0000","last_became_peered":"2024-09-12T12:21:30.205726+0000","last_unstale":"2024-09-12T12:21:30.210164+0000","last_undegraded":"2024-09-12T12:21:30.210164+0000","last_fullsized":"2024-09-12T12:21:30.210164+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:14:53.863676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209831+0000","last_change":"2024-09-12T12:21:29.207503+0000","last_active":"2024-09-12T12:21:31.209831+0000","last_peered":"2024-09-12T12:21:31.209831+0000","last_clean":"2024-09-12T12:21:31.209831+0000","last_became_active":"2024-09-12T12:21:29.206715+0000","last_became_peered":"2024-09-12T12:21:29.206715+0000","last_unstale":"2024-09-12T12:21:31.209831+0000","last_undegraded":"2024-09-12T12:21:31.209831+0000","last_fullsized":"2024-09-12T12:21:31.209831+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:18:52.540391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225377+0000","last_change":"2024-09-12T12:21:30.220672+0000","last_active":"2024-09-12T12:21:30.225377+0000","last_peered":"2024-09-12T12:21:30.225377+0000","last_clean":"2024-09-12T12:21:30.225377+0000","last_became_active":"2024-09-12T12:21:30.219208+0000","last_became_peered":"2024-09-12T12:21:30.219208+0000","last_unstale":"2024-09-12T12:21:30.225377+0000","last_undegraded":"2024-09-12T12:21:30.225377+0000","last_fullsized":"2024-09-12T12:21:30.225377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:38.730945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207603+0000","last_change":"2024-09-12T12:21:29.204647+0000","last_active":"2024-09-12T12:21:31.207603+0000","last_peered":"2024-09-12T12:21:31.207603+0000","last_clean":"2024-09-12T12:21:31.207603+0000","last_became_active":"2024-09-12T12:21:29.203941+0000","last_became_peered":"2024-09-12T12:21:29.203941+0000","last_unstale":"2024-09-12T12:21:31.207603+0000","last_undegraded":"2024-09-12T12:21:31.207603+0000","last_fullsized":"2024-09-12T12:21:31.207603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:51.045605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919837+0000","last_change":"2024-09-12T12:21:30.216996+0000","last_active":"2024-09-12T12:21:31.919837+0000","last_peered":"2024-09-12T12:21:31.919837+0000","last_clean":"2024-09-12T12:21:31.919837+0000","last_became_active":"2024-09-12T12:21:30.216631+0000","last_became_peered":"2024-09-12T12:21:30.216631+0000","last_unstale":"2024-09-12T12:21:31.919837+0000","last_undegraded":"2024-09-12T12:21:31.919837+0000","last_fullsized":"2024-09-12T12:21:31.919837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:40.449036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191140+0000","last_change":"2024-09-12T12:21:29.195971+0000","last_active":"2024-09-12T12:21:30.191140+0000","last_peered":"2024-09-12T12:21:30.191140+0000","last_clean":"2024-09-12T12:21:30.191140+0000","last_became_active":"2024-09-12T12:21:29.195461+0000","last_became_peered":"2024-09-12T12:21:29.195461+0000","last_unstale":"2024-09-12T12:21:30.191140+0000","last_undegraded":"2024-09-12T12:21:30.191140+0000","last_fullsized":"2024-09-12T12:21:30.191140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:16.138730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210986+0000","last_change":"2024-09-12T12:21:30.213037+0000","last_active":"2024-09-12T12:21:31.210986+0000","last_peered":"2024-09-12T12:21:31.210986+0000","last_clean":"2024-09-12T12:21:31.210986+0000","last_became_active":"2024-09-12T12:21:30.212297+0000","last_became_peered":"2024-09-12T12:21:30.212297+0000","last_unstale":"2024-09-12T12:21:31.210986+0000","last_undegraded":"2024-09-12T12:21:31.210986+0000","last_fullsized":"2024-09-12T12:21:31.210986+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210988+0000","last_change":"2024-09-12T12:21:29.209621+0000","last_active":"2024-09-12T12:21:31.210988+0000","last_peered":"2024-09-12T12:21:31.210988+0000","last_clean":"2024-09-12T12:21:31.210988+0000","last_became_active":"2024-09-12T12:21:29.208221+0000","last_became_peered":"2024-09-12T12:21:29.208221+0000","last_unstale":"2024-09-12T12:21:31.210988+0000","last_undegraded":"2024-09-12T12:21:31.210988+0000","last_fullsized":"2024-09-12T12:21:31.210988+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210773+0000","last_change":"2024-09-12T12:21:30.208783+0000","last_active":"2024-09-12T12:21:30.210773+0000","last_peered":"2024-09-12T12:21:30.210773+0000","last_clean":"2024-09-12T12:21:30.210773+0000","last_became_active":"2024-09-12T12:21:30.202586+0000","last_became_peered":"2024-09-12T12:21:30.202586+0000","last_unstale":"2024-09-12T12:21:30.210773+0000","last_undegraded":"2024-09-12T12:21:30.210773+0000","last_fullsized":"2024-09-12T12:21:30.210773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193677+0000","last_change":"2024-09-12T12:21:29.221269+0000","last_active":"2024-09-12T12:21:30.193677+0000","last_peered":"2024-09-12T12:21:30.193677+0000","last_clean":"2024-09-12T12:21:30.193677+0000","last_became_active":"2024-09-12T12:21:29.220934+0000","last_became_peered":"2024-09-12T12:21:29.220934+0000","last_unstale":"2024-09-12T12:21:30.193677+0000","last_undegraded":"2024-09-12T12:21:30.193677+0000","last_fullsized":"2024-09-12T12:21:30.193677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:05.831499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232623+0000","last_change":"2024-09-12T12:21:29.225993+0000","last_active":"2024-09-12T12:21:30.232623+0000","last_peered":"2024-09-12T12:21:30.232623+0000","last_clean":"2024-09-12T12:21:30.232623+0000","last_became_active":"2024-09-12T12:21:29.225250+0000","last_became_peered":"2024-09-12T12:21:29.225250+0000","last_unstale":"2024-09-12T12:21:30.232623+0000","last_undegraded":"2024-09-12T12:21:30.232623+0000","last_fullsized":"2024-09-12T12:21:30.232623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:19.415105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210545+0000","last_change":"2024-09-12T12:21:30.201827+0000","last_active":"2024-09-12T12:21:30.210545+0000","last_peered":"2024-09-12T12:21:30.210545+0000","last_clean":"2024-09-12T12:21:30.210545+0000","last_became_active":"2024-09-12T12:21:30.200750+0000","last_became_peered":"2024-09-12T12:21:30.200750+0000","last_unstale":"2024-09-12T12:21:30.210545+0000","last_undegraded":"2024-09-12T12:21:30.210545+0000","last_fullsized":"2024-09-12T12:21:30.210545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:18.349581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896201+0000","last_change":"2024-09-12T12:21:29.206800+0000","last_active":"2024-09-12T12:21:31.896201+0000","last_peered":"2024-09-12T12:21:31.896201+0000","last_clean":"2024-09-12T12:21:31.896201+0000","last_became_active":"2024-09-12T12:21:29.206391+0000","last_became_peered":"2024-09-12T12:21:29.206391+0000","last_unstale":"2024-09-12T12:21:31.896201+0000","last_undegraded":"2024-09-12T12:21:31.896201+0000","last_fullsized":"2024-09-12T12:21:31.896201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:44.852326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920202+0000","last_change":"2024-09-12T12:21:30.222358+0000","last_active":"2024-09-12T12:21:31.920202+0000","last_peered":"2024-09-12T12:21:31.920202+0000","last_clean":"2024-09-12T12:21:31.920202+0000","last_became_active":"2024-09-12T12:21:30.221945+0000","last_became_peered":"2024-09-12T12:21:30.221945+0000","last_unstale":"2024-09-12T12:21:31.920202+0000","last_undegraded":"2024-09-12T12:21:31.920202+0000","last_fullsized":"2024-09-12T12:21:31.920202+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.176104+0000","last_change":"2024-09-12T12:21:28.176104+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.176104+0000","last_undegraded":"2024-09-12T12:21:28.176104+0000","last_fullsized":"2024-09-12T12:21:28.176104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210138+0000","last_change":"2024-09-12T12:21:30.206097+0000","last_active":"2024-09-12T12:21:30.210138+0000","last_peered":"2024-09-12T12:21:30.210138+0000","last_clean":"2024-09-12T12:21:30.210138+0000","last_became_active":"2024-09-12T12:21:30.205062+0000","last_became_peered":"2024-09-12T12:21:30.205062+0000","last_unstale":"2024-09-12T12:21:30.210138+0000","last_undegraded":"2024-09-12T12:21:30.210138+0000","last_fullsized":"2024-09-12T12:21:30.210138+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:29.640882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184063+0000","last_change":"2024-09-12T12:21:29.226014+0000","last_active":"2024-09-12T12:21:30.184063+0000","last_peered":"2024-09-12T12:21:30.184063+0000","last_clean":"2024-09-12T12:21:30.184063+0000","last_became_active":"2024-09-12T12:21:29.225416+0000","last_became_peered":"2024-09-12T12:21:29.225416+0000","last_unstale":"2024-09-12T12:21:30.184063+0000","last_undegraded":"2024-09-12T12:21:30.184063+0000","last_fullsized":"2024-09-12T12:21:30.184063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174130+0000","last_change":"2024-09-12T12:21:28.174130+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174130+0000","last_undegraded":"2024-09-12T12:21:28.174130+0000","last_fullsized":"2024-09-12T12:21:28.174130+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.221978+0000","last_change":"2024-09-12T12:21:30.218431+0000","last_active":"2024-09-12T12:21:30.221978+0000","last_peered":"2024-09-12T12:21:30.221978+0000","last_clean":"2024-09-12T12:21:30.221978+0000","last_became_active":"2024-09-12T12:21:30.216964+0000","last_became_peered":"2024-09-12T12:21:30.216964+0000","last_unstale":"2024-09-12T12:21:30.221978+0000","last_undegraded":"2024-09-12T12:21:30.221978+0000","last_fullsized":"2024-09-12T12:21:30.221978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.241841+0000","last_change":"2024-09-12T12:21:29.192499+0000","last_active":"2024-09-12T12:21:30.241841+0000","last_peered":"2024-09-12T12:21:30.241841+0000","last_clean":"2024-09-12T12:21:30.241841+0000","last_became_active":"2024-09-12T12:21:29.192004+0000","last_became_peered":"2024-09-12T12:21:29.192004+0000","last_unstale":"2024-09-12T12:21:30.241841+0000","last_undegraded":"2024-09-12T12:21:30.241841+0000","last_fullsized":"2024-09-12T12:21:30.241841+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:34.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896555+0000","last_change":"2024-09-12T12:21:30.205201+0000","last_active":"2024-09-12T12:21:31.896555+0000","last_peered":"2024-09-12T12:21:31.896555+0000","last_clean":"2024-09-12T12:21:31.896555+0000","last_became_active":"2024-09-12T12:21:30.204816+0000","last_became_peered":"2024-09-12T12:21:30.204816+0000","last_unstale":"2024-09-12T12:21:31.896555+0000","last_undegraded":"2024-09-12T12:21:31.896555+0000","last_fullsized":"2024-09-12T12:21:31.896555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:15.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896398+0000","last_change":"2024-09-12T12:21:29.188763+0000","last_active":"2024-09-12T12:21:31.896398+0000","last_peered":"2024-09-12T12:21:31.896398+0000","last_clean":"2024-09-12T12:21:31.896398+0000","last_became_active":"2024-09-12T12:21:29.188359+0000","last_became_peered":"2024-09-12T12:21:29.188359+0000","last_unstale":"2024-09-12T12:21:31.896398+0000","last_undegraded":"2024-09-12T12:21:31.896398+0000","last_fullsized":"2024-09-12T12:21:31.896398+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:01:56.346195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.243691+0000","last_change":"2024-09-12T12:21:30.239463+0000","last_active":"2024-09-12T12:21:30.243691+0000","last_peered":"2024-09-12T12:21:30.243691+0000","last_clean":"2024-09-12T12:21:30.243691+0000","last_became_active":"2024-09-12T12:21:30.239104+0000","last_became_peered":"2024-09-12T12:21:30.239104+0000","last_unstale":"2024-09-12T12:21:30.243691+0000","last_undegraded":"2024-09-12T12:21:30.243691+0000","last_fullsized":"2024-09-12T12:21:30.243691+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:26.091079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.211047+0000","last_change":"2024-09-12T12:21:29.207118+0000","last_active":"2024-09-12T12:21:31.211047+0000","last_peered":"2024-09-12T12:21:31.211047+0000","last_clean":"2024-09-12T12:21:31.211047+0000","last_became_active":"2024-09-12T12:21:29.206326+0000","last_became_peered":"2024-09-12T12:21:29.206326+0000","last_unstale":"2024-09-12T12:21:31.211047+0000","last_undegraded":"2024-09-12T12:21:31.211047+0000","last_fullsized":"2024-09-12T12:21:31.211047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920288+0000","last_change":"2024-09-12T12:21:30.240598+0000","last_active":"2024-09-12T12:21:31.920288+0000","last_peered":"2024-09-12T12:21:31.920288+0000","last_clean":"2024-09-12T12:21:31.920288+0000","last_became_active":"2024-09-12T12:21:30.240256+0000","last_became_peered":"2024-09-12T12:21:30.240256+0000","last_unstale":"2024-09-12T12:21:31.920288+0000","last_undegraded":"2024-09-12T12:21:31.920288+0000","last_fullsized":"2024-09-12T12:21:31.920288+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209227+0000","last_change":"2024-09-12T12:21:29.199199+0000","last_active":"2024-09-12T12:21:31.209227+0000","last_peered":"2024-09-12T12:21:31.209227+0000","last_clean":"2024-09-12T12:21:31.209227+0000","last_became_active":"2024-09-12T12:21:29.198335+0000","last_became_peered":"2024-09-12T12:21:29.198335+0000","last_unstale":"2024-09-12T12:21:31.209227+0000","last_undegraded":"2024-09-12T12:21:31.209227+0000","last_fullsized":"2024-09-12T12:21:31.209227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:21.550250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920668+0000","last_change":"2024-09-12T12:21:30.228570+0000","last_active":"2024-09-12T12:21:31.920668+0000","last_peered":"2024-09-12T12:21:31.920668+0000","last_clean":"2024-09-12T12:21:31.920668+0000","last_became_active":"2024-09-12T12:21:30.226661+0000","last_became_peered":"2024-09-12T12:21:30.226661+0000","last_unstale":"2024-09-12T12:21:31.920668+0000","last_undegraded":"2024-09-12T12:21:31.920668+0000","last_fullsized":"2024-09-12T12:21:31.920668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:22.534750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174471+0000","last_change":"2024-09-12T12:21:28.174471+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174471+0000","last_undegraded":"2024-09-12T12:21:28.174471+0000","last_fullsized":"2024-09-12T12:21:28.174471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.175594+0000","last_change":"2024-09-12T12:21:28.175594+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.175594+0000","last_undegraded":"2024-09-12T12:21:28.175594+0000","last_fullsized":"2024-09-12T12:21:28.175594+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.223948+0000","last_change":"2024-09-12T12:21:30.213940+0000","last_active":"2024-09-12T12:21:30.223948+0000","last_peered":"2024-09-12T12:21:30.223948+0000","last_clean":"2024-09-12T12:21:30.223948+0000","last_became_active":"2024-09-12T12:21:30.204295+0000","last_became_peered":"2024-09-12T12:21:30.204295+0000","last_unstale":"2024-09-12T12:21:30.223948+0000","last_undegraded":"2024-09-12T12:21:30.223948+0000","last_fullsized":"2024-09-12T12:21:30.223948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207755+0000","last_change":"2024-09-12T12:21:29.221505+0000","last_active":"2024-09-12T12:21:31.207755+0000","last_peered":"2024-09-12T12:21:31.207755+0000","last_clean":"2024-09-12T12:21:31.207755+0000","last_became_active":"2024-09-12T12:21:29.220806+0000","last_became_peered":"2024-09-12T12:21:29.220806+0000","last_unstale":"2024-09-12T12:21:31.207755+0000","last_undegraded":"2024-09-12T12:21:31.207755+0000","last_fullsized":"2024-09-12T12:21:31.207755+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896182+0000","last_change":"2024-09-12T12:21:30.202543+0000","last_active":"2024-09-12T12:21:31.896182+0000","last_peered":"2024-09-12T12:21:31.896182+0000","last_clean":"2024-09-12T12:21:31.896182+0000","last_became_active":"2024-09-12T12:21:30.201969+0000","last_became_peered":"2024-09-12T12:21:30.201969+0000","last_unstale":"2024-09-12T12:21:31.896182+0000","last_undegraded":"2024-09-12T12:21:31.896182+0000","last_fullsized":"2024-09-12T12:21:31.896182+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:17.869098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193913+0000","last_change":"2024-09-12T12:21:29.221416+0000","last_active":"2024-09-12T12:21:30.193913+0000","last_peered":"2024-09-12T12:21:30.193913+0000","last_clean":"2024-09-12T12:21:30.193913+0000","last_became_active":"2024-09-12T12:21:29.220920+0000","last_became_peered":"2024-09-12T12:21:29.220920+0000","last_unstale":"2024-09-12T12:21:30.193913+0000","last_undegraded":"2024-09-12T12:21:30.193913+0000","last_fullsized":"2024-09-12T12:21:30.193913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:22:16.215986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.230398+0000","last_change":"2024-09-12T12:21:30.228710+0000","last_active":"2024-09-12T12:21:30.230398+0000","last_peered":"2024-09-12T12:21:30.230398+0000","last_clean":"2024-09-12T12:21:30.230398+0000","last_became_active":"2024-09-12T12:21:30.228353+0000","last_became_peered":"2024-09-12T12:21:30.228353+0000","last_unstale":"2024-09-12T12:21:30.230398+0000","last_undegraded":"2024-09-12T12:21:30.230398+0000","last_fullsized":"2024-09-12T12:21:30.230398+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:52.048471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183945+0000","last_change":"2024-09-12T12:21:29.220023+0000","last_active":"2024-09-12T12:21:30.183945+0000","last_peered":"2024-09-12T12:21:30.183945+0000","last_clean":"2024-09-12T12:21:30.183945+0000","last_became_active":"2024-09-12T12:21:29.219318+0000","last_became_peered":"2024-09-12T12:21:29.219318+0000","last_unstale":"2024-09-12T12:21:30.183945+0000","last_undegraded":"2024-09-12T12:21:30.183945+0000","last_fullsized":"2024-09-12T12:21:30.183945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920942+0000","last_change":"2024-09-12T12:21:30.226661+0000","last_active":"2024-09-12T12:21:31.920942+0000","last_peered":"2024-09-12T12:21:31.920942+0000","last_clean":"2024-09-12T12:21:31.920942+0000","last_became_active":"2024-09-12T12:21:30.225756+0000","last_became_peered":"2024-09-12T12:21:30.225756+0000","last_unstale":"2024-09-12T12:21:31.920942+0000","last_undegraded":"2024-09-12T12:21:31.920942+0000","last_fullsized":"2024-09-12T12:21:31.920942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174633+0000","last_change":"2024-09-12T12:21:28.174633+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174633+0000","last_undegraded":"2024-09-12T12:21:28.174633+0000","last_fullsized":"2024-09-12T12:21:28.174633+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.220107+0000","last_change":"2024-09-12T12:21:30.212519+0000","last_active":"2024-09-12T12:21:30.220107+0000","last_peered":"2024-09-12T12:21:30.220107+0000","last_clean":"2024-09-12T12:21:30.220107+0000","last_became_active":"2024-09-12T12:21:30.211566+0000","last_became_peered":"2024-09-12T12:21:30.211566+0000","last_unstale":"2024-09-12T12:21:30.220107+0000","last_undegraded":"2024-09-12T12:21:30.220107+0000","last_fullsized":"2024-09-12T12:21:30.220107+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:30:37.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.186811+0000","last_change":"2024-09-12T12:21:29.227541+0000","last_active":"2024-09-12T12:21:30.186811+0000","last_peered":"2024-09-12T12:21:30.186811+0000","last_clean":"2024-09-12T12:21:30.186811+0000","last_became_active":"2024-09-12T12:21:29.226501+0000","last_became_peered":"2024-09-12T12:21:29.226501+0000","last_unstale":"2024-09-12T12:21:30.186811+0000","last_undegraded":"2024-09-12T12:21:30.186811+0000","last_fullsized":"2024-09-12T12:21:30.186811+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:21.714121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210598+0000","last_change":"2024-09-12T12:21:30.204222+0000","last_active":"2024-09-12T12:21:30.210598+0000","last_peered":"2024-09-12T12:21:30.210598+0000","last_clean":"2024-09-12T12:21:30.210598+0000","last_became_active":"2024-09-12T12:21:30.203608+0000","last_became_peered":"2024-09-12T12:21:30.203608+0000","last_unstale":"2024-09-12T12:21:30.210598+0000","last_undegraded":"2024-09-12T12:21:30.210598+0000","last_fullsized":"2024-09-12T12:21:30.210598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181466+0000","last_change":"2024-09-12T12:21:29.188153+0000","last_active":"2024-09-12T12:21:30.181466+0000","last_peered":"2024-09-12T12:21:30.181466+0000","last_clean":"2024-09-12T12:21:30.181466+0000","last_became_active":"2024-09-12T12:21:29.187754+0000","last_became_peered":"2024-09-12T12:21:29.187754+0000","last_unstale":"2024-09-12T12:21:30.181466+0000","last_undegraded":"2024-09-12T12:21:30.181466+0000","last_fullsized":"2024-09-12T12:21:30.181466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184473+0000","last_change":"2024-09-12T12:21:29.227844+0000","last_active":"2024-09-12T12:21:30.184473+0000","last_peered":"2024-09-12T12:21:30.184473+0000","last_clean":"2024-09-12T12:21:30.184473+0000","last_became_active":"2024-09-12T12:21:29.227189+0000","last_became_peered":"2024-09-12T12:21:29.227189+0000","last_unstale":"2024-09-12T12:21:30.184473+0000","last_undegraded":"2024-09-12T12:21:30.184473+0000","last_fullsized":"2024-09-12T12:21:30.184473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:29.888392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209825+0000","last_change":"2024-09-12T12:21:29.203168+0000","last_active":"2024-09-12T12:21:31.209825+0000","last_peered":"2024-09-12T12:21:31.209825+0000","last_clean":"2024-09-12T12:21:31.209825+0000","last_became_active":"2024-09-12T12:21:29.201201+0000","last_became_peered":"2024-09-12T12:21:29.201201+0000","last_unstale":"2024-09-12T12:21:31.209825+0000","last_undegraded":"2024-09-12T12:21:31.209825+0000","last_fullsized":"2024-09-12T12:21:31.209825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:14.394911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207540+0000","last_change":"2024-09-12T12:21:30.221515+0000","last_active":"2024-09-12T12:21:31.207540+0000","last_peered":"2024-09-12T12:21:31.207540+0000","last_clean":"2024-09-12T12:21:31.207540+0000","last_became_active":"2024-09-12T12:21:30.221202+0000","last_became_peered":"2024-09-12T12:21:30.221202+0000","last_unstale":"2024-09-12T12:21:31.207540+0000","last_undegraded":"2024-09-12T12:21:31.207540+0000","last_fullsized":"2024-09-12T12:21:31.207540+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181458+0000","last_change":"2024-09-12T12:21:29.215337+0000","last_active":"2024-09-12T12:21:30.181458+0000","last_peered":"2024-09-12T12:21:30.181458+0000","last_clean":"2024-09-12T12:21:30.181458+0000","last_became_active":"2024-09-12T12:21:29.214887+0000","last_became_peered":"2024-09-12T12:21:29.214887+0000","last_unstale":"2024-09-12T12:21:30.181458+0000","last_undegraded":"2024-09-12T12:21:30.181458+0000","last_fullsized":"2024-09-12T12:21:30.181458+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208962+0000","last_change":"2024-09-12T12:21:30.228674+0000","last_active":"2024-09-12T12:21:31.208962+0000","last_peered":"2024-09-12T12:21:31.208962+0000","last_clean":"2024-09-12T12:21:31.208962+0000","last_became_active":"2024-09-12T12:21:30.228376+0000","last_became_peered":"2024-09-12T12:21:30.228376+0000","last_unstale":"2024-09-12T12:21:31.208962+0000","last_undegraded":"2024-09-12T12:21:31.208962+0000","last_fullsized":"2024-09-12T12:21:31.208962+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191879+0000","last_change":"2024-09-12T12:21:29.193465+0000","last_active":"2024-09-12T12:21:30.191879+0000","last_peered":"2024-09-12T12:21:30.191879+0000","last_clean":"2024-09-12T12:21:30.191879+0000","last_became_active":"2024-09-12T12:21:29.193077+0000","last_became_peered":"2024-09-12T12:21:29.193077+0000","last_unstale":"2024-09-12T12:21:30.191879+0000","last_undegraded":"2024-09-12T12:21:30.191879+0000","last_fullsized":"2024-09-12T12:21:30.191879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:13.069402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210273+0000","last_change":"2024-09-12T12:21:30.206965+0000","last_active":"2024-09-12T12:21:30.210273+0000","last_peered":"2024-09-12T12:21:30.210273+0000","last_clean":"2024-09-12T12:21:30.210273+0000","last_became_active":"2024-09-12T12:21:30.204731+0000","last_became_peered":"2024-09-12T12:21:30.204731+0000","last_unstale":"2024-09-12T12:21:30.210273+0000","last_undegraded":"2024-09-12T12:21:30.210273+0000","last_fullsized":"2024-09-12T12:21:30.210273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:55.697639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209341+0000","last_change":"2024-09-12T12:21:29.203115+0000","last_active":"2024-09-12T12:21:31.209341+0000","last_peered":"2024-09-12T12:21:31.209341+0000","last_clean":"2024-09-12T12:21:31.209341+0000","last_became_active":"2024-09-12T12:21:29.202388+0000","last_became_peered":"2024-09-12T12:21:29.202388+0000","last_unstale":"2024-09-12T12:21:31.209341+0000","last_undegraded":"2024-09-12T12:21:31.209341+0000","last_fullsized":"2024-09-12T12:21:31.209341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183862+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:21:30.183862+0000","last_peered":"2024-09-12T12:21:30.183862+0000","last_clean":"2024-09-12T12:21:30.183862+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:21:30.183862+0000","last_undegraded":"2024-09-12T12:21:30.183862+0000","last_fullsized":"2024-09-12T12:21:30.183862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":180925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1085440,"data_stored":784425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":200337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":201879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":201901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":194037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":784369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:21:34.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:21:34.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 Processor -- start 2024-09-12T12:21:34.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 -- start start 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7bdc15ac30 0x7f7bdc06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc144110 0x7f7bdc0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7bdc0727b0 0x7f7bdc072c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7bdc072490 con 0x7f7bdc15ac30 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7bdc073140 con 0x7f7bdc0727b0 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7be2019640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc15b010 con 0x7f7bdc144110 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7bdc15ac30 0x7f7bdc06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7bdc15ac30 0x7f7bdc06f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47252/0 (socket says 172.21.15.3:47252) 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/287726186 learned_addr learned my addr 172.21.15.3:0/287726186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdbfff640 1 --2- 172.21.15.3:0/287726186 >> v2:172.21.15.142:3300/0 conn(0x7f7bdc0727b0 0x7f7bdc072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdaffd640 1 --2- 172.21.15.3:0/287726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc144110 0x7f7bdc0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/287726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc144110 msgr2=0x7f7bdc0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 --2- 172.21.15.3:0/287726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc144110 0x7f7bdc0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/287726186 >> v2:172.21.15.142:3300/0 conn(0x7f7bdc0727b0 msgr2=0x7f7bdc072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 --2- 172.21.15.3:0/287726186 >> v2:172.21.15.142:3300/0 conn(0x7f7bdc0727b0 0x7f7bdc072c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.417+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/287726186 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc071ed0 con 0x7f7bdc15ac30 2024-09-12T12:21:34.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7bdb7fe640 1 --2- 172.21.15.3:0/287726186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc15ac30 0x7f7bdc06f6b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc074230 tx=0x7f7bcc01d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e51964243b327f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7bda7fc640 1 -- 172.21.15.3:0/287726186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00b040 con 0x7f7bdc15ac30 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7bda7fc640 1 -- 172.21.15.3:0/287726186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0a9650 con 0x7f7bdc15ac30 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7bda7fc640 1 -- 172.21.15.3:0/287726186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0a9a20 con 0x7f7bdc15ac30 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 -- 172.21.15.3:0/287726186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc15ac30 msgr2=0x7f7bdc06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 --2- 172.21.15.3:0/287726186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc15ac30 0x7f7bdc06f6b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc074230 tx=0x7f7bcc01d7d0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 -- 172.21.15.3:0/287726186 shutdown_connections 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 --2- 172.21.15.3:0/287726186 >> v2:172.21.15.142:3300/0 conn(0x7f7bdc0727b0 0x7f7bdc072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 --2- 172.21.15.3:0/287726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc144110 0x7f7bdc0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 --2- 172.21.15.3:0/287726186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc15ac30 0x7f7bdc06f6b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 -- 172.21.15.3:0/287726186 >> 172.21.15.3:0/287726186 conn(0x7f7bdc0b1d80 msgr2=0x7f7bdc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 -- 172.21.15.3:0/287726186 shutdown_connections 2024-09-12T12:21:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.421+0000 7f7be2019640 1 -- 172.21.15.3:0/287726186 wait complete. 2024-09-12T12:21:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 Processor -- start 2024-09-12T12:21:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 -- start start 2024-09-12T12:21:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc0727b0 0x7f7bdc1f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bdc144110 0x7f7bdc098b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc15ac30 0x7f7bdc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc0ad6c0 con 0x7f7bdc0727b0 2024-09-12T12:21:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc0ad540 con 0x7f7bdc144110 2024-09-12T12:21:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc0ad840 con 0x7f7bdc15ac30 2024-09-12T12:21:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc0727b0 0x7f7bdc1f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc0727b0 0x7f7bdc1f84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47266/0 (socket says 172.21.15.3:47266) 2024-09-12T12:21:34.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/1430042733 learned_addr learned my addr 172.21.15.3:0/1430042733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdaffd640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bdc144110 0x7f7bdc098b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdbfff640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc15ac30 0x7f7bdc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc15ac30 msgr2=0x7f7bdc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc15ac30 0x7f7bdc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bdc144110 msgr2=0x7f7bdc098b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bdc144110 0x7f7bdc098b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 -- 172.21.15.3:0/1430042733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc099070 con 0x7f7bdc0727b0 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdaffd640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bdc144110 0x7f7bdc098b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bdb7fe640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc0727b0 0x7f7bdc1f84b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00a1c0 tx=0x7f7bcc01a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bc3fff640 1 -- 172.21.15.3:0/1430042733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0b7070 con 0x7f7bdc0727b0 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bc3fff640 1 -- 172.21.15.3:0/1430042733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0b1610 con 0x7f7bdc0727b0 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc1f6780 con 0x7f7bdc0727b0 2024-09-12T12:21:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7bc3fff640 1 -- 172.21.15.3:0/1430042733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0b1a00 con 0x7f7bdc0727b0 2024-09-12T12:21:34.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc09a310 con 0x7f7bdc0727b0 2024-09-12T12:21:34.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.425+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bdc0b4a30 con 0x7f7bdc0727b0 2024-09-12T12:21:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.429+0000 7f7bc3fff640 1 -- 172.21.15.3:0/1430042733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7bcc0c2060 con 0x7f7bdc0727b0 2024-09-12T12:21:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.429+0000 7f7bc3fff640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7bc80678c0 0x7f7bc8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.429+0000 7f7bc3fff640 1 -- 172.21.15.3:0/1430042733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7bcc1200c0 con 0x7f7bdc0727b0 2024-09-12T12:21:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.429+0000 7f7bdaffd640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7bc80678c0 0x7f7bc8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.429+0000 7f7bdaffd640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7bc80678c0 0x7f7bc8069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0b0520 tx=0x7f7bd0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.433+0000 7f7bc3fff640 1 -- 172.21.15.3:0/1430042733 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7bcc0ad030 con 0x7f7bdc0727b0 2024-09-12T12:21:34.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.589+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bdc157ed0 con 0x7f7bc80678c0 2024-09-12T12:21:34.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.609+0000 7f7bc3fff640 1 -- 172.21.15.3:0/1430042733 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250086 (secure 0 0 0) 0x7f7bdc157ed0 con 0x7f7bc80678c0 2024-09-12T12:21:34.614 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:34.616 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:21:34.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7bc80678c0 msgr2=0x7f7bc8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7bc80678c0 0x7f7bc8069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0b0520 tx=0x7f7bd0008040 comp rx=0 tx=0).stop 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc0727b0 msgr2=0x7f7bdc1f84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc0727b0 0x7f7bdc1f84b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00a1c0 tx=0x7f7bcc01a570 comp rx=0 tx=0).stop 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 shutdown_connections 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7bc80678c0 0x7f7bc8069d80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7bdc15ac30 0x7f7bdc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bdc144110 0x7f7bdc098b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 --2- 172.21.15.3:0/1430042733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bdc0727b0 0x7f7bdc1f84b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 >> 172.21.15.3:0/1430042733 conn(0x7f7bdc0b1d80 msgr2=0x7f7bdc06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:34.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 shutdown_connections 2024-09-12T12:21:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.613+0000 7f7be2019640 1 -- 172.21.15.3:0/1430042733 wait complete. 2024-09-12T12:21:34.634 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-12T12:21:33.691853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225428,"kb_used_data":5032,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749292,"statfs":{"total":773094113280,"available":772863275008,"internally_reserved":0,"allocated":5152768,"data_stored":2742512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526019"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.208627+0000","last_change":"2024-09-12T12:21:30.200850+0000","last_active":"2024-09-12T12:21:30.208627+0000","last_peered":"2024-09-12T12:21:30.208627+0000","last_clean":"2024-09-12T12:21:30.208627+0000","last_became_active":"2024-09-12T12:21:30.200142+0000","last_became_peered":"2024-09-12T12:21:30.200142+0000","last_unstale":"2024-09-12T12:21:30.208627+0000","last_undegraded":"2024-09-12T12:21:30.208627+0000","last_fullsized":"2024-09-12T12:21:30.208627+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209219+0000","last_change":"2024-09-12T12:21:30.208435+0000","last_active":"2024-09-12T12:21:31.209219+0000","last_peered":"2024-09-12T12:21:31.209219+0000","last_clean":"2024-09-12T12:21:31.209219+0000","last_became_active":"2024-09-12T12:21:30.208134+0000","last_became_peered":"2024-09-12T12:21:30.208134+0000","last_unstale":"2024-09-12T12:21:31.209219+0000","last_undegraded":"2024-09-12T12:21:31.209219+0000","last_fullsized":"2024-09-12T12:21:31.209219+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:33.718518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210339+0000","last_change":"2024-09-12T12:21:30.212483+0000","last_active":"2024-09-12T12:21:31.210339+0000","last_peered":"2024-09-12T12:21:31.210339+0000","last_clean":"2024-09-12T12:21:31.210339+0000","last_became_active":"2024-09-12T12:21:30.211973+0000","last_became_peered":"2024-09-12T12:21:30.211973+0000","last_unstale":"2024-09-12T12:21:31.210339+0000","last_undegraded":"2024-09-12T12:21:31.210339+0000","last_fullsized":"2024-09-12T12:21:31.210339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:30.898304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227851+0000","last_change":"2024-09-12T12:21:30.222772+0000","last_active":"2024-09-12T12:21:30.227851+0000","last_peered":"2024-09-12T12:21:30.227851+0000","last_clean":"2024-09-12T12:21:30.227851+0000","last_became_active":"2024-09-12T12:21:30.222582+0000","last_became_peered":"2024-09-12T12:21:30.222582+0000","last_unstale":"2024-09-12T12:21:30.227851+0000","last_undegraded":"2024-09-12T12:21:30.227851+0000","last_fullsized":"2024-09-12T12:21:30.227851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:53.948158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207968+0000","last_change":"2024-09-12T12:21:30.208596+0000","last_active":"2024-09-12T12:21:31.207968+0000","last_peered":"2024-09-12T12:21:31.207968+0000","last_clean":"2024-09-12T12:21:31.207968+0000","last_became_active":"2024-09-12T12:21:30.208202+0000","last_became_peered":"2024-09-12T12:21:30.208202+0000","last_unstale":"2024-09-12T12:21:31.207968+0000","last_undegraded":"2024-09-12T12:21:31.207968+0000","last_fullsized":"2024-09-12T12:21:31.207968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210712+0000","last_change":"2024-09-12T12:21:30.204887+0000","last_active":"2024-09-12T12:21:30.210712+0000","last_peered":"2024-09-12T12:21:30.210712+0000","last_clean":"2024-09-12T12:21:30.210712+0000","last_became_active":"2024-09-12T12:21:30.204053+0000","last_became_peered":"2024-09-12T12:21:30.204053+0000","last_unstale":"2024-09-12T12:21:30.210712+0000","last_undegraded":"2024-09-12T12:21:30.210712+0000","last_fullsized":"2024-09-12T12:21:30.210712+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225399+0000","last_change":"2024-09-12T12:21:30.220921+0000","last_active":"2024-09-12T12:21:30.225399+0000","last_peered":"2024-09-12T12:21:30.225399+0000","last_clean":"2024-09-12T12:21:30.225399+0000","last_became_active":"2024-09-12T12:21:30.220376+0000","last_became_peered":"2024-09-12T12:21:30.220376+0000","last_unstale":"2024-09-12T12:21:30.225399+0000","last_undegraded":"2024-09-12T12:21:30.225399+0000","last_fullsized":"2024-09-12T12:21:30.225399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:49.354057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233115+0000","last_change":"2024-09-12T12:21:30.229582+0000","last_active":"2024-09-12T12:21:30.233115+0000","last_peered":"2024-09-12T12:21:30.233115+0000","last_clean":"2024-09-12T12:21:30.233115+0000","last_became_active":"2024-09-12T12:21:30.229229+0000","last_became_peered":"2024-09-12T12:21:30.229229+0000","last_unstale":"2024-09-12T12:21:30.233115+0000","last_undegraded":"2024-09-12T12:21:30.233115+0000","last_fullsized":"2024-09-12T12:21:30.233115+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:42:40.496362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227913+0000","last_change":"2024-09-12T12:21:30.222613+0000","last_active":"2024-09-12T12:21:30.227913+0000","last_peered":"2024-09-12T12:21:30.227913+0000","last_clean":"2024-09-12T12:21:30.227913+0000","last_became_active":"2024-09-12T12:21:30.222213+0000","last_became_peered":"2024-09-12T12:21:30.222213+0000","last_unstale":"2024-09-12T12:21:30.227913+0000","last_undegraded":"2024-09-12T12:21:30.227913+0000","last_fullsized":"2024-09-12T12:21:30.227913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:22.314764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233213+0000","last_change":"2024-09-12T12:21:30.229703+0000","last_active":"2024-09-12T12:21:30.233213+0000","last_peered":"2024-09-12T12:21:30.233213+0000","last_clean":"2024-09-12T12:21:30.233213+0000","last_became_active":"2024-09-12T12:21:30.229280+0000","last_became_peered":"2024-09-12T12:21:30.229280+0000","last_unstale":"2024-09-12T12:21:30.233213+0000","last_undegraded":"2024-09-12T12:21:30.233213+0000","last_fullsized":"2024-09-12T12:21:30.233213+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:06:44.348610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208772+0000","last_change":"2024-09-12T12:21:30.208502+0000","last_active":"2024-09-12T12:21:31.208772+0000","last_peered":"2024-09-12T12:21:31.208772+0000","last_clean":"2024-09-12T12:21:31.208772+0000","last_became_active":"2024-09-12T12:21:30.207768+0000","last_became_peered":"2024-09-12T12:21:30.207768+0000","last_unstale":"2024-09-12T12:21:31.208772+0000","last_undegraded":"2024-09-12T12:21:31.208772+0000","last_fullsized":"2024-09-12T12:21:31.208772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896095+0000","last_change":"2024-09-12T12:21:30.227759+0000","last_active":"2024-09-12T12:21:31.896095+0000","last_peered":"2024-09-12T12:21:31.896095+0000","last_clean":"2024-09-12T12:21:31.896095+0000","last_became_active":"2024-09-12T12:21:30.227324+0000","last_became_peered":"2024-09-12T12:21:30.227324+0000","last_unstale":"2024-09-12T12:21:31.896095+0000","last_undegraded":"2024-09-12T12:21:31.896095+0000","last_fullsized":"2024-09-12T12:21:31.896095+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:39.438927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.218203+0000","last_change":"2024-09-12T12:21:30.213039+0000","last_active":"2024-09-12T12:21:30.218203+0000","last_peered":"2024-09-12T12:21:30.218203+0000","last_clean":"2024-09-12T12:21:30.218203+0000","last_became_active":"2024-09-12T12:21:30.212232+0000","last_became_peered":"2024-09-12T12:21:30.212232+0000","last_unstale":"2024-09-12T12:21:30.218203+0000","last_undegraded":"2024-09-12T12:21:30.218203+0000","last_fullsized":"2024-09-12T12:21:30.218203+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:04.883109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919869+0000","last_change":"2024-09-12T12:21:30.216647+0000","last_active":"2024-09-12T12:21:31.919869+0000","last_peered":"2024-09-12T12:21:31.919869+0000","last_clean":"2024-09-12T12:21:31.919869+0000","last_became_active":"2024-09-12T12:21:30.216256+0000","last_became_peered":"2024-09-12T12:21:30.216256+0000","last_unstale":"2024-09-12T12:21:31.919869+0000","last_undegraded":"2024-09-12T12:21:31.919869+0000","last_fullsized":"2024-09-12T12:21:31.919869+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:02.339672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210550+0000","last_change":"2024-09-12T12:21:30.214244+0000","last_active":"2024-09-12T12:21:31.210550+0000","last_peered":"2024-09-12T12:21:31.210550+0000","last_clean":"2024-09-12T12:21:31.210550+0000","last_became_active":"2024-09-12T12:21:30.213464+0000","last_became_peered":"2024-09-12T12:21:30.213464+0000","last_unstale":"2024-09-12T12:21:31.210550+0000","last_undegraded":"2024-09-12T12:21:31.210550+0000","last_fullsized":"2024-09-12T12:21:31.210550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:22.724033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210473+0000","last_change":"2024-09-12T12:21:30.213222+0000","last_active":"2024-09-12T12:21:31.210473+0000","last_peered":"2024-09-12T12:21:31.210473+0000","last_clean":"2024-09-12T12:21:31.210473+0000","last_became_active":"2024-09-12T12:21:30.212819+0000","last_became_peered":"2024-09-12T12:21:30.212819+0000","last_unstale":"2024-09-12T12:21:31.210473+0000","last_undegraded":"2024-09-12T12:21:31.210473+0000","last_fullsized":"2024-09-12T12:21:31.210473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:20.425017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920825+0000","last_change":"2024-09-12T12:21:30.216805+0000","last_active":"2024-09-12T12:21:31.920825+0000","last_peered":"2024-09-12T12:21:31.920825+0000","last_clean":"2024-09-12T12:21:31.920825+0000","last_became_active":"2024-09-12T12:21:30.216287+0000","last_became_peered":"2024-09-12T12:21:30.216287+0000","last_unstale":"2024-09-12T12:21:31.920825+0000","last_undegraded":"2024-09-12T12:21:31.920825+0000","last_fullsized":"2024-09-12T12:21:31.920825+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896373+0000","last_change":"2024-09-12T12:21:30.211007+0000","last_active":"2024-09-12T12:21:31.896373+0000","last_peered":"2024-09-12T12:21:31.896373+0000","last_clean":"2024-09-12T12:21:31.896373+0000","last_became_active":"2024-09-12T12:21:30.207831+0000","last_became_peered":"2024-09-12T12:21:30.207831+0000","last_unstale":"2024-09-12T12:21:31.896373+0000","last_undegraded":"2024-09-12T12:21:31.896373+0000","last_fullsized":"2024-09-12T12:21:31.896373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:49.469026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224229+0000","last_change":"2024-09-12T12:21:30.216387+0000","last_active":"2024-09-12T12:21:30.224229+0000","last_peered":"2024-09-12T12:21:30.224229+0000","last_clean":"2024-09-12T12:21:30.224229+0000","last_became_active":"2024-09-12T12:21:30.215852+0000","last_became_peered":"2024-09-12T12:21:30.215852+0000","last_unstale":"2024-09-12T12:21:30.224229+0000","last_undegraded":"2024-09-12T12:21:30.224229+0000","last_fullsized":"2024-09-12T12:21:30.224229+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:38.107312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920593+0000","last_change":"2024-09-12T12:21:30.225623+0000","last_active":"2024-09-12T12:21:31.920593+0000","last_peered":"2024-09-12T12:21:31.920593+0000","last_clean":"2024-09-12T12:21:31.920593+0000","last_became_active":"2024-09-12T12:21:30.222578+0000","last_became_peered":"2024-09-12T12:21:30.222578+0000","last_unstale":"2024-09-12T12:21:31.920593+0000","last_undegraded":"2024-09-12T12:21:31.920593+0000","last_fullsized":"2024-09-12T12:21:31.920593+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:28.867805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919887+0000","last_change":"2024-09-12T12:21:30.217472+0000","last_active":"2024-09-12T12:21:31.919887+0000","last_peered":"2024-09-12T12:21:31.919887+0000","last_clean":"2024-09-12T12:21:31.919887+0000","last_became_active":"2024-09-12T12:21:30.217177+0000","last_became_peered":"2024-09-12T12:21:30.217177+0000","last_unstale":"2024-09-12T12:21:31.919887+0000","last_undegraded":"2024-09-12T12:21:31.919887+0000","last_fullsized":"2024-09-12T12:21:31.919887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920545+0000","last_change":"2024-09-12T12:21:30.226600+0000","last_active":"2024-09-12T12:21:31.920545+0000","last_peered":"2024-09-12T12:21:31.920545+0000","last_clean":"2024-09-12T12:21:31.920545+0000","last_became_active":"2024-09-12T12:21:30.226195+0000","last_became_peered":"2024-09-12T12:21:30.226195+0000","last_unstale":"2024-09-12T12:21:31.920545+0000","last_undegraded":"2024-09-12T12:21:31.920545+0000","last_fullsized":"2024-09-12T12:21:31.920545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:11.697205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896333+0000","last_change":"2024-09-12T12:21:30.227132+0000","last_active":"2024-09-12T12:21:31.896333+0000","last_peered":"2024-09-12T12:21:31.896333+0000","last_clean":"2024-09-12T12:21:31.896333+0000","last_became_active":"2024-09-12T12:21:30.226820+0000","last_became_peered":"2024-09-12T12:21:30.226820+0000","last_unstale":"2024-09-12T12:21:31.896333+0000","last_undegraded":"2024-09-12T12:21:31.896333+0000","last_fullsized":"2024-09-12T12:21:31.896333+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:14.450296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210369+0000","last_change":"2024-09-12T12:21:30.207017+0000","last_active":"2024-09-12T12:21:31.210369+0000","last_peered":"2024-09-12T12:21:31.210369+0000","last_clean":"2024-09-12T12:21:31.210369+0000","last_became_active":"2024-09-12T12:21:30.206281+0000","last_became_peered":"2024-09-12T12:21:30.206281+0000","last_unstale":"2024-09-12T12:21:31.210369+0000","last_undegraded":"2024-09-12T12:21:31.210369+0000","last_fullsized":"2024-09-12T12:21:31.210369+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:59.576497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209179+0000","last_change":"2024-09-12T12:21:30.208170+0000","last_active":"2024-09-12T12:21:31.209179+0000","last_peered":"2024-09-12T12:21:31.209179+0000","last_clean":"2024-09-12T12:21:31.209179+0000","last_became_active":"2024-09-12T12:21:30.207326+0000","last_became_peered":"2024-09-12T12:21:30.207326+0000","last_unstale":"2024-09-12T12:21:31.209179+0000","last_undegraded":"2024-09-12T12:21:31.209179+0000","last_fullsized":"2024-09-12T12:21:31.209179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920532+0000","last_change":"2024-09-12T12:21:30.226334+0000","last_active":"2024-09-12T12:21:31.920532+0000","last_peered":"2024-09-12T12:21:31.920532+0000","last_clean":"2024-09-12T12:21:31.920532+0000","last_became_active":"2024-09-12T12:21:30.225985+0000","last_became_peered":"2024-09-12T12:21:30.225985+0000","last_unstale":"2024-09-12T12:21:31.920532+0000","last_undegraded":"2024-09-12T12:21:31.920532+0000","last_fullsized":"2024-09-12T12:21:31.920532+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:49.456119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208773+0000","last_change":"2024-09-12T12:21:30.209539+0000","last_active":"2024-09-12T12:21:31.208773+0000","last_peered":"2024-09-12T12:21:31.208773+0000","last_clean":"2024-09-12T12:21:31.208773+0000","last_became_active":"2024-09-12T12:21:30.209224+0000","last_became_peered":"2024-09-12T12:21:30.209224+0000","last_unstale":"2024-09-12T12:21:31.208773+0000","last_undegraded":"2024-09-12T12:21:31.208773+0000","last_fullsized":"2024-09-12T12:21:31.208773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896155+0000","last_change":"2024-09-12T12:21:30.225947+0000","last_active":"2024-09-12T12:21:31.896155+0000","last_peered":"2024-09-12T12:21:31.896155+0000","last_clean":"2024-09-12T12:21:31.896155+0000","last_became_active":"2024-09-12T12:21:30.225590+0000","last_became_peered":"2024-09-12T12:21:30.225590+0000","last_unstale":"2024-09-12T12:21:31.896155+0000","last_undegraded":"2024-09-12T12:21:31.896155+0000","last_fullsized":"2024-09-12T12:21:31.896155+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.919842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232731+0000","last_change":"2024-09-12T12:21:30.224922+0000","last_active":"2024-09-12T12:21:30.232731+0000","last_peered":"2024-09-12T12:21:30.232731+0000","last_clean":"2024-09-12T12:21:30.232731+0000","last_became_active":"2024-09-12T12:21:30.224326+0000","last_became_peered":"2024-09-12T12:21:30.224326+0000","last_unstale":"2024-09-12T12:21:30.232731+0000","last_undegraded":"2024-09-12T12:21:30.232731+0000","last_fullsized":"2024-09-12T12:21:30.232731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174745+0000","last_change":"2024-09-12T12:21:28.174745+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174745+0000","last_undegraded":"2024-09-12T12:21:28.174745+0000","last_fullsized":"2024-09-12T12:21:28.174745+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920715+0000","last_change":"2024-09-12T12:21:30.222316+0000","last_active":"2024-09-12T12:21:31.920715+0000","last_peered":"2024-09-12T12:21:31.920715+0000","last_clean":"2024-09-12T12:21:31.920715+0000","last_became_active":"2024-09-12T12:21:30.221990+0000","last_became_peered":"2024-09-12T12:21:30.221990+0000","last_unstale":"2024-09-12T12:21:31.920715+0000","last_undegraded":"2024-09-12T12:21:31.920715+0000","last_fullsized":"2024-09-12T12:21:31.920715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:52.055517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210844+0000","last_change":"2024-09-12T12:21:29.219231+0000","last_active":"2024-09-12T12:21:31.210844+0000","last_peered":"2024-09-12T12:21:31.210844+0000","last_clean":"2024-09-12T12:21:31.210844+0000","last_became_active":"2024-09-12T12:21:29.218147+0000","last_became_peered":"2024-09-12T12:21:29.218147+0000","last_unstale":"2024-09-12T12:21:31.210844+0000","last_undegraded":"2024-09-12T12:21:31.210844+0000","last_fullsized":"2024-09-12T12:21:31.210844+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209841+0000","last_change":"2024-09-12T12:21:30.227814+0000","last_active":"2024-09-12T12:21:31.209841+0000","last_peered":"2024-09-12T12:21:31.209841+0000","last_clean":"2024-09-12T12:21:31.209841+0000","last_became_active":"2024-09-12T12:21:30.227419+0000","last_became_peered":"2024-09-12T12:21:30.227419+0000","last_unstale":"2024-09-12T12:21:31.209841+0000","last_undegraded":"2024-09-12T12:21:31.209841+0000","last_fullsized":"2024-09-12T12:21:31.209841+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:06:39.410795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233639+0000","last_change":"2024-09-12T12:21:29.188168+0000","last_active":"2024-09-12T12:21:30.233639+0000","last_peered":"2024-09-12T12:21:30.233639+0000","last_clean":"2024-09-12T12:21:30.233639+0000","last_became_active":"2024-09-12T12:21:29.187846+0000","last_became_peered":"2024-09-12T12:21:29.187846+0000","last_unstale":"2024-09-12T12:21:30.233639+0000","last_undegraded":"2024-09-12T12:21:30.233639+0000","last_fullsized":"2024-09-12T12:21:30.233639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224153+0000","last_change":"2024-09-12T12:21:30.217678+0000","last_active":"2024-09-12T12:21:30.224153+0000","last_peered":"2024-09-12T12:21:30.224153+0000","last_clean":"2024-09-12T12:21:30.224153+0000","last_became_active":"2024-09-12T12:21:30.216952+0000","last_became_peered":"2024-09-12T12:21:30.216952+0000","last_unstale":"2024-09-12T12:21:30.224153+0000","last_undegraded":"2024-09-12T12:21:30.224153+0000","last_fullsized":"2024-09-12T12:21:30.224153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.229864+0000","last_change":"2024-09-12T12:21:29.225871+0000","last_active":"2024-09-12T12:21:30.229864+0000","last_peered":"2024-09-12T12:21:30.229864+0000","last_clean":"2024-09-12T12:21:30.229864+0000","last_became_active":"2024-09-12T12:21:29.225182+0000","last_became_peered":"2024-09-12T12:21:29.225182+0000","last_unstale":"2024-09-12T12:21:30.229864+0000","last_undegraded":"2024-09-12T12:21:30.229864+0000","last_fullsized":"2024-09-12T12:21:30.229864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:58.566585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209173+0000","last_change":"2024-09-12T12:21:30.217628+0000","last_active":"2024-09-12T12:21:31.209173+0000","last_peered":"2024-09-12T12:21:31.209173+0000","last_clean":"2024-09-12T12:21:31.209173+0000","last_became_active":"2024-09-12T12:21:30.217335+0000","last_became_peered":"2024-09-12T12:21:30.217335+0000","last_unstale":"2024-09-12T12:21:31.209173+0000","last_undegraded":"2024-09-12T12:21:31.209173+0000","last_fullsized":"2024-09-12T12:21:31.209173+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:08.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.176529+0000","last_change":"2024-09-12T12:21:28.176529+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.176529+0000","last_undegraded":"2024-09-12T12:21:28.176529+0000","last_fullsized":"2024-09-12T12:21:28.176529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210164+0000","last_change":"2024-09-12T12:21:30.206432+0000","last_active":"2024-09-12T12:21:30.210164+0000","last_peered":"2024-09-12T12:21:30.210164+0000","last_clean":"2024-09-12T12:21:30.210164+0000","last_became_active":"2024-09-12T12:21:30.205726+0000","last_became_peered":"2024-09-12T12:21:30.205726+0000","last_unstale":"2024-09-12T12:21:30.210164+0000","last_undegraded":"2024-09-12T12:21:30.210164+0000","last_fullsized":"2024-09-12T12:21:30.210164+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:14:53.863676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209831+0000","last_change":"2024-09-12T12:21:29.207503+0000","last_active":"2024-09-12T12:21:31.209831+0000","last_peered":"2024-09-12T12:21:31.209831+0000","last_clean":"2024-09-12T12:21:31.209831+0000","last_became_active":"2024-09-12T12:21:29.206715+0000","last_became_peered":"2024-09-12T12:21:29.206715+0000","last_unstale":"2024-09-12T12:21:31.209831+0000","last_undegraded":"2024-09-12T12:21:31.209831+0000","last_fullsized":"2024-09-12T12:21:31.209831+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:18:52.540391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225377+0000","last_change":"2024-09-12T12:21:30.220672+0000","last_active":"2024-09-12T12:21:30.225377+0000","last_peered":"2024-09-12T12:21:30.225377+0000","last_clean":"2024-09-12T12:21:30.225377+0000","last_became_active":"2024-09-12T12:21:30.219208+0000","last_became_peered":"2024-09-12T12:21:30.219208+0000","last_unstale":"2024-09-12T12:21:30.225377+0000","last_undegraded":"2024-09-12T12:21:30.225377+0000","last_fullsized":"2024-09-12T12:21:30.225377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:38.730945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207603+0000","last_change":"2024-09-12T12:21:29.204647+0000","last_active":"2024-09-12T12:21:31.207603+0000","last_peered":"2024-09-12T12:21:31.207603+0000","last_clean":"2024-09-12T12:21:31.207603+0000","last_became_active":"2024-09-12T12:21:29.203941+0000","last_became_peered":"2024-09-12T12:21:29.203941+0000","last_unstale":"2024-09-12T12:21:31.207603+0000","last_undegraded":"2024-09-12T12:21:31.207603+0000","last_fullsized":"2024-09-12T12:21:31.207603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:51.045605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919837+0000","last_change":"2024-09-12T12:21:30.216996+0000","last_active":"2024-09-12T12:21:31.919837+0000","last_peered":"2024-09-12T12:21:31.919837+0000","last_clean":"2024-09-12T12:21:31.919837+0000","last_became_active":"2024-09-12T12:21:30.216631+0000","last_became_peered":"2024-09-12T12:21:30.216631+0000","last_unstale":"2024-09-12T12:21:31.919837+0000","last_undegraded":"2024-09-12T12:21:31.919837+0000","last_fullsized":"2024-09-12T12:21:31.919837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:40.449036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191140+0000","last_change":"2024-09-12T12:21:29.195971+0000","last_active":"2024-09-12T12:21:30.191140+0000","last_peered":"2024-09-12T12:21:30.191140+0000","last_clean":"2024-09-12T12:21:30.191140+0000","last_became_active":"2024-09-12T12:21:29.195461+0000","last_became_peered":"2024-09-12T12:21:29.195461+0000","last_unstale":"2024-09-12T12:21:30.191140+0000","last_undegraded":"2024-09-12T12:21:30.191140+0000","last_fullsized":"2024-09-12T12:21:30.191140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:16.138730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210986+0000","last_change":"2024-09-12T12:21:30.213037+0000","last_active":"2024-09-12T12:21:31.210986+0000","last_peered":"2024-09-12T12:21:31.210986+0000","last_clean":"2024-09-12T12:21:31.210986+0000","last_became_active":"2024-09-12T12:21:30.212297+0000","last_became_peered":"2024-09-12T12:21:30.212297+0000","last_unstale":"2024-09-12T12:21:31.210986+0000","last_undegraded":"2024-09-12T12:21:31.210986+0000","last_fullsized":"2024-09-12T12:21:31.210986+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210988+0000","last_change":"2024-09-12T12:21:29.209621+0000","last_active":"2024-09-12T12:21:31.210988+0000","last_peered":"2024-09-12T12:21:31.210988+0000","last_clean":"2024-09-12T12:21:31.210988+0000","last_became_active":"2024-09-12T12:21:29.208221+0000","last_became_peered":"2024-09-12T12:21:29.208221+0000","last_unstale":"2024-09-12T12:21:31.210988+0000","last_undegraded":"2024-09-12T12:21:31.210988+0000","last_fullsized":"2024-09-12T12:21:31.210988+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210773+0000","last_change":"2024-09-12T12:21:30.208783+0000","last_active":"2024-09-12T12:21:30.210773+0000","last_peered":"2024-09-12T12:21:30.210773+0000","last_clean":"2024-09-12T12:21:30.210773+0000","last_became_active":"2024-09-12T12:21:30.202586+0000","last_became_peered":"2024-09-12T12:21:30.202586+0000","last_unstale":"2024-09-12T12:21:30.210773+0000","last_undegraded":"2024-09-12T12:21:30.210773+0000","last_fullsized":"2024-09-12T12:21:30.210773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193677+0000","last_change":"2024-09-12T12:21:29.221269+0000","last_active":"2024-09-12T12:21:30.193677+0000","last_peered":"2024-09-12T12:21:30.193677+0000","last_clean":"2024-09-12T12:21:30.193677+0000","last_became_active":"2024-09-12T12:21:29.220934+0000","last_became_peered":"2024-09-12T12:21:29.220934+0000","last_unstale":"2024-09-12T12:21:30.193677+0000","last_undegraded":"2024-09-12T12:21:30.193677+0000","last_fullsized":"2024-09-12T12:21:30.193677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:05.831499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232623+0000","last_change":"2024-09-12T12:21:29.225993+0000","last_active":"2024-09-12T12:21:30.232623+0000","last_peered":"2024-09-12T12:21:30.232623+0000","last_clean":"2024-09-12T12:21:30.232623+0000","last_became_active":"2024-09-12T12:21:29.225250+0000","last_became_peered":"2024-09-12T12:21:29.225250+0000","last_unstale":"2024-09-12T12:21:30.232623+0000","last_undegraded":"2024-09-12T12:21:30.232623+0000","last_fullsized":"2024-09-12T12:21:30.232623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:19.415105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210545+0000","last_change":"2024-09-12T12:21:30.201827+0000","last_active":"2024-09-12T12:21:30.210545+0000","last_peered":"2024-09-12T12:21:30.210545+0000","last_clean":"2024-09-12T12:21:30.210545+0000","last_became_active":"2024-09-12T12:21:30.200750+0000","last_became_peered":"2024-09-12T12:21:30.200750+0000","last_unstale":"2024-09-12T12:21:30.210545+0000","last_undegraded":"2024-09-12T12:21:30.210545+0000","last_fullsized":"2024-09-12T12:21:30.210545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:18.349581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896201+0000","last_change":"2024-09-12T12:21:29.206800+0000","last_active":"2024-09-12T12:21:31.896201+0000","last_peered":"2024-09-12T12:21:31.896201+0000","last_clean":"2024-09-12T12:21:31.896201+0000","last_became_active":"2024-09-12T12:21:29.206391+0000","last_became_peered":"2024-09-12T12:21:29.206391+0000","last_unstale":"2024-09-12T12:21:31.896201+0000","last_undegraded":"2024-09-12T12:21:31.896201+0000","last_fullsized":"2024-09-12T12:21:31.896201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:44.852326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920202+0000","last_change":"2024-09-12T12:21:30.222358+0000","last_active":"2024-09-12T12:21:31.920202+0000","last_peered":"2024-09-12T12:21:31.920202+0000","last_clean":"2024-09-12T12:21:31.920202+0000","last_became_active":"2024-09-12T12:21:30.221945+0000","last_became_peered":"2024-09-12T12:21:30.221945+0000","last_unstale":"2024-09-12T12:21:31.920202+0000","last_undegraded":"2024-09-12T12:21:31.920202+0000","last_fullsized":"2024-09-12T12:21:31.920202+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.176104+0000","last_change":"2024-09-12T12:21:28.176104+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.176104+0000","last_undegraded":"2024-09-12T12:21:28.176104+0000","last_fullsized":"2024-09-12T12:21:28.176104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210138+0000","last_change":"2024-09-12T12:21:30.206097+0000","last_active":"2024-09-12T12:21:30.210138+0000","last_peered":"2024-09-12T12:21:30.210138+0000","last_clean":"2024-09-12T12:21:30.210138+0000","last_became_active":"2024-09-12T12:21:30.205062+0000","last_became_peered":"2024-09-12T12:21:30.205062+0000","last_unstale":"2024-09-12T12:21:30.210138+0000","last_undegraded":"2024-09-12T12:21:30.210138+0000","last_fullsized":"2024-09-12T12:21:30.210138+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:29.640882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184063+0000","last_change":"2024-09-12T12:21:29.226014+0000","last_active":"2024-09-12T12:21:30.184063+0000","last_peered":"2024-09-12T12:21:30.184063+0000","last_clean":"2024-09-12T12:21:30.184063+0000","last_became_active":"2024-09-12T12:21:29.225416+0000","last_became_peered":"2024-09-12T12:21:29.225416+0000","last_unstale":"2024-09-12T12:21:30.184063+0000","last_undegraded":"2024-09-12T12:21:30.184063+0000","last_fullsized":"2024-09-12T12:21:30.184063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174130+0000","last_change":"2024-09-12T12:21:28.174130+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174130+0000","last_undegraded":"2024-09-12T12:21:28.174130+0000","last_fullsized":"2024-09-12T12:21:28.174130+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.221978+0000","last_change":"2024-09-12T12:21:30.218431+0000","last_active":"2024-09-12T12:21:30.221978+0000","last_peered":"2024-09-12T12:21:30.221978+0000","last_clean":"2024-09-12T12:21:30.221978+0000","last_became_active":"2024-09-12T12:21:30.216964+0000","last_became_peered":"2024-09-12T12:21:30.216964+0000","last_unstale":"2024-09-12T12:21:30.221978+0000","last_undegraded":"2024-09-12T12:21:30.221978+0000","last_fullsized":"2024-09-12T12:21:30.221978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.241841+0000","last_change":"2024-09-12T12:21:29.192499+0000","last_active":"2024-09-12T12:21:30.241841+0000","last_peered":"2024-09-12T12:21:30.241841+0000","last_clean":"2024-09-12T12:21:30.241841+0000","last_became_active":"2024-09-12T12:21:29.192004+0000","last_became_peered":"2024-09-12T12:21:29.192004+0000","last_unstale":"2024-09-12T12:21:30.241841+0000","last_undegraded":"2024-09-12T12:21:30.241841+0000","last_fullsized":"2024-09-12T12:21:30.241841+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:34.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896555+0000","last_change":"2024-09-12T12:21:30.205201+0000","last_active":"2024-09-12T12:21:31.896555+0000","last_peered":"2024-09-12T12:21:31.896555+0000","last_clean":"2024-09-12T12:21:31.896555+0000","last_became_active":"2024-09-12T12:21:30.204816+0000","last_became_peered":"2024-09-12T12:21:30.204816+0000","last_unstale":"2024-09-12T12:21:31.896555+0000","last_undegraded":"2024-09-12T12:21:31.896555+0000","last_fullsized":"2024-09-12T12:21:31.896555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:15.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896398+0000","last_change":"2024-09-12T12:21:29.188763+0000","last_active":"2024-09-12T12:21:31.896398+0000","last_peered":"2024-09-12T12:21:31.896398+0000","last_clean":"2024-09-12T12:21:31.896398+0000","last_became_active":"2024-09-12T12:21:29.188359+0000","last_became_peered":"2024-09-12T12:21:29.188359+0000","last_unstale":"2024-09-12T12:21:31.896398+0000","last_undegraded":"2024-09-12T12:21:31.896398+0000","last_fullsized":"2024-09-12T12:21:31.896398+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:01:56.346195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.243691+0000","last_change":"2024-09-12T12:21:30.239463+0000","last_active":"2024-09-12T12:21:30.243691+0000","last_peered":"2024-09-12T12:21:30.243691+0000","last_clean":"2024-09-12T12:21:30.243691+0000","last_became_active":"2024-09-12T12:21:30.239104+0000","last_became_peered":"2024-09-12T12:21:30.239104+0000","last_unstale":"2024-09-12T12:21:30.243691+0000","last_undegraded":"2024-09-12T12:21:30.243691+0000","last_fullsized":"2024-09-12T12:21:30.243691+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:26.091079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.211047+0000","last_change":"2024-09-12T12:21:29.207118+0000","last_active":"2024-09-12T12:21:31.211047+0000","last_peered":"2024-09-12T12:21:31.211047+0000","last_clean":"2024-09-12T12:21:31.211047+0000","last_became_active":"2024-09-12T12:21:29.206326+0000","last_became_peered":"2024-09-12T12:21:29.206326+0000","last_unstale":"2024-09-12T12:21:31.211047+0000","last_undegraded":"2024-09-12T12:21:31.211047+0000","last_fullsized":"2024-09-12T12:21:31.211047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920288+0000","last_change":"2024-09-12T12:21:30.240598+0000","last_active":"2024-09-12T12:21:31.920288+0000","last_peered":"2024-09-12T12:21:31.920288+0000","last_clean":"2024-09-12T12:21:31.920288+0000","last_became_active":"2024-09-12T12:21:30.240256+0000","last_became_peered":"2024-09-12T12:21:30.240256+0000","last_unstale":"2024-09-12T12:21:31.920288+0000","last_undegraded":"2024-09-12T12:21:31.920288+0000","last_fullsized":"2024-09-12T12:21:31.920288+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209227+0000","last_change":"2024-09-12T12:21:29.199199+0000","last_active":"2024-09-12T12:21:31.209227+0000","last_peered":"2024-09-12T12:21:31.209227+0000","last_clean":"2024-09-12T12:21:31.209227+0000","last_became_active":"2024-09-12T12:21:29.198335+0000","last_became_peered":"2024-09-12T12:21:29.198335+0000","last_unstale":"2024-09-12T12:21:31.209227+0000","last_undegraded":"2024-09-12T12:21:31.209227+0000","last_fullsized":"2024-09-12T12:21:31.209227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:21.550250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920668+0000","last_change":"2024-09-12T12:21:30.228570+0000","last_active":"2024-09-12T12:21:31.920668+0000","last_peered":"2024-09-12T12:21:31.920668+0000","last_clean":"2024-09-12T12:21:31.920668+0000","last_became_active":"2024-09-12T12:21:30.226661+0000","last_became_peered":"2024-09-12T12:21:30.226661+0000","last_unstale":"2024-09-12T12:21:31.920668+0000","last_undegraded":"2024-09-12T12:21:31.920668+0000","last_fullsized":"2024-09-12T12:21:31.920668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:22.534750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174471+0000","last_change":"2024-09-12T12:21:28.174471+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174471+0000","last_undegraded":"2024-09-12T12:21:28.174471+0000","last_fullsized":"2024-09-12T12:21:28.174471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.175594+0000","last_change":"2024-09-12T12:21:28.175594+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.175594+0000","last_undegraded":"2024-09-12T12:21:28.175594+0000","last_fullsized":"2024-09-12T12:21:28.175594+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.223948+0000","last_change":"2024-09-12T12:21:30.213940+0000","last_active":"2024-09-12T12:21:30.223948+0000","last_peered":"2024-09-12T12:21:30.223948+0000","last_clean":"2024-09-12T12:21:30.223948+0000","last_became_active":"2024-09-12T12:21:30.204295+0000","last_became_peered":"2024-09-12T12:21:30.204295+0000","last_unstale":"2024-09-12T12:21:30.223948+0000","last_undegraded":"2024-09-12T12:21:30.223948+0000","last_fullsized":"2024-09-12T12:21:30.223948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207755+0000","last_change":"2024-09-12T12:21:29.221505+0000","last_active":"2024-09-12T12:21:31.207755+0000","last_peered":"2024-09-12T12:21:31.207755+0000","last_clean":"2024-09-12T12:21:31.207755+0000","last_became_active":"2024-09-12T12:21:29.220806+0000","last_became_peered":"2024-09-12T12:21:29.220806+0000","last_unstale":"2024-09-12T12:21:31.207755+0000","last_undegraded":"2024-09-12T12:21:31.207755+0000","last_fullsized":"2024-09-12T12:21:31.207755+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896182+0000","last_change":"2024-09-12T12:21:30.202543+0000","last_active":"2024-09-12T12:21:31.896182+0000","last_peered":"2024-09-12T12:21:31.896182+0000","last_clean":"2024-09-12T12:21:31.896182+0000","last_became_active":"2024-09-12T12:21:30.201969+0000","last_became_peered":"2024-09-12T12:21:30.201969+0000","last_unstale":"2024-09-12T12:21:31.896182+0000","last_undegraded":"2024-09-12T12:21:31.896182+0000","last_fullsized":"2024-09-12T12:21:31.896182+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:17.869098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193913+0000","last_change":"2024-09-12T12:21:29.221416+0000","last_active":"2024-09-12T12:21:30.193913+0000","last_peered":"2024-09-12T12:21:30.193913+0000","last_clean":"2024-09-12T12:21:30.193913+0000","last_became_active":"2024-09-12T12:21:29.220920+0000","last_became_peered":"2024-09-12T12:21:29.220920+0000","last_unstale":"2024-09-12T12:21:30.193913+0000","last_undegraded":"2024-09-12T12:21:30.193913+0000","last_fullsized":"2024-09-12T12:21:30.193913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:22:16.215986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.230398+0000","last_change":"2024-09-12T12:21:30.228710+0000","last_active":"2024-09-12T12:21:30.230398+0000","last_peered":"2024-09-12T12:21:30.230398+0000","last_clean":"2024-09-12T12:21:30.230398+0000","last_became_active":"2024-09-12T12:21:30.228353+0000","last_became_peered":"2024-09-12T12:21:30.228353+0000","last_unstale":"2024-09-12T12:21:30.230398+0000","last_undegraded":"2024-09-12T12:21:30.230398+0000","last_fullsized":"2024-09-12T12:21:30.230398+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:52.048471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183945+0000","last_change":"2024-09-12T12:21:29.220023+0000","last_active":"2024-09-12T12:21:30.183945+0000","last_peered":"2024-09-12T12:21:30.183945+0000","last_clean":"2024-09-12T12:21:30.183945+0000","last_became_active":"2024-09-12T12:21:29.219318+0000","last_became_peered":"2024-09-12T12:21:29.219318+0000","last_unstale":"2024-09-12T12:21:30.183945+0000","last_undegraded":"2024-09-12T12:21:30.183945+0000","last_fullsized":"2024-09-12T12:21:30.183945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920942+0000","last_change":"2024-09-12T12:21:30.226661+0000","last_active":"2024-09-12T12:21:31.920942+0000","last_peered":"2024-09-12T12:21:31.920942+0000","last_clean":"2024-09-12T12:21:31.920942+0000","last_became_active":"2024-09-12T12:21:30.225756+0000","last_became_peered":"2024-09-12T12:21:30.225756+0000","last_unstale":"2024-09-12T12:21:31.920942+0000","last_undegraded":"2024-09-12T12:21:31.920942+0000","last_fullsized":"2024-09-12T12:21:31.920942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174633+0000","last_change":"2024-09-12T12:21:28.174633+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174633+0000","last_undegraded":"2024-09-12T12:21:28.174633+0000","last_fullsized":"2024-09-12T12:21:28.174633+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.220107+0000","last_change":"2024-09-12T12:21:30.212519+0000","last_active":"2024-09-12T12:21:30.220107+0000","last_peered":"2024-09-12T12:21:30.220107+0000","last_clean":"2024-09-12T12:21:30.220107+0000","last_became_active":"2024-09-12T12:21:30.211566+0000","last_became_peered":"2024-09-12T12:21:30.211566+0000","last_unstale":"2024-09-12T12:21:30.220107+0000","last_undegraded":"2024-09-12T12:21:30.220107+0000","last_fullsized":"2024-09-12T12:21:30.220107+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:30:37.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.186811+0000","last_change":"2024-09-12T12:21:29.227541+0000","last_active":"2024-09-12T12:21:30.186811+0000","last_peered":"2024-09-12T12:21:30.186811+0000","last_clean":"2024-09-12T12:21:30.186811+0000","last_became_active":"2024-09-12T12:21:29.226501+0000","last_became_peered":"2024-09-12T12:21:29.226501+0000","last_unstale":"2024-09-12T12:21:30.186811+0000","last_undegraded":"2024-09-12T12:21:30.186811+0000","last_fullsized":"2024-09-12T12:21:30.186811+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:21.714121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210598+0000","last_change":"2024-09-12T12:21:30.204222+0000","last_active":"2024-09-12T12:21:30.210598+0000","last_peered":"2024-09-12T12:21:30.210598+0000","last_clean":"2024-09-12T12:21:30.210598+0000","last_became_active":"2024-09-12T12:21:30.203608+0000","last_became_peered":"2024-09-12T12:21:30.203608+0000","last_unstale":"2024-09-12T12:21:30.210598+0000","last_undegraded":"2024-09-12T12:21:30.210598+0000","last_fullsized":"2024-09-12T12:21:30.210598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181466+0000","last_change":"2024-09-12T12:21:29.188153+0000","last_active":"2024-09-12T12:21:30.181466+0000","last_peered":"2024-09-12T12:21:30.181466+0000","last_clean":"2024-09-12T12:21:30.181466+0000","last_became_active":"2024-09-12T12:21:29.187754+0000","last_became_peered":"2024-09-12T12:21:29.187754+0000","last_unstale":"2024-09-12T12:21:30.181466+0000","last_undegraded":"2024-09-12T12:21:30.181466+0000","last_fullsized":"2024-09-12T12:21:30.181466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184473+0000","last_change":"2024-09-12T12:21:29.227844+0000","last_active":"2024-09-12T12:21:30.184473+0000","last_peered":"2024-09-12T12:21:30.184473+0000","last_clean":"2024-09-12T12:21:30.184473+0000","last_became_active":"2024-09-12T12:21:29.227189+0000","last_became_peered":"2024-09-12T12:21:29.227189+0000","last_unstale":"2024-09-12T12:21:30.184473+0000","last_undegraded":"2024-09-12T12:21:30.184473+0000","last_fullsized":"2024-09-12T12:21:30.184473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:29.888392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209825+0000","last_change":"2024-09-12T12:21:29.203168+0000","last_active":"2024-09-12T12:21:31.209825+0000","last_peered":"2024-09-12T12:21:31.209825+0000","last_clean":"2024-09-12T12:21:31.209825+0000","last_became_active":"2024-09-12T12:21:29.201201+0000","last_became_peered":"2024-09-12T12:21:29.201201+0000","last_unstale":"2024-09-12T12:21:31.209825+0000","last_undegraded":"2024-09-12T12:21:31.209825+0000","last_fullsized":"2024-09-12T12:21:31.209825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:14.394911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207540+0000","last_change":"2024-09-12T12:21:30.221515+0000","last_active":"2024-09-12T12:21:31.207540+0000","last_peered":"2024-09-12T12:21:31.207540+0000","last_clean":"2024-09-12T12:21:31.207540+0000","last_became_active":"2024-09-12T12:21:30.221202+0000","last_became_peered":"2024-09-12T12:21:30.221202+0000","last_unstale":"2024-09-12T12:21:31.207540+0000","last_undegraded":"2024-09-12T12:21:31.207540+0000","last_fullsized":"2024-09-12T12:21:31.207540+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181458+0000","last_change":"2024-09-12T12:21:29.215337+0000","last_active":"2024-09-12T12:21:30.181458+0000","last_peered":"2024-09-12T12:21:30.181458+0000","last_clean":"2024-09-12T12:21:30.181458+0000","last_became_active":"2024-09-12T12:21:29.214887+0000","last_became_peered":"2024-09-12T12:21:29.214887+0000","last_unstale":"2024-09-12T12:21:30.181458+0000","last_undegraded":"2024-09-12T12:21:30.181458+0000","last_fullsized":"2024-09-12T12:21:30.181458+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208962+0000","last_change":"2024-09-12T12:21:30.228674+0000","last_active":"2024-09-12T12:21:31.208962+0000","last_peered":"2024-09-12T12:21:31.208962+0000","last_clean":"2024-09-12T12:21:31.208962+0000","last_became_active":"2024-09-12T12:21:30.228376+0000","last_became_peered":"2024-09-12T12:21:30.228376+0000","last_unstale":"2024-09-12T12:21:31.208962+0000","last_undegraded":"2024-09-12T12:21:31.208962+0000","last_fullsized":"2024-09-12T12:21:31.208962+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191879+0000","last_change":"2024-09-12T12:21:29.193465+0000","last_active":"2024-09-12T12:21:30.191879+0000","last_peered":"2024-09-12T12:21:30.191879+0000","last_clean":"2024-09-12T12:21:30.191879+0000","last_became_active":"2024-09-12T12:21:29.193077+0000","last_became_peered":"2024-09-12T12:21:29.193077+0000","last_unstale":"2024-09-12T12:21:30.191879+0000","last_undegraded":"2024-09-12T12:21:30.191879+0000","last_fullsized":"2024-09-12T12:21:30.191879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:13.069402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210273+0000","last_change":"2024-09-12T12:21:30.206965+0000","last_active":"2024-09-12T12:21:30.210273+0000","last_peered":"2024-09-12T12:21:30.210273+0000","last_clean":"2024-09-12T12:21:30.210273+0000","last_became_active":"2024-09-12T12:21:30.204731+0000","last_became_peered":"2024-09-12T12:21:30.204731+0000","last_unstale":"2024-09-12T12:21:30.210273+0000","last_undegraded":"2024-09-12T12:21:30.210273+0000","last_fullsized":"2024-09-12T12:21:30.210273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:55.697639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209341+0000","last_change":"2024-09-12T12:21:29.203115+0000","last_active":"2024-09-12T12:21:31.209341+0000","last_peered":"2024-09-12T12:21:31.209341+0000","last_clean":"2024-09-12T12:21:31.209341+0000","last_became_active":"2024-09-12T12:21:29.202388+0000","last_became_peered":"2024-09-12T12:21:29.202388+0000","last_unstale":"2024-09-12T12:21:31.209341+0000","last_undegraded":"2024-09-12T12:21:31.209341+0000","last_fullsized":"2024-09-12T12:21:31.209341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183862+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:21:30.183862+0000","last_peered":"2024-09-12T12:21:30.183862+0000","last_clean":"2024-09-12T12:21:30.183862+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:21:30.183862+0000","last_undegraded":"2024-09-12T12:21:30.183862+0000","last_fullsized":"2024-09-12T12:21:30.183862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":180925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1085440,"data_stored":784425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":200337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":201879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":201901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":194037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":784369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:21:34.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:21:34.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c2b97640 1 Processor -- start 2024-09-12T12:21:34.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c2b97640 1 -- start start 2024-09-12T12:21:34.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c2b97640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5bc15e5d0 0x7fb5bc0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c2b97640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c2b97640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5bc0d31a0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c2b97640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb5bc0b1060 con 0x7fb5bc15e5d0 2024-09-12T12:21:34.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c2b97640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5bc0d3480 con 0x7fb5bc0cfbe0 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c0933640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5bc15e5d0 0x7fb5bc0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5b3fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb5bc0d5050 0x7fb5bc0cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35516/0 (socket says 172.21.15.3:35516) 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5b3fff640 1 -- 172.21.15.3:0/3384527590 learned_addr learned my addr 172.21.15.3:0/3384527590 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c1134640 1 --2- 172.21.15.3:0/3384527590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5b37fe640 1 -- 172.21.15.3:0/3384527590 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307947147 0 0) 0x7fb5bc0d31a0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5b37fe640 1 -- 172.21.15.3:0/3384527590 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5a4003530 con 0x7fb5bc0d5050 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c0933640 1 -- 172.21.15.3:0/3384527590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 msgr2=0x7fb5bc0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c0933640 1 --2- 172.21.15.3:0/3384527590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c0933640 1 -- 172.21.15.3:0/3384527590 >> v1:172.21.15.3:6789/0 conn(0x7fb5bc0d5050 legacy=0x7fb5bc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.825+0000 7fb5c0933640 1 -- 172.21.15.3:0/3384527590 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc0d3760 con 0x7fb5bc15e5d0 2024-09-12T12:21:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c0933640 1 --2- 172.21.15.3:0/3384527590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc0ceb60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80097c0 tx=0x7fb5b801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe047c6139220c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b37fe640 1 -- 172.21.15.3:0/3384527590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80ab430 con 0x7fb5bc15e5d0 2024-09-12T12:21:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b37fe640 1 -- 172.21.15.3:0/3384527590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b80b0070 con 0x7fb5bc15e5d0 2024-09-12T12:21:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b37fe640 1 -- 172.21.15.3:0/3384527590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80ab7f0 con 0x7fb5bc15e5d0 2024-09-12T12:21:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3384527590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc15e5d0 msgr2=0x7fb5bc0ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3384527590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc0ceb60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80097c0 tx=0x7fb5b801c410 comp rx=0 tx=0).stop 2024-09-12T12:21:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3384527590 shutdown_connections 2024-09-12T12:21:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3384527590 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3384527590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc0ceb60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3384527590 >> 172.21.15.3:0/3384527590 conn(0x7fb5bc08c000 msgr2=0x7fb5bc0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3384527590 shutdown_connections 2024-09-12T12:21:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3384527590 wait complete. 2024-09-12T12:21:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 Processor -- start 2024-09-12T12:21:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- start start 2024-09-12T12:21:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc1fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc0d5050 0x7fb5bc0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc0b25d0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc0b2450 con 0x7fb5bc15e5d0 2024-09-12T12:21:34.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c2b97640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5bc0b2750 con 0x7fb5bc0cfbe0 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc0d5050 0x7fb5bc0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc0d5050 0x7fb5bc0c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47290/0 (socket says 172.21.15.3:47290) 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b3fff640 1 -- 172.21.15.3:0/3159518008 learned_addr learned my addr 172.21.15.3:0/3159518008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c1134640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5c0933640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc1fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b3fff640 1 -- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 msgr2=0x7fb5bc1fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b3fff640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc1fb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.829+0000 7fb5b3fff640 1 -- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5bc15e5d0 msgr2=0x7fb5bc1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b3fff640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b3fff640 1 -- 172.21.15.3:0/3159518008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc159250 con 0x7fb5bc0d5050 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5c1134640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b3fff640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc0d5050 0x7fb5bc0c9420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c3c0 tx=0x7fb5ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b17fa640 1 -- 172.21.15.3:0/3159518008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac017070 con 0x7fb5bc0d5050 2024-09-12T12:21:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b17fa640 1 -- 172.21.15.3:0/3159518008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ac005cf0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc1594f0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b17fa640 1 -- 172.21.15.3:0/3159518008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0072f0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc159ab0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc0b14f0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b17fa640 1 -- 172.21.15.3:0/3159518008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5ac01a070 con 0x7fb5bc0d5050 2024-09-12T12:21:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b17fa640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5b17fa640 1 -- 172.21.15.3:0/3159518008 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb5ac0850a0 con 0x7fb5bc0d5050 2024-09-12T12:21:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5c0933640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.833+0000 7fb5c0933640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5a80678c0 0x7fb5a8069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80af000 tx=0x7fb5b80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:34.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:34.841+0000 7fb5b17fa640 1 -- 172.21.15.3:0/3159518008 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb5ac055be0 con 0x7fb5bc0d5050 2024-09-12T12:21:35.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.001+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5bc0ba490 con 0x7fb5a80678c0 2024-09-12T12:21:35.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.017+0000 7fb5b17fa640 1 -- 172.21.15.3:0/3159518008 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250086 (secure 0 0 0) 0x7fb5bc0ba490 con 0x7fb5a80678c0 2024-09-12T12:21:35.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:35.024 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:21:35.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5a80678c0 msgr2=0x7fb5a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5a80678c0 0x7fb5a8069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80af000 tx=0x7fb5b80a7000 comp rx=0 tx=0).stop 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc0d5050 msgr2=0x7fb5bc0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc0d5050 0x7fb5bc0c9420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00c3c0 tx=0x7fb5ac00c880 comp rx=0 tx=0).stop 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 shutdown_connections 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5bc15e5d0 0x7fb5bc1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5bc0d5050 0x7fb5bc0c9420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 --2- 172.21.15.3:0/3159518008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5bc0cfbe0 0x7fb5bc1fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 >> 172.21.15.3:0/3159518008 conn(0x7fb5bc08c000 msgr2=0x7fb5bc0b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 shutdown_connections 2024-09-12T12:21:35.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.021+0000 7fb5c2b97640 1 -- 172.21.15.3:0/3159518008 wait complete. 2024-09-12T12:21:35.042 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-12T12:21:33.691853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225428,"kb_used_data":5032,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749292,"statfs":{"total":773094113280,"available":772863275008,"internally_reserved":0,"allocated":5152768,"data_stored":2742512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526019"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.208627+0000","last_change":"2024-09-12T12:21:30.200850+0000","last_active":"2024-09-12T12:21:30.208627+0000","last_peered":"2024-09-12T12:21:30.208627+0000","last_clean":"2024-09-12T12:21:30.208627+0000","last_became_active":"2024-09-12T12:21:30.200142+0000","last_became_peered":"2024-09-12T12:21:30.200142+0000","last_unstale":"2024-09-12T12:21:30.208627+0000","last_undegraded":"2024-09-12T12:21:30.208627+0000","last_fullsized":"2024-09-12T12:21:30.208627+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209219+0000","last_change":"2024-09-12T12:21:30.208435+0000","last_active":"2024-09-12T12:21:31.209219+0000","last_peered":"2024-09-12T12:21:31.209219+0000","last_clean":"2024-09-12T12:21:31.209219+0000","last_became_active":"2024-09-12T12:21:30.208134+0000","last_became_peered":"2024-09-12T12:21:30.208134+0000","last_unstale":"2024-09-12T12:21:31.209219+0000","last_undegraded":"2024-09-12T12:21:31.209219+0000","last_fullsized":"2024-09-12T12:21:31.209219+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:33.718518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210339+0000","last_change":"2024-09-12T12:21:30.212483+0000","last_active":"2024-09-12T12:21:31.210339+0000","last_peered":"2024-09-12T12:21:31.210339+0000","last_clean":"2024-09-12T12:21:31.210339+0000","last_became_active":"2024-09-12T12:21:30.211973+0000","last_became_peered":"2024-09-12T12:21:30.211973+0000","last_unstale":"2024-09-12T12:21:31.210339+0000","last_undegraded":"2024-09-12T12:21:31.210339+0000","last_fullsized":"2024-09-12T12:21:31.210339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:30.898304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227851+0000","last_change":"2024-09-12T12:21:30.222772+0000","last_active":"2024-09-12T12:21:30.227851+0000","last_peered":"2024-09-12T12:21:30.227851+0000","last_clean":"2024-09-12T12:21:30.227851+0000","last_became_active":"2024-09-12T12:21:30.222582+0000","last_became_peered":"2024-09-12T12:21:30.222582+0000","last_unstale":"2024-09-12T12:21:30.227851+0000","last_undegraded":"2024-09-12T12:21:30.227851+0000","last_fullsized":"2024-09-12T12:21:30.227851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:53.948158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207968+0000","last_change":"2024-09-12T12:21:30.208596+0000","last_active":"2024-09-12T12:21:31.207968+0000","last_peered":"2024-09-12T12:21:31.207968+0000","last_clean":"2024-09-12T12:21:31.207968+0000","last_became_active":"2024-09-12T12:21:30.208202+0000","last_became_peered":"2024-09-12T12:21:30.208202+0000","last_unstale":"2024-09-12T12:21:31.207968+0000","last_undegraded":"2024-09-12T12:21:31.207968+0000","last_fullsized":"2024-09-12T12:21:31.207968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210712+0000","last_change":"2024-09-12T12:21:30.204887+0000","last_active":"2024-09-12T12:21:30.210712+0000","last_peered":"2024-09-12T12:21:30.210712+0000","last_clean":"2024-09-12T12:21:30.210712+0000","last_became_active":"2024-09-12T12:21:30.204053+0000","last_became_peered":"2024-09-12T12:21:30.204053+0000","last_unstale":"2024-09-12T12:21:30.210712+0000","last_undegraded":"2024-09-12T12:21:30.210712+0000","last_fullsized":"2024-09-12T12:21:30.210712+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225399+0000","last_change":"2024-09-12T12:21:30.220921+0000","last_active":"2024-09-12T12:21:30.225399+0000","last_peered":"2024-09-12T12:21:30.225399+0000","last_clean":"2024-09-12T12:21:30.225399+0000","last_became_active":"2024-09-12T12:21:30.220376+0000","last_became_peered":"2024-09-12T12:21:30.220376+0000","last_unstale":"2024-09-12T12:21:30.225399+0000","last_undegraded":"2024-09-12T12:21:30.225399+0000","last_fullsized":"2024-09-12T12:21:30.225399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:49.354057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233115+0000","last_change":"2024-09-12T12:21:30.229582+0000","last_active":"2024-09-12T12:21:30.233115+0000","last_peered":"2024-09-12T12:21:30.233115+0000","last_clean":"2024-09-12T12:21:30.233115+0000","last_became_active":"2024-09-12T12:21:30.229229+0000","last_became_peered":"2024-09-12T12:21:30.229229+0000","last_unstale":"2024-09-12T12:21:30.233115+0000","last_undegraded":"2024-09-12T12:21:30.233115+0000","last_fullsized":"2024-09-12T12:21:30.233115+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:42:40.496362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227913+0000","last_change":"2024-09-12T12:21:30.222613+0000","last_active":"2024-09-12T12:21:30.227913+0000","last_peered":"2024-09-12T12:21:30.227913+0000","last_clean":"2024-09-12T12:21:30.227913+0000","last_became_active":"2024-09-12T12:21:30.222213+0000","last_became_peered":"2024-09-12T12:21:30.222213+0000","last_unstale":"2024-09-12T12:21:30.227913+0000","last_undegraded":"2024-09-12T12:21:30.227913+0000","last_fullsized":"2024-09-12T12:21:30.227913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:22.314764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233213+0000","last_change":"2024-09-12T12:21:30.229703+0000","last_active":"2024-09-12T12:21:30.233213+0000","last_peered":"2024-09-12T12:21:30.233213+0000","last_clean":"2024-09-12T12:21:30.233213+0000","last_became_active":"2024-09-12T12:21:30.229280+0000","last_became_peered":"2024-09-12T12:21:30.229280+0000","last_unstale":"2024-09-12T12:21:30.233213+0000","last_undegraded":"2024-09-12T12:21:30.233213+0000","last_fullsized":"2024-09-12T12:21:30.233213+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:06:44.348610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208772+0000","last_change":"2024-09-12T12:21:30.208502+0000","last_active":"2024-09-12T12:21:31.208772+0000","last_peered":"2024-09-12T12:21:31.208772+0000","last_clean":"2024-09-12T12:21:31.208772+0000","last_became_active":"2024-09-12T12:21:30.207768+0000","last_became_peered":"2024-09-12T12:21:30.207768+0000","last_unstale":"2024-09-12T12:21:31.208772+0000","last_undegraded":"2024-09-12T12:21:31.208772+0000","last_fullsized":"2024-09-12T12:21:31.208772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896095+0000","last_change":"2024-09-12T12:21:30.227759+0000","last_active":"2024-09-12T12:21:31.896095+0000","last_peered":"2024-09-12T12:21:31.896095+0000","last_clean":"2024-09-12T12:21:31.896095+0000","last_became_active":"2024-09-12T12:21:30.227324+0000","last_became_peered":"2024-09-12T12:21:30.227324+0000","last_unstale":"2024-09-12T12:21:31.896095+0000","last_undegraded":"2024-09-12T12:21:31.896095+0000","last_fullsized":"2024-09-12T12:21:31.896095+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:39.438927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.218203+0000","last_change":"2024-09-12T12:21:30.213039+0000","last_active":"2024-09-12T12:21:30.218203+0000","last_peered":"2024-09-12T12:21:30.218203+0000","last_clean":"2024-09-12T12:21:30.218203+0000","last_became_active":"2024-09-12T12:21:30.212232+0000","last_became_peered":"2024-09-12T12:21:30.212232+0000","last_unstale":"2024-09-12T12:21:30.218203+0000","last_undegraded":"2024-09-12T12:21:30.218203+0000","last_fullsized":"2024-09-12T12:21:30.218203+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:04.883109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919869+0000","last_change":"2024-09-12T12:21:30.216647+0000","last_active":"2024-09-12T12:21:31.919869+0000","last_peered":"2024-09-12T12:21:31.919869+0000","last_clean":"2024-09-12T12:21:31.919869+0000","last_became_active":"2024-09-12T12:21:30.216256+0000","last_became_peered":"2024-09-12T12:21:30.216256+0000","last_unstale":"2024-09-12T12:21:31.919869+0000","last_undegraded":"2024-09-12T12:21:31.919869+0000","last_fullsized":"2024-09-12T12:21:31.919869+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:02.339672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210550+0000","last_change":"2024-09-12T12:21:30.214244+0000","last_active":"2024-09-12T12:21:31.210550+0000","last_peered":"2024-09-12T12:21:31.210550+0000","last_clean":"2024-09-12T12:21:31.210550+0000","last_became_active":"2024-09-12T12:21:30.213464+0000","last_became_peered":"2024-09-12T12:21:30.213464+0000","last_unstale":"2024-09-12T12:21:31.210550+0000","last_undegraded":"2024-09-12T12:21:31.210550+0000","last_fullsized":"2024-09-12T12:21:31.210550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:22.724033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210473+0000","last_change":"2024-09-12T12:21:30.213222+0000","last_active":"2024-09-12T12:21:31.210473+0000","last_peered":"2024-09-12T12:21:31.210473+0000","last_clean":"2024-09-12T12:21:31.210473+0000","last_became_active":"2024-09-12T12:21:30.212819+0000","last_became_peered":"2024-09-12T12:21:30.212819+0000","last_unstale":"2024-09-12T12:21:31.210473+0000","last_undegraded":"2024-09-12T12:21:31.210473+0000","last_fullsized":"2024-09-12T12:21:31.210473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:20.425017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920825+0000","last_change":"2024-09-12T12:21:30.216805+0000","last_active":"2024-09-12T12:21:31.920825+0000","last_peered":"2024-09-12T12:21:31.920825+0000","last_clean":"2024-09-12T12:21:31.920825+0000","last_became_active":"2024-09-12T12:21:30.216287+0000","last_became_peered":"2024-09-12T12:21:30.216287+0000","last_unstale":"2024-09-12T12:21:31.920825+0000","last_undegraded":"2024-09-12T12:21:31.920825+0000","last_fullsized":"2024-09-12T12:21:31.920825+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896373+0000","last_change":"2024-09-12T12:21:30.211007+0000","last_active":"2024-09-12T12:21:31.896373+0000","last_peered":"2024-09-12T12:21:31.896373+0000","last_clean":"2024-09-12T12:21:31.896373+0000","last_became_active":"2024-09-12T12:21:30.207831+0000","last_became_peered":"2024-09-12T12:21:30.207831+0000","last_unstale":"2024-09-12T12:21:31.896373+0000","last_undegraded":"2024-09-12T12:21:31.896373+0000","last_fullsized":"2024-09-12T12:21:31.896373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:49.469026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224229+0000","last_change":"2024-09-12T12:21:30.216387+0000","last_active":"2024-09-12T12:21:30.224229+0000","last_peered":"2024-09-12T12:21:30.224229+0000","last_clean":"2024-09-12T12:21:30.224229+0000","last_became_active":"2024-09-12T12:21:30.215852+0000","last_became_peered":"2024-09-12T12:21:30.215852+0000","last_unstale":"2024-09-12T12:21:30.224229+0000","last_undegraded":"2024-09-12T12:21:30.224229+0000","last_fullsized":"2024-09-12T12:21:30.224229+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:38.107312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920593+0000","last_change":"2024-09-12T12:21:30.225623+0000","last_active":"2024-09-12T12:21:31.920593+0000","last_peered":"2024-09-12T12:21:31.920593+0000","last_clean":"2024-09-12T12:21:31.920593+0000","last_became_active":"2024-09-12T12:21:30.222578+0000","last_became_peered":"2024-09-12T12:21:30.222578+0000","last_unstale":"2024-09-12T12:21:31.920593+0000","last_undegraded":"2024-09-12T12:21:31.920593+0000","last_fullsized":"2024-09-12T12:21:31.920593+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:28.867805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919887+0000","last_change":"2024-09-12T12:21:30.217472+0000","last_active":"2024-09-12T12:21:31.919887+0000","last_peered":"2024-09-12T12:21:31.919887+0000","last_clean":"2024-09-12T12:21:31.919887+0000","last_became_active":"2024-09-12T12:21:30.217177+0000","last_became_peered":"2024-09-12T12:21:30.217177+0000","last_unstale":"2024-09-12T12:21:31.919887+0000","last_undegraded":"2024-09-12T12:21:31.919887+0000","last_fullsized":"2024-09-12T12:21:31.919887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920545+0000","last_change":"2024-09-12T12:21:30.226600+0000","last_active":"2024-09-12T12:21:31.920545+0000","last_peered":"2024-09-12T12:21:31.920545+0000","last_clean":"2024-09-12T12:21:31.920545+0000","last_became_active":"2024-09-12T12:21:30.226195+0000","last_became_peered":"2024-09-12T12:21:30.226195+0000","last_unstale":"2024-09-12T12:21:31.920545+0000","last_undegraded":"2024-09-12T12:21:31.920545+0000","last_fullsized":"2024-09-12T12:21:31.920545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:11.697205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896333+0000","last_change":"2024-09-12T12:21:30.227132+0000","last_active":"2024-09-12T12:21:31.896333+0000","last_peered":"2024-09-12T12:21:31.896333+0000","last_clean":"2024-09-12T12:21:31.896333+0000","last_became_active":"2024-09-12T12:21:30.226820+0000","last_became_peered":"2024-09-12T12:21:30.226820+0000","last_unstale":"2024-09-12T12:21:31.896333+0000","last_undegraded":"2024-09-12T12:21:31.896333+0000","last_fullsized":"2024-09-12T12:21:31.896333+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:14.450296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210369+0000","last_change":"2024-09-12T12:21:30.207017+0000","last_active":"2024-09-12T12:21:31.210369+0000","last_peered":"2024-09-12T12:21:31.210369+0000","last_clean":"2024-09-12T12:21:31.210369+0000","last_became_active":"2024-09-12T12:21:30.206281+0000","last_became_peered":"2024-09-12T12:21:30.206281+0000","last_unstale":"2024-09-12T12:21:31.210369+0000","last_undegraded":"2024-09-12T12:21:31.210369+0000","last_fullsized":"2024-09-12T12:21:31.210369+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:59.576497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209179+0000","last_change":"2024-09-12T12:21:30.208170+0000","last_active":"2024-09-12T12:21:31.209179+0000","last_peered":"2024-09-12T12:21:31.209179+0000","last_clean":"2024-09-12T12:21:31.209179+0000","last_became_active":"2024-09-12T12:21:30.207326+0000","last_became_peered":"2024-09-12T12:21:30.207326+0000","last_unstale":"2024-09-12T12:21:31.209179+0000","last_undegraded":"2024-09-12T12:21:31.209179+0000","last_fullsized":"2024-09-12T12:21:31.209179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920532+0000","last_change":"2024-09-12T12:21:30.226334+0000","last_active":"2024-09-12T12:21:31.920532+0000","last_peered":"2024-09-12T12:21:31.920532+0000","last_clean":"2024-09-12T12:21:31.920532+0000","last_became_active":"2024-09-12T12:21:30.225985+0000","last_became_peered":"2024-09-12T12:21:30.225985+0000","last_unstale":"2024-09-12T12:21:31.920532+0000","last_undegraded":"2024-09-12T12:21:31.920532+0000","last_fullsized":"2024-09-12T12:21:31.920532+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:49.456119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208773+0000","last_change":"2024-09-12T12:21:30.209539+0000","last_active":"2024-09-12T12:21:31.208773+0000","last_peered":"2024-09-12T12:21:31.208773+0000","last_clean":"2024-09-12T12:21:31.208773+0000","last_became_active":"2024-09-12T12:21:30.209224+0000","last_became_peered":"2024-09-12T12:21:30.209224+0000","last_unstale":"2024-09-12T12:21:31.208773+0000","last_undegraded":"2024-09-12T12:21:31.208773+0000","last_fullsized":"2024-09-12T12:21:31.208773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896155+0000","last_change":"2024-09-12T12:21:30.225947+0000","last_active":"2024-09-12T12:21:31.896155+0000","last_peered":"2024-09-12T12:21:31.896155+0000","last_clean":"2024-09-12T12:21:31.896155+0000","last_became_active":"2024-09-12T12:21:30.225590+0000","last_became_peered":"2024-09-12T12:21:30.225590+0000","last_unstale":"2024-09-12T12:21:31.896155+0000","last_undegraded":"2024-09-12T12:21:31.896155+0000","last_fullsized":"2024-09-12T12:21:31.896155+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.919842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232731+0000","last_change":"2024-09-12T12:21:30.224922+0000","last_active":"2024-09-12T12:21:30.232731+0000","last_peered":"2024-09-12T12:21:30.232731+0000","last_clean":"2024-09-12T12:21:30.232731+0000","last_became_active":"2024-09-12T12:21:30.224326+0000","last_became_peered":"2024-09-12T12:21:30.224326+0000","last_unstale":"2024-09-12T12:21:30.232731+0000","last_undegraded":"2024-09-12T12:21:30.232731+0000","last_fullsized":"2024-09-12T12:21:30.232731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174745+0000","last_change":"2024-09-12T12:21:28.174745+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174745+0000","last_undegraded":"2024-09-12T12:21:28.174745+0000","last_fullsized":"2024-09-12T12:21:28.174745+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920715+0000","last_change":"2024-09-12T12:21:30.222316+0000","last_active":"2024-09-12T12:21:31.920715+0000","last_peered":"2024-09-12T12:21:31.920715+0000","last_clean":"2024-09-12T12:21:31.920715+0000","last_became_active":"2024-09-12T12:21:30.221990+0000","last_became_peered":"2024-09-12T12:21:30.221990+0000","last_unstale":"2024-09-12T12:21:31.920715+0000","last_undegraded":"2024-09-12T12:21:31.920715+0000","last_fullsized":"2024-09-12T12:21:31.920715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:52.055517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210844+0000","last_change":"2024-09-12T12:21:29.219231+0000","last_active":"2024-09-12T12:21:31.210844+0000","last_peered":"2024-09-12T12:21:31.210844+0000","last_clean":"2024-09-12T12:21:31.210844+0000","last_became_active":"2024-09-12T12:21:29.218147+0000","last_became_peered":"2024-09-12T12:21:29.218147+0000","last_unstale":"2024-09-12T12:21:31.210844+0000","last_undegraded":"2024-09-12T12:21:31.210844+0000","last_fullsized":"2024-09-12T12:21:31.210844+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209841+0000","last_change":"2024-09-12T12:21:30.227814+0000","last_active":"2024-09-12T12:21:31.209841+0000","last_peered":"2024-09-12T12:21:31.209841+0000","last_clean":"2024-09-12T12:21:31.209841+0000","last_became_active":"2024-09-12T12:21:30.227419+0000","last_became_peered":"2024-09-12T12:21:30.227419+0000","last_unstale":"2024-09-12T12:21:31.209841+0000","last_undegraded":"2024-09-12T12:21:31.209841+0000","last_fullsized":"2024-09-12T12:21:31.209841+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:06:39.410795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233639+0000","last_change":"2024-09-12T12:21:29.188168+0000","last_active":"2024-09-12T12:21:30.233639+0000","last_peered":"2024-09-12T12:21:30.233639+0000","last_clean":"2024-09-12T12:21:30.233639+0000","last_became_active":"2024-09-12T12:21:29.187846+0000","last_became_peered":"2024-09-12T12:21:29.187846+0000","last_unstale":"2024-09-12T12:21:30.233639+0000","last_undegraded":"2024-09-12T12:21:30.233639+0000","last_fullsized":"2024-09-12T12:21:30.233639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224153+0000","last_change":"2024-09-12T12:21:30.217678+0000","last_active":"2024-09-12T12:21:30.224153+0000","last_peered":"2024-09-12T12:21:30.224153+0000","last_clean":"2024-09-12T12:21:30.224153+0000","last_became_active":"2024-09-12T12:21:30.216952+0000","last_became_peered":"2024-09-12T12:21:30.216952+0000","last_unstale":"2024-09-12T12:21:30.224153+0000","last_undegraded":"2024-09-12T12:21:30.224153+0000","last_fullsized":"2024-09-12T12:21:30.224153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.229864+0000","last_change":"2024-09-12T12:21:29.225871+0000","last_active":"2024-09-12T12:21:30.229864+0000","last_peered":"2024-09-12T12:21:30.229864+0000","last_clean":"2024-09-12T12:21:30.229864+0000","last_became_active":"2024-09-12T12:21:29.225182+0000","last_became_peered":"2024-09-12T12:21:29.225182+0000","last_unstale":"2024-09-12T12:21:30.229864+0000","last_undegraded":"2024-09-12T12:21:30.229864+0000","last_fullsized":"2024-09-12T12:21:30.229864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:58.566585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209173+0000","last_change":"2024-09-12T12:21:30.217628+0000","last_active":"2024-09-12T12:21:31.209173+0000","last_peered":"2024-09-12T12:21:31.209173+0000","last_clean":"2024-09-12T12:21:31.209173+0000","last_became_active":"2024-09-12T12:21:30.217335+0000","last_became_peered":"2024-09-12T12:21:30.217335+0000","last_unstale":"2024-09-12T12:21:31.209173+0000","last_undegraded":"2024-09-12T12:21:31.209173+0000","last_fullsized":"2024-09-12T12:21:31.209173+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:08.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.176529+0000","last_change":"2024-09-12T12:21:28.176529+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.176529+0000","last_undegraded":"2024-09-12T12:21:28.176529+0000","last_fullsized":"2024-09-12T12:21:28.176529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210164+0000","last_change":"2024-09-12T12:21:30.206432+0000","last_active":"2024-09-12T12:21:30.210164+0000","last_peered":"2024-09-12T12:21:30.210164+0000","last_clean":"2024-09-12T12:21:30.210164+0000","last_became_active":"2024-09-12T12:21:30.205726+0000","last_became_peered":"2024-09-12T12:21:30.205726+0000","last_unstale":"2024-09-12T12:21:30.210164+0000","last_undegraded":"2024-09-12T12:21:30.210164+0000","last_fullsized":"2024-09-12T12:21:30.210164+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:14:53.863676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209831+0000","last_change":"2024-09-12T12:21:29.207503+0000","last_active":"2024-09-12T12:21:31.209831+0000","last_peered":"2024-09-12T12:21:31.209831+0000","last_clean":"2024-09-12T12:21:31.209831+0000","last_became_active":"2024-09-12T12:21:29.206715+0000","last_became_peered":"2024-09-12T12:21:29.206715+0000","last_unstale":"2024-09-12T12:21:31.209831+0000","last_undegraded":"2024-09-12T12:21:31.209831+0000","last_fullsized":"2024-09-12T12:21:31.209831+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:18:52.540391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225377+0000","last_change":"2024-09-12T12:21:30.220672+0000","last_active":"2024-09-12T12:21:30.225377+0000","last_peered":"2024-09-12T12:21:30.225377+0000","last_clean":"2024-09-12T12:21:30.225377+0000","last_became_active":"2024-09-12T12:21:30.219208+0000","last_became_peered":"2024-09-12T12:21:30.219208+0000","last_unstale":"2024-09-12T12:21:30.225377+0000","last_undegraded":"2024-09-12T12:21:30.225377+0000","last_fullsized":"2024-09-12T12:21:30.225377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:38.730945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207603+0000","last_change":"2024-09-12T12:21:29.204647+0000","last_active":"2024-09-12T12:21:31.207603+0000","last_peered":"2024-09-12T12:21:31.207603+0000","last_clean":"2024-09-12T12:21:31.207603+0000","last_became_active":"2024-09-12T12:21:29.203941+0000","last_became_peered":"2024-09-12T12:21:29.203941+0000","last_unstale":"2024-09-12T12:21:31.207603+0000","last_undegraded":"2024-09-12T12:21:31.207603+0000","last_fullsized":"2024-09-12T12:21:31.207603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:51.045605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919837+0000","last_change":"2024-09-12T12:21:30.216996+0000","last_active":"2024-09-12T12:21:31.919837+0000","last_peered":"2024-09-12T12:21:31.919837+0000","last_clean":"2024-09-12T12:21:31.919837+0000","last_became_active":"2024-09-12T12:21:30.216631+0000","last_became_peered":"2024-09-12T12:21:30.216631+0000","last_unstale":"2024-09-12T12:21:31.919837+0000","last_undegraded":"2024-09-12T12:21:31.919837+0000","last_fullsized":"2024-09-12T12:21:31.919837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:40.449036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191140+0000","last_change":"2024-09-12T12:21:29.195971+0000","last_active":"2024-09-12T12:21:30.191140+0000","last_peered":"2024-09-12T12:21:30.191140+0000","last_clean":"2024-09-12T12:21:30.191140+0000","last_became_active":"2024-09-12T12:21:29.195461+0000","last_became_peered":"2024-09-12T12:21:29.195461+0000","last_unstale":"2024-09-12T12:21:30.191140+0000","last_undegraded":"2024-09-12T12:21:30.191140+0000","last_fullsized":"2024-09-12T12:21:30.191140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:16.138730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210986+0000","last_change":"2024-09-12T12:21:30.213037+0000","last_active":"2024-09-12T12:21:31.210986+0000","last_peered":"2024-09-12T12:21:31.210986+0000","last_clean":"2024-09-12T12:21:31.210986+0000","last_became_active":"2024-09-12T12:21:30.212297+0000","last_became_peered":"2024-09-12T12:21:30.212297+0000","last_unstale":"2024-09-12T12:21:31.210986+0000","last_undegraded":"2024-09-12T12:21:31.210986+0000","last_fullsized":"2024-09-12T12:21:31.210986+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210988+0000","last_change":"2024-09-12T12:21:29.209621+0000","last_active":"2024-09-12T12:21:31.210988+0000","last_peered":"2024-09-12T12:21:31.210988+0000","last_clean":"2024-09-12T12:21:31.210988+0000","last_became_active":"2024-09-12T12:21:29.208221+0000","last_became_peered":"2024-09-12T12:21:29.208221+0000","last_unstale":"2024-09-12T12:21:31.210988+0000","last_undegraded":"2024-09-12T12:21:31.210988+0000","last_fullsized":"2024-09-12T12:21:31.210988+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210773+0000","last_change":"2024-09-12T12:21:30.208783+0000","last_active":"2024-09-12T12:21:30.210773+0000","last_peered":"2024-09-12T12:21:30.210773+0000","last_clean":"2024-09-12T12:21:30.210773+0000","last_became_active":"2024-09-12T12:21:30.202586+0000","last_became_peered":"2024-09-12T12:21:30.202586+0000","last_unstale":"2024-09-12T12:21:30.210773+0000","last_undegraded":"2024-09-12T12:21:30.210773+0000","last_fullsized":"2024-09-12T12:21:30.210773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193677+0000","last_change":"2024-09-12T12:21:29.221269+0000","last_active":"2024-09-12T12:21:30.193677+0000","last_peered":"2024-09-12T12:21:30.193677+0000","last_clean":"2024-09-12T12:21:30.193677+0000","last_became_active":"2024-09-12T12:21:29.220934+0000","last_became_peered":"2024-09-12T12:21:29.220934+0000","last_unstale":"2024-09-12T12:21:30.193677+0000","last_undegraded":"2024-09-12T12:21:30.193677+0000","last_fullsized":"2024-09-12T12:21:30.193677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:05.831499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232623+0000","last_change":"2024-09-12T12:21:29.225993+0000","last_active":"2024-09-12T12:21:30.232623+0000","last_peered":"2024-09-12T12:21:30.232623+0000","last_clean":"2024-09-12T12:21:30.232623+0000","last_became_active":"2024-09-12T12:21:29.225250+0000","last_became_peered":"2024-09-12T12:21:29.225250+0000","last_unstale":"2024-09-12T12:21:30.232623+0000","last_undegraded":"2024-09-12T12:21:30.232623+0000","last_fullsized":"2024-09-12T12:21:30.232623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:19.415105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210545+0000","last_change":"2024-09-12T12:21:30.201827+0000","last_active":"2024-09-12T12:21:30.210545+0000","last_peered":"2024-09-12T12:21:30.210545+0000","last_clean":"2024-09-12T12:21:30.210545+0000","last_became_active":"2024-09-12T12:21:30.200750+0000","last_became_peered":"2024-09-12T12:21:30.200750+0000","last_unstale":"2024-09-12T12:21:30.210545+0000","last_undegraded":"2024-09-12T12:21:30.210545+0000","last_fullsized":"2024-09-12T12:21:30.210545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:18.349581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896201+0000","last_change":"2024-09-12T12:21:29.206800+0000","last_active":"2024-09-12T12:21:31.896201+0000","last_peered":"2024-09-12T12:21:31.896201+0000","last_clean":"2024-09-12T12:21:31.896201+0000","last_became_active":"2024-09-12T12:21:29.206391+0000","last_became_peered":"2024-09-12T12:21:29.206391+0000","last_unstale":"2024-09-12T12:21:31.896201+0000","last_undegraded":"2024-09-12T12:21:31.896201+0000","last_fullsized":"2024-09-12T12:21:31.896201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:44.852326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920202+0000","last_change":"2024-09-12T12:21:30.222358+0000","last_active":"2024-09-12T12:21:31.920202+0000","last_peered":"2024-09-12T12:21:31.920202+0000","last_clean":"2024-09-12T12:21:31.920202+0000","last_became_active":"2024-09-12T12:21:30.221945+0000","last_became_peered":"2024-09-12T12:21:30.221945+0000","last_unstale":"2024-09-12T12:21:31.920202+0000","last_undegraded":"2024-09-12T12:21:31.920202+0000","last_fullsized":"2024-09-12T12:21:31.920202+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.176104+0000","last_change":"2024-09-12T12:21:28.176104+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.176104+0000","last_undegraded":"2024-09-12T12:21:28.176104+0000","last_fullsized":"2024-09-12T12:21:28.176104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210138+0000","last_change":"2024-09-12T12:21:30.206097+0000","last_active":"2024-09-12T12:21:30.210138+0000","last_peered":"2024-09-12T12:21:30.210138+0000","last_clean":"2024-09-12T12:21:30.210138+0000","last_became_active":"2024-09-12T12:21:30.205062+0000","last_became_peered":"2024-09-12T12:21:30.205062+0000","last_unstale":"2024-09-12T12:21:30.210138+0000","last_undegraded":"2024-09-12T12:21:30.210138+0000","last_fullsized":"2024-09-12T12:21:30.210138+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:29.640882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184063+0000","last_change":"2024-09-12T12:21:29.226014+0000","last_active":"2024-09-12T12:21:30.184063+0000","last_peered":"2024-09-12T12:21:30.184063+0000","last_clean":"2024-09-12T12:21:30.184063+0000","last_became_active":"2024-09-12T12:21:29.225416+0000","last_became_peered":"2024-09-12T12:21:29.225416+0000","last_unstale":"2024-09-12T12:21:30.184063+0000","last_undegraded":"2024-09-12T12:21:30.184063+0000","last_fullsized":"2024-09-12T12:21:30.184063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174130+0000","last_change":"2024-09-12T12:21:28.174130+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174130+0000","last_undegraded":"2024-09-12T12:21:28.174130+0000","last_fullsized":"2024-09-12T12:21:28.174130+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.221978+0000","last_change":"2024-09-12T12:21:30.218431+0000","last_active":"2024-09-12T12:21:30.221978+0000","last_peered":"2024-09-12T12:21:30.221978+0000","last_clean":"2024-09-12T12:21:30.221978+0000","last_became_active":"2024-09-12T12:21:30.216964+0000","last_became_peered":"2024-09-12T12:21:30.216964+0000","last_unstale":"2024-09-12T12:21:30.221978+0000","last_undegraded":"2024-09-12T12:21:30.221978+0000","last_fullsized":"2024-09-12T12:21:30.221978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.241841+0000","last_change":"2024-09-12T12:21:29.192499+0000","last_active":"2024-09-12T12:21:30.241841+0000","last_peered":"2024-09-12T12:21:30.241841+0000","last_clean":"2024-09-12T12:21:30.241841+0000","last_became_active":"2024-09-12T12:21:29.192004+0000","last_became_peered":"2024-09-12T12:21:29.192004+0000","last_unstale":"2024-09-12T12:21:30.241841+0000","last_undegraded":"2024-09-12T12:21:30.241841+0000","last_fullsized":"2024-09-12T12:21:30.241841+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:34.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896555+0000","last_change":"2024-09-12T12:21:30.205201+0000","last_active":"2024-09-12T12:21:31.896555+0000","last_peered":"2024-09-12T12:21:31.896555+0000","last_clean":"2024-09-12T12:21:31.896555+0000","last_became_active":"2024-09-12T12:21:30.204816+0000","last_became_peered":"2024-09-12T12:21:30.204816+0000","last_unstale":"2024-09-12T12:21:31.896555+0000","last_undegraded":"2024-09-12T12:21:31.896555+0000","last_fullsized":"2024-09-12T12:21:31.896555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:15.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896398+0000","last_change":"2024-09-12T12:21:29.188763+0000","last_active":"2024-09-12T12:21:31.896398+0000","last_peered":"2024-09-12T12:21:31.896398+0000","last_clean":"2024-09-12T12:21:31.896398+0000","last_became_active":"2024-09-12T12:21:29.188359+0000","last_became_peered":"2024-09-12T12:21:29.188359+0000","last_unstale":"2024-09-12T12:21:31.896398+0000","last_undegraded":"2024-09-12T12:21:31.896398+0000","last_fullsized":"2024-09-12T12:21:31.896398+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:01:56.346195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.243691+0000","last_change":"2024-09-12T12:21:30.239463+0000","last_active":"2024-09-12T12:21:30.243691+0000","last_peered":"2024-09-12T12:21:30.243691+0000","last_clean":"2024-09-12T12:21:30.243691+0000","last_became_active":"2024-09-12T12:21:30.239104+0000","last_became_peered":"2024-09-12T12:21:30.239104+0000","last_unstale":"2024-09-12T12:21:30.243691+0000","last_undegraded":"2024-09-12T12:21:30.243691+0000","last_fullsized":"2024-09-12T12:21:30.243691+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:26.091079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.211047+0000","last_change":"2024-09-12T12:21:29.207118+0000","last_active":"2024-09-12T12:21:31.211047+0000","last_peered":"2024-09-12T12:21:31.211047+0000","last_clean":"2024-09-12T12:21:31.211047+0000","last_became_active":"2024-09-12T12:21:29.206326+0000","last_became_peered":"2024-09-12T12:21:29.206326+0000","last_unstale":"2024-09-12T12:21:31.211047+0000","last_undegraded":"2024-09-12T12:21:31.211047+0000","last_fullsized":"2024-09-12T12:21:31.211047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920288+0000","last_change":"2024-09-12T12:21:30.240598+0000","last_active":"2024-09-12T12:21:31.920288+0000","last_peered":"2024-09-12T12:21:31.920288+0000","last_clean":"2024-09-12T12:21:31.920288+0000","last_became_active":"2024-09-12T12:21:30.240256+0000","last_became_peered":"2024-09-12T12:21:30.240256+0000","last_unstale":"2024-09-12T12:21:31.920288+0000","last_undegraded":"2024-09-12T12:21:31.920288+0000","last_fullsized":"2024-09-12T12:21:31.920288+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209227+0000","last_change":"2024-09-12T12:21:29.199199+0000","last_active":"2024-09-12T12:21:31.209227+0000","last_peered":"2024-09-12T12:21:31.209227+0000","last_clean":"2024-09-12T12:21:31.209227+0000","last_became_active":"2024-09-12T12:21:29.198335+0000","last_became_peered":"2024-09-12T12:21:29.198335+0000","last_unstale":"2024-09-12T12:21:31.209227+0000","last_undegraded":"2024-09-12T12:21:31.209227+0000","last_fullsized":"2024-09-12T12:21:31.209227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:21.550250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920668+0000","last_change":"2024-09-12T12:21:30.228570+0000","last_active":"2024-09-12T12:21:31.920668+0000","last_peered":"2024-09-12T12:21:31.920668+0000","last_clean":"2024-09-12T12:21:31.920668+0000","last_became_active":"2024-09-12T12:21:30.226661+0000","last_became_peered":"2024-09-12T12:21:30.226661+0000","last_unstale":"2024-09-12T12:21:31.920668+0000","last_undegraded":"2024-09-12T12:21:31.920668+0000","last_fullsized":"2024-09-12T12:21:31.920668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:22.534750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174471+0000","last_change":"2024-09-12T12:21:28.174471+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174471+0000","last_undegraded":"2024-09-12T12:21:28.174471+0000","last_fullsized":"2024-09-12T12:21:28.174471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.175594+0000","last_change":"2024-09-12T12:21:28.175594+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.175594+0000","last_undegraded":"2024-09-12T12:21:28.175594+0000","last_fullsized":"2024-09-12T12:21:28.175594+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.223948+0000","last_change":"2024-09-12T12:21:30.213940+0000","last_active":"2024-09-12T12:21:30.223948+0000","last_peered":"2024-09-12T12:21:30.223948+0000","last_clean":"2024-09-12T12:21:30.223948+0000","last_became_active":"2024-09-12T12:21:30.204295+0000","last_became_peered":"2024-09-12T12:21:30.204295+0000","last_unstale":"2024-09-12T12:21:30.223948+0000","last_undegraded":"2024-09-12T12:21:30.223948+0000","last_fullsized":"2024-09-12T12:21:30.223948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207755+0000","last_change":"2024-09-12T12:21:29.221505+0000","last_active":"2024-09-12T12:21:31.207755+0000","last_peered":"2024-09-12T12:21:31.207755+0000","last_clean":"2024-09-12T12:21:31.207755+0000","last_became_active":"2024-09-12T12:21:29.220806+0000","last_became_peered":"2024-09-12T12:21:29.220806+0000","last_unstale":"2024-09-12T12:21:31.207755+0000","last_undegraded":"2024-09-12T12:21:31.207755+0000","last_fullsized":"2024-09-12T12:21:31.207755+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896182+0000","last_change":"2024-09-12T12:21:30.202543+0000","last_active":"2024-09-12T12:21:31.896182+0000","last_peered":"2024-09-12T12:21:31.896182+0000","last_clean":"2024-09-12T12:21:31.896182+0000","last_became_active":"2024-09-12T12:21:30.201969+0000","last_became_peered":"2024-09-12T12:21:30.201969+0000","last_unstale":"2024-09-12T12:21:31.896182+0000","last_undegraded":"2024-09-12T12:21:31.896182+0000","last_fullsized":"2024-09-12T12:21:31.896182+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:17.869098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193913+0000","last_change":"2024-09-12T12:21:29.221416+0000","last_active":"2024-09-12T12:21:30.193913+0000","last_peered":"2024-09-12T12:21:30.193913+0000","last_clean":"2024-09-12T12:21:30.193913+0000","last_became_active":"2024-09-12T12:21:29.220920+0000","last_became_peered":"2024-09-12T12:21:29.220920+0000","last_unstale":"2024-09-12T12:21:30.193913+0000","last_undegraded":"2024-09-12T12:21:30.193913+0000","last_fullsized":"2024-09-12T12:21:30.193913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:22:16.215986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.230398+0000","last_change":"2024-09-12T12:21:30.228710+0000","last_active":"2024-09-12T12:21:30.230398+0000","last_peered":"2024-09-12T12:21:30.230398+0000","last_clean":"2024-09-12T12:21:30.230398+0000","last_became_active":"2024-09-12T12:21:30.228353+0000","last_became_peered":"2024-09-12T12:21:30.228353+0000","last_unstale":"2024-09-12T12:21:30.230398+0000","last_undegraded":"2024-09-12T12:21:30.230398+0000","last_fullsized":"2024-09-12T12:21:30.230398+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:52.048471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183945+0000","last_change":"2024-09-12T12:21:29.220023+0000","last_active":"2024-09-12T12:21:30.183945+0000","last_peered":"2024-09-12T12:21:30.183945+0000","last_clean":"2024-09-12T12:21:30.183945+0000","last_became_active":"2024-09-12T12:21:29.219318+0000","last_became_peered":"2024-09-12T12:21:29.219318+0000","last_unstale":"2024-09-12T12:21:30.183945+0000","last_undegraded":"2024-09-12T12:21:30.183945+0000","last_fullsized":"2024-09-12T12:21:30.183945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920942+0000","last_change":"2024-09-12T12:21:30.226661+0000","last_active":"2024-09-12T12:21:31.920942+0000","last_peered":"2024-09-12T12:21:31.920942+0000","last_clean":"2024-09-12T12:21:31.920942+0000","last_became_active":"2024-09-12T12:21:30.225756+0000","last_became_peered":"2024-09-12T12:21:30.225756+0000","last_unstale":"2024-09-12T12:21:31.920942+0000","last_undegraded":"2024-09-12T12:21:31.920942+0000","last_fullsized":"2024-09-12T12:21:31.920942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-12T12:21:28.174633+0000","last_change":"2024-09-12T12:21:28.174633+0000","last_active":"2024-09-12T12:21:28.153915+0000","last_peered":"2024-09-12T12:21:28.153915+0000","last_clean":"2024-09-12T12:21:28.153915+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:28.174633+0000","last_undegraded":"2024-09-12T12:21:28.174633+0000","last_fullsized":"2024-09-12T12:21:28.174633+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.220107+0000","last_change":"2024-09-12T12:21:30.212519+0000","last_active":"2024-09-12T12:21:30.220107+0000","last_peered":"2024-09-12T12:21:30.220107+0000","last_clean":"2024-09-12T12:21:30.220107+0000","last_became_active":"2024-09-12T12:21:30.211566+0000","last_became_peered":"2024-09-12T12:21:30.211566+0000","last_unstale":"2024-09-12T12:21:30.220107+0000","last_undegraded":"2024-09-12T12:21:30.220107+0000","last_fullsized":"2024-09-12T12:21:30.220107+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:30:37.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.186811+0000","last_change":"2024-09-12T12:21:29.227541+0000","last_active":"2024-09-12T12:21:30.186811+0000","last_peered":"2024-09-12T12:21:30.186811+0000","last_clean":"2024-09-12T12:21:30.186811+0000","last_became_active":"2024-09-12T12:21:29.226501+0000","last_became_peered":"2024-09-12T12:21:29.226501+0000","last_unstale":"2024-09-12T12:21:30.186811+0000","last_undegraded":"2024-09-12T12:21:30.186811+0000","last_fullsized":"2024-09-12T12:21:30.186811+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:21.714121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210598+0000","last_change":"2024-09-12T12:21:30.204222+0000","last_active":"2024-09-12T12:21:30.210598+0000","last_peered":"2024-09-12T12:21:30.210598+0000","last_clean":"2024-09-12T12:21:30.210598+0000","last_became_active":"2024-09-12T12:21:30.203608+0000","last_became_peered":"2024-09-12T12:21:30.203608+0000","last_unstale":"2024-09-12T12:21:30.210598+0000","last_undegraded":"2024-09-12T12:21:30.210598+0000","last_fullsized":"2024-09-12T12:21:30.210598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181466+0000","last_change":"2024-09-12T12:21:29.188153+0000","last_active":"2024-09-12T12:21:30.181466+0000","last_peered":"2024-09-12T12:21:30.181466+0000","last_clean":"2024-09-12T12:21:30.181466+0000","last_became_active":"2024-09-12T12:21:29.187754+0000","last_became_peered":"2024-09-12T12:21:29.187754+0000","last_unstale":"2024-09-12T12:21:30.181466+0000","last_undegraded":"2024-09-12T12:21:30.181466+0000","last_fullsized":"2024-09-12T12:21:30.181466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184473+0000","last_change":"2024-09-12T12:21:29.227844+0000","last_active":"2024-09-12T12:21:30.184473+0000","last_peered":"2024-09-12T12:21:30.184473+0000","last_clean":"2024-09-12T12:21:30.184473+0000","last_became_active":"2024-09-12T12:21:29.227189+0000","last_became_peered":"2024-09-12T12:21:29.227189+0000","last_unstale":"2024-09-12T12:21:30.184473+0000","last_undegraded":"2024-09-12T12:21:30.184473+0000","last_fullsized":"2024-09-12T12:21:30.184473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:29.888392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:21:29.163827+0000","last_change":"2024-09-12T12:21:29.163827+0000","last_active":"2024-09-12T12:21:29.163827+0000","last_peered":"2024-09-12T12:21:29.163827+0000","last_clean":"2024-09-12T12:21:29.163827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:21:29.163827+0000","last_undegraded":"2024-09-12T12:21:29.163827+0000","last_fullsized":"2024-09-12T12:21:29.163827+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209825+0000","last_change":"2024-09-12T12:21:29.203168+0000","last_active":"2024-09-12T12:21:31.209825+0000","last_peered":"2024-09-12T12:21:31.209825+0000","last_clean":"2024-09-12T12:21:31.209825+0000","last_became_active":"2024-09-12T12:21:29.201201+0000","last_became_peered":"2024-09-12T12:21:29.201201+0000","last_unstale":"2024-09-12T12:21:31.209825+0000","last_undegraded":"2024-09-12T12:21:31.209825+0000","last_fullsized":"2024-09-12T12:21:31.209825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:14.394911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207540+0000","last_change":"2024-09-12T12:21:30.221515+0000","last_active":"2024-09-12T12:21:31.207540+0000","last_peered":"2024-09-12T12:21:31.207540+0000","last_clean":"2024-09-12T12:21:31.207540+0000","last_became_active":"2024-09-12T12:21:30.221202+0000","last_became_peered":"2024-09-12T12:21:30.221202+0000","last_unstale":"2024-09-12T12:21:31.207540+0000","last_undegraded":"2024-09-12T12:21:31.207540+0000","last_fullsized":"2024-09-12T12:21:31.207540+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181458+0000","last_change":"2024-09-12T12:21:29.215337+0000","last_active":"2024-09-12T12:21:30.181458+0000","last_peered":"2024-09-12T12:21:30.181458+0000","last_clean":"2024-09-12T12:21:30.181458+0000","last_became_active":"2024-09-12T12:21:29.214887+0000","last_became_peered":"2024-09-12T12:21:29.214887+0000","last_unstale":"2024-09-12T12:21:30.181458+0000","last_undegraded":"2024-09-12T12:21:30.181458+0000","last_fullsized":"2024-09-12T12:21:30.181458+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208962+0000","last_change":"2024-09-12T12:21:30.228674+0000","last_active":"2024-09-12T12:21:31.208962+0000","last_peered":"2024-09-12T12:21:31.208962+0000","last_clean":"2024-09-12T12:21:31.208962+0000","last_became_active":"2024-09-12T12:21:30.228376+0000","last_became_peered":"2024-09-12T12:21:30.228376+0000","last_unstale":"2024-09-12T12:21:31.208962+0000","last_undegraded":"2024-09-12T12:21:31.208962+0000","last_fullsized":"2024-09-12T12:21:31.208962+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191879+0000","last_change":"2024-09-12T12:21:29.193465+0000","last_active":"2024-09-12T12:21:30.191879+0000","last_peered":"2024-09-12T12:21:30.191879+0000","last_clean":"2024-09-12T12:21:30.191879+0000","last_became_active":"2024-09-12T12:21:29.193077+0000","last_became_peered":"2024-09-12T12:21:29.193077+0000","last_unstale":"2024-09-12T12:21:30.191879+0000","last_undegraded":"2024-09-12T12:21:30.191879+0000","last_fullsized":"2024-09-12T12:21:30.191879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:13.069402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210273+0000","last_change":"2024-09-12T12:21:30.206965+0000","last_active":"2024-09-12T12:21:30.210273+0000","last_peered":"2024-09-12T12:21:30.210273+0000","last_clean":"2024-09-12T12:21:30.210273+0000","last_became_active":"2024-09-12T12:21:30.204731+0000","last_became_peered":"2024-09-12T12:21:30.204731+0000","last_unstale":"2024-09-12T12:21:30.210273+0000","last_undegraded":"2024-09-12T12:21:30.210273+0000","last_fullsized":"2024-09-12T12:21:30.210273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:55.697639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209341+0000","last_change":"2024-09-12T12:21:29.203115+0000","last_active":"2024-09-12T12:21:31.209341+0000","last_peered":"2024-09-12T12:21:31.209341+0000","last_clean":"2024-09-12T12:21:31.209341+0000","last_became_active":"2024-09-12T12:21:29.202388+0000","last_became_peered":"2024-09-12T12:21:29.202388+0000","last_unstale":"2024-09-12T12:21:31.209341+0000","last_undegraded":"2024-09-12T12:21:31.209341+0000","last_fullsized":"2024-09-12T12:21:31.209341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183862+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:21:30.183862+0000","last_peered":"2024-09-12T12:21:30.183862+0000","last_clean":"2024-09-12T12:21:30.183862+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:21:30.183862+0000","last_undegraded":"2024-09-12T12:21:30.183862+0000","last_fullsized":"2024-09-12T12:21:30.183862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":180925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1085440,"data_stored":784425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":200337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":201879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":201901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":194037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":784369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:21:35.046 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-12T12:21:35.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.046 INFO:tasks.ceph.ceph_manager.ceph:PG 7.31 is not active+clean 2024-09-12T12:21:35.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.046 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2b is not active+clean 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:21:28.174745+0000', 'last_change': '2024-09-12T12:21:28.174745+0000', 'last_active': '2024-09-12T12:21:28.153915+0000', 'last_peered': '2024-09-12T12:21:28.153915+0000', 'last_clean': '2024-09-12T12:21:28.153915+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:28.174745+0000', 'last_undegraded': '2024-09-12T12:21:28.174745+0000', 'last_fullsized': '2024-09-12T12:21:28.174745+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:21:28.176529+0000', 'last_change': '2024-09-12T12:21:28.176529+0000', 'last_active': '2024-09-12T12:21:28.153915+0000', 'last_peered': '2024-09-12T12:21:28.153915+0000', 'last_clean': '2024-09-12T12:21:28.153915+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:28.176529+0000', 'last_undegraded': '2024-09-12T12:21:28.176529+0000', 'last_fullsized': '2024-09-12T12:21:28.176529+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-09-12T12:21:35.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:21:28.176104+0000', 'last_change': '2024-09-12T12:21:28.176104+0000', 'last_active': '2024-09-12T12:21:28.153915+0000', 'last_peered': '2024-09-12T12:21:28.153915+0000', 'last_clean': '2024-09-12T12:21:28.153915+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:28.176104+0000', 'last_undegraded': '2024-09-12T12:21:28.176104+0000', 'last_fullsized': '2024-09-12T12:21:28.176104+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:21:28.174130+0000', 'last_change': '2024-09-12T12:21:28.174130+0000', 'last_active': '2024-09-12T12:21:28.153915+0000', 'last_peered': '2024-09-12T12:21:28.153915+0000', 'last_clean': '2024-09-12T12:21:28.153915+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:28.174130+0000', 'last_undegraded': '2024-09-12T12:21:28.174130+0000', 'last_fullsized': '2024-09-12T12:21:28.174130+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:21:28.174471+0000', 'last_change': '2024-09-12T12:21:28.174471+0000', 'last_active': '2024-09-12T12:21:28.153915+0000', 'last_peered': '2024-09-12T12:21:28.153915+0000', 'last_clean': '2024-09-12T12:21:28.153915+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:28.174471+0000', 'last_undegraded': '2024-09-12T12:21:28.174471+0000', 'last_fullsized': '2024-09-12T12:21:28.174471+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:PG 7.a is not active+clean 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.048 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-09-12T12:21:35.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:21:28.175594+0000', 'last_change': '2024-09-12T12:21:28.175594+0000', 'last_active': '2024-09-12T12:21:28.153915+0000', 'last_peered': '2024-09-12T12:21:28.153915+0000', 'last_clean': '2024-09-12T12:21:28.153915+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:28.175594+0000', 'last_undegraded': '2024-09-12T12:21:28.175594+0000', 'last_fullsized': '2024-09-12T12:21:28.175594+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:21:35.049 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-09-12T12:21:35.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:21:28.174633+0000', 'last_change': '2024-09-12T12:21:28.174633+0000', 'last_active': '2024-09-12T12:21:28.153915+0000', 'last_peered': '2024-09-12T12:21:28.153915+0000', 'last_clean': '2024-09-12T12:21:28.153915+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:28.174633+0000', 'last_undegraded': '2024-09-12T12:21:28.174633+0000', 'last_fullsized': '2024-09-12T12:21:28.174633+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:28.153915+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-12T12:21:35.049 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2 is not active+clean 2024-09-12T12:21:35.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.049 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-09-12T12:21:35.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-12T12:21:29.163827+0000', 'last_change': '2024-09-12T12:21:29.163827+0000', 'last_active': '2024-09-12T12:21:29.163827+0000', 'last_peered': '2024-09-12T12:21:29.163827+0000', 'last_clean': '2024-09-12T12:21:29.163827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:21:29.163827+0000', 'last_undegraded': '2024-09-12T12:21:29.163827+0000', 'last_fullsized': '2024-09-12T12:21:29.163827+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'last_clean_scrub_stamp': '2024-09-12T12:21:29.163827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-12T12:21:35.050 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T12:21:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.205+0000 7f4ce190d640 1 Processor -- start 2024-09-12T12:21:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.205+0000 7f4ce190d640 1 -- start start 2024-09-12T12:21:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc15ac30 0x7f4cdc06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cdc144110 0x7f4cdc0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cdc071ed0 con 0x7f4cdc0727b0 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4cdc072490 con 0x7f4cdc144110 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc15b010 con 0x7f4cdc15ac30 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cdc144110 0x7f4cdc0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cdc144110 0x7f4cdc0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47294/0 (socket says 172.21.15.3:47294) 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1416687452 learned_addr learned my addr 172.21.15.3:0/1416687452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1416687452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc15ac30 msgr2=0x7f4cdc06f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 --2- 172.21.15.3:0/1416687452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc15ac30 0x7f4cdc06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1416687452 >> v1:172.21.15.142:6789/0 conn(0x7f4cdc0727b0 legacy=0x7f4cdc0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:35.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1416687452 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc0721b0 con 0x7f4cdc144110 2024-09-12T12:21:35.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cd3fff640 1 --2- 172.21.15.3:0/1416687452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc0b4a30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009870 tx=0x7f4cc401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccaba5c8ff8dbaa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:35.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cda7fc640 1 -- 172.21.15.3:0/1416687452 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40b0070 con 0x7f4cdc144110 2024-09-12T12:21:35.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cda7fc640 1 -- 172.21.15.3:0/1416687452 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc40a4e50 con 0x7f4cdc144110 2024-09-12T12:21:35.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4cda7fc640 1 -- 172.21.15.3:0/1416687452 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40ab820 con 0x7f4cdc144110 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- 172.21.15.3:0/1416687452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 msgr2=0x7f4cdc0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1416687452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc0b4a30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009870 tx=0x7f4cc401c400 comp rx=0 tx=0).stop 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- 172.21.15.3:0/1416687452 shutdown_connections 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1416687452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc0b4a30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1416687452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc15ac30 0x7f4cdc06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- 172.21.15.3:0/1416687452 >> 172.21.15.3:0/1416687452 conn(0x7f4cdc0b1d80 msgr2=0x7f4cdc0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- 172.21.15.3:0/1416687452 shutdown_connections 2024-09-12T12:21:35.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.209+0000 7f4ce190d640 1 -- 172.21.15.3:0/1416687452 wait complete. 2024-09-12T12:21:35.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 Processor -- start 2024-09-12T12:21:35.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 -- start start 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc0727b0 0x7f4cdc1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc14c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc15ac30 0x7f4cdc14cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc165700 con 0x7f4cdc144110 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc073010 con 0x7f4cdc15ac30 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4cdc165880 con 0x7f4cdc0727b0 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc14c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47302/0 (socket says 172.21.15.3:47302) 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1755938904 learned_addr learned my addr 172.21.15.3:0/1755938904 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc0727b0 msgr2=0x7f4cdc1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc0727b0 0x7f4cdc1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc15ac30 msgr2=0x7f4cdc14cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cdb7fe640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc15ac30 0x7f4cdc14cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc15ac30 0x7f4cdc14cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 -- 172.21.15.3:0/1755938904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc14e930 con 0x7f4cdc144110 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cdb7fe640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc15ac30 0x7f4cdc14cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd3fff640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc14c850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ae040 tx=0x7f4cc40191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd8ff9640 1 -- 172.21.15.3:0/1755938904 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40b0040 con 0x7f4cdc144110 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd8ff9640 1 -- 172.21.15.3:0/1755938904 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc40b9070 con 0x7f4cdc144110 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc14c0f0 con 0x7f4cdc144110 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4cd8ff9640 1 -- 172.21.15.3:0/1755938904 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40b4680 con 0x7f4cdc144110 2024-09-12T12:21:35.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc14ebd0 con 0x7f4cdc144110 2024-09-12T12:21:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.213+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cdc0b4a30 con 0x7f4cdc144110 2024-09-12T12:21:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.217+0000 7f4cd8ff9640 1 -- 172.21.15.3:0/1755938904 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4cc40b4820 con 0x7f4cdc144110 2024-09-12T12:21:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.217+0000 7f4cd8ff9640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cbc0678c0 0x7f4cbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.217+0000 7f4cd8ff9640 1 -- 172.21.15.3:0/1755938904 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f4cc411f980 con 0x7f4cdc144110 2024-09-12T12:21:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.217+0000 7f4cdaffd640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cbc0678c0 0x7f4cbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.217+0000 7f4cdaffd640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cbc0678c0 0x7f4cbc069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009690 tx=0x7f4ccc006c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:35.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.221+0000 7f4cd8ff9640 1 -- 172.21.15.3:0/1755938904 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4cc4016110 con 0x7f4cdc144110 2024-09-12T12:21:35.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.441+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4cdc14c5f0 con 0x7f4cdc144110 2024-09-12T12:21:35.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4cd8ff9640 1 -- 172.21.15.3:0/1755938904 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1636 (secure 0 0 0) 0x7f4cc40f04c0 con 0x7f4cdc144110 2024-09-12T12:21:35.446 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:35.446 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":130,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":8,"num_up_osds":8,"osd_up_since":1726143578,"num_in_osds":8,"osd_in_since":1726143567,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":81},{"state_name":"unknown","count":9},{"state_name":"creating+peering","count":7}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592162,"bytes_used":230838272,"bytes_avail":772863275008,"bytes_total":773094113280,"unknown_pgs_ratio":0.092783503234386444,"inactive_pgs_ratio":0.072164945304393768,"write_bytes_sec":1111,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":23,"btime":"2024-09-12T12:21:33:226221+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5262}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-12T12:21:17.687512+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T12:21:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cbc0678c0 msgr2=0x7f4cbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cbc0678c0 0x7f4cbc069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009690 tx=0x7f4ccc006c00 comp rx=0 tx=0).stop 2024-09-12T12:21:35.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 msgr2=0x7f4cdc14c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc14c850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ae040 tx=0x7f4cc40191a0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 shutdown_connections 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cbc0678c0 0x7f4cbc069d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cdc15ac30 0x7f4cdc14cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cdc144110 0x7f4cdc14c850 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 --2- 172.21.15.3:0/1755938904 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cdc0727b0 0x7f4cdc1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 >> 172.21.15.3:0/1755938904 conn(0x7f4cdc0b1d80 msgr2=0x7f4cdc166030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 shutdown_connections 2024-09-12T12:21:35.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.445+0000 7f4ce190d640 1 -- 172.21.15.3:0/1755938904 wait complete. 2024-09-12T12:21:35.465 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T12:21:35.465 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:21:35.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe71a4cc640 1 Processor -- start 2024-09-12T12:21:35.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe71a4cc640 1 -- start start 2024-09-12T12:21:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe71a4cc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe714159bd0 0x7fe71406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe71a4cc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71406fbb0 con 0x7fe71415d2d0 2024-09-12T12:21:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe71a4cc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71415d760 con 0x7fe714159fb0 2024-09-12T12:21:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe71a4cc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe71406f010 con 0x7fe714159bd0 2024-09-12T12:21:35.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe718a69640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe71415d2d0 0x7fe7140ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35518/0 (socket says 172.21.15.3:35518) 2024-09-12T12:21:35.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe718a69640 1 -- 172.21.15.3:0/644293881 learned_addr learned my addr 172.21.15.3:0/644293881 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe713fff640 1 --2- 172.21.15.3:0/644293881 >> v2:172.21.15.142:3300/0 conn(0x7fe714159bd0 0x7fe71406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542127780 0 0) 0x7fe71406fbb0 con 0x7fe71415d2d0 2024-09-12T12:21:35.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6fc003530 con 0x7fe71415d2d0 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1138116180 0 0) 0x7fe6fc003530 con 0x7fe71415d2d0 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 >> v2:172.21.15.142:3300/0 conn(0x7fe714159bd0 msgr2=0x7fe71406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 --2- 172.21.15.3:0/644293881 >> v2:172.21.15.142:3300/0 conn(0x7fe714159bd0 0x7fe71406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 >> v1:172.21.15.142:6789/0 conn(0x7fe714159fb0 legacy=0x7fe71406f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71406e400 con 0x7fe71415d2d0 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fe70c001a00 con 0x7fe71415d2d0 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 >> v1:172.21.15.3:6789/0 conn(0x7fe71415d2d0 legacy=0x7fe7140ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe6fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6fc004810 0x7fe6fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6fc005220 0x7fe6fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe71406f010 con 0x7fe6fc0039c0 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00c150 con 0x7fe6fc004810 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc00c350 con 0x7fe6fc005220 2024-09-12T12:21:35.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe713fff640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe6fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe713fff640 1 -- 172.21.15.3:0/644293881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6fc005220 msgr2=0x7fe6fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe713fff640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe6fc005220 0x7fe6fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe713fff640 1 -- 172.21.15.3:0/644293881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6fc004810 msgr2=0x7fe6fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe713fff640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6fc004810 0x7fe6fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe713fff640 1 -- 172.21.15.3:0/644293881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc003530 con 0x7fe6fc0039c0 2024-09-12T12:21:35.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.577+0000 7fe718a69640 1 -- 172.21.15.3:0/644293881 reap_dead start 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe6fc003e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe70400ba30 tx=0x7fe70401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bbd64c7cf76b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70400b600 con 0x7fe6fc0039c0 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7040b0070 con 0x7fe6fc0039c0 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe712ffd640 1 -- 172.21.15.3:0/644293881 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040ab8c0 con 0x7fe6fc0039c0 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- 172.21.15.3:0/644293881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 msgr2=0x7fe6fc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe6fc003e00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe70400ba30 tx=0x7fe70401c670 comp rx=0 tx=0).stop 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- 172.21.15.3:0/644293881 shutdown_connections 2024-09-12T12:21:35.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/644293881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe6fc003e00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- 172.21.15.3:0/644293881 >> 172.21.15.3:0/644293881 conn(0x7fe7140b1d80 msgr2=0x7fe7140b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:35.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- 172.21.15.3:0/644293881 shutdown_connections 2024-09-12T12:21:35.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- 172.21.15.3:0/644293881 wait complete. 2024-09-12T12:21:35.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 Processor -- start 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- start start 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe714073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7141587c0 0x7fe7140afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7140734c0 0x7fe714145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe71406f890 con 0x7fe6fc0039c0 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe71406eca0 con 0x7fe7141587c0 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe71a4cc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe71406fa10 con 0x7fe7140734c0 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe714073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe714073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47322/0 (socket says 172.21.15.3:47322) 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 -- 172.21.15.3:0/64602220 learned_addr learned my addr 172.21.15.3:0/64602220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 -- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7140734c0 msgr2=0x7fe714145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7140734c0 0x7fe714145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 -- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7141587c0 msgr2=0x7fe7140afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7141587c0 0x7fe7140afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.581+0000 7fe713fff640 1 -- 172.21.15.3:0/64602220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe714158d10 con 0x7fe6fc0039c0 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe713fff640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe714073a00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ad040 tx=0x7fe6fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe710ff9640 1 -- 172.21.15.3:0/64602220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040ba070 con 0x7fe6fc0039c0 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe710ff9640 1 -- 172.21.15.3:0/64602220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe704004220 con 0x7fe6fc0039c0 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe714072a00 con 0x7fe6fc0039c0 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe710ff9640 1 -- 172.21.15.3:0/64602220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040ab740 con 0x7fe6fc0039c0 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe714072fc0 con 0x7fe6fc0039c0 2024-09-12T12:21:35.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7140ad870 con 0x7fe6fc0039c0 2024-09-12T12:21:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe710ff9640 1 -- 172.21.15.3:0/64602220 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7040ab8e0 con 0x7fe6fc0039c0 2024-09-12T12:21:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe710ff9640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe7000678c0 0x7fe700069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe710ff9640 1 -- 172.21.15.3:0/64602220 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fe704120260 con 0x7fe6fc0039c0 2024-09-12T12:21:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe7137fe640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe7000678c0 0x7fe700069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:35.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.585+0000 7fe7137fe640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe7000678c0 0x7fe700069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe708007fd0 tx=0x7fe7080079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:35.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.589+0000 7fe710ff9640 1 -- 172.21.15.3:0/64602220 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7040c4060 con 0x7fe6fc0039c0 2024-09-12T12:21:35.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.745+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7140b2b00 con 0x7fe7000678c0 2024-09-12T12:21:35.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.765+0000 7fe710ff9640 1 -- 172.21.15.3:0/64602220 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7fe7140b2b00 con 0x7fe7000678c0 2024-09-12T12:21:35.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:35.771 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe7000678c0 msgr2=0x7fe700069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe7000678c0 0x7fe700069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe708007fd0 tx=0x7fe7080079a0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 msgr2=0x7fe714073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe714073a00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ad040 tx=0x7fe6fc005840 comp rx=0 tx=0).stop 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 shutdown_connections 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe7000678c0 0x7fe700069d80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7140734c0 0x7fe714145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7141587c0 0x7fe7140afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 --2- 172.21.15.3:0/64602220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6fc0039c0 0x7fe714073a00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 >> 172.21.15.3:0/64602220 conn(0x7fe7140b1d80 msgr2=0x7fe71415ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 shutdown_connections 2024-09-12T12:21:35.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:35.769+0000 7fe71a4cc640 1 -- 172.21.15.3:0/64602220 wait complete. 2024-09-12T12:21:35.788 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-12T12:21:35.692783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225476,"kb_used_data":5080,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749244,"statfs":{"total":773094113280,"available":772863225856,"internally_reserved":0,"allocated":5201920,"data_stored":2762492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516209"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.208627+0000","last_change":"2024-09-12T12:21:30.200850+0000","last_active":"2024-09-12T12:21:30.208627+0000","last_peered":"2024-09-12T12:21:30.208627+0000","last_clean":"2024-09-12T12:21:30.208627+0000","last_became_active":"2024-09-12T12:21:30.200142+0000","last_became_peered":"2024-09-12T12:21:30.200142+0000","last_unstale":"2024-09-12T12:21:30.208627+0000","last_undegraded":"2024-09-12T12:21:30.208627+0000","last_fullsized":"2024-09-12T12:21:30.208627+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209219+0000","last_change":"2024-09-12T12:21:30.208435+0000","last_active":"2024-09-12T12:21:31.209219+0000","last_peered":"2024-09-12T12:21:31.209219+0000","last_clean":"2024-09-12T12:21:31.209219+0000","last_became_active":"2024-09-12T12:21:30.208134+0000","last_became_peered":"2024-09-12T12:21:30.208134+0000","last_unstale":"2024-09-12T12:21:31.209219+0000","last_undegraded":"2024-09-12T12:21:31.209219+0000","last_fullsized":"2024-09-12T12:21:31.209219+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:33.718518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210339+0000","last_change":"2024-09-12T12:21:30.212483+0000","last_active":"2024-09-12T12:21:31.210339+0000","last_peered":"2024-09-12T12:21:31.210339+0000","last_clean":"2024-09-12T12:21:31.210339+0000","last_became_active":"2024-09-12T12:21:30.211973+0000","last_became_peered":"2024-09-12T12:21:30.211973+0000","last_unstale":"2024-09-12T12:21:31.210339+0000","last_undegraded":"2024-09-12T12:21:31.210339+0000","last_fullsized":"2024-09-12T12:21:31.210339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:30.898304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227851+0000","last_change":"2024-09-12T12:21:30.222772+0000","last_active":"2024-09-12T12:21:30.227851+0000","last_peered":"2024-09-12T12:21:30.227851+0000","last_clean":"2024-09-12T12:21:30.227851+0000","last_became_active":"2024-09-12T12:21:30.222582+0000","last_became_peered":"2024-09-12T12:21:30.222582+0000","last_unstale":"2024-09-12T12:21:30.227851+0000","last_undegraded":"2024-09-12T12:21:30.227851+0000","last_fullsized":"2024-09-12T12:21:30.227851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:53.948158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207968+0000","last_change":"2024-09-12T12:21:30.208596+0000","last_active":"2024-09-12T12:21:31.207968+0000","last_peered":"2024-09-12T12:21:31.207968+0000","last_clean":"2024-09-12T12:21:31.207968+0000","last_became_active":"2024-09-12T12:21:30.208202+0000","last_became_peered":"2024-09-12T12:21:30.208202+0000","last_unstale":"2024-09-12T12:21:31.207968+0000","last_undegraded":"2024-09-12T12:21:31.207968+0000","last_fullsized":"2024-09-12T12:21:31.207968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214106+0000","last_change":"2024-09-12T12:21:30.227649+0000","last_active":"2024-09-12T12:21:32.214106+0000","last_peered":"2024-09-12T12:21:32.214106+0000","last_clean":"2024-09-12T12:21:32.214106+0000","last_became_active":"2024-09-12T12:21:30.226708+0000","last_became_peered":"2024-09-12T12:21:30.226708+0000","last_unstale":"2024-09-12T12:21:32.214106+0000","last_undegraded":"2024-09-12T12:21:32.214106+0000","last_fullsized":"2024-09-12T12:21:32.214106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:14.331393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210712+0000","last_change":"2024-09-12T12:21:30.204887+0000","last_active":"2024-09-12T12:21:30.210712+0000","last_peered":"2024-09-12T12:21:30.210712+0000","last_clean":"2024-09-12T12:21:30.210712+0000","last_became_active":"2024-09-12T12:21:30.204053+0000","last_became_peered":"2024-09-12T12:21:30.204053+0000","last_unstale":"2024-09-12T12:21:30.210712+0000","last_undegraded":"2024-09-12T12:21:30.210712+0000","last_fullsized":"2024-09-12T12:21:30.210712+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225399+0000","last_change":"2024-09-12T12:21:30.220921+0000","last_active":"2024-09-12T12:21:30.225399+0000","last_peered":"2024-09-12T12:21:30.225399+0000","last_clean":"2024-09-12T12:21:30.225399+0000","last_became_active":"2024-09-12T12:21:30.220376+0000","last_became_peered":"2024-09-12T12:21:30.220376+0000","last_unstale":"2024-09-12T12:21:30.225399+0000","last_undegraded":"2024-09-12T12:21:30.225399+0000","last_fullsized":"2024-09-12T12:21:30.225399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:49.354057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233115+0000","last_change":"2024-09-12T12:21:30.229582+0000","last_active":"2024-09-12T12:21:30.233115+0000","last_peered":"2024-09-12T12:21:30.233115+0000","last_clean":"2024-09-12T12:21:30.233115+0000","last_became_active":"2024-09-12T12:21:30.229229+0000","last_became_peered":"2024-09-12T12:21:30.229229+0000","last_unstale":"2024-09-12T12:21:30.233115+0000","last_undegraded":"2024-09-12T12:21:30.233115+0000","last_fullsized":"2024-09-12T12:21:30.233115+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:42:40.496362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227913+0000","last_change":"2024-09-12T12:21:30.222613+0000","last_active":"2024-09-12T12:21:30.227913+0000","last_peered":"2024-09-12T12:21:30.227913+0000","last_clean":"2024-09-12T12:21:30.227913+0000","last_became_active":"2024-09-12T12:21:30.222213+0000","last_became_peered":"2024-09-12T12:21:30.222213+0000","last_unstale":"2024-09-12T12:21:30.227913+0000","last_undegraded":"2024-09-12T12:21:30.227913+0000","last_fullsized":"2024-09-12T12:21:30.227913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:22.314764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233213+0000","last_change":"2024-09-12T12:21:30.229703+0000","last_active":"2024-09-12T12:21:30.233213+0000","last_peered":"2024-09-12T12:21:30.233213+0000","last_clean":"2024-09-12T12:21:30.233213+0000","last_became_active":"2024-09-12T12:21:30.229280+0000","last_became_peered":"2024-09-12T12:21:30.229280+0000","last_unstale":"2024-09-12T12:21:30.233213+0000","last_undegraded":"2024-09-12T12:21:30.233213+0000","last_fullsized":"2024-09-12T12:21:30.233213+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:06:44.348610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208772+0000","last_change":"2024-09-12T12:21:30.208502+0000","last_active":"2024-09-12T12:21:31.208772+0000","last_peered":"2024-09-12T12:21:31.208772+0000","last_clean":"2024-09-12T12:21:31.208772+0000","last_became_active":"2024-09-12T12:21:30.207768+0000","last_became_peered":"2024-09-12T12:21:30.207768+0000","last_unstale":"2024-09-12T12:21:31.208772+0000","last_undegraded":"2024-09-12T12:21:31.208772+0000","last_fullsized":"2024-09-12T12:21:31.208772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896095+0000","last_change":"2024-09-12T12:21:30.227759+0000","last_active":"2024-09-12T12:21:31.896095+0000","last_peered":"2024-09-12T12:21:31.896095+0000","last_clean":"2024-09-12T12:21:31.896095+0000","last_became_active":"2024-09-12T12:21:30.227324+0000","last_became_peered":"2024-09-12T12:21:30.227324+0000","last_unstale":"2024-09-12T12:21:31.896095+0000","last_undegraded":"2024-09-12T12:21:31.896095+0000","last_fullsized":"2024-09-12T12:21:31.896095+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:39.438927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.218203+0000","last_change":"2024-09-12T12:21:30.213039+0000","last_active":"2024-09-12T12:21:30.218203+0000","last_peered":"2024-09-12T12:21:30.218203+0000","last_clean":"2024-09-12T12:21:30.218203+0000","last_became_active":"2024-09-12T12:21:30.212232+0000","last_became_peered":"2024-09-12T12:21:30.212232+0000","last_unstale":"2024-09-12T12:21:30.218203+0000","last_undegraded":"2024-09-12T12:21:30.218203+0000","last_fullsized":"2024-09-12T12:21:30.218203+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:04.883109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214020+0000","last_change":"2024-09-12T12:21:30.232536+0000","last_active":"2024-09-12T12:21:32.214020+0000","last_peered":"2024-09-12T12:21:32.214020+0000","last_clean":"2024-09-12T12:21:32.214020+0000","last_became_active":"2024-09-12T12:21:30.225903+0000","last_became_peered":"2024-09-12T12:21:30.225903+0000","last_unstale":"2024-09-12T12:21:32.214020+0000","last_undegraded":"2024-09-12T12:21:32.214020+0000","last_fullsized":"2024-09-12T12:21:32.214020+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:39:45.100624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919869+0000","last_change":"2024-09-12T12:21:30.216647+0000","last_active":"2024-09-12T12:21:31.919869+0000","last_peered":"2024-09-12T12:21:31.919869+0000","last_clean":"2024-09-12T12:21:31.919869+0000","last_became_active":"2024-09-12T12:21:30.216256+0000","last_became_peered":"2024-09-12T12:21:30.216256+0000","last_unstale":"2024-09-12T12:21:31.919869+0000","last_undegraded":"2024-09-12T12:21:31.919869+0000","last_fullsized":"2024-09-12T12:21:31.919869+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:02.339672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210550+0000","last_change":"2024-09-12T12:21:30.214244+0000","last_active":"2024-09-12T12:21:31.210550+0000","last_peered":"2024-09-12T12:21:31.210550+0000","last_clean":"2024-09-12T12:21:31.210550+0000","last_became_active":"2024-09-12T12:21:30.213464+0000","last_became_peered":"2024-09-12T12:21:30.213464+0000","last_unstale":"2024-09-12T12:21:31.210550+0000","last_undegraded":"2024-09-12T12:21:31.210550+0000","last_fullsized":"2024-09-12T12:21:31.210550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:22.724033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210473+0000","last_change":"2024-09-12T12:21:30.213222+0000","last_active":"2024-09-12T12:21:31.210473+0000","last_peered":"2024-09-12T12:21:31.210473+0000","last_clean":"2024-09-12T12:21:31.210473+0000","last_became_active":"2024-09-12T12:21:30.212819+0000","last_became_peered":"2024-09-12T12:21:30.212819+0000","last_unstale":"2024-09-12T12:21:31.210473+0000","last_undegraded":"2024-09-12T12:21:31.210473+0000","last_fullsized":"2024-09-12T12:21:31.210473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:20.425017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920825+0000","last_change":"2024-09-12T12:21:30.216805+0000","last_active":"2024-09-12T12:21:31.920825+0000","last_peered":"2024-09-12T12:21:31.920825+0000","last_clean":"2024-09-12T12:21:31.920825+0000","last_became_active":"2024-09-12T12:21:30.216287+0000","last_became_peered":"2024-09-12T12:21:30.216287+0000","last_unstale":"2024-09-12T12:21:31.920825+0000","last_undegraded":"2024-09-12T12:21:31.920825+0000","last_fullsized":"2024-09-12T12:21:31.920825+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896373+0000","last_change":"2024-09-12T12:21:30.211007+0000","last_active":"2024-09-12T12:21:31.896373+0000","last_peered":"2024-09-12T12:21:31.896373+0000","last_clean":"2024-09-12T12:21:31.896373+0000","last_became_active":"2024-09-12T12:21:30.207831+0000","last_became_peered":"2024-09-12T12:21:30.207831+0000","last_unstale":"2024-09-12T12:21:31.896373+0000","last_undegraded":"2024-09-12T12:21:31.896373+0000","last_fullsized":"2024-09-12T12:21:31.896373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:49.469026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214153+0000","last_change":"2024-09-12T12:21:30.224643+0000","last_active":"2024-09-12T12:21:32.214153+0000","last_peered":"2024-09-12T12:21:32.214153+0000","last_clean":"2024-09-12T12:21:32.214153+0000","last_became_active":"2024-09-12T12:21:30.224374+0000","last_became_peered":"2024-09-12T12:21:30.224374+0000","last_unstale":"2024-09-12T12:21:32.214153+0000","last_undegraded":"2024-09-12T12:21:32.214153+0000","last_fullsized":"2024-09-12T12:21:32.214153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:56:27.586549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224229+0000","last_change":"2024-09-12T12:21:30.216387+0000","last_active":"2024-09-12T12:21:30.224229+0000","last_peered":"2024-09-12T12:21:30.224229+0000","last_clean":"2024-09-12T12:21:30.224229+0000","last_became_active":"2024-09-12T12:21:30.215852+0000","last_became_peered":"2024-09-12T12:21:30.215852+0000","last_unstale":"2024-09-12T12:21:30.224229+0000","last_undegraded":"2024-09-12T12:21:30.224229+0000","last_fullsized":"2024-09-12T12:21:30.224229+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:38.107312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920593+0000","last_change":"2024-09-12T12:21:30.225623+0000","last_active":"2024-09-12T12:21:31.920593+0000","last_peered":"2024-09-12T12:21:31.920593+0000","last_clean":"2024-09-12T12:21:31.920593+0000","last_became_active":"2024-09-12T12:21:30.222578+0000","last_became_peered":"2024-09-12T12:21:30.222578+0000","last_unstale":"2024-09-12T12:21:31.920593+0000","last_undegraded":"2024-09-12T12:21:31.920593+0000","last_fullsized":"2024-09-12T12:21:31.920593+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:28.867805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919887+0000","last_change":"2024-09-12T12:21:30.217472+0000","last_active":"2024-09-12T12:21:31.919887+0000","last_peered":"2024-09-12T12:21:31.919887+0000","last_clean":"2024-09-12T12:21:31.919887+0000","last_became_active":"2024-09-12T12:21:30.217177+0000","last_became_peered":"2024-09-12T12:21:30.217177+0000","last_unstale":"2024-09-12T12:21:31.919887+0000","last_undegraded":"2024-09-12T12:21:31.919887+0000","last_fullsized":"2024-09-12T12:21:31.919887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920545+0000","last_change":"2024-09-12T12:21:30.226600+0000","last_active":"2024-09-12T12:21:31.920545+0000","last_peered":"2024-09-12T12:21:31.920545+0000","last_clean":"2024-09-12T12:21:31.920545+0000","last_became_active":"2024-09-12T12:21:30.226195+0000","last_became_peered":"2024-09-12T12:21:30.226195+0000","last_unstale":"2024-09-12T12:21:31.920545+0000","last_undegraded":"2024-09-12T12:21:31.920545+0000","last_fullsized":"2024-09-12T12:21:31.920545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:11.697205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896333+0000","last_change":"2024-09-12T12:21:30.227132+0000","last_active":"2024-09-12T12:21:31.896333+0000","last_peered":"2024-09-12T12:21:31.896333+0000","last_clean":"2024-09-12T12:21:31.896333+0000","last_became_active":"2024-09-12T12:21:30.226820+0000","last_became_peered":"2024-09-12T12:21:30.226820+0000","last_unstale":"2024-09-12T12:21:31.896333+0000","last_undegraded":"2024-09-12T12:21:31.896333+0000","last_fullsized":"2024-09-12T12:21:31.896333+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:14.450296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210369+0000","last_change":"2024-09-12T12:21:30.207017+0000","last_active":"2024-09-12T12:21:31.210369+0000","last_peered":"2024-09-12T12:21:31.210369+0000","last_clean":"2024-09-12T12:21:31.210369+0000","last_became_active":"2024-09-12T12:21:30.206281+0000","last_became_peered":"2024-09-12T12:21:30.206281+0000","last_unstale":"2024-09-12T12:21:31.210369+0000","last_undegraded":"2024-09-12T12:21:31.210369+0000","last_fullsized":"2024-09-12T12:21:31.210369+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:59.576497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209179+0000","last_change":"2024-09-12T12:21:30.208170+0000","last_active":"2024-09-12T12:21:31.209179+0000","last_peered":"2024-09-12T12:21:31.209179+0000","last_clean":"2024-09-12T12:21:31.209179+0000","last_became_active":"2024-09-12T12:21:30.207326+0000","last_became_peered":"2024-09-12T12:21:30.207326+0000","last_unstale":"2024-09-12T12:21:31.209179+0000","last_undegraded":"2024-09-12T12:21:31.209179+0000","last_fullsized":"2024-09-12T12:21:31.209179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920532+0000","last_change":"2024-09-12T12:21:30.226334+0000","last_active":"2024-09-12T12:21:31.920532+0000","last_peered":"2024-09-12T12:21:31.920532+0000","last_clean":"2024-09-12T12:21:31.920532+0000","last_became_active":"2024-09-12T12:21:30.225985+0000","last_became_peered":"2024-09-12T12:21:30.225985+0000","last_unstale":"2024-09-12T12:21:31.920532+0000","last_undegraded":"2024-09-12T12:21:31.920532+0000","last_fullsized":"2024-09-12T12:21:31.920532+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:49.456119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208773+0000","last_change":"2024-09-12T12:21:30.209539+0000","last_active":"2024-09-12T12:21:31.208773+0000","last_peered":"2024-09-12T12:21:31.208773+0000","last_clean":"2024-09-12T12:21:31.208773+0000","last_became_active":"2024-09-12T12:21:30.209224+0000","last_became_peered":"2024-09-12T12:21:30.209224+0000","last_unstale":"2024-09-12T12:21:31.208773+0000","last_undegraded":"2024-09-12T12:21:31.208773+0000","last_fullsized":"2024-09-12T12:21:31.208773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214202+0000","last_change":"2024-09-12T12:21:30.232530+0000","last_active":"2024-09-12T12:21:32.214202+0000","last_peered":"2024-09-12T12:21:32.214202+0000","last_clean":"2024-09-12T12:21:32.214202+0000","last_became_active":"2024-09-12T12:21:30.222768+0000","last_became_peered":"2024-09-12T12:21:30.222768+0000","last_unstale":"2024-09-12T12:21:32.214202+0000","last_undegraded":"2024-09-12T12:21:32.214202+0000","last_fullsized":"2024-09-12T12:21:32.214202+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:05.290854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896155+0000","last_change":"2024-09-12T12:21:30.225947+0000","last_active":"2024-09-12T12:21:31.896155+0000","last_peered":"2024-09-12T12:21:31.896155+0000","last_clean":"2024-09-12T12:21:31.896155+0000","last_became_active":"2024-09-12T12:21:30.225590+0000","last_became_peered":"2024-09-12T12:21:30.225590+0000","last_unstale":"2024-09-12T12:21:31.896155+0000","last_undegraded":"2024-09-12T12:21:31.896155+0000","last_fullsized":"2024-09-12T12:21:31.896155+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.919842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232731+0000","last_change":"2024-09-12T12:21:30.224922+0000","last_active":"2024-09-12T12:21:30.232731+0000","last_peered":"2024-09-12T12:21:30.232731+0000","last_clean":"2024-09-12T12:21:30.232731+0000","last_became_active":"2024-09-12T12:21:30.224326+0000","last_became_peered":"2024-09-12T12:21:30.224326+0000","last_unstale":"2024-09-12T12:21:30.232731+0000","last_undegraded":"2024-09-12T12:21:30.232731+0000","last_fullsized":"2024-09-12T12:21:30.232731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.215024+0000","last_change":"2024-09-12T12:21:29.216451+0000","last_active":"2024-09-12T12:21:32.215024+0000","last_peered":"2024-09-12T12:21:32.215024+0000","last_clean":"2024-09-12T12:21:32.215024+0000","last_became_active":"2024-09-12T12:21:29.215960+0000","last_became_peered":"2024-09-12T12:21:29.215960+0000","last_unstale":"2024-09-12T12:21:32.215024+0000","last_undegraded":"2024-09-12T12:21:32.215024+0000","last_fullsized":"2024-09-12T12:21:32.215024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:47:22.554148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920715+0000","last_change":"2024-09-12T12:21:30.222316+0000","last_active":"2024-09-12T12:21:31.920715+0000","last_peered":"2024-09-12T12:21:31.920715+0000","last_clean":"2024-09-12T12:21:31.920715+0000","last_became_active":"2024-09-12T12:21:30.221990+0000","last_became_peered":"2024-09-12T12:21:30.221990+0000","last_unstale":"2024-09-12T12:21:31.920715+0000","last_undegraded":"2024-09-12T12:21:31.920715+0000","last_fullsized":"2024-09-12T12:21:31.920715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:52.055517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210844+0000","last_change":"2024-09-12T12:21:29.219231+0000","last_active":"2024-09-12T12:21:31.210844+0000","last_peered":"2024-09-12T12:21:31.210844+0000","last_clean":"2024-09-12T12:21:31.210844+0000","last_became_active":"2024-09-12T12:21:29.218147+0000","last_became_peered":"2024-09-12T12:21:29.218147+0000","last_unstale":"2024-09-12T12:21:31.210844+0000","last_undegraded":"2024-09-12T12:21:31.210844+0000","last_fullsized":"2024-09-12T12:21:31.210844+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209841+0000","last_change":"2024-09-12T12:21:30.227814+0000","last_active":"2024-09-12T12:21:31.209841+0000","last_peered":"2024-09-12T12:21:31.209841+0000","last_clean":"2024-09-12T12:21:31.209841+0000","last_became_active":"2024-09-12T12:21:30.227419+0000","last_became_peered":"2024-09-12T12:21:30.227419+0000","last_unstale":"2024-09-12T12:21:31.209841+0000","last_undegraded":"2024-09-12T12:21:31.209841+0000","last_fullsized":"2024-09-12T12:21:31.209841+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:06:39.410795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233639+0000","last_change":"2024-09-12T12:21:29.188168+0000","last_active":"2024-09-12T12:21:30.233639+0000","last_peered":"2024-09-12T12:21:30.233639+0000","last_clean":"2024-09-12T12:21:30.233639+0000","last_became_active":"2024-09-12T12:21:29.187846+0000","last_became_peered":"2024-09-12T12:21:29.187846+0000","last_unstale":"2024-09-12T12:21:30.233639+0000","last_undegraded":"2024-09-12T12:21:30.233639+0000","last_fullsized":"2024-09-12T12:21:30.233639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224153+0000","last_change":"2024-09-12T12:21:30.217678+0000","last_active":"2024-09-12T12:21:30.224153+0000","last_peered":"2024-09-12T12:21:30.224153+0000","last_clean":"2024-09-12T12:21:30.224153+0000","last_became_active":"2024-09-12T12:21:30.216952+0000","last_became_peered":"2024-09-12T12:21:30.216952+0000","last_unstale":"2024-09-12T12:21:30.224153+0000","last_undegraded":"2024-09-12T12:21:30.224153+0000","last_fullsized":"2024-09-12T12:21:30.224153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.229864+0000","last_change":"2024-09-12T12:21:29.225871+0000","last_active":"2024-09-12T12:21:30.229864+0000","last_peered":"2024-09-12T12:21:30.229864+0000","last_clean":"2024-09-12T12:21:30.229864+0000","last_became_active":"2024-09-12T12:21:29.225182+0000","last_became_peered":"2024-09-12T12:21:29.225182+0000","last_unstale":"2024-09-12T12:21:30.229864+0000","last_undegraded":"2024-09-12T12:21:30.229864+0000","last_fullsized":"2024-09-12T12:21:30.229864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:58.566585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209173+0000","last_change":"2024-09-12T12:21:30.217628+0000","last_active":"2024-09-12T12:21:31.209173+0000","last_peered":"2024-09-12T12:21:31.209173+0000","last_clean":"2024-09-12T12:21:31.209173+0000","last_became_active":"2024-09-12T12:21:30.217335+0000","last_became_peered":"2024-09-12T12:21:30.217335+0000","last_unstale":"2024-09-12T12:21:31.209173+0000","last_undegraded":"2024-09-12T12:21:31.209173+0000","last_fullsized":"2024-09-12T12:21:31.209173+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:08.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.215095+0000","last_change":"2024-09-12T12:21:29.221255+0000","last_active":"2024-09-12T12:21:32.215095+0000","last_peered":"2024-09-12T12:21:32.215095+0000","last_clean":"2024-09-12T12:21:32.215095+0000","last_became_active":"2024-09-12T12:21:29.220798+0000","last_became_peered":"2024-09-12T12:21:29.220798+0000","last_unstale":"2024-09-12T12:21:32.215095+0000","last_undegraded":"2024-09-12T12:21:32.215095+0000","last_fullsized":"2024-09-12T12:21:32.215095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:59.351871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210164+0000","last_change":"2024-09-12T12:21:30.206432+0000","last_active":"2024-09-12T12:21:30.210164+0000","last_peered":"2024-09-12T12:21:30.210164+0000","last_clean":"2024-09-12T12:21:30.210164+0000","last_became_active":"2024-09-12T12:21:30.205726+0000","last_became_peered":"2024-09-12T12:21:30.205726+0000","last_unstale":"2024-09-12T12:21:30.210164+0000","last_undegraded":"2024-09-12T12:21:30.210164+0000","last_fullsized":"2024-09-12T12:21:30.210164+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:14:53.863676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209831+0000","last_change":"2024-09-12T12:21:29.207503+0000","last_active":"2024-09-12T12:21:31.209831+0000","last_peered":"2024-09-12T12:21:31.209831+0000","last_clean":"2024-09-12T12:21:31.209831+0000","last_became_active":"2024-09-12T12:21:29.206715+0000","last_became_peered":"2024-09-12T12:21:29.206715+0000","last_unstale":"2024-09-12T12:21:31.209831+0000","last_undegraded":"2024-09-12T12:21:31.209831+0000","last_fullsized":"2024-09-12T12:21:31.209831+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:18:52.540391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225377+0000","last_change":"2024-09-12T12:21:30.220672+0000","last_active":"2024-09-12T12:21:30.225377+0000","last_peered":"2024-09-12T12:21:30.225377+0000","last_clean":"2024-09-12T12:21:30.225377+0000","last_became_active":"2024-09-12T12:21:30.219208+0000","last_became_peered":"2024-09-12T12:21:30.219208+0000","last_unstale":"2024-09-12T12:21:30.225377+0000","last_undegraded":"2024-09-12T12:21:30.225377+0000","last_fullsized":"2024-09-12T12:21:30.225377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:38.730945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207603+0000","last_change":"2024-09-12T12:21:29.204647+0000","last_active":"2024-09-12T12:21:31.207603+0000","last_peered":"2024-09-12T12:21:31.207603+0000","last_clean":"2024-09-12T12:21:31.207603+0000","last_became_active":"2024-09-12T12:21:29.203941+0000","last_became_peered":"2024-09-12T12:21:29.203941+0000","last_unstale":"2024-09-12T12:21:31.207603+0000","last_undegraded":"2024-09-12T12:21:31.207603+0000","last_fullsized":"2024-09-12T12:21:31.207603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:51.045605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919837+0000","last_change":"2024-09-12T12:21:30.216996+0000","last_active":"2024-09-12T12:21:31.919837+0000","last_peered":"2024-09-12T12:21:31.919837+0000","last_clean":"2024-09-12T12:21:31.919837+0000","last_became_active":"2024-09-12T12:21:30.216631+0000","last_became_peered":"2024-09-12T12:21:30.216631+0000","last_unstale":"2024-09-12T12:21:31.919837+0000","last_undegraded":"2024-09-12T12:21:31.919837+0000","last_fullsized":"2024-09-12T12:21:31.919837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:40.449036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191140+0000","last_change":"2024-09-12T12:21:29.195971+0000","last_active":"2024-09-12T12:21:30.191140+0000","last_peered":"2024-09-12T12:21:30.191140+0000","last_clean":"2024-09-12T12:21:30.191140+0000","last_became_active":"2024-09-12T12:21:29.195461+0000","last_became_peered":"2024-09-12T12:21:29.195461+0000","last_unstale":"2024-09-12T12:21:30.191140+0000","last_undegraded":"2024-09-12T12:21:30.191140+0000","last_fullsized":"2024-09-12T12:21:30.191140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:16.138730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210986+0000","last_change":"2024-09-12T12:21:30.213037+0000","last_active":"2024-09-12T12:21:31.210986+0000","last_peered":"2024-09-12T12:21:31.210986+0000","last_clean":"2024-09-12T12:21:31.210986+0000","last_became_active":"2024-09-12T12:21:30.212297+0000","last_became_peered":"2024-09-12T12:21:30.212297+0000","last_unstale":"2024-09-12T12:21:31.210986+0000","last_undegraded":"2024-09-12T12:21:31.210986+0000","last_fullsized":"2024-09-12T12:21:31.210986+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210988+0000","last_change":"2024-09-12T12:21:29.209621+0000","last_active":"2024-09-12T12:21:31.210988+0000","last_peered":"2024-09-12T12:21:31.210988+0000","last_clean":"2024-09-12T12:21:31.210988+0000","last_became_active":"2024-09-12T12:21:29.208221+0000","last_became_peered":"2024-09-12T12:21:29.208221+0000","last_unstale":"2024-09-12T12:21:31.210988+0000","last_undegraded":"2024-09-12T12:21:31.210988+0000","last_fullsized":"2024-09-12T12:21:31.210988+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210773+0000","last_change":"2024-09-12T12:21:30.208783+0000","last_active":"2024-09-12T12:21:30.210773+0000","last_peered":"2024-09-12T12:21:30.210773+0000","last_clean":"2024-09-12T12:21:30.210773+0000","last_became_active":"2024-09-12T12:21:30.202586+0000","last_became_peered":"2024-09-12T12:21:30.202586+0000","last_unstale":"2024-09-12T12:21:30.210773+0000","last_undegraded":"2024-09-12T12:21:30.210773+0000","last_fullsized":"2024-09-12T12:21:30.210773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193677+0000","last_change":"2024-09-12T12:21:29.221269+0000","last_active":"2024-09-12T12:21:30.193677+0000","last_peered":"2024-09-12T12:21:30.193677+0000","last_clean":"2024-09-12T12:21:30.193677+0000","last_became_active":"2024-09-12T12:21:29.220934+0000","last_became_peered":"2024-09-12T12:21:29.220934+0000","last_unstale":"2024-09-12T12:21:30.193677+0000","last_undegraded":"2024-09-12T12:21:30.193677+0000","last_fullsized":"2024-09-12T12:21:30.193677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:05.831499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214887+0000","last_change":"2024-09-12T12:21:30.224755+0000","last_active":"2024-09-12T12:21:32.214887+0000","last_peered":"2024-09-12T12:21:32.214887+0000","last_clean":"2024-09-12T12:21:32.214887+0000","last_became_active":"2024-09-12T12:21:30.222276+0000","last_became_peered":"2024-09-12T12:21:30.222276+0000","last_unstale":"2024-09-12T12:21:32.214887+0000","last_undegraded":"2024-09-12T12:21:32.214887+0000","last_fullsized":"2024-09-12T12:21:32.214887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:18.814846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232623+0000","last_change":"2024-09-12T12:21:29.225993+0000","last_active":"2024-09-12T12:21:30.232623+0000","last_peered":"2024-09-12T12:21:30.232623+0000","last_clean":"2024-09-12T12:21:30.232623+0000","last_became_active":"2024-09-12T12:21:29.225250+0000","last_became_peered":"2024-09-12T12:21:29.225250+0000","last_unstale":"2024-09-12T12:21:30.232623+0000","last_undegraded":"2024-09-12T12:21:30.232623+0000","last_fullsized":"2024-09-12T12:21:30.232623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:19.415105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210545+0000","last_change":"2024-09-12T12:21:30.201827+0000","last_active":"2024-09-12T12:21:30.210545+0000","last_peered":"2024-09-12T12:21:30.210545+0000","last_clean":"2024-09-12T12:21:30.210545+0000","last_became_active":"2024-09-12T12:21:30.200750+0000","last_became_peered":"2024-09-12T12:21:30.200750+0000","last_unstale":"2024-09-12T12:21:30.210545+0000","last_undegraded":"2024-09-12T12:21:30.210545+0000","last_fullsized":"2024-09-12T12:21:30.210545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:18.349581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896201+0000","last_change":"2024-09-12T12:21:29.206800+0000","last_active":"2024-09-12T12:21:31.896201+0000","last_peered":"2024-09-12T12:21:31.896201+0000","last_clean":"2024-09-12T12:21:31.896201+0000","last_became_active":"2024-09-12T12:21:29.206391+0000","last_became_peered":"2024-09-12T12:21:29.206391+0000","last_unstale":"2024-09-12T12:21:31.896201+0000","last_undegraded":"2024-09-12T12:21:31.896201+0000","last_fullsized":"2024-09-12T12:21:31.896201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:44.852326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920202+0000","last_change":"2024-09-12T12:21:30.222358+0000","last_active":"2024-09-12T12:21:31.920202+0000","last_peered":"2024-09-12T12:21:31.920202+0000","last_clean":"2024-09-12T12:21:31.920202+0000","last_became_active":"2024-09-12T12:21:30.221945+0000","last_became_peered":"2024-09-12T12:21:30.221945+0000","last_unstale":"2024-09-12T12:21:31.920202+0000","last_undegraded":"2024-09-12T12:21:31.920202+0000","last_fullsized":"2024-09-12T12:21:31.920202+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214814+0000","last_change":"2024-09-12T12:21:29.199711+0000","last_active":"2024-09-12T12:21:32.214814+0000","last_peered":"2024-09-12T12:21:32.214814+0000","last_clean":"2024-09-12T12:21:32.214814+0000","last_became_active":"2024-09-12T12:21:29.199226+0000","last_became_peered":"2024-09-12T12:21:29.199226+0000","last_unstale":"2024-09-12T12:21:32.214814+0000","last_undegraded":"2024-09-12T12:21:32.214814+0000","last_fullsized":"2024-09-12T12:21:32.214814+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:36:17.189325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210138+0000","last_change":"2024-09-12T12:21:30.206097+0000","last_active":"2024-09-12T12:21:30.210138+0000","last_peered":"2024-09-12T12:21:30.210138+0000","last_clean":"2024-09-12T12:21:30.210138+0000","last_became_active":"2024-09-12T12:21:30.205062+0000","last_became_peered":"2024-09-12T12:21:30.205062+0000","last_unstale":"2024-09-12T12:21:30.210138+0000","last_undegraded":"2024-09-12T12:21:30.210138+0000","last_fullsized":"2024-09-12T12:21:30.210138+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:29.640882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184063+0000","last_change":"2024-09-12T12:21:29.226014+0000","last_active":"2024-09-12T12:21:30.184063+0000","last_peered":"2024-09-12T12:21:30.184063+0000","last_clean":"2024-09-12T12:21:30.184063+0000","last_became_active":"2024-09-12T12:21:29.225416+0000","last_became_peered":"2024-09-12T12:21:29.225416+0000","last_unstale":"2024-09-12T12:21:30.184063+0000","last_undegraded":"2024-09-12T12:21:30.184063+0000","last_fullsized":"2024-09-12T12:21:30.184063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.213949+0000","last_change":"2024-09-12T12:21:30.223400+0000","last_active":"2024-09-12T12:21:32.213949+0000","last_peered":"2024-09-12T12:21:32.213949+0000","last_clean":"2024-09-12T12:21:32.213949+0000","last_became_active":"2024-09-12T12:21:30.223016+0000","last_became_peered":"2024-09-12T12:21:30.223016+0000","last_unstale":"2024-09-12T12:21:32.213949+0000","last_undegraded":"2024-09-12T12:21:32.213949+0000","last_fullsized":"2024-09-12T12:21:32.213949+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:58:18.019451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214187+0000","last_change":"2024-09-12T12:21:29.193758+0000","last_active":"2024-09-12T12:21:32.214187+0000","last_peered":"2024-09-12T12:21:32.214187+0000","last_clean":"2024-09-12T12:21:32.214187+0000","last_became_active":"2024-09-12T12:21:29.193474+0000","last_became_peered":"2024-09-12T12:21:29.193474+0000","last_unstale":"2024-09-12T12:21:32.214187+0000","last_undegraded":"2024-09-12T12:21:32.214187+0000","last_fullsized":"2024-09-12T12:21:32.214187+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:17:33.654013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.221978+0000","last_change":"2024-09-12T12:21:30.218431+0000","last_active":"2024-09-12T12:21:30.221978+0000","last_peered":"2024-09-12T12:21:30.221978+0000","last_clean":"2024-09-12T12:21:30.221978+0000","last_became_active":"2024-09-12T12:21:30.216964+0000","last_became_peered":"2024-09-12T12:21:30.216964+0000","last_unstale":"2024-09-12T12:21:30.221978+0000","last_undegraded":"2024-09-12T12:21:30.221978+0000","last_fullsized":"2024-09-12T12:21:30.221978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.241841+0000","last_change":"2024-09-12T12:21:29.192499+0000","last_active":"2024-09-12T12:21:30.241841+0000","last_peered":"2024-09-12T12:21:30.241841+0000","last_clean":"2024-09-12T12:21:30.241841+0000","last_became_active":"2024-09-12T12:21:29.192004+0000","last_became_peered":"2024-09-12T12:21:29.192004+0000","last_unstale":"2024-09-12T12:21:30.241841+0000","last_undegraded":"2024-09-12T12:21:30.241841+0000","last_fullsized":"2024-09-12T12:21:30.241841+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:34.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896555+0000","last_change":"2024-09-12T12:21:30.205201+0000","last_active":"2024-09-12T12:21:31.896555+0000","last_peered":"2024-09-12T12:21:31.896555+0000","last_clean":"2024-09-12T12:21:31.896555+0000","last_became_active":"2024-09-12T12:21:30.204816+0000","last_became_peered":"2024-09-12T12:21:30.204816+0000","last_unstale":"2024-09-12T12:21:31.896555+0000","last_undegraded":"2024-09-12T12:21:31.896555+0000","last_fullsized":"2024-09-12T12:21:31.896555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:15.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896398+0000","last_change":"2024-09-12T12:21:29.188763+0000","last_active":"2024-09-12T12:21:31.896398+0000","last_peered":"2024-09-12T12:21:31.896398+0000","last_clean":"2024-09-12T12:21:31.896398+0000","last_became_active":"2024-09-12T12:21:29.188359+0000","last_became_peered":"2024-09-12T12:21:29.188359+0000","last_unstale":"2024-09-12T12:21:31.896398+0000","last_undegraded":"2024-09-12T12:21:31.896398+0000","last_fullsized":"2024-09-12T12:21:31.896398+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:01:56.346195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.243691+0000","last_change":"2024-09-12T12:21:30.239463+0000","last_active":"2024-09-12T12:21:30.243691+0000","last_peered":"2024-09-12T12:21:30.243691+0000","last_clean":"2024-09-12T12:21:30.243691+0000","last_became_active":"2024-09-12T12:21:30.239104+0000","last_became_peered":"2024-09-12T12:21:30.239104+0000","last_unstale":"2024-09-12T12:21:30.243691+0000","last_undegraded":"2024-09-12T12:21:30.243691+0000","last_fullsized":"2024-09-12T12:21:30.243691+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:26.091079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.211047+0000","last_change":"2024-09-12T12:21:29.207118+0000","last_active":"2024-09-12T12:21:31.211047+0000","last_peered":"2024-09-12T12:21:31.211047+0000","last_clean":"2024-09-12T12:21:31.211047+0000","last_became_active":"2024-09-12T12:21:29.206326+0000","last_became_peered":"2024-09-12T12:21:29.206326+0000","last_unstale":"2024-09-12T12:21:31.211047+0000","last_undegraded":"2024-09-12T12:21:31.211047+0000","last_fullsized":"2024-09-12T12:21:31.211047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920288+0000","last_change":"2024-09-12T12:21:30.240598+0000","last_active":"2024-09-12T12:21:31.920288+0000","last_peered":"2024-09-12T12:21:31.920288+0000","last_clean":"2024-09-12T12:21:31.920288+0000","last_became_active":"2024-09-12T12:21:30.240256+0000","last_became_peered":"2024-09-12T12:21:30.240256+0000","last_unstale":"2024-09-12T12:21:31.920288+0000","last_undegraded":"2024-09-12T12:21:31.920288+0000","last_fullsized":"2024-09-12T12:21:31.920288+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209227+0000","last_change":"2024-09-12T12:21:29.199199+0000","last_active":"2024-09-12T12:21:31.209227+0000","last_peered":"2024-09-12T12:21:31.209227+0000","last_clean":"2024-09-12T12:21:31.209227+0000","last_became_active":"2024-09-12T12:21:29.198335+0000","last_became_peered":"2024-09-12T12:21:29.198335+0000","last_unstale":"2024-09-12T12:21:31.209227+0000","last_undegraded":"2024-09-12T12:21:31.209227+0000","last_fullsized":"2024-09-12T12:21:31.209227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:21.550250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920668+0000","last_change":"2024-09-12T12:21:30.228570+0000","last_active":"2024-09-12T12:21:31.920668+0000","last_peered":"2024-09-12T12:21:31.920668+0000","last_clean":"2024-09-12T12:21:31.920668+0000","last_became_active":"2024-09-12T12:21:30.226661+0000","last_became_peered":"2024-09-12T12:21:30.226661+0000","last_unstale":"2024-09-12T12:21:31.920668+0000","last_undegraded":"2024-09-12T12:21:31.920668+0000","last_fullsized":"2024-09-12T12:21:31.920668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:22.534750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214612+0000","last_change":"2024-09-12T12:21:29.209464+0000","last_active":"2024-09-12T12:21:32.214612+0000","last_peered":"2024-09-12T12:21:32.214612+0000","last_clean":"2024-09-12T12:21:32.214612+0000","last_became_active":"2024-09-12T12:21:29.209126+0000","last_became_peered":"2024-09-12T12:21:29.209126+0000","last_unstale":"2024-09-12T12:21:32.214612+0000","last_undegraded":"2024-09-12T12:21:32.214612+0000","last_fullsized":"2024-09-12T12:21:32.214612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:08:18.009203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214399+0000","last_change":"2024-09-12T12:21:30.227383+0000","last_active":"2024-09-12T12:21:32.214399+0000","last_peered":"2024-09-12T12:21:32.214399+0000","last_clean":"2024-09-12T12:21:32.214399+0000","last_became_active":"2024-09-12T12:21:30.226337+0000","last_became_peered":"2024-09-12T12:21:30.226337+0000","last_unstale":"2024-09-12T12:21:32.214399+0000","last_undegraded":"2024-09-12T12:21:32.214399+0000","last_fullsized":"2024-09-12T12:21:32.214399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:07.482331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.215169+0000","last_change":"2024-09-12T12:21:29.212953+0000","last_active":"2024-09-12T12:21:32.215169+0000","last_peered":"2024-09-12T12:21:32.215169+0000","last_clean":"2024-09-12T12:21:32.215169+0000","last_became_active":"2024-09-12T12:21:29.212586+0000","last_became_peered":"2024-09-12T12:21:29.212586+0000","last_unstale":"2024-09-12T12:21:32.215169+0000","last_undegraded":"2024-09-12T12:21:32.215169+0000","last_fullsized":"2024-09-12T12:21:32.215169+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:55.364737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.223948+0000","last_change":"2024-09-12T12:21:30.213940+0000","last_active":"2024-09-12T12:21:30.223948+0000","last_peered":"2024-09-12T12:21:30.223948+0000","last_clean":"2024-09-12T12:21:30.223948+0000","last_became_active":"2024-09-12T12:21:30.204295+0000","last_became_peered":"2024-09-12T12:21:30.204295+0000","last_unstale":"2024-09-12T12:21:30.223948+0000","last_undegraded":"2024-09-12T12:21:30.223948+0000","last_fullsized":"2024-09-12T12:21:30.223948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207755+0000","last_change":"2024-09-12T12:21:29.221505+0000","last_active":"2024-09-12T12:21:31.207755+0000","last_peered":"2024-09-12T12:21:31.207755+0000","last_clean":"2024-09-12T12:21:31.207755+0000","last_became_active":"2024-09-12T12:21:29.220806+0000","last_became_peered":"2024-09-12T12:21:29.220806+0000","last_unstale":"2024-09-12T12:21:31.207755+0000","last_undegraded":"2024-09-12T12:21:31.207755+0000","last_fullsized":"2024-09-12T12:21:31.207755+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896182+0000","last_change":"2024-09-12T12:21:30.202543+0000","last_active":"2024-09-12T12:21:31.896182+0000","last_peered":"2024-09-12T12:21:31.896182+0000","last_clean":"2024-09-12T12:21:31.896182+0000","last_became_active":"2024-09-12T12:21:30.201969+0000","last_became_peered":"2024-09-12T12:21:30.201969+0000","last_unstale":"2024-09-12T12:21:31.896182+0000","last_undegraded":"2024-09-12T12:21:31.896182+0000","last_fullsized":"2024-09-12T12:21:31.896182+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:17.869098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193913+0000","last_change":"2024-09-12T12:21:29.221416+0000","last_active":"2024-09-12T12:21:30.193913+0000","last_peered":"2024-09-12T12:21:30.193913+0000","last_clean":"2024-09-12T12:21:30.193913+0000","last_became_active":"2024-09-12T12:21:29.220920+0000","last_became_peered":"2024-09-12T12:21:29.220920+0000","last_unstale":"2024-09-12T12:21:30.193913+0000","last_undegraded":"2024-09-12T12:21:30.193913+0000","last_fullsized":"2024-09-12T12:21:30.193913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:22:16.215986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.230398+0000","last_change":"2024-09-12T12:21:30.228710+0000","last_active":"2024-09-12T12:21:30.230398+0000","last_peered":"2024-09-12T12:21:30.230398+0000","last_clean":"2024-09-12T12:21:30.230398+0000","last_became_active":"2024-09-12T12:21:30.228353+0000","last_became_peered":"2024-09-12T12:21:30.228353+0000","last_unstale":"2024-09-12T12:21:30.230398+0000","last_undegraded":"2024-09-12T12:21:30.230398+0000","last_fullsized":"2024-09-12T12:21:30.230398+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:52.048471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183945+0000","last_change":"2024-09-12T12:21:29.220023+0000","last_active":"2024-09-12T12:21:30.183945+0000","last_peered":"2024-09-12T12:21:30.183945+0000","last_clean":"2024-09-12T12:21:30.183945+0000","last_became_active":"2024-09-12T12:21:29.219318+0000","last_became_peered":"2024-09-12T12:21:29.219318+0000","last_unstale":"2024-09-12T12:21:30.183945+0000","last_undegraded":"2024-09-12T12:21:30.183945+0000","last_fullsized":"2024-09-12T12:21:30.183945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920942+0000","last_change":"2024-09-12T12:21:30.226661+0000","last_active":"2024-09-12T12:21:31.920942+0000","last_peered":"2024-09-12T12:21:31.920942+0000","last_clean":"2024-09-12T12:21:31.920942+0000","last_became_active":"2024-09-12T12:21:30.225756+0000","last_became_peered":"2024-09-12T12:21:30.225756+0000","last_unstale":"2024-09-12T12:21:31.920942+0000","last_undegraded":"2024-09-12T12:21:31.920942+0000","last_fullsized":"2024-09-12T12:21:31.920942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214947+0000","last_change":"2024-09-12T12:21:29.206612+0000","last_active":"2024-09-12T12:21:32.214947+0000","last_peered":"2024-09-12T12:21:32.214947+0000","last_clean":"2024-09-12T12:21:32.214947+0000","last_became_active":"2024-09-12T12:21:29.206303+0000","last_became_peered":"2024-09-12T12:21:29.206303+0000","last_unstale":"2024-09-12T12:21:32.214947+0000","last_undegraded":"2024-09-12T12:21:32.214947+0000","last_fullsized":"2024-09-12T12:21:32.214947+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:51.121788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.220107+0000","last_change":"2024-09-12T12:21:30.212519+0000","last_active":"2024-09-12T12:21:30.220107+0000","last_peered":"2024-09-12T12:21:30.220107+0000","last_clean":"2024-09-12T12:21:30.220107+0000","last_became_active":"2024-09-12T12:21:30.211566+0000","last_became_peered":"2024-09-12T12:21:30.211566+0000","last_unstale":"2024-09-12T12:21:30.220107+0000","last_undegraded":"2024-09-12T12:21:30.220107+0000","last_fullsized":"2024-09-12T12:21:30.220107+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:30:37.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.186811+0000","last_change":"2024-09-12T12:21:29.227541+0000","last_active":"2024-09-12T12:21:30.186811+0000","last_peered":"2024-09-12T12:21:30.186811+0000","last_clean":"2024-09-12T12:21:30.186811+0000","last_became_active":"2024-09-12T12:21:29.226501+0000","last_became_peered":"2024-09-12T12:21:29.226501+0000","last_unstale":"2024-09-12T12:21:30.186811+0000","last_undegraded":"2024-09-12T12:21:30.186811+0000","last_fullsized":"2024-09-12T12:21:30.186811+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:21.714121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210598+0000","last_change":"2024-09-12T12:21:30.204222+0000","last_active":"2024-09-12T12:21:30.210598+0000","last_peered":"2024-09-12T12:21:30.210598+0000","last_clean":"2024-09-12T12:21:30.210598+0000","last_became_active":"2024-09-12T12:21:30.203608+0000","last_became_peered":"2024-09-12T12:21:30.203608+0000","last_unstale":"2024-09-12T12:21:30.210598+0000","last_undegraded":"2024-09-12T12:21:30.210598+0000","last_fullsized":"2024-09-12T12:21:30.210598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181466+0000","last_change":"2024-09-12T12:21:29.188153+0000","last_active":"2024-09-12T12:21:30.181466+0000","last_peered":"2024-09-12T12:21:30.181466+0000","last_clean":"2024-09-12T12:21:30.181466+0000","last_became_active":"2024-09-12T12:21:29.187754+0000","last_became_peered":"2024-09-12T12:21:29.187754+0000","last_unstale":"2024-09-12T12:21:30.181466+0000","last_undegraded":"2024-09-12T12:21:30.181466+0000","last_fullsized":"2024-09-12T12:21:30.181466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214578+0000","last_change":"2024-09-12T12:21:30.227067+0000","last_active":"2024-09-12T12:21:32.214578+0000","last_peered":"2024-09-12T12:21:32.214578+0000","last_clean":"2024-09-12T12:21:32.214578+0000","last_became_active":"2024-09-12T12:21:30.225956+0000","last_became_peered":"2024-09-12T12:21:30.225956+0000","last_unstale":"2024-09-12T12:21:32.214578+0000","last_undegraded":"2024-09-12T12:21:32.214578+0000","last_fullsized":"2024-09-12T12:21:32.214578+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:52.370860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184473+0000","last_change":"2024-09-12T12:21:29.227844+0000","last_active":"2024-09-12T12:21:30.184473+0000","last_peered":"2024-09-12T12:21:30.184473+0000","last_clean":"2024-09-12T12:21:30.184473+0000","last_became_active":"2024-09-12T12:21:29.227189+0000","last_became_peered":"2024-09-12T12:21:29.227189+0000","last_unstale":"2024-09-12T12:21:30.184473+0000","last_undegraded":"2024-09-12T12:21:30.184473+0000","last_fullsized":"2024-09-12T12:21:30.184473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:29.888392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214592+0000","last_change":"2024-09-12T12:21:30.224043+0000","last_active":"2024-09-12T12:21:32.214592+0000","last_peered":"2024-09-12T12:21:32.214592+0000","last_clean":"2024-09-12T12:21:32.214592+0000","last_became_active":"2024-09-12T12:21:30.222387+0000","last_became_peered":"2024-09-12T12:21:30.222387+0000","last_unstale":"2024-09-12T12:21:32.214592+0000","last_undegraded":"2024-09-12T12:21:32.214592+0000","last_fullsized":"2024-09-12T12:21:32.214592+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:11:28.175498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209825+0000","last_change":"2024-09-12T12:21:29.203168+0000","last_active":"2024-09-12T12:21:31.209825+0000","last_peered":"2024-09-12T12:21:31.209825+0000","last_clean":"2024-09-12T12:21:31.209825+0000","last_became_active":"2024-09-12T12:21:29.201201+0000","last_became_peered":"2024-09-12T12:21:29.201201+0000","last_unstale":"2024-09-12T12:21:31.209825+0000","last_undegraded":"2024-09-12T12:21:31.209825+0000","last_fullsized":"2024-09-12T12:21:31.209825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:14.394911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207540+0000","last_change":"2024-09-12T12:21:30.221515+0000","last_active":"2024-09-12T12:21:31.207540+0000","last_peered":"2024-09-12T12:21:31.207540+0000","last_clean":"2024-09-12T12:21:31.207540+0000","last_became_active":"2024-09-12T12:21:30.221202+0000","last_became_peered":"2024-09-12T12:21:30.221202+0000","last_unstale":"2024-09-12T12:21:31.207540+0000","last_undegraded":"2024-09-12T12:21:31.207540+0000","last_fullsized":"2024-09-12T12:21:31.207540+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181458+0000","last_change":"2024-09-12T12:21:29.215337+0000","last_active":"2024-09-12T12:21:30.181458+0000","last_peered":"2024-09-12T12:21:30.181458+0000","last_clean":"2024-09-12T12:21:30.181458+0000","last_became_active":"2024-09-12T12:21:29.214887+0000","last_became_peered":"2024-09-12T12:21:29.214887+0000","last_unstale":"2024-09-12T12:21:30.181458+0000","last_undegraded":"2024-09-12T12:21:30.181458+0000","last_fullsized":"2024-09-12T12:21:30.181458+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208962+0000","last_change":"2024-09-12T12:21:30.228674+0000","last_active":"2024-09-12T12:21:31.208962+0000","last_peered":"2024-09-12T12:21:31.208962+0000","last_clean":"2024-09-12T12:21:31.208962+0000","last_became_active":"2024-09-12T12:21:30.228376+0000","last_became_peered":"2024-09-12T12:21:30.228376+0000","last_unstale":"2024-09-12T12:21:31.208962+0000","last_undegraded":"2024-09-12T12:21:31.208962+0000","last_fullsized":"2024-09-12T12:21:31.208962+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191879+0000","last_change":"2024-09-12T12:21:29.193465+0000","last_active":"2024-09-12T12:21:30.191879+0000","last_peered":"2024-09-12T12:21:30.191879+0000","last_clean":"2024-09-12T12:21:30.191879+0000","last_became_active":"2024-09-12T12:21:29.193077+0000","last_became_peered":"2024-09-12T12:21:29.193077+0000","last_unstale":"2024-09-12T12:21:30.191879+0000","last_undegraded":"2024-09-12T12:21:30.191879+0000","last_fullsized":"2024-09-12T12:21:30.191879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:13.069402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210273+0000","last_change":"2024-09-12T12:21:30.206965+0000","last_active":"2024-09-12T12:21:30.210273+0000","last_peered":"2024-09-12T12:21:30.210273+0000","last_clean":"2024-09-12T12:21:30.210273+0000","last_became_active":"2024-09-12T12:21:30.204731+0000","last_became_peered":"2024-09-12T12:21:30.204731+0000","last_unstale":"2024-09-12T12:21:30.210273+0000","last_undegraded":"2024-09-12T12:21:30.210273+0000","last_fullsized":"2024-09-12T12:21:30.210273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:55.697639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209341+0000","last_change":"2024-09-12T12:21:29.203115+0000","last_active":"2024-09-12T12:21:31.209341+0000","last_peered":"2024-09-12T12:21:31.209341+0000","last_clean":"2024-09-12T12:21:31.209341+0000","last_became_active":"2024-09-12T12:21:29.202388+0000","last_became_peered":"2024-09-12T12:21:29.202388+0000","last_unstale":"2024-09-12T12:21:31.209341+0000","last_undegraded":"2024-09-12T12:21:31.209341+0000","last_fullsized":"2024-09-12T12:21:31.209341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183862+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:21:30.183862+0000","last_peered":"2024-09-12T12:21:30.183862+0000","last_clean":"2024-09-12T12:21:30.183862+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:21:30.183862+0000","last_undegraded":"2024-09-12T12:21:30.183862+0000","last_fullsized":"2024-09-12T12:21:30.183862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":200905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1085440,"data_stored":784425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":200337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":201879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":201901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":194037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":784369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:21:38.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:21:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.929+0000 7fb45e65f640 1 Processor -- start 2024-09-12T12:21:38.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45e65f640 1 -- start start 2024-09-12T12:21:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45e65f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45815b2d0 0x7fb4580b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45e65f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb45806f6e0 0x7fb458072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45e65f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45815f150 con 0x7fb4580cc300 2024-09-12T12:21:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45e65f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb4580ae920 con 0x7fb45806f6e0 2024-09-12T12:21:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45e65f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb45815b6b0 con 0x7fb45815b2d0 2024-09-12T12:21:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45cbfc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb4580cc300 0x7fb4580aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35046/0 (socket says 172.21.15.3:35046) 2024-09-12T12:21:38.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb45cbfc640 1 -- 172.21.15.3:0/2385275185 learned_addr learned my addr 172.21.15.3:0/2385275185 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb4577fe640 1 --2- 172.21.15.3:0/2385275185 >> v2:172.21.15.3:3300/0 conn(0x7fb45806f6e0 0x7fb458072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb457fff640 1 --2- 172.21.15.3:0/2385275185 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45815b2d0 0x7fb4580b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb456ffd640 1 -- 172.21.15.3:0/2385275185 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440392361 0 0) 0x7fb45815f150 con 0x7fb4580cc300 2024-09-12T12:21:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb456ffd640 1 -- 172.21.15.3:0/2385275185 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb438005740 con 0x7fb4580cc300 2024-09-12T12:21:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb4577fe640 1 -- 172.21.15.3:0/2385275185 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45815b2d0 msgr2=0x7fb4580b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb4577fe640 1 --2- 172.21.15.3:0/2385275185 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45815b2d0 0x7fb4580b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:38.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb4577fe640 1 -- 172.21.15.3:0/2385275185 >> v1:172.21.15.3:6789/0 conn(0x7fb4580cc300 legacy=0x7fb4580aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:38.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb4577fe640 1 -- 172.21.15.3:0/2385275185 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45815ee70 con 0x7fb45806f6e0 2024-09-12T12:21:38.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.933+0000 7fb457fff640 1 --2- 172.21.15.3:0/2385275185 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45815b2d0 0x7fb4580b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb4577fe640 1 --2- 172.21.15.3:0/2385275185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45806f6e0 0x7fb458072f30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4400097c0 tx=0x7fb44001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7807f621c5424f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb456ffd640 1 -- 172.21.15.3:0/2385275185 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400b0070 con 0x7fb45806f6e0 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb456ffd640 1 -- 172.21.15.3:0/2385275185 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4400ab600 con 0x7fb45806f6e0 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb456ffd640 1 -- 172.21.15.3:0/2385275185 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400ab9b0 con 0x7fb45806f6e0 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- 172.21.15.3:0/2385275185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45806f6e0 msgr2=0x7fb458072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 --2- 172.21.15.3:0/2385275185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45806f6e0 0x7fb458072f30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4400097c0 tx=0x7fb44001c470 comp rx=0 tx=0).stop 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- 172.21.15.3:0/2385275185 shutdown_connections 2024-09-12T12:21:38.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 --2- 172.21.15.3:0/2385275185 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45806f6e0 0x7fb458072f30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 --2- 172.21.15.3:0/2385275185 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45815b2d0 0x7fb4580b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- 172.21.15.3:0/2385275185 >> 172.21.15.3:0/2385275185 conn(0x7fb4580b1d80 msgr2=0x7fb4580b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- 172.21.15.3:0/2385275185 shutdown_connections 2024-09-12T12:21:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- 172.21.15.3:0/2385275185 wait complete. 2024-09-12T12:21:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 Processor -- start 2024-09-12T12:21:38.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- start start 2024-09-12T12:21:38.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45806f6e0 0x7fb458157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4580cc300 0x7fb45814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45815b2d0 0x7fb458155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb45806d8b0 con 0x7fb45815b2d0 2024-09-12T12:21:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb458073880 con 0x7fb4580cc300 2024-09-12T12:21:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45e65f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb4580b35c0 con 0x7fb45806f6e0 2024-09-12T12:21:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45cbfc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45815b2d0 0x7fb458155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:38.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45cbfc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45815b2d0 0x7fb458155b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60664/0 (socket says 172.21.15.3:60664) 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb45cbfc640 1 -- 172.21.15.3:0/230356226 learned_addr learned my addr 172.21.15.3:0/230356226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.937+0000 7fb4577fe640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4580cc300 0x7fb45814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb457fff640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45806f6e0 0x7fb458157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45cbfc640 1 -- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45806f6e0 msgr2=0x7fb458157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45cbfc640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45806f6e0 0x7fb458157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45cbfc640 1 -- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4580cc300 msgr2=0x7fb45814e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45cbfc640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4580cc300 0x7fb45814e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45cbfc640 1 -- 172.21.15.3:0/230356226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb458154ad0 con 0x7fb45815b2d0 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb457fff640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45806f6e0 0x7fb458157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb4577fe640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4580cc300 0x7fb45814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45cbfc640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45815b2d0 0x7fb458155b70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb45000cad0 tx=0x7fb45000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:38.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb454ff9640 1 -- 172.21.15.3:0/230356226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450017070 con 0x7fb45815b2d0 2024-09-12T12:21:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb454ff9640 1 -- 172.21.15.3:0/230356226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45000ac30 con 0x7fb45815b2d0 2024-09-12T12:21:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb458154db0 con 0x7fb45815b2d0 2024-09-12T12:21:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb454ff9640 1 -- 172.21.15.3:0/230356226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450012400 con 0x7fb45815b2d0 2024-09-12T12:21:38.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb458155370 con 0x7fb45815b2d0 2024-09-12T12:21:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb458075550 con 0x7fb45815b2d0 2024-09-12T12:21:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb454ff9640 1 -- 172.21.15.3:0/230356226 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb45001a070 con 0x7fb45815b2d0 2024-09-12T12:21:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb454ff9640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb43c0678c0 0x7fb43c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb454ff9640 1 -- 172.21.15.3:0/230356226 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb450085ae0 con 0x7fb45815b2d0 2024-09-12T12:21:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb457fff640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb43c0678c0 0x7fb43c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:38.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.941+0000 7fb457fff640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb43c0678c0 0x7fb43c069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb448009980 tx=0x7fb448008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:38.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:38.945+0000 7fb454ff9640 1 -- 172.21.15.3:0/230356226 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb450056620 con 0x7fb45815b2d0 2024-09-12T12:21:39.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.109+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb458072eb0 con 0x7fb43c0678c0 2024-09-12T12:21:39.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.125+0000 7fb454ff9640 1 -- 172.21.15.3:0/230356226 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251933 (secure 0 0 0) 0x7fb458072eb0 con 0x7fb43c0678c0 2024-09-12T12:21:39.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:39.134 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb43c0678c0 msgr2=0x7fb43c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb43c0678c0 0x7fb43c069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb448009980 tx=0x7fb448008040 comp rx=0 tx=0).stop 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45815b2d0 msgr2=0x7fb458155b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45815b2d0 0x7fb458155b70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb45000cad0 tx=0x7fb45000cf90 comp rx=0 tx=0).stop 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 shutdown_connections 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb43c0678c0 0x7fb43c069d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb45815b2d0 0x7fb458155b70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4580cc300 0x7fb45814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 --2- 172.21.15.3:0/230356226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb45806f6e0 0x7fb458157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 >> 172.21.15.3:0/230356226 conn(0x7fb4580b1d80 msgr2=0x7fb4580b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 shutdown_connections 2024-09-12T12:21:39.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.129+0000 7fb45e65f640 1 -- 172.21.15.3:0/230356226 wait complete. 2024-09-12T12:21:39.151 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-12T12:21:37.693432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225476,"kb_used_data":5080,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749244,"statfs":{"total":773094113280,"available":772863225856,"internally_reserved":0,"allocated":5201920,"data_stored":2762492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002814"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.208627+0000","last_change":"2024-09-12T12:21:30.200850+0000","last_active":"2024-09-12T12:21:30.208627+0000","last_peered":"2024-09-12T12:21:30.208627+0000","last_clean":"2024-09-12T12:21:30.208627+0000","last_became_active":"2024-09-12T12:21:30.200142+0000","last_became_peered":"2024-09-12T12:21:30.200142+0000","last_unstale":"2024-09-12T12:21:30.208627+0000","last_undegraded":"2024-09-12T12:21:30.208627+0000","last_fullsized":"2024-09-12T12:21:30.208627+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209219+0000","last_change":"2024-09-12T12:21:30.208435+0000","last_active":"2024-09-12T12:21:31.209219+0000","last_peered":"2024-09-12T12:21:31.209219+0000","last_clean":"2024-09-12T12:21:31.209219+0000","last_became_active":"2024-09-12T12:21:30.208134+0000","last_became_peered":"2024-09-12T12:21:30.208134+0000","last_unstale":"2024-09-12T12:21:31.209219+0000","last_undegraded":"2024-09-12T12:21:31.209219+0000","last_fullsized":"2024-09-12T12:21:31.209219+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:27:33.718518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210339+0000","last_change":"2024-09-12T12:21:30.212483+0000","last_active":"2024-09-12T12:21:31.210339+0000","last_peered":"2024-09-12T12:21:31.210339+0000","last_clean":"2024-09-12T12:21:31.210339+0000","last_became_active":"2024-09-12T12:21:30.211973+0000","last_became_peered":"2024-09-12T12:21:30.211973+0000","last_unstale":"2024-09-12T12:21:31.210339+0000","last_undegraded":"2024-09-12T12:21:31.210339+0000","last_fullsized":"2024-09-12T12:21:31.210339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:30.898304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227851+0000","last_change":"2024-09-12T12:21:30.222772+0000","last_active":"2024-09-12T12:21:30.227851+0000","last_peered":"2024-09-12T12:21:30.227851+0000","last_clean":"2024-09-12T12:21:30.227851+0000","last_became_active":"2024-09-12T12:21:30.222582+0000","last_became_peered":"2024-09-12T12:21:30.222582+0000","last_unstale":"2024-09-12T12:21:30.227851+0000","last_undegraded":"2024-09-12T12:21:30.227851+0000","last_fullsized":"2024-09-12T12:21:30.227851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:53.948158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207968+0000","last_change":"2024-09-12T12:21:30.208596+0000","last_active":"2024-09-12T12:21:31.207968+0000","last_peered":"2024-09-12T12:21:31.207968+0000","last_clean":"2024-09-12T12:21:31.207968+0000","last_became_active":"2024-09-12T12:21:30.208202+0000","last_became_peered":"2024-09-12T12:21:30.208202+0000","last_unstale":"2024-09-12T12:21:31.207968+0000","last_undegraded":"2024-09-12T12:21:31.207968+0000","last_fullsized":"2024-09-12T12:21:31.207968+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214106+0000","last_change":"2024-09-12T12:21:30.227649+0000","last_active":"2024-09-12T12:21:32.214106+0000","last_peered":"2024-09-12T12:21:32.214106+0000","last_clean":"2024-09-12T12:21:32.214106+0000","last_became_active":"2024-09-12T12:21:30.226708+0000","last_became_peered":"2024-09-12T12:21:30.226708+0000","last_unstale":"2024-09-12T12:21:32.214106+0000","last_undegraded":"2024-09-12T12:21:32.214106+0000","last_fullsized":"2024-09-12T12:21:32.214106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:14.331393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210712+0000","last_change":"2024-09-12T12:21:30.204887+0000","last_active":"2024-09-12T12:21:30.210712+0000","last_peered":"2024-09-12T12:21:30.210712+0000","last_clean":"2024-09-12T12:21:30.210712+0000","last_became_active":"2024-09-12T12:21:30.204053+0000","last_became_peered":"2024-09-12T12:21:30.204053+0000","last_unstale":"2024-09-12T12:21:30.210712+0000","last_undegraded":"2024-09-12T12:21:30.210712+0000","last_fullsized":"2024-09-12T12:21:30.210712+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:55:48.432049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225399+0000","last_change":"2024-09-12T12:21:30.220921+0000","last_active":"2024-09-12T12:21:30.225399+0000","last_peered":"2024-09-12T12:21:30.225399+0000","last_clean":"2024-09-12T12:21:30.225399+0000","last_became_active":"2024-09-12T12:21:30.220376+0000","last_became_peered":"2024-09-12T12:21:30.220376+0000","last_unstale":"2024-09-12T12:21:30.225399+0000","last_undegraded":"2024-09-12T12:21:30.225399+0000","last_fullsized":"2024-09-12T12:21:30.225399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:35:49.354057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233115+0000","last_change":"2024-09-12T12:21:30.229582+0000","last_active":"2024-09-12T12:21:30.233115+0000","last_peered":"2024-09-12T12:21:30.233115+0000","last_clean":"2024-09-12T12:21:30.233115+0000","last_became_active":"2024-09-12T12:21:30.229229+0000","last_became_peered":"2024-09-12T12:21:30.229229+0000","last_unstale":"2024-09-12T12:21:30.233115+0000","last_undegraded":"2024-09-12T12:21:30.233115+0000","last_fullsized":"2024-09-12T12:21:30.233115+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:42:40.496362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.227913+0000","last_change":"2024-09-12T12:21:30.222613+0000","last_active":"2024-09-12T12:21:30.227913+0000","last_peered":"2024-09-12T12:21:30.227913+0000","last_clean":"2024-09-12T12:21:30.227913+0000","last_became_active":"2024-09-12T12:21:30.222213+0000","last_became_peered":"2024-09-12T12:21:30.222213+0000","last_unstale":"2024-09-12T12:21:30.227913+0000","last_undegraded":"2024-09-12T12:21:30.227913+0000","last_fullsized":"2024-09-12T12:21:30.227913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:22.314764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233213+0000","last_change":"2024-09-12T12:21:30.229703+0000","last_active":"2024-09-12T12:21:30.233213+0000","last_peered":"2024-09-12T12:21:30.233213+0000","last_clean":"2024-09-12T12:21:30.233213+0000","last_became_active":"2024-09-12T12:21:30.229280+0000","last_became_peered":"2024-09-12T12:21:30.229280+0000","last_unstale":"2024-09-12T12:21:30.233213+0000","last_undegraded":"2024-09-12T12:21:30.233213+0000","last_fullsized":"2024-09-12T12:21:30.233213+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:06:44.348610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208772+0000","last_change":"2024-09-12T12:21:30.208502+0000","last_active":"2024-09-12T12:21:31.208772+0000","last_peered":"2024-09-12T12:21:31.208772+0000","last_clean":"2024-09-12T12:21:31.208772+0000","last_became_active":"2024-09-12T12:21:30.207768+0000","last_became_peered":"2024-09-12T12:21:30.207768+0000","last_unstale":"2024-09-12T12:21:31.208772+0000","last_undegraded":"2024-09-12T12:21:31.208772+0000","last_fullsized":"2024-09-12T12:21:31.208772+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896095+0000","last_change":"2024-09-12T12:21:30.227759+0000","last_active":"2024-09-12T12:21:31.896095+0000","last_peered":"2024-09-12T12:21:31.896095+0000","last_clean":"2024-09-12T12:21:31.896095+0000","last_became_active":"2024-09-12T12:21:30.227324+0000","last_became_peered":"2024-09-12T12:21:30.227324+0000","last_unstale":"2024-09-12T12:21:31.896095+0000","last_undegraded":"2024-09-12T12:21:31.896095+0000","last_fullsized":"2024-09-12T12:21:31.896095+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:39.438927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.218203+0000","last_change":"2024-09-12T12:21:30.213039+0000","last_active":"2024-09-12T12:21:30.218203+0000","last_peered":"2024-09-12T12:21:30.218203+0000","last_clean":"2024-09-12T12:21:30.218203+0000","last_became_active":"2024-09-12T12:21:30.212232+0000","last_became_peered":"2024-09-12T12:21:30.212232+0000","last_unstale":"2024-09-12T12:21:30.218203+0000","last_undegraded":"2024-09-12T12:21:30.218203+0000","last_fullsized":"2024-09-12T12:21:30.218203+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:42:04.883109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214020+0000","last_change":"2024-09-12T12:21:30.232536+0000","last_active":"2024-09-12T12:21:32.214020+0000","last_peered":"2024-09-12T12:21:32.214020+0000","last_clean":"2024-09-12T12:21:32.214020+0000","last_became_active":"2024-09-12T12:21:30.225903+0000","last_became_peered":"2024-09-12T12:21:30.225903+0000","last_unstale":"2024-09-12T12:21:32.214020+0000","last_undegraded":"2024-09-12T12:21:32.214020+0000","last_fullsized":"2024-09-12T12:21:32.214020+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:39:45.100624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919869+0000","last_change":"2024-09-12T12:21:30.216647+0000","last_active":"2024-09-12T12:21:31.919869+0000","last_peered":"2024-09-12T12:21:31.919869+0000","last_clean":"2024-09-12T12:21:31.919869+0000","last_became_active":"2024-09-12T12:21:30.216256+0000","last_became_peered":"2024-09-12T12:21:30.216256+0000","last_unstale":"2024-09-12T12:21:31.919869+0000","last_undegraded":"2024-09-12T12:21:31.919869+0000","last_fullsized":"2024-09-12T12:21:31.919869+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:02.339672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210550+0000","last_change":"2024-09-12T12:21:30.214244+0000","last_active":"2024-09-12T12:21:31.210550+0000","last_peered":"2024-09-12T12:21:31.210550+0000","last_clean":"2024-09-12T12:21:31.210550+0000","last_became_active":"2024-09-12T12:21:30.213464+0000","last_became_peered":"2024-09-12T12:21:30.213464+0000","last_unstale":"2024-09-12T12:21:31.210550+0000","last_undegraded":"2024-09-12T12:21:31.210550+0000","last_fullsized":"2024-09-12T12:21:31.210550+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:22.724033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210473+0000","last_change":"2024-09-12T12:21:30.213222+0000","last_active":"2024-09-12T12:21:31.210473+0000","last_peered":"2024-09-12T12:21:31.210473+0000","last_clean":"2024-09-12T12:21:31.210473+0000","last_became_active":"2024-09-12T12:21:30.212819+0000","last_became_peered":"2024-09-12T12:21:30.212819+0000","last_unstale":"2024-09-12T12:21:31.210473+0000","last_undegraded":"2024-09-12T12:21:31.210473+0000","last_fullsized":"2024-09-12T12:21:31.210473+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:20.425017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920825+0000","last_change":"2024-09-12T12:21:30.216805+0000","last_active":"2024-09-12T12:21:31.920825+0000","last_peered":"2024-09-12T12:21:31.920825+0000","last_clean":"2024-09-12T12:21:31.920825+0000","last_became_active":"2024-09-12T12:21:30.216287+0000","last_became_peered":"2024-09-12T12:21:30.216287+0000","last_unstale":"2024-09-12T12:21:31.920825+0000","last_undegraded":"2024-09-12T12:21:31.920825+0000","last_fullsized":"2024-09-12T12:21:31.920825+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896373+0000","last_change":"2024-09-12T12:21:30.211007+0000","last_active":"2024-09-12T12:21:31.896373+0000","last_peered":"2024-09-12T12:21:31.896373+0000","last_clean":"2024-09-12T12:21:31.896373+0000","last_became_active":"2024-09-12T12:21:30.207831+0000","last_became_peered":"2024-09-12T12:21:30.207831+0000","last_unstale":"2024-09-12T12:21:31.896373+0000","last_undegraded":"2024-09-12T12:21:31.896373+0000","last_fullsized":"2024-09-12T12:21:31.896373+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:35:49.469026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214153+0000","last_change":"2024-09-12T12:21:30.224643+0000","last_active":"2024-09-12T12:21:32.214153+0000","last_peered":"2024-09-12T12:21:32.214153+0000","last_clean":"2024-09-12T12:21:32.214153+0000","last_became_active":"2024-09-12T12:21:30.224374+0000","last_became_peered":"2024-09-12T12:21:30.224374+0000","last_unstale":"2024-09-12T12:21:32.214153+0000","last_undegraded":"2024-09-12T12:21:32.214153+0000","last_fullsized":"2024-09-12T12:21:32.214153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:56:27.586549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224229+0000","last_change":"2024-09-12T12:21:30.216387+0000","last_active":"2024-09-12T12:21:30.224229+0000","last_peered":"2024-09-12T12:21:30.224229+0000","last_clean":"2024-09-12T12:21:30.224229+0000","last_became_active":"2024-09-12T12:21:30.215852+0000","last_became_peered":"2024-09-12T12:21:30.215852+0000","last_unstale":"2024-09-12T12:21:30.224229+0000","last_undegraded":"2024-09-12T12:21:30.224229+0000","last_fullsized":"2024-09-12T12:21:30.224229+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:57:38.107312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920593+0000","last_change":"2024-09-12T12:21:30.225623+0000","last_active":"2024-09-12T12:21:31.920593+0000","last_peered":"2024-09-12T12:21:31.920593+0000","last_clean":"2024-09-12T12:21:31.920593+0000","last_became_active":"2024-09-12T12:21:30.222578+0000","last_became_peered":"2024-09-12T12:21:30.222578+0000","last_unstale":"2024-09-12T12:21:31.920593+0000","last_undegraded":"2024-09-12T12:21:31.920593+0000","last_fullsized":"2024-09-12T12:21:31.920593+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:28.867805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919887+0000","last_change":"2024-09-12T12:21:30.217472+0000","last_active":"2024-09-12T12:21:31.919887+0000","last_peered":"2024-09-12T12:21:31.919887+0000","last_clean":"2024-09-12T12:21:31.919887+0000","last_became_active":"2024-09-12T12:21:30.217177+0000","last_became_peered":"2024-09-12T12:21:30.217177+0000","last_unstale":"2024-09-12T12:21:31.919887+0000","last_undegraded":"2024-09-12T12:21:31.919887+0000","last_fullsized":"2024-09-12T12:21:31.919887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920545+0000","last_change":"2024-09-12T12:21:30.226600+0000","last_active":"2024-09-12T12:21:31.920545+0000","last_peered":"2024-09-12T12:21:31.920545+0000","last_clean":"2024-09-12T12:21:31.920545+0000","last_became_active":"2024-09-12T12:21:30.226195+0000","last_became_peered":"2024-09-12T12:21:30.226195+0000","last_unstale":"2024-09-12T12:21:31.920545+0000","last_undegraded":"2024-09-12T12:21:31.920545+0000","last_fullsized":"2024-09-12T12:21:31.920545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:11.697205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896333+0000","last_change":"2024-09-12T12:21:30.227132+0000","last_active":"2024-09-12T12:21:31.896333+0000","last_peered":"2024-09-12T12:21:31.896333+0000","last_clean":"2024-09-12T12:21:31.896333+0000","last_became_active":"2024-09-12T12:21:30.226820+0000","last_became_peered":"2024-09-12T12:21:30.226820+0000","last_unstale":"2024-09-12T12:21:31.896333+0000","last_undegraded":"2024-09-12T12:21:31.896333+0000","last_fullsized":"2024-09-12T12:21:31.896333+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:53:14.450296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210369+0000","last_change":"2024-09-12T12:21:30.207017+0000","last_active":"2024-09-12T12:21:31.210369+0000","last_peered":"2024-09-12T12:21:31.210369+0000","last_clean":"2024-09-12T12:21:31.210369+0000","last_became_active":"2024-09-12T12:21:30.206281+0000","last_became_peered":"2024-09-12T12:21:30.206281+0000","last_unstale":"2024-09-12T12:21:31.210369+0000","last_undegraded":"2024-09-12T12:21:31.210369+0000","last_fullsized":"2024-09-12T12:21:31.210369+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:59.576497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209179+0000","last_change":"2024-09-12T12:21:30.208170+0000","last_active":"2024-09-12T12:21:31.209179+0000","last_peered":"2024-09-12T12:21:31.209179+0000","last_clean":"2024-09-12T12:21:31.209179+0000","last_became_active":"2024-09-12T12:21:30.207326+0000","last_became_peered":"2024-09-12T12:21:30.207326+0000","last_unstale":"2024-09-12T12:21:31.209179+0000","last_undegraded":"2024-09-12T12:21:31.209179+0000","last_fullsized":"2024-09-12T12:21:31.209179+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920532+0000","last_change":"2024-09-12T12:21:30.226334+0000","last_active":"2024-09-12T12:21:31.920532+0000","last_peered":"2024-09-12T12:21:31.920532+0000","last_clean":"2024-09-12T12:21:31.920532+0000","last_became_active":"2024-09-12T12:21:30.225985+0000","last_became_peered":"2024-09-12T12:21:30.225985+0000","last_unstale":"2024-09-12T12:21:31.920532+0000","last_undegraded":"2024-09-12T12:21:31.920532+0000","last_fullsized":"2024-09-12T12:21:31.920532+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:42:49.456119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208773+0000","last_change":"2024-09-12T12:21:30.209539+0000","last_active":"2024-09-12T12:21:31.208773+0000","last_peered":"2024-09-12T12:21:31.208773+0000","last_clean":"2024-09-12T12:21:31.208773+0000","last_became_active":"2024-09-12T12:21:30.209224+0000","last_became_peered":"2024-09-12T12:21:30.209224+0000","last_unstale":"2024-09-12T12:21:31.208773+0000","last_undegraded":"2024-09-12T12:21:31.208773+0000","last_fullsized":"2024-09-12T12:21:31.208773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:22.560162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214202+0000","last_change":"2024-09-12T12:21:30.232530+0000","last_active":"2024-09-12T12:21:32.214202+0000","last_peered":"2024-09-12T12:21:32.214202+0000","last_clean":"2024-09-12T12:21:32.214202+0000","last_became_active":"2024-09-12T12:21:30.222768+0000","last_became_peered":"2024-09-12T12:21:30.222768+0000","last_unstale":"2024-09-12T12:21:32.214202+0000","last_undegraded":"2024-09-12T12:21:32.214202+0000","last_fullsized":"2024-09-12T12:21:32.214202+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:05.290854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896155+0000","last_change":"2024-09-12T12:21:30.225947+0000","last_active":"2024-09-12T12:21:31.896155+0000","last_peered":"2024-09-12T12:21:31.896155+0000","last_clean":"2024-09-12T12:21:31.896155+0000","last_became_active":"2024-09-12T12:21:30.225590+0000","last_became_peered":"2024-09-12T12:21:30.225590+0000","last_unstale":"2024-09-12T12:21:31.896155+0000","last_undegraded":"2024-09-12T12:21:31.896155+0000","last_fullsized":"2024-09-12T12:21:31.896155+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:29.919842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232731+0000","last_change":"2024-09-12T12:21:30.224922+0000","last_active":"2024-09-12T12:21:30.232731+0000","last_peered":"2024-09-12T12:21:30.232731+0000","last_clean":"2024-09-12T12:21:30.232731+0000","last_became_active":"2024-09-12T12:21:30.224326+0000","last_became_peered":"2024-09-12T12:21:30.224326+0000","last_unstale":"2024-09-12T12:21:30.232731+0000","last_undegraded":"2024-09-12T12:21:30.232731+0000","last_fullsized":"2024-09-12T12:21:30.232731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.215024+0000","last_change":"2024-09-12T12:21:29.216451+0000","last_active":"2024-09-12T12:21:32.215024+0000","last_peered":"2024-09-12T12:21:32.215024+0000","last_clean":"2024-09-12T12:21:32.215024+0000","last_became_active":"2024-09-12T12:21:29.215960+0000","last_became_peered":"2024-09-12T12:21:29.215960+0000","last_unstale":"2024-09-12T12:21:32.215024+0000","last_undegraded":"2024-09-12T12:21:32.215024+0000","last_fullsized":"2024-09-12T12:21:32.215024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:47:22.554148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920715+0000","last_change":"2024-09-12T12:21:30.222316+0000","last_active":"2024-09-12T12:21:31.920715+0000","last_peered":"2024-09-12T12:21:31.920715+0000","last_clean":"2024-09-12T12:21:31.920715+0000","last_became_active":"2024-09-12T12:21:30.221990+0000","last_became_peered":"2024-09-12T12:21:30.221990+0000","last_unstale":"2024-09-12T12:21:31.920715+0000","last_undegraded":"2024-09-12T12:21:31.920715+0000","last_fullsized":"2024-09-12T12:21:31.920715+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:52.055517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210844+0000","last_change":"2024-09-12T12:21:29.219231+0000","last_active":"2024-09-12T12:21:31.210844+0000","last_peered":"2024-09-12T12:21:31.210844+0000","last_clean":"2024-09-12T12:21:31.210844+0000","last_became_active":"2024-09-12T12:21:29.218147+0000","last_became_peered":"2024-09-12T12:21:29.218147+0000","last_unstale":"2024-09-12T12:21:31.210844+0000","last_undegraded":"2024-09-12T12:21:31.210844+0000","last_fullsized":"2024-09-12T12:21:31.210844+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209841+0000","last_change":"2024-09-12T12:21:30.227814+0000","last_active":"2024-09-12T12:21:31.209841+0000","last_peered":"2024-09-12T12:21:31.209841+0000","last_clean":"2024-09-12T12:21:31.209841+0000","last_became_active":"2024-09-12T12:21:30.227419+0000","last_became_peered":"2024-09-12T12:21:30.227419+0000","last_unstale":"2024-09-12T12:21:31.209841+0000","last_undegraded":"2024-09-12T12:21:31.209841+0000","last_fullsized":"2024-09-12T12:21:31.209841+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:06:39.410795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.233639+0000","last_change":"2024-09-12T12:21:29.188168+0000","last_active":"2024-09-12T12:21:30.233639+0000","last_peered":"2024-09-12T12:21:30.233639+0000","last_clean":"2024-09-12T12:21:30.233639+0000","last_became_active":"2024-09-12T12:21:29.187846+0000","last_became_peered":"2024-09-12T12:21:29.187846+0000","last_unstale":"2024-09-12T12:21:30.233639+0000","last_undegraded":"2024-09-12T12:21:30.233639+0000","last_fullsized":"2024-09-12T12:21:30.233639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.224153+0000","last_change":"2024-09-12T12:21:30.217678+0000","last_active":"2024-09-12T12:21:30.224153+0000","last_peered":"2024-09-12T12:21:30.224153+0000","last_clean":"2024-09-12T12:21:30.224153+0000","last_became_active":"2024-09-12T12:21:30.216952+0000","last_became_peered":"2024-09-12T12:21:30.216952+0000","last_unstale":"2024-09-12T12:21:30.224153+0000","last_undegraded":"2024-09-12T12:21:30.224153+0000","last_fullsized":"2024-09-12T12:21:30.224153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.229864+0000","last_change":"2024-09-12T12:21:29.225871+0000","last_active":"2024-09-12T12:21:30.229864+0000","last_peered":"2024-09-12T12:21:30.229864+0000","last_clean":"2024-09-12T12:21:30.229864+0000","last_became_active":"2024-09-12T12:21:29.225182+0000","last_became_peered":"2024-09-12T12:21:29.225182+0000","last_unstale":"2024-09-12T12:21:30.229864+0000","last_undegraded":"2024-09-12T12:21:30.229864+0000","last_fullsized":"2024-09-12T12:21:30.229864+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:58.566585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209173+0000","last_change":"2024-09-12T12:21:30.217628+0000","last_active":"2024-09-12T12:21:31.209173+0000","last_peered":"2024-09-12T12:21:31.209173+0000","last_clean":"2024-09-12T12:21:31.209173+0000","last_became_active":"2024-09-12T12:21:30.217335+0000","last_became_peered":"2024-09-12T12:21:30.217335+0000","last_unstale":"2024-09-12T12:21:31.209173+0000","last_undegraded":"2024-09-12T12:21:31.209173+0000","last_fullsized":"2024-09-12T12:21:31.209173+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:08.753929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.215095+0000","last_change":"2024-09-12T12:21:29.221255+0000","last_active":"2024-09-12T12:21:32.215095+0000","last_peered":"2024-09-12T12:21:32.215095+0000","last_clean":"2024-09-12T12:21:32.215095+0000","last_became_active":"2024-09-12T12:21:29.220798+0000","last_became_peered":"2024-09-12T12:21:29.220798+0000","last_unstale":"2024-09-12T12:21:32.215095+0000","last_undegraded":"2024-09-12T12:21:32.215095+0000","last_fullsized":"2024-09-12T12:21:32.215095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:59.351871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210164+0000","last_change":"2024-09-12T12:21:30.206432+0000","last_active":"2024-09-12T12:21:30.210164+0000","last_peered":"2024-09-12T12:21:30.210164+0000","last_clean":"2024-09-12T12:21:30.210164+0000","last_became_active":"2024-09-12T12:21:30.205726+0000","last_became_peered":"2024-09-12T12:21:30.205726+0000","last_unstale":"2024-09-12T12:21:30.210164+0000","last_undegraded":"2024-09-12T12:21:30.210164+0000","last_fullsized":"2024-09-12T12:21:30.210164+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:14:53.863676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209831+0000","last_change":"2024-09-12T12:21:29.207503+0000","last_active":"2024-09-12T12:21:31.209831+0000","last_peered":"2024-09-12T12:21:31.209831+0000","last_clean":"2024-09-12T12:21:31.209831+0000","last_became_active":"2024-09-12T12:21:29.206715+0000","last_became_peered":"2024-09-12T12:21:29.206715+0000","last_unstale":"2024-09-12T12:21:31.209831+0000","last_undegraded":"2024-09-12T12:21:31.209831+0000","last_fullsized":"2024-09-12T12:21:31.209831+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:18:52.540391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.225377+0000","last_change":"2024-09-12T12:21:30.220672+0000","last_active":"2024-09-12T12:21:30.225377+0000","last_peered":"2024-09-12T12:21:30.225377+0000","last_clean":"2024-09-12T12:21:30.225377+0000","last_became_active":"2024-09-12T12:21:30.219208+0000","last_became_peered":"2024-09-12T12:21:30.219208+0000","last_unstale":"2024-09-12T12:21:30.225377+0000","last_undegraded":"2024-09-12T12:21:30.225377+0000","last_fullsized":"2024-09-12T12:21:30.225377+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:38.730945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207603+0000","last_change":"2024-09-12T12:21:29.204647+0000","last_active":"2024-09-12T12:21:31.207603+0000","last_peered":"2024-09-12T12:21:31.207603+0000","last_clean":"2024-09-12T12:21:31.207603+0000","last_became_active":"2024-09-12T12:21:29.203941+0000","last_became_peered":"2024-09-12T12:21:29.203941+0000","last_unstale":"2024-09-12T12:21:31.207603+0000","last_undegraded":"2024-09-12T12:21:31.207603+0000","last_fullsized":"2024-09-12T12:21:31.207603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:51.045605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.919837+0000","last_change":"2024-09-12T12:21:30.216996+0000","last_active":"2024-09-12T12:21:31.919837+0000","last_peered":"2024-09-12T12:21:31.919837+0000","last_clean":"2024-09-12T12:21:31.919837+0000","last_became_active":"2024-09-12T12:21:30.216631+0000","last_became_peered":"2024-09-12T12:21:30.216631+0000","last_unstale":"2024-09-12T12:21:31.919837+0000","last_undegraded":"2024-09-12T12:21:31.919837+0000","last_fullsized":"2024-09-12T12:21:31.919837+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:40.449036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191140+0000","last_change":"2024-09-12T12:21:29.195971+0000","last_active":"2024-09-12T12:21:30.191140+0000","last_peered":"2024-09-12T12:21:30.191140+0000","last_clean":"2024-09-12T12:21:30.191140+0000","last_became_active":"2024-09-12T12:21:29.195461+0000","last_became_peered":"2024-09-12T12:21:29.195461+0000","last_unstale":"2024-09-12T12:21:30.191140+0000","last_undegraded":"2024-09-12T12:21:30.191140+0000","last_fullsized":"2024-09-12T12:21:30.191140+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:16.138730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210986+0000","last_change":"2024-09-12T12:21:30.213037+0000","last_active":"2024-09-12T12:21:31.210986+0000","last_peered":"2024-09-12T12:21:31.210986+0000","last_clean":"2024-09-12T12:21:31.210986+0000","last_became_active":"2024-09-12T12:21:30.212297+0000","last_became_peered":"2024-09-12T12:21:30.212297+0000","last_unstale":"2024-09-12T12:21:31.210986+0000","last_undegraded":"2024-09-12T12:21:31.210986+0000","last_fullsized":"2024-09-12T12:21:31.210986+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:38.156948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.210988+0000","last_change":"2024-09-12T12:21:29.209621+0000","last_active":"2024-09-12T12:21:31.210988+0000","last_peered":"2024-09-12T12:21:31.210988+0000","last_clean":"2024-09-12T12:21:31.210988+0000","last_became_active":"2024-09-12T12:21:29.208221+0000","last_became_peered":"2024-09-12T12:21:29.208221+0000","last_unstale":"2024-09-12T12:21:31.210988+0000","last_undegraded":"2024-09-12T12:21:31.210988+0000","last_fullsized":"2024-09-12T12:21:31.210988+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:01.329760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210773+0000","last_change":"2024-09-12T12:21:30.208783+0000","last_active":"2024-09-12T12:21:30.210773+0000","last_peered":"2024-09-12T12:21:30.210773+0000","last_clean":"2024-09-12T12:21:30.210773+0000","last_became_active":"2024-09-12T12:21:30.202586+0000","last_became_peered":"2024-09-12T12:21:30.202586+0000","last_unstale":"2024-09-12T12:21:30.210773+0000","last_undegraded":"2024-09-12T12:21:30.210773+0000","last_fullsized":"2024-09-12T12:21:30.210773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:52:19.105596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193677+0000","last_change":"2024-09-12T12:21:29.221269+0000","last_active":"2024-09-12T12:21:30.193677+0000","last_peered":"2024-09-12T12:21:30.193677+0000","last_clean":"2024-09-12T12:21:30.193677+0000","last_became_active":"2024-09-12T12:21:29.220934+0000","last_became_peered":"2024-09-12T12:21:29.220934+0000","last_unstale":"2024-09-12T12:21:30.193677+0000","last_undegraded":"2024-09-12T12:21:30.193677+0000","last_fullsized":"2024-09-12T12:21:30.193677+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:47:05.831499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214887+0000","last_change":"2024-09-12T12:21:30.224755+0000","last_active":"2024-09-12T12:21:32.214887+0000","last_peered":"2024-09-12T12:21:32.214887+0000","last_clean":"2024-09-12T12:21:32.214887+0000","last_became_active":"2024-09-12T12:21:30.222276+0000","last_became_peered":"2024-09-12T12:21:30.222276+0000","last_unstale":"2024-09-12T12:21:32.214887+0000","last_undegraded":"2024-09-12T12:21:32.214887+0000","last_fullsized":"2024-09-12T12:21:32.214887+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:18.814846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.232623+0000","last_change":"2024-09-12T12:21:29.225993+0000","last_active":"2024-09-12T12:21:30.232623+0000","last_peered":"2024-09-12T12:21:30.232623+0000","last_clean":"2024-09-12T12:21:30.232623+0000","last_became_active":"2024-09-12T12:21:29.225250+0000","last_became_peered":"2024-09-12T12:21:29.225250+0000","last_unstale":"2024-09-12T12:21:30.232623+0000","last_undegraded":"2024-09-12T12:21:30.232623+0000","last_fullsized":"2024-09-12T12:21:30.232623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:05:19.415105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210545+0000","last_change":"2024-09-12T12:21:30.201827+0000","last_active":"2024-09-12T12:21:30.210545+0000","last_peered":"2024-09-12T12:21:30.210545+0000","last_clean":"2024-09-12T12:21:30.210545+0000","last_became_active":"2024-09-12T12:21:30.200750+0000","last_became_peered":"2024-09-12T12:21:30.200750+0000","last_unstale":"2024-09-12T12:21:30.210545+0000","last_undegraded":"2024-09-12T12:21:30.210545+0000","last_fullsized":"2024-09-12T12:21:30.210545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:18.349581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896201+0000","last_change":"2024-09-12T12:21:29.206800+0000","last_active":"2024-09-12T12:21:31.896201+0000","last_peered":"2024-09-12T12:21:31.896201+0000","last_clean":"2024-09-12T12:21:31.896201+0000","last_became_active":"2024-09-12T12:21:29.206391+0000","last_became_peered":"2024-09-12T12:21:29.206391+0000","last_unstale":"2024-09-12T12:21:31.896201+0000","last_undegraded":"2024-09-12T12:21:31.896201+0000","last_fullsized":"2024-09-12T12:21:31.896201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:44.852326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920202+0000","last_change":"2024-09-12T12:21:30.222358+0000","last_active":"2024-09-12T12:21:31.920202+0000","last_peered":"2024-09-12T12:21:31.920202+0000","last_clean":"2024-09-12T12:21:31.920202+0000","last_became_active":"2024-09-12T12:21:30.221945+0000","last_became_peered":"2024-09-12T12:21:30.221945+0000","last_unstale":"2024-09-12T12:21:31.920202+0000","last_undegraded":"2024-09-12T12:21:31.920202+0000","last_fullsized":"2024-09-12T12:21:31.920202+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:17.898996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214814+0000","last_change":"2024-09-12T12:21:29.199711+0000","last_active":"2024-09-12T12:21:32.214814+0000","last_peered":"2024-09-12T12:21:32.214814+0000","last_clean":"2024-09-12T12:21:32.214814+0000","last_became_active":"2024-09-12T12:21:29.199226+0000","last_became_peered":"2024-09-12T12:21:29.199226+0000","last_unstale":"2024-09-12T12:21:32.214814+0000","last_undegraded":"2024-09-12T12:21:32.214814+0000","last_fullsized":"2024-09-12T12:21:32.214814+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:36:17.189325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210138+0000","last_change":"2024-09-12T12:21:30.206097+0000","last_active":"2024-09-12T12:21:30.210138+0000","last_peered":"2024-09-12T12:21:30.210138+0000","last_clean":"2024-09-12T12:21:30.210138+0000","last_became_active":"2024-09-12T12:21:30.205062+0000","last_became_peered":"2024-09-12T12:21:30.205062+0000","last_unstale":"2024-09-12T12:21:30.210138+0000","last_undegraded":"2024-09-12T12:21:30.210138+0000","last_fullsized":"2024-09-12T12:21:30.210138+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:29.640882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184063+0000","last_change":"2024-09-12T12:21:29.226014+0000","last_active":"2024-09-12T12:21:30.184063+0000","last_peered":"2024-09-12T12:21:30.184063+0000","last_clean":"2024-09-12T12:21:30.184063+0000","last_became_active":"2024-09-12T12:21:29.225416+0000","last_became_peered":"2024-09-12T12:21:29.225416+0000","last_unstale":"2024-09-12T12:21:30.184063+0000","last_undegraded":"2024-09-12T12:21:30.184063+0000","last_fullsized":"2024-09-12T12:21:30.184063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.213949+0000","last_change":"2024-09-12T12:21:30.223400+0000","last_active":"2024-09-12T12:21:32.213949+0000","last_peered":"2024-09-12T12:21:32.213949+0000","last_clean":"2024-09-12T12:21:32.213949+0000","last_became_active":"2024-09-12T12:21:30.223016+0000","last_became_peered":"2024-09-12T12:21:30.223016+0000","last_unstale":"2024-09-12T12:21:32.213949+0000","last_undegraded":"2024-09-12T12:21:32.213949+0000","last_fullsized":"2024-09-12T12:21:32.213949+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:58:18.019451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214187+0000","last_change":"2024-09-12T12:21:29.193758+0000","last_active":"2024-09-12T12:21:32.214187+0000","last_peered":"2024-09-12T12:21:32.214187+0000","last_clean":"2024-09-12T12:21:32.214187+0000","last_became_active":"2024-09-12T12:21:29.193474+0000","last_became_peered":"2024-09-12T12:21:29.193474+0000","last_unstale":"2024-09-12T12:21:32.214187+0000","last_undegraded":"2024-09-12T12:21:32.214187+0000","last_fullsized":"2024-09-12T12:21:32.214187+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:17:33.654013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.221978+0000","last_change":"2024-09-12T12:21:30.218431+0000","last_active":"2024-09-12T12:21:30.221978+0000","last_peered":"2024-09-12T12:21:30.221978+0000","last_clean":"2024-09-12T12:21:30.221978+0000","last_became_active":"2024-09-12T12:21:30.216964+0000","last_became_peered":"2024-09-12T12:21:30.216964+0000","last_unstale":"2024-09-12T12:21:30.221978+0000","last_undegraded":"2024-09-12T12:21:30.221978+0000","last_fullsized":"2024-09-12T12:21:30.221978+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:27.508577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.241841+0000","last_change":"2024-09-12T12:21:29.192499+0000","last_active":"2024-09-12T12:21:30.241841+0000","last_peered":"2024-09-12T12:21:30.241841+0000","last_clean":"2024-09-12T12:21:30.241841+0000","last_became_active":"2024-09-12T12:21:29.192004+0000","last_became_peered":"2024-09-12T12:21:29.192004+0000","last_unstale":"2024-09-12T12:21:30.241841+0000","last_undegraded":"2024-09-12T12:21:30.241841+0000","last_fullsized":"2024-09-12T12:21:30.241841+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:34.754926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896555+0000","last_change":"2024-09-12T12:21:30.205201+0000","last_active":"2024-09-12T12:21:31.896555+0000","last_peered":"2024-09-12T12:21:31.896555+0000","last_clean":"2024-09-12T12:21:31.896555+0000","last_became_active":"2024-09-12T12:21:30.204816+0000","last_became_peered":"2024-09-12T12:21:30.204816+0000","last_unstale":"2024-09-12T12:21:31.896555+0000","last_undegraded":"2024-09-12T12:21:31.896555+0000","last_fullsized":"2024-09-12T12:21:31.896555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:15.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896398+0000","last_change":"2024-09-12T12:21:29.188763+0000","last_active":"2024-09-12T12:21:31.896398+0000","last_peered":"2024-09-12T12:21:31.896398+0000","last_clean":"2024-09-12T12:21:31.896398+0000","last_became_active":"2024-09-12T12:21:29.188359+0000","last_became_peered":"2024-09-12T12:21:29.188359+0000","last_unstale":"2024-09-12T12:21:31.896398+0000","last_undegraded":"2024-09-12T12:21:31.896398+0000","last_fullsized":"2024-09-12T12:21:31.896398+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:01:56.346195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.243691+0000","last_change":"2024-09-12T12:21:30.239463+0000","last_active":"2024-09-12T12:21:30.243691+0000","last_peered":"2024-09-12T12:21:30.243691+0000","last_clean":"2024-09-12T12:21:30.243691+0000","last_became_active":"2024-09-12T12:21:30.239104+0000","last_became_peered":"2024-09-12T12:21:30.239104+0000","last_unstale":"2024-09-12T12:21:30.243691+0000","last_undegraded":"2024-09-12T12:21:30.243691+0000","last_fullsized":"2024-09-12T12:21:30.243691+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:26.091079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.211047+0000","last_change":"2024-09-12T12:21:29.207118+0000","last_active":"2024-09-12T12:21:31.211047+0000","last_peered":"2024-09-12T12:21:31.211047+0000","last_clean":"2024-09-12T12:21:31.211047+0000","last_became_active":"2024-09-12T12:21:29.206326+0000","last_became_peered":"2024-09-12T12:21:29.206326+0000","last_unstale":"2024-09-12T12:21:31.211047+0000","last_undegraded":"2024-09-12T12:21:31.211047+0000","last_fullsized":"2024-09-12T12:21:31.211047+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920288+0000","last_change":"2024-09-12T12:21:30.240598+0000","last_active":"2024-09-12T12:21:31.920288+0000","last_peered":"2024-09-12T12:21:31.920288+0000","last_clean":"2024-09-12T12:21:31.920288+0000","last_became_active":"2024-09-12T12:21:30.240256+0000","last_became_peered":"2024-09-12T12:21:30.240256+0000","last_unstale":"2024-09-12T12:21:31.920288+0000","last_undegraded":"2024-09-12T12:21:31.920288+0000","last_fullsized":"2024-09-12T12:21:31.920288+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:14:34.006877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209227+0000","last_change":"2024-09-12T12:21:29.199199+0000","last_active":"2024-09-12T12:21:31.209227+0000","last_peered":"2024-09-12T12:21:31.209227+0000","last_clean":"2024-09-12T12:21:31.209227+0000","last_became_active":"2024-09-12T12:21:29.198335+0000","last_became_peered":"2024-09-12T12:21:29.198335+0000","last_unstale":"2024-09-12T12:21:31.209227+0000","last_undegraded":"2024-09-12T12:21:31.209227+0000","last_fullsized":"2024-09-12T12:21:31.209227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:35:21.550250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920668+0000","last_change":"2024-09-12T12:21:30.228570+0000","last_active":"2024-09-12T12:21:31.920668+0000","last_peered":"2024-09-12T12:21:31.920668+0000","last_clean":"2024-09-12T12:21:31.920668+0000","last_became_active":"2024-09-12T12:21:30.226661+0000","last_became_peered":"2024-09-12T12:21:30.226661+0000","last_unstale":"2024-09-12T12:21:31.920668+0000","last_undegraded":"2024-09-12T12:21:31.920668+0000","last_fullsized":"2024-09-12T12:21:31.920668+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:22.534750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214612+0000","last_change":"2024-09-12T12:21:29.209464+0000","last_active":"2024-09-12T12:21:32.214612+0000","last_peered":"2024-09-12T12:21:32.214612+0000","last_clean":"2024-09-12T12:21:32.214612+0000","last_became_active":"2024-09-12T12:21:29.209126+0000","last_became_peered":"2024-09-12T12:21:29.209126+0000","last_unstale":"2024-09-12T12:21:32.214612+0000","last_undegraded":"2024-09-12T12:21:32.214612+0000","last_fullsized":"2024-09-12T12:21:32.214612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:08:18.009203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214399+0000","last_change":"2024-09-12T12:21:30.227383+0000","last_active":"2024-09-12T12:21:32.214399+0000","last_peered":"2024-09-12T12:21:32.214399+0000","last_clean":"2024-09-12T12:21:32.214399+0000","last_became_active":"2024-09-12T12:21:30.226337+0000","last_became_peered":"2024-09-12T12:21:30.226337+0000","last_unstale":"2024-09-12T12:21:32.214399+0000","last_undegraded":"2024-09-12T12:21:32.214399+0000","last_fullsized":"2024-09-12T12:21:32.214399+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:07.482331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.215169+0000","last_change":"2024-09-12T12:21:29.212953+0000","last_active":"2024-09-12T12:21:32.215169+0000","last_peered":"2024-09-12T12:21:32.215169+0000","last_clean":"2024-09-12T12:21:32.215169+0000","last_became_active":"2024-09-12T12:21:29.212586+0000","last_became_peered":"2024-09-12T12:21:29.212586+0000","last_unstale":"2024-09-12T12:21:32.215169+0000","last_undegraded":"2024-09-12T12:21:32.215169+0000","last_fullsized":"2024-09-12T12:21:32.215169+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:55.364737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.223948+0000","last_change":"2024-09-12T12:21:30.213940+0000","last_active":"2024-09-12T12:21:30.223948+0000","last_peered":"2024-09-12T12:21:30.223948+0000","last_clean":"2024-09-12T12:21:30.223948+0000","last_became_active":"2024-09-12T12:21:30.204295+0000","last_became_peered":"2024-09-12T12:21:30.204295+0000","last_unstale":"2024-09-12T12:21:30.223948+0000","last_undegraded":"2024-09-12T12:21:30.223948+0000","last_fullsized":"2024-09-12T12:21:30.223948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:28.363357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207755+0000","last_change":"2024-09-12T12:21:29.221505+0000","last_active":"2024-09-12T12:21:31.207755+0000","last_peered":"2024-09-12T12:21:31.207755+0000","last_clean":"2024-09-12T12:21:31.207755+0000","last_became_active":"2024-09-12T12:21:29.220806+0000","last_became_peered":"2024-09-12T12:21:29.220806+0000","last_unstale":"2024-09-12T12:21:31.207755+0000","last_undegraded":"2024-09-12T12:21:31.207755+0000","last_fullsized":"2024-09-12T12:21:31.207755+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.896182+0000","last_change":"2024-09-12T12:21:30.202543+0000","last_active":"2024-09-12T12:21:31.896182+0000","last_peered":"2024-09-12T12:21:31.896182+0000","last_clean":"2024-09-12T12:21:31.896182+0000","last_became_active":"2024-09-12T12:21:30.201969+0000","last_became_peered":"2024-09-12T12:21:30.201969+0000","last_unstale":"2024-09-12T12:21:31.896182+0000","last_undegraded":"2024-09-12T12:21:31.896182+0000","last_fullsized":"2024-09-12T12:21:31.896182+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:17.869098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.193913+0000","last_change":"2024-09-12T12:21:29.221416+0000","last_active":"2024-09-12T12:21:30.193913+0000","last_peered":"2024-09-12T12:21:30.193913+0000","last_clean":"2024-09-12T12:21:30.193913+0000","last_became_active":"2024-09-12T12:21:29.220920+0000","last_became_peered":"2024-09-12T12:21:29.220920+0000","last_unstale":"2024-09-12T12:21:30.193913+0000","last_undegraded":"2024-09-12T12:21:30.193913+0000","last_fullsized":"2024-09-12T12:21:30.193913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:22:16.215986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.230398+0000","last_change":"2024-09-12T12:21:30.228710+0000","last_active":"2024-09-12T12:21:30.230398+0000","last_peered":"2024-09-12T12:21:30.230398+0000","last_clean":"2024-09-12T12:21:30.230398+0000","last_became_active":"2024-09-12T12:21:30.228353+0000","last_became_peered":"2024-09-12T12:21:30.228353+0000","last_unstale":"2024-09-12T12:21:30.230398+0000","last_undegraded":"2024-09-12T12:21:30.230398+0000","last_fullsized":"2024-09-12T12:21:30.230398+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:52.048471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183945+0000","last_change":"2024-09-12T12:21:29.220023+0000","last_active":"2024-09-12T12:21:30.183945+0000","last_peered":"2024-09-12T12:21:30.183945+0000","last_clean":"2024-09-12T12:21:30.183945+0000","last_became_active":"2024-09-12T12:21:29.219318+0000","last_became_peered":"2024-09-12T12:21:29.219318+0000","last_unstale":"2024-09-12T12:21:30.183945+0000","last_undegraded":"2024-09-12T12:21:30.183945+0000","last_fullsized":"2024-09-12T12:21:30.183945+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:49.717098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.920942+0000","last_change":"2024-09-12T12:21:30.226661+0000","last_active":"2024-09-12T12:21:31.920942+0000","last_peered":"2024-09-12T12:21:31.920942+0000","last_clean":"2024-09-12T12:21:31.920942+0000","last_became_active":"2024-09-12T12:21:30.225756+0000","last_became_peered":"2024-09-12T12:21:30.225756+0000","last_unstale":"2024-09-12T12:21:31.920942+0000","last_undegraded":"2024-09-12T12:21:31.920942+0000","last_fullsized":"2024-09-12T12:21:31.920942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214947+0000","last_change":"2024-09-12T12:21:29.206612+0000","last_active":"2024-09-12T12:21:32.214947+0000","last_peered":"2024-09-12T12:21:32.214947+0000","last_clean":"2024-09-12T12:21:32.214947+0000","last_became_active":"2024-09-12T12:21:29.206303+0000","last_became_peered":"2024-09-12T12:21:29.206303+0000","last_unstale":"2024-09-12T12:21:32.214947+0000","last_undegraded":"2024-09-12T12:21:32.214947+0000","last_fullsized":"2024-09-12T12:21:32.214947+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:51.121788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.220107+0000","last_change":"2024-09-12T12:21:30.212519+0000","last_active":"2024-09-12T12:21:30.220107+0000","last_peered":"2024-09-12T12:21:30.220107+0000","last_clean":"2024-09-12T12:21:30.220107+0000","last_became_active":"2024-09-12T12:21:30.211566+0000","last_became_peered":"2024-09-12T12:21:30.211566+0000","last_unstale":"2024-09-12T12:21:30.220107+0000","last_undegraded":"2024-09-12T12:21:30.220107+0000","last_fullsized":"2024-09-12T12:21:30.220107+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:30:37.991300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.186811+0000","last_change":"2024-09-12T12:21:29.227541+0000","last_active":"2024-09-12T12:21:30.186811+0000","last_peered":"2024-09-12T12:21:30.186811+0000","last_clean":"2024-09-12T12:21:30.186811+0000","last_became_active":"2024-09-12T12:21:29.226501+0000","last_became_peered":"2024-09-12T12:21:29.226501+0000","last_unstale":"2024-09-12T12:21:30.186811+0000","last_undegraded":"2024-09-12T12:21:30.186811+0000","last_fullsized":"2024-09-12T12:21:30.186811+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:21.714121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210598+0000","last_change":"2024-09-12T12:21:30.204222+0000","last_active":"2024-09-12T12:21:30.210598+0000","last_peered":"2024-09-12T12:21:30.210598+0000","last_clean":"2024-09-12T12:21:30.210598+0000","last_became_active":"2024-09-12T12:21:30.203608+0000","last_became_peered":"2024-09-12T12:21:30.203608+0000","last_unstale":"2024-09-12T12:21:30.210598+0000","last_undegraded":"2024-09-12T12:21:30.210598+0000","last_fullsized":"2024-09-12T12:21:30.210598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181466+0000","last_change":"2024-09-12T12:21:29.188153+0000","last_active":"2024-09-12T12:21:30.181466+0000","last_peered":"2024-09-12T12:21:30.181466+0000","last_clean":"2024-09-12T12:21:30.181466+0000","last_became_active":"2024-09-12T12:21:29.187754+0000","last_became_peered":"2024-09-12T12:21:29.187754+0000","last_unstale":"2024-09-12T12:21:30.181466+0000","last_undegraded":"2024-09-12T12:21:30.181466+0000","last_fullsized":"2024-09-12T12:21:30.181466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:15:39.439125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214578+0000","last_change":"2024-09-12T12:21:30.227067+0000","last_active":"2024-09-12T12:21:32.214578+0000","last_peered":"2024-09-12T12:21:32.214578+0000","last_clean":"2024-09-12T12:21:32.214578+0000","last_became_active":"2024-09-12T12:21:30.225956+0000","last_became_peered":"2024-09-12T12:21:30.225956+0000","last_unstale":"2024-09-12T12:21:32.214578+0000","last_undegraded":"2024-09-12T12:21:32.214578+0000","last_fullsized":"2024-09-12T12:21:32.214578+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:50:52.370860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.184473+0000","last_change":"2024-09-12T12:21:29.227844+0000","last_active":"2024-09-12T12:21:30.184473+0000","last_peered":"2024-09-12T12:21:30.184473+0000","last_clean":"2024-09-12T12:21:30.184473+0000","last_became_active":"2024-09-12T12:21:29.227189+0000","last_became_peered":"2024-09-12T12:21:29.227189+0000","last_unstale":"2024-09-12T12:21:30.184473+0000","last_undegraded":"2024-09-12T12:21:30.184473+0000","last_fullsized":"2024-09-12T12:21:30.184473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:29.888392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:32.214592+0000","last_change":"2024-09-12T12:21:30.224043+0000","last_active":"2024-09-12T12:21:32.214592+0000","last_peered":"2024-09-12T12:21:32.214592+0000","last_clean":"2024-09-12T12:21:32.214592+0000","last_became_active":"2024-09-12T12:21:30.222387+0000","last_became_peered":"2024-09-12T12:21:30.222387+0000","last_unstale":"2024-09-12T12:21:32.214592+0000","last_undegraded":"2024-09-12T12:21:32.214592+0000","last_fullsized":"2024-09-12T12:21:32.214592+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:11:28.175498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209825+0000","last_change":"2024-09-12T12:21:29.203168+0000","last_active":"2024-09-12T12:21:31.209825+0000","last_peered":"2024-09-12T12:21:31.209825+0000","last_clean":"2024-09-12T12:21:31.209825+0000","last_became_active":"2024-09-12T12:21:29.201201+0000","last_became_peered":"2024-09-12T12:21:29.201201+0000","last_unstale":"2024-09-12T12:21:31.209825+0000","last_undegraded":"2024-09-12T12:21:31.209825+0000","last_fullsized":"2024-09-12T12:21:31.209825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:11:14.394911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.207540+0000","last_change":"2024-09-12T12:21:30.221515+0000","last_active":"2024-09-12T12:21:31.207540+0000","last_peered":"2024-09-12T12:21:31.207540+0000","last_clean":"2024-09-12T12:21:31.207540+0000","last_became_active":"2024-09-12T12:21:30.221202+0000","last_became_peered":"2024-09-12T12:21:30.221202+0000","last_unstale":"2024-09-12T12:21:31.207540+0000","last_undegraded":"2024-09-12T12:21:31.207540+0000","last_fullsized":"2024-09-12T12:21:31.207540+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:59:38.803306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.181458+0000","last_change":"2024-09-12T12:21:29.215337+0000","last_active":"2024-09-12T12:21:30.181458+0000","last_peered":"2024-09-12T12:21:30.181458+0000","last_clean":"2024-09-12T12:21:30.181458+0000","last_became_active":"2024-09-12T12:21:29.214887+0000","last_became_peered":"2024-09-12T12:21:29.214887+0000","last_unstale":"2024-09-12T12:21:30.181458+0000","last_undegraded":"2024-09-12T12:21:30.181458+0000","last_fullsized":"2024-09-12T12:21:30.181458+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:36.683929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.208962+0000","last_change":"2024-09-12T12:21:30.228674+0000","last_active":"2024-09-12T12:21:31.208962+0000","last_peered":"2024-09-12T12:21:31.208962+0000","last_clean":"2024-09-12T12:21:31.208962+0000","last_became_active":"2024-09-12T12:21:30.228376+0000","last_became_peered":"2024-09-12T12:21:30.228376+0000","last_unstale":"2024-09-12T12:21:31.208962+0000","last_undegraded":"2024-09-12T12:21:31.208962+0000","last_fullsized":"2024-09-12T12:21:31.208962+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:40:50.727010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.191879+0000","last_change":"2024-09-12T12:21:29.193465+0000","last_active":"2024-09-12T12:21:30.191879+0000","last_peered":"2024-09-12T12:21:30.191879+0000","last_clean":"2024-09-12T12:21:30.191879+0000","last_became_active":"2024-09-12T12:21:29.193077+0000","last_became_peered":"2024-09-12T12:21:29.193077+0000","last_unstale":"2024-09-12T12:21:30.191879+0000","last_undegraded":"2024-09-12T12:21:30.191879+0000","last_fullsized":"2024-09-12T12:21:30.191879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:13.069402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.210273+0000","last_change":"2024-09-12T12:21:30.206965+0000","last_active":"2024-09-12T12:21:30.210273+0000","last_peered":"2024-09-12T12:21:30.210273+0000","last_clean":"2024-09-12T12:21:30.210273+0000","last_became_active":"2024-09-12T12:21:30.204731+0000","last_became_peered":"2024-09-12T12:21:30.204731+0000","last_unstale":"2024-09-12T12:21:30.210273+0000","last_undegraded":"2024-09-12T12:21:30.210273+0000","last_fullsized":"2024-09-12T12:21:30.210273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:29.163827+0000","last_clean_scrub_stamp":"2024-09-12T12:21:29.163827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:49:55.697639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-12T12:21:31.209341+0000","last_change":"2024-09-12T12:21:29.203115+0000","last_active":"2024-09-12T12:21:31.209341+0000","last_peered":"2024-09-12T12:21:31.209341+0000","last_clean":"2024-09-12T12:21:31.209341+0000","last_became_active":"2024-09-12T12:21:29.202388+0000","last_became_peered":"2024-09-12T12:21:29.202388+0000","last_unstale":"2024-09-12T12:21:31.209341+0000","last_undegraded":"2024-09-12T12:21:31.209341+0000","last_fullsized":"2024-09-12T12:21:31.209341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:21:28.153915+0000","last_clean_scrub_stamp":"2024-09-12T12:21:28.153915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.744017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-12T12:21:30.183862+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:21:30.183862+0000","last_peered":"2024-09-12T12:21:30.183862+0000","last_clean":"2024-09-12T12:21:30.183862+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:21:30.183862+0000","last_undegraded":"2024-09-12T12:21:30.183862+0000","last_fullsized":"2024-09-12T12:21:30.183862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":200905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1085440,"data_stored":784425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":495616,"data_stored":194639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":200337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":512000,"data_stored":201879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":201901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":194037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":784369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:21:39.155 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:21:39.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T12:21:39.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.321+0000 7fb028f35640 1 Processor -- start 2024-09-12T12:21:39.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.321+0000 7fb028f35640 1 -- start start 2024-09-12T12:21:39.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb028f35640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb0240d05a0 0x7fb02406d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb028f35640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0240d0980 con 0x7fb0240d2920 2024-09-12T12:21:39.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb028f35640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02415fa90 con 0x7fb0240720d0 2024-09-12T12:21:39.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb028f35640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb02406eb90 con 0x7fb0240d05a0 2024-09-12T12:21:39.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb022d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb0240d2920 0x7fb024162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35048/0 (socket says 172.21.15.3:35048) 2024-09-12T12:21:39.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb022d9d640 1 -- 172.21.15.3:0/3426152829 learned_addr learned my addr 172.21.15.3:0/3426152829 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:39.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 --2- 172.21.15.3:0/3426152829 >> v2:172.21.15.142:3300/0 conn(0x7fb0240d05a0 0x7fb02406d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374201251 0 0) 0x7fb0240d0980 con 0x7fb0240d2920 2024-09-12T12:21:39.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb010003530 con 0x7fb0240d2920 2024-09-12T12:21:39.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3817785356 0 0) 0x7fb010003530 con 0x7fb0240d2920 2024-09-12T12:21:39.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 >> v2:172.21.15.142:3300/0 conn(0x7fb0240d05a0 msgr2=0x7fb02406d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:39.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 --2- 172.21.15.3:0/3426152829 >> v2:172.21.15.142:3300/0 conn(0x7fb0240d05a0 0x7fb02406d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 >> v1:172.21.15.142:6789/0 conn(0x7fb0240720d0 legacy=0x7fb0240d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02415ee80 con 0x7fb0240d2920 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fb01c001a20 con 0x7fb0240d2920 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 >> v1:172.21.15.3:6789/0 conn(0x7fb0240d2920 legacy=0x7fb024162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0100039c0 0x7fb010003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb010004810 0x7fb010004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100051d0 0x7fb01000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb02406eb90 con 0x7fb0100039c0 2024-09-12T12:21:39.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb01000c170 con 0x7fb010004810 2024-09-12T12:21:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb01000c370 con 0x7fb0100051d0 2024-09-12T12:21:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0100039c0 0x7fb010003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb01bfff640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb010004810 0x7fb010004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb022d9d640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100051d0 0x7fb01000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 -- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100051d0 msgr2=0x7fb01000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100051d0 0x7fb01000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 -- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb010004810 msgr2=0x7fb010004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:39.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb010004810 0x7fb010004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 -- 172.21.15.3:0/3426152829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb010003530 con 0x7fb0100039c0 2024-09-12T12:21:39.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb022d9d640 1 -- 172.21.15.3:0/3426152829 reap_dead start 2024-09-12T12:21:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb02259c640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0100039c0 0x7fb010003de0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb014009ac0 tx=0x7fb01401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa8150177569666 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014009690 con 0x7fb0100039c0 2024-09-12T12:21:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0140a8460 con 0x7fb0100039c0 2024-09-12T12:21:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.325+0000 7fb021d9b640 1 -- 172.21.15.3:0/3426152829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140a8830 con 0x7fb0100039c0 2024-09-12T12:21:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- 172.21.15.3:0/3426152829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0100039c0 msgr2=0x7fb010003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0100039c0 0x7fb010003de0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb014009ac0 tx=0x7fb01401ca50 comp rx=0 tx=0).stop 2024-09-12T12:21:39.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- 172.21.15.3:0/3426152829 shutdown_connections 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 --2- 172.21.15.3:0/3426152829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0100039c0 0x7fb010003de0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- 172.21.15.3:0/3426152829 >> 172.21.15.3:0/3426152829 conn(0x7fb024070060 msgr2=0x7fb024070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- 172.21.15.3:0/3426152829 shutdown_connections 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- 172.21.15.3:0/3426152829 wait complete. 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 Processor -- start 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- start start 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100039c0 0x7fb0241508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02415de50 0x7fb0240ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02415e2d0 0x7fb0240cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb02415eac0 con 0x7fb02415e2d0 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb02415f720 con 0x7fb02415de50 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb02415ec40 con 0x7fb0100039c0 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02415e2d0 0x7fb0240cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02415e2d0 0x7fb0240cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60678/0 (socket says 172.21.15.3:60678) 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 -- 172.21.15.3:0/2054567039 learned_addr learned my addr 172.21.15.3:0/2054567039 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb02259c640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100039c0 0x7fb0241508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb01bfff640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02415de50 0x7fb0240ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 -- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100039c0 msgr2=0x7fb0241508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100039c0 0x7fb0241508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 -- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02415de50 msgr2=0x7fb0240ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02415de50 0x7fb0240ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 -- 172.21.15.3:0/2054567039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0240c96c0 con 0x7fb02415e2d0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb02259c640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100039c0 0x7fb0241508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb022d9d640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02415e2d0 0x7fb0240cad80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb01c001380 tx=0x7fb01c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb01affd640 1 -- 172.21.15.3:0/2054567039 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c004b70 con 0x7fb02415e2d0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb01affd640 1 -- 172.21.15.3:0/2054567039 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb01c004d10 con 0x7fb02415e2d0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0240c99a0 con 0x7fb02415e2d0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb01affd640 1 -- 172.21.15.3:0/2054567039 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c00edf0 con 0x7fb02415e2d0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.329+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0240c9f60 con 0x7fb02415e2d0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.333+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb024162b40 con 0x7fb02415e2d0 2024-09-12T12:21:39.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.333+0000 7fb01affd640 1 -- 172.21.15.3:0/2054567039 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb01c01b020 con 0x7fb02415e2d0 2024-09-12T12:21:39.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.333+0000 7fb01affd640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0040678f0 0x7fb004069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.333+0000 7fb01affd640 1 -- 172.21.15.3:0/2054567039 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb01c085590 con 0x7fb02415e2d0 2024-09-12T12:21:39.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.333+0000 7fb02259c640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0040678f0 0x7fb004069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.333+0000 7fb02259c640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0040678f0 0x7fb004069db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0140af000 tx=0x7fb0140046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:39.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.337+0000 7fb01affd640 1 -- 172.21.15.3:0/2054567039 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb01c0560b0 con 0x7fb02415e2d0 2024-09-12T12:21:39.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.561+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb0240b8200 con 0x7fb02415e2d0 2024-09-12T12:21:39.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.565+0000 7fb01affd640 1 -- 172.21.15.3:0/2054567039 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fb01c0559b0 con 0x7fb02415e2d0 2024-09-12T12:21:39.568 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T12:21:39.568 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:21:39.569 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-12T12:21:39.569 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-12T12:21:39.569 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T12:21:39.569 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-12T12:21:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0040678f0 msgr2=0x7fb004069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0040678f0 0x7fb004069db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0140af000 tx=0x7fb0140046d0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02415e2d0 msgr2=0x7fb0240cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02415e2d0 0x7fb0240cad80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb01c001380 tx=0x7fb01c00e530 comp rx=0 tx=0).stop 2024-09-12T12:21:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 shutdown_connections 2024-09-12T12:21:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0040678f0 0x7fb004069db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02415e2d0 0x7fb0240cad80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb02415de50 0x7fb0240ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 --2- 172.21.15.3:0/2054567039 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0100039c0 0x7fb0241508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 >> 172.21.15.3:0/2054567039 conn(0x7fb024070060 msgr2=0x7fb0240cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 shutdown_connections 2024-09-12T12:21:39.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.569+0000 7fb028f35640 1 -- 172.21.15.3:0/2054567039 wait complete. 2024-09-12T12:21:39.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:39.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.693+0000 7f1d4ad6b640 1 Processor -- start 2024-09-12T12:21:39.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.693+0000 7f1d4ad6b640 1 -- start start 2024-09-12T12:21:39.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d4ad6b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d4415e4b0 0x7f1d440ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d4ad6b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d440724b0 0x7f1d4406eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d4ad6b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440ad5a0 con 0x7f1d440b42b0 2024-09-12T12:21:39.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d4ad6b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1d4406e810 con 0x7f1d440724b0 2024-09-12T12:21:39.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d4ad6b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1d4406dee0 con 0x7f1d4415e4b0 2024-09-12T12:21:39.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d440724b0 0x7f1d4406eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1d440724b0 0x7f1d4406eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60692/0 (socket says 172.21.15.3:60692) 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 -- 172.21.15.3:0/1786697040 learned_addr learned my addr 172.21.15.3:0/1786697040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d43fff640 1 --1- 172.21.15.3:0/1786697040 >> v1:172.21.15.3:6789/0 conn(0x7f1d440b42b0 0x7f1d44071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35062/0 (socket says 172.21.15.3:35062) 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d48b07640 1 --2- 172.21.15.3:0/1786697040 >> v2:172.21.15.142:3300/0 conn(0x7f1d4415e4b0 0x7f1d440ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 -- 172.21.15.3:0/1786697040 >> v2:172.21.15.142:3300/0 conn(0x7f1d4415e4b0 msgr2=0x7f1d440ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 --2- 172.21.15.3:0/1786697040 >> v2:172.21.15.142:3300/0 conn(0x7f1d4415e4b0 0x7f1d440ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 -- 172.21.15.3:0/1786697040 >> v1:172.21.15.3:6789/0 conn(0x7f1d440b42b0 legacy=0x7f1d44071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d437fe640 1 -- 172.21.15.3:0/1786697040 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196631884 0 0) 0x7f1d440ad5a0 con 0x7f1d440b42b0 2024-09-12T12:21:39.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 -- 172.21.15.3:0/1786697040 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4406e1c0 con 0x7f1d440724b0 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d49308640 1 --2- 172.21.15.3:0/1786697040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d4406eca0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d44071350 tx=0x7f1d4406f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e89daa97ebfab20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d437fe640 1 -- 172.21.15.3:0/1786697040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340b2070 con 0x7f1d440724b0 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d437fe640 1 -- 172.21.15.3:0/1786697040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d340a9070 con 0x7f1d440724b0 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d437fe640 1 -- 172.21.15.3:0/1786697040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d340ad880 con 0x7f1d440724b0 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/1786697040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 msgr2=0x7f1d4406eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.697+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/1786697040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d4406eca0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d44071350 tx=0x7f1d4406f9f0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/1786697040 shutdown_connections 2024-09-12T12:21:39.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/1786697040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d4406eca0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/1786697040 >> v2:172.21.15.142:3300/0 conn(0x7f1d4415e4b0 0x7f1d440ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/1786697040 >> 172.21.15.3:0/1786697040 conn(0x7f1d440b1d80 msgr2=0x7f1d440b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:39.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/1786697040 shutdown_connections 2024-09-12T12:21:39.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/1786697040 wait complete. 2024-09-12T12:21:39.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 Processor -- start 2024-09-12T12:21:39.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- start start 2024-09-12T12:21:39.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d441513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d4415e4b0 0x7f1d44151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d44154c90 0x7f1d4414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4416b1f0 con 0x7f1d440724b0 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440b3390 con 0x7f1d4415e4b0 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4416b370 con 0x7f1d44154c90 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d441513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d441513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60698/0 (socket says 172.21.15.3:60698) 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 -- 172.21.15.3:0/2138115602 learned_addr learned my addr 172.21.15.3:0/2138115602 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 -- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d44154c90 msgr2=0x7f1d4414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d49308640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d44154c90 0x7f1d4414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d44154c90 0x7f1d4414dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 -- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d4415e4b0 msgr2=0x7f1d44151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d4415e4b0 0x7f1d44151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 -- 172.21.15.3:0/2138115602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d44152240 con 0x7f1d440724b0 2024-09-12T12:21:39.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d48b07640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d441513f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c007e40 tx=0x7f1d3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:39.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d417fa640 1 -- 172.21.15.3:0/2138115602 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c014620 con 0x7f1d440724b0 2024-09-12T12:21:39.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d417fa640 1 -- 172.21.15.3:0/2138115602 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3c019070 con 0x7f1d440724b0 2024-09-12T12:21:39.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441519d0 con 0x7f1d440724b0 2024-09-12T12:21:39.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d417fa640 1 -- 172.21.15.3:0/2138115602 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c014a00 con 0x7f1d440724b0 2024-09-12T12:21:39.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d44155ba0 con 0x7f1d440724b0 2024-09-12T12:21:39.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.701+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4406eca0 con 0x7f1d440724b0 2024-09-12T12:21:39.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.705+0000 7f1d417fa640 1 -- 172.21.15.3:0/2138115602 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1d3c014ba0 con 0x7f1d440724b0 2024-09-12T12:21:39.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.705+0000 7f1d417fa640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1d240678c0 0x7f1d24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:39.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.705+0000 7f1d417fa640 1 -- 172.21.15.3:0/2138115602 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f1d3c086570 con 0x7f1d440724b0 2024-09-12T12:21:39.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.705+0000 7f1d43fff640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1d240678c0 0x7f1d24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:39.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.705+0000 7f1d43fff640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1d240678c0 0x7f1d24069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c000c10 tx=0x7f1d2c007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:39.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.709+0000 7f1d417fa640 1 -- 172.21.15.3:0/2138115602 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1d3c0570b0 con 0x7f1d440724b0 2024-09-12T12:21:39.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.897+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d4409e3d0 con 0x7f1d440724b0 2024-09-12T12:21:39.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d417fa640 1 -- 172.21.15.3:0/2138115602 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5220 (secure 0 0 0) 0x7f1d3c0569b0 con 0x7f1d440724b0 2024-09-12T12:21:39.903 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:39.903 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"btime":"2024-09-12T12:21:33:226221+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5256,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:32.604624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5262},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5262]},"id":3}]} 2024-09-12T12:21:39.903 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-09-12T12:21:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1d240678c0 msgr2=0x7f1d24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1d240678c0 0x7f1d24069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c000c10 tx=0x7f1d2c007910 comp rx=0 tx=0).stop 2024-09-12T12:21:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 msgr2=0x7f1d441513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:39.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d441513f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c007e40 tx=0x7f1d3c00e4b0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 shutdown_connections 2024-09-12T12:21:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1d240678c0 0x7f1d24069d80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d44154c90 0x7f1d4414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d4415e4b0 0x7f1d44151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 --2- 172.21.15.3:0/2138115602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d440724b0 0x7f1d441513f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 >> 172.21.15.3:0/2138115602 conn(0x7f1d440b1d80 msgr2=0x7f1d440ae190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:39.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 shutdown_connections 2024-09-12T12:21:39.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:39.901+0000 7f1d4ad6b640 1 -- 172.21.15.3:0/2138115602 wait complete. 2024-09-12T12:21:40.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:21:29.563260+0000', 'modified': '2024-09-12T12:21:32.604624+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5262}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5262': {'gid': 5262, 'name': 'd', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6837/2796023141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6835', 'nonce': 2796023141}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2796023141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5262, 'qdb_cluster': [5262]} 2024-09-12T12:21:40.324 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:21:40.324 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-12T12:21:40.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d09768640 1 Processor -- start 2024-09-12T12:21:40.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d09768640 1 -- start start 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d09768640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8d040d0a90 0x7f8d04072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d09768640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d0406f010 con 0x7f8d040af550 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d09768640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d0406f2f0 con 0x7f8d04073320 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d09768640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8d04073ad0 con 0x7f8d040d0a90 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d02ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8d040af550 0x7f8d040d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35072/0 (socket says 172.21.15.3:35072) 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d02ffd640 1 -- 172.21.15.3:0/637293125 learned_addr learned my addr 172.21.15.3:0/637293125 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d027fc640 1 --2- 172.21.15.3:0/637293125 >> v2:172.21.15.3:3300/0 conn(0x7f8d040d0a90 0x7f8d04072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d01ffb640 1 -- 172.21.15.3:0/637293125 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348695103 0 0) 0x7f8d0406f010 con 0x7f8d040af550 2024-09-12T12:21:40.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d01ffb640 1 -- 172.21.15.3:0/637293125 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cec003530 con 0x7f8d040af550 2024-09-12T12:21:40.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d027fc640 1 -- 172.21.15.3:0/637293125 >> v1:172.21.15.142:6789/0 conn(0x7f8d04073320 legacy=0x7f8d0415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d027fc640 1 -- 172.21.15.3:0/637293125 >> v1:172.21.15.3:6789/0 conn(0x7f8d040af550 legacy=0x7f8d040d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.457+0000 7f8d027fc640 1 -- 172.21.15.3:0/637293125 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0406f5d0 con 0x7f8d040d0a90 2024-09-12T12:21:40.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d027fc640 1 --2- 172.21.15.3:0/637293125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d040d0a90 0x7f8d04072de0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40097c0 tx=0x7f8cf401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8db9f3451f7b414 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d01ffb640 1 -- 172.21.15.3:0/637293125 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40ab600 con 0x7f8d040d0a90 2024-09-12T12:21:40.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d01ffb640 1 -- 172.21.15.3:0/637293125 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf40b0070 con 0x7f8d040d0a90 2024-09-12T12:21:40.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d01ffb640 1 -- 172.21.15.3:0/637293125 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40ab9c0 con 0x7f8d040d0a90 2024-09-12T12:21:40.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 -- 172.21.15.3:0/637293125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d040d0a90 msgr2=0x7f8d04072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 --2- 172.21.15.3:0/637293125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d040d0a90 0x7f8d04072de0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40097c0 tx=0x7f8cf401c3e0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 -- 172.21.15.3:0/637293125 shutdown_connections 2024-09-12T12:21:40.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 --2- 172.21.15.3:0/637293125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d040d0a90 0x7f8d04072de0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 -- 172.21.15.3:0/637293125 >> 172.21.15.3:0/637293125 conn(0x7f8d0406be90 msgr2=0x7f8d0406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 -- 172.21.15.3:0/637293125 shutdown_connections 2024-09-12T12:21:40.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 -- 172.21.15.3:0/637293125 wait complete. 2024-09-12T12:21:40.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 Processor -- start 2024-09-12T12:21:40.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.461+0000 7f8d09768640 1 -- start start 2024-09-12T12:21:40.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d04073320 0x7f8d0414af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d040af550 0x7f8d040afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d040d0a90 0x7f8d040b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0415eda0 con 0x7f8d04073320 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0415ec20 con 0x7f8d040d0a90 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0406dd40 con 0x7f8d040af550 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d04073320 0x7f8d0414af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d04073320 0x7f8d0414af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60726/0 (socket says 172.21.15.3:60726) 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 -- 172.21.15.3:0/2727901182 learned_addr learned my addr 172.21.15.3:0/2727901182 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d037fe640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d040d0a90 0x7f8d040b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d027fc640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d040af550 0x7f8d040afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 -- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d040af550 msgr2=0x7f8d040afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d040af550 0x7f8d040afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 -- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d040d0a90 msgr2=0x7f8d040b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d040d0a90 0x7f8d040b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 -- 172.21.15.3:0/2727901182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d04159d30 con 0x7f8d04073320 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d037fe640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d040d0a90 0x7f8d040b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d027fc640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d040af550 0x7f8d040afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d04073320 0x7f8d0414af10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c4d0 tx=0x7f8cf800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2727901182 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8017070 con 0x7f8d04073320 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2727901182 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf8005cf0 con 0x7f8d04073320 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2727901182 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8007270 con 0x7f8d04073320 2024-09-12T12:21:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0415a010 con 0x7f8d04073320 2024-09-12T12:21:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0414b450 con 0x7f8d04073320 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2727901182 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8cf801a070 con 0x7f8d04073320 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8ceb7fe640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cf00678c0 0x7f8cf0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8ceb7fe640 1 -- 172.21.15.3:0/2727901182 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8cf8085090 con 0x7f8d04073320 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.465+0000 7f8d027fc640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cf00678c0 0x7f8cf0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 Processor -- start 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- start start 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cf00678c0 0x7f8cf0069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40af000 tx=0x7f8cf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498dfb20 0x5600498dff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x560049895d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560049896290 0x560049896670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56004977db60 con 0x5600498e0440 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5600496e4c70 con 0x560049896290 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x56004968aed0 con 0x5600498dfb20 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x560049895d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x560049895d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60736/0 (socket says 172.21.15.3:60736) 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 -- 172.21.15.3:0/2302380171 learned_addr learned my addr 172.21.15.3:0/2302380171 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d037fe640 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560049896290 0x560049896670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498dfb20 0x5600498dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 -- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498dfb20 msgr2=0x5600498dff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498dfb20 0x5600498dff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 -- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560049896290 msgr2=0x560049896670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560049896290 0x560049896670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 -- 172.21.15.3:0/2302380171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56004969ce50 con 0x5600498e0440 2024-09-12T12:21:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d027fc640 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x560049895d50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000c10 tx=0x7f8cf4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0527c9b2da9e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/2302380171 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40c7070 con 0x5600498e0440 2024-09-12T12:21:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/2302380171 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf40b0040 con 0x5600498e0440 2024-09-12T12:21:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8ce8ff9640 1 -- 172.21.15.3:0/2302380171 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40b5c50 con 0x5600498e0440 2024-09-12T12:21:40.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- 172.21.15.3:0/2302380171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 msgr2=0x560049895d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x560049895d50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4000c10 tx=0x7f8cf4004740 comp rx=0 tx=0).stop 2024-09-12T12:21:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- 172.21.15.3:0/2302380171 shutdown_connections 2024-09-12T12:21:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560049896290 0x560049896670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x560049895d50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 --2- 172.21.15.3:0/2302380171 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498dfb20 0x5600498dff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- 172.21.15.3:0/2302380171 >> 172.21.15.3:0/2302380171 conn(0x56004989c600 msgr2=0x560049894630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- 172.21.15.3:0/2302380171 shutdown_connections 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- 172.21.15.3:0/2302380171 wait complete. 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 -- start start 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.469+0000 7f8d0c824000 1 Processor -- start 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 0x56004989adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498e0440 0x5600498a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56004989b330 0x5600498a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560049746b00 con 0x56004989b330 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5600496bf360 con 0x5600498dfb20 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5600496bc2d0 con 0x5600498e0440 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 10 client.0 _ll_register_callbacks cb 0x560049881820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:40.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56004989b330 0x5600498a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56004989b330 0x5600498a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60740/0 (socket says 172.21.15.3:60740) 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 -- 172.21.15.3:0/1411100202 learned_addr learned my addr 172.21.15.3:0/1411100202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d027fc640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498e0440 0x5600498a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 0x56004989adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 -- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498e0440 msgr2=0x5600498a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498e0440 0x5600498a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 -- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 msgr2=0x56004989adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 0x56004989adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 -- 172.21.15.3:0/1411100202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56004968b120 con 0x56004989b330 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d027fc640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498e0440 0x5600498a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d037fe640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56004989b330 0x5600498a1230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc007dc0 tx=0x7f8cfc00f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc66e83d1e9968b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1411100202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc017020 con 0x56004989b330 2024-09-12T12:21:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1411100202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8cfc003b90 con 0x56004989b330 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf40b0040 con 0x56004989b330 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1411100202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8cfc021050 con 0x56004989b330 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 10 client.5406 fetch_fsmap learned FSMap version 23 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x560049684450 con 0x56004989b330 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1411100202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf40b0040 con 0x56004989b330 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1411100202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f8cfc024020 con 0x56004989b330 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 10 client.5406 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 10 client.5406 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 20 client.5406 populate_metadata read hostname 'smithi003' 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498ee280 0x5600498f0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 4 client.5406 mds_command: new command op to 5262 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 --> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56004967ec00 con 0x5600498ee280 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498ee280 0x5600498f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498ee280 0x5600498f0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0e74cb17f50474 server_cookie=a9a473fa30e52f77 in_seq=0 out_seq=0 2024-09-12T12:21:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.473+0000 7f8ce1ffb640 10 client.5406 ms_handle_connect on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1411100202 <== mds.0 v2:172.21.15.142:6835/2796023141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56004967ec00 con 0x5600498ee280 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8ce1ffb640 10 client.5406 handle_command_reply: tid=0 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 client.5406 shutdown 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56004989b330 msgr2=0x5600498a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56004989b330 0x5600498a1230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc007dc0 tx=0x7f8cfc00f980 comp rx=0 tx=0).stop 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 shutdown_connections 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498ee280 0x5600498f0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56004989b330 0x5600498a1230 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8ce1ffb640 0 client.5406 ms_handle_reset on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5600498e0440 0x5600498a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1411100202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 0x56004989adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 >> 172.21.15.3:0/1411100202 conn(0x560049896290 msgr2=0x560049894d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 shutdown_connections 2024-09-12T12:21:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 1 -- 172.21.15.3:0/1411100202 wait complete. 2024-09-12T12:21:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.477+0000 7f8d0c824000 20 client.5406 trim_cache size 0 max 16384 2024-09-12T12:21:40.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.493+0000 7f8d0c824000 1 Processor -- start 2024-09-12T12:21:40.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.493+0000 7f8d0c824000 1 -- start start 2024-09-12T12:21:40.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 0x5600498ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 0x5600498de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x5600498de700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d042004a0 con 0x5600498e0440 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041ff880 con 0x5600498dfb20 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8d041ffe50 con 0x56004989b330 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x5600498de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x5600498de700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60744/0 (socket says 172.21.15.3:60744) 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 -- 172.21.15.3:0/4021276147 learned_addr learned my addr 172.21.15.3:0/4021276147 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 -- 172.21.15.3:0/4021276147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 msgr2=0x5600498ddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 --2- 172.21.15.3:0/4021276147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 0x5600498ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 -- 172.21.15.3:0/4021276147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 msgr2=0x5600498de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 --2- 172.21.15.3:0/4021276147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 0x5600498de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 -- 172.21.15.3:0/4021276147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56004966bfd0 con 0x5600498e0440 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 --2- 172.21.15.3:0/4021276147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x5600498de700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005260 tx=0x7f8cfc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc28154166ac3e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8ce3fff640 1 -- 172.21.15.3:0/4021276147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc017030 con 0x5600498e0440 2024-09-12T12:21:40.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8ce3fff640 1 -- 172.21.15.3:0/4021276147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc0052e0 con 0x5600498e0440 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8ce3fff640 1 -- 172.21.15.3:0/4021276147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0056d0 con 0x5600498e0440 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- 172.21.15.3:0/4021276147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 msgr2=0x5600498de700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- 172.21.15.3:0/4021276147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x5600498de700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005260 tx=0x7f8cfc013000 comp rx=0 tx=0).stop 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- 172.21.15.3:0/4021276147 shutdown_connections 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- 172.21.15.3:0/4021276147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498e0440 0x5600498de700 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- 172.21.15.3:0/4021276147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498dfb20 0x5600498de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- 172.21.15.3:0/4021276147 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 0x5600498ddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- 172.21.15.3:0/4021276147 >> 172.21.15.3:0/4021276147 conn(0x560049896290 msgr2=0x56004989a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- 172.21.15.3:0/4021276147 shutdown_connections 2024-09-12T12:21:40.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- 172.21.15.3:0/4021276147 wait complete. 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- start start 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 Processor -- start 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 0x56004995b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498dfb20 0x56004995bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498e0440 0x5600498f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5600496e3f80 con 0x5600498dfb20 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56004969cca0 con 0x5600498e0440 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5600496e92a0 con 0x56004989b330 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 10 client.0 _ll_register_callbacks cb 0x7f8cfc001670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498dfb20 0x56004995bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498dfb20 0x56004995bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60752/0 (socket says 172.21.15.3:60752) 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 -- 172.21.15.3:0/1287312615 learned_addr learned my addr 172.21.15.3:0/1287312615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d037fe640 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498e0440 0x5600498f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 -- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 msgr2=0x56004995b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 0x56004995b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 -- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498e0440 msgr2=0x5600498f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498e0440 0x5600498f1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 -- 172.21.15.3:0/1287312615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560049767750 con 0x5600498dfb20 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d027fc640 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498dfb20 0x56004995bf30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4008010 tx=0x7f8cf4019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27628fa3d1fedb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8ce1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1287312615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40c7040 con 0x5600498dfb20 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1287312615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8cf40b5850 con 0x5600498dfb20 2024-09-12T12:21:40.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.497+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cfc0052e0 con 0x5600498dfb20 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1287312615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8cf40d7050 con 0x5600498dfb20 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 10 client.5412 fetch_fsmap learned FSMap version 23 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x560049672a30 con 0x5600498dfb20 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1287312615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cfc0052e0 con 0x5600498dfb20 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1287312615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f8cf40da020 con 0x5600498dfb20 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 10 client.5412 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 10 client.5412 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 20 client.5412 populate_metadata read hostname 'smithi003' 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498f1850 0x5600498f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 4 client.5412 mds_command: new command op to 5262 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:21:40.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 --> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] -- command(tid 0: {"prefix": "status"}) -- 0x5600496f23a0 con 0x5600498f1850 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498f1850 0x5600498f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8ce1ffb640 10 client.5412 ms_handle_connect on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d02ffd640 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498f1850 0x5600498f1c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdafcc464a96678 server_cookie=b515bf7a7947cfd9 in_seq=0 out_seq=0 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8ce1ffb640 1 -- 172.21.15.3:0/1287312615 <== mds.0 v2:172.21.15.142:6835/2796023141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5600496f23a0 con 0x5600498f1850 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8ce1ffb640 10 client.5412 handle_command_reply: tid=0 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 client.5412 shutdown 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498dfb20 msgr2=0x56004995bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498dfb20 0x56004995bf30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4008010 tx=0x7f8cf4019180 comp rx=0 tx=0).stop 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 shutdown_connections 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x5600498f1850 0x5600498f1c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600498e0440 0x5600498f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5600498dfb20 0x56004995bf30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 --2- 172.21.15.3:0/1287312615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x56004989b330 0x56004995b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8ce1ffb640 0 client.5412 ms_handle_reset on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 >> 172.21.15.3:0/1287312615 conn(0x560049896290 msgr2=0x5600498ddc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 shutdown_connections 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 1 -- 172.21.15.3:0/1287312615 wait complete. 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d0c824000 20 client.5412 trim_cache size 0 max 16384 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5262, 2024-09-12T12:21:40.509 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.300620871, 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 23, 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 43, 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 43, 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.303818516 2024-09-12T12:21:40.510 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cf00678c0 msgr2=0x7f8cf0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cf00678c0 0x7f8cf0069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40af000 tx=0x7f8cf4008040 comp rx=0 tx=0).stop 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d04073320 msgr2=0x7f8d0414af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d04073320 0x7f8d0414af10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c4d0 tx=0x7f8cf800c990 comp rx=0 tx=0).stop 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 shutdown_connections 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8cf00678c0 0x7f8cf0069d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d040d0a90 0x7f8d040b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d040af550 0x7f8d040afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 --2- 172.21.15.3:0/2727901182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d04073320 0x7f8d0414af10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 >> 172.21.15.3:0/2727901182 conn(0x7f8d0406be90 msgr2=0x7f8d040b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 shutdown_connections 2024-09-12T12:21:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.501+0000 7f8d09768640 1 -- 172.21.15.3:0/2727901182 wait complete. 2024-09-12T12:21:40.520 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:21:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.633+0000 7f9040168640 1 Processor -- start 2024-09-12T12:21:40.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f9040168640 1 -- start start 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f9040168640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f903815e5d0 0x7f90380ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f9040168640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90380d5050 0x7f90380cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f9040168640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90380d31a0 con 0x7f90380cfa10 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f9040168640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f90380b1060 con 0x7f90380d5050 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f9040168640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f90380cfe40 con 0x7f903815e5d0 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90380d5050 0x7f90380cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f90380d5050 0x7f90380cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60758/0 (socket says 172.21.15.3:60758) 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 -- 172.21.15.3:0/2974568855 learned_addr learned my addr 172.21.15.3:0/2974568855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903df04640 1 --2- 172.21.15.3:0/2974568855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f903815e5d0 0x7f90380ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 -- 172.21.15.3:0/2974568855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f903815e5d0 msgr2=0x7f90380ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 --2- 172.21.15.3:0/2974568855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f903815e5d0 0x7f90380ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 -- 172.21.15.3:0/2974568855 >> v1:172.21.15.142:6789/0 conn(0x7f90380cfa10 legacy=0x7f9038162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:40.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 -- 172.21.15.3:0/2974568855 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90380d3480 con 0x7f90380d5050 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.637+0000 7f903d703640 1 --2- 172.21.15.3:0/2974568855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380cf4d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9034009870 tx=0x7f903401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6295429160371e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903cf02640 1 -- 172.21.15.3:0/2974568855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340aa620 con 0x7f90380d5050 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903cf02640 1 -- 172.21.15.3:0/2974568855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90340af070 con 0x7f90380d5050 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903cf02640 1 -- 172.21.15.3:0/2974568855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340aa9e0 con 0x7f90380d5050 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- 172.21.15.3:0/2974568855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 msgr2=0x7f90380cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 --2- 172.21.15.3:0/2974568855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380cf4d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9034009870 tx=0x7f903401c520 comp rx=0 tx=0).stop 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- 172.21.15.3:0/2974568855 shutdown_connections 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 --2- 172.21.15.3:0/2974568855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380cf4d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 --2- 172.21.15.3:0/2974568855 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f903815e5d0 0x7f90380ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- 172.21.15.3:0/2974568855 >> 172.21.15.3:0/2974568855 conn(0x7f903808c000 msgr2=0x7f90380d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- 172.21.15.3:0/2974568855 shutdown_connections 2024-09-12T12:21:40.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- 172.21.15.3:0/2974568855 wait complete. 2024-09-12T12:21:40.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 Processor -- start 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- start start 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380cfa10 0x7f90380caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f903815e5d0 0x7f9038154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f903816b5c0 con 0x7f90380d5050 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f903815ebc0 con 0x7f903815e5d0 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f903816b740 con 0x7f90380cfa10 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60770/0 (socket says 172.21.15.3:60770) 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 -- 172.21.15.3:0/1684376274 learned_addr learned my addr 172.21.15.3:0/1684376274 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903e705640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f903815e5d0 0x7f9038154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903df04640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380cfa10 0x7f90380caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 -- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380cfa10 msgr2=0x7f90380caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380cfa10 0x7f90380caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 -- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f903815e5d0 msgr2=0x7f9038154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f903815e5d0 0x7f9038154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 -- 172.21.15.3:0/1684376274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9038156fb0 con 0x7f90380d5050 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903e705640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f903815e5d0 0x7f9038154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f903d703640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380c8010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f903401cdc0 tx=0x7f903401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f902e7fc640 1 -- 172.21.15.3:0/1684376274 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340aad00 con 0x7f90380d5050 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f902e7fc640 1 -- 172.21.15.3:0/1684376274 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90340bd3f0 con 0x7f90380d5050 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90380b49d0 con 0x7f90380d5050 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f902e7fc640 1 -- 172.21.15.3:0/1684376274 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340bd7e0 con 0x7f90380d5050 2024-09-12T12:21:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.641+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90381567f0 con 0x7f90380d5050 2024-09-12T12:21:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.645+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90380cf4d0 con 0x7f90380d5050 2024-09-12T12:21:40.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.645+0000 7f902e7fc640 1 -- 172.21.15.3:0/1684376274 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f90340b33f0 con 0x7f90380d5050 2024-09-12T12:21:40.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.645+0000 7f902e7fc640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f901c0678c0 0x7f901c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:40.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.645+0000 7f902e7fc640 1 -- 172.21.15.3:0/1684376274 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f903411fd40 con 0x7f90380d5050 2024-09-12T12:21:40.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.645+0000 7f903df04640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f901c0678c0 0x7f901c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:40.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.645+0000 7f903df04640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f901c0678c0 0x7f901c069d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90380d2cd0 tx=0x7f9028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:40.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.649+0000 7f902e7fc640 1 -- 172.21.15.3:0/1684376274 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f90340c3060 con 0x7f90380d5050 2024-09-12T12:21:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90380ba490 con 0x7f90380d5050 2024-09-12T12:21:40.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f902e7fc640 1 -- 172.21.15.3:0/1684376274 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f9034016110 con 0x7f90380d5050 2024-09-12T12:21:40.847 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f901c0678c0 msgr2=0x7f901c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f901c0678c0 0x7f901c069d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90380d2cd0 tx=0x7f9028008040 comp rx=0 tx=0).stop 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 msgr2=0x7f90380c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380c8010 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f903401cdc0 tx=0x7f903401ad60 comp rx=0 tx=0).stop 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 shutdown_connections 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f901c0678c0 0x7f901c069d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f903815e5d0 0x7f9038154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f90380d5050 0x7f90380c8010 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 --2- 172.21.15.3:0/1684376274 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90380cfa10 0x7f90380caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:40.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 >> 172.21.15.3:0/1684376274 conn(0x7f903808c000 msgr2=0x7f903806e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:40.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 shutdown_connections 2024-09-12T12:21:40.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:40.845+0000 7f9040168640 1 -- 172.21.15.3:0/1684376274 wait complete. 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:21:40.865 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:21:40.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:40.866 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:40.871 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42386sec preferred_lft 42386sec 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:21:40.872 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:21:40.873 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.874 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:21:40.874 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:40.874 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:21:40.874 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:40.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:40.874 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:21:40.924 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:21:40.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:40.926 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:21:40.979 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:21:40.979 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:21:40.979 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:21:40.979 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:21:40.979 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:21:40.980 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:21:40.980 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:21:40.980 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:21:40.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:21:40.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:21:40.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:21:40.981 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:21:41.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:21:41.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:21:41.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:41.104 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:21:41.109 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:21:41.109 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:21:41.109 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:21:41.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:41.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:21:41.163 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:21:41.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:21:41.163 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:41.215 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:21:41.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:21:41.215 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:41.263 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:21:41.320 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:41.367 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:41.368 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:21:41.369 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:21:41.370 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:21:41.370 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:41.370 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:41.370 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:41.370 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:41.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:41.371 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:21:41.419 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:21:41.419 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:21:41.463 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:21:41.480 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:21:41.527 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:41.528 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:41.529 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:41.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:41.530 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:21:41.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:21:41.549+0000 7fa4acf86580 -1 init, newargv = 0x5567ba13a5d0 newargc=15 2024-09-12T12:21:41.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[38387]: starting ceph client 2024-09-12T12:21:41.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[38387]: starting fuse 2024-09-12T12:21:42.656 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:21:42.672 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:42.721 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:21:42.721 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:21:42.721 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:42.722 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:21:42.723 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:21:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:21:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:21:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:21:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:21:42.724 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:42.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:42.725 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:21:42.775 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:21:42.775 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:21:42.775 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:42.776 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:21:42.777 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:21:42.778 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:21:42.778 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:21:42.903 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.38387.asok 2024-09-12T12:21:42.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:21:42.916 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38387.asok 2024-09-12T12:21:42.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:42.916 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38387.asok status 2024-09-12T12:21:43.074 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:21:43.075 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:21:43.075 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:21:43.075 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:21:43.075 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:21:43.075 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:21:43.076 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:21:43.076 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "38387", 2024-09-12T12:21:43.076 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:21:43.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:43.076 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:21:43.076 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:21:43.077 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5424, 2024-09-12T12:21:43.077 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:21:43.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:21:43.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:21:43.077 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5424 2024-09-12T12:21:43.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:43.078 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:21:43.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:21:43.078 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:21:43.078 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2872188260 2024-09-12T12:21:43.078 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:21:43.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:43.079 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:21:43.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:21:43.079 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:21:43.079 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2872188260 2024-09-12T12:21:43.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:21:43.079 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5424 192.168.0.1:0/2872188260", 2024-09-12T12:21:43.079 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/2872188260", 2024-09-12T12:21:43.080 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:21:43.080 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 23, 2024-09-12T12:21:43.080 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 43, 2024-09-12T12:21:43.080 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:21:43.080 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:21:43.080 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:21:43.081 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:21:43.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:43.088 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:21:43.088 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:43.095 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:21:43.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:43.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:43.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:21:43.152 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:21:43.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.317+0000 7f2575cc2640 1 Processor -- start 2024-09-12T12:21:43.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.317+0000 7f2575cc2640 1 -- start start 2024-09-12T12:21:43.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f2575cc2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f257015e5d0 0x7f25700ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:43.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f2575cc2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700cfbe0 0x7f25700b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:43.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f2575cc2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25700d31a0 con 0x7f25700d5050 2024-09-12T12:21:43.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f2575cc2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f25700b1060 con 0x7f257015e5d0 2024-09-12T12:21:43.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f2575cc2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25700d3480 con 0x7f25700cfbe0 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f257015e5d0 0x7f25700ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256effd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f25700d5050 0x7f25700cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35084/0 (socket says 172.21.15.3:35084) 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256effd640 1 -- 172.21.15.3:0/3615081516 learned_addr learned my addr 172.21.15.3:0/3615081516 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256ffff640 1 --2- 172.21.15.3:0/3615081516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700cfbe0 0x7f25700b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256f7fe640 1 -- 172.21.15.3:0/3615081516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700cfbe0 msgr2=0x7f25700b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256f7fe640 1 --2- 172.21.15.3:0/3615081516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700cfbe0 0x7f25700b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256f7fe640 1 -- 172.21.15.3:0/3615081516 >> v1:172.21.15.3:6789/0 conn(0x7f25700d5050 legacy=0x7f25700cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:43.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256f7fe640 1 -- 172.21.15.3:0/3615081516 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25700d3760 con 0x7f257015e5d0 2024-09-12T12:21:43.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256ffff640 1 --2- 172.21.15.3:0/3615081516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700cfbe0 0x7f25700b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:43.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256f7fe640 1 --2- 172.21.15.3:0/3615081516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f257015e5d0 0x7f25700ceb60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f25680097c0 tx=0x7f256801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bcb3fd00a7e0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:43.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256e7fc640 1 -- 172.21.15.3:0/3615081516 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680ab430 con 0x7f257015e5d0 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256e7fc640 1 -- 172.21.15.3:0/3615081516 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25680b0070 con 0x7f257015e5d0 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.321+0000 7f256e7fc640 1 -- 172.21.15.3:0/3615081516 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680ab7f0 con 0x7f257015e5d0 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- 172.21.15.3:0/3615081516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f257015e5d0 msgr2=0x7f25700ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3615081516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f257015e5d0 0x7f25700ceb60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f25680097c0 tx=0x7f256801c410 comp rx=0 tx=0).stop 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- 172.21.15.3:0/3615081516 shutdown_connections 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3615081516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700cfbe0 0x7f25700b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3615081516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f257015e5d0 0x7f25700ceb60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- 172.21.15.3:0/3615081516 >> 172.21.15.3:0/3615081516 conn(0x7f257008c000 msgr2=0x7f25700d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- 172.21.15.3:0/3615081516 shutdown_connections 2024-09-12T12:21:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- 172.21.15.3:0/3615081516 wait complete. 2024-09-12T12:21:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 Processor -- start 2024-09-12T12:21:43.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- start start 2024-09-12T12:21:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25700cfbe0 0x7f2570152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700d5050 0x7f257006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257015e5d0 0x7f2570159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25700b25d0 con 0x7f25700cfbe0 2024-09-12T12:21:43.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25700b2450 con 0x7f257015e5d0 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25700b2750 con 0x7f25700d5050 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25700cfbe0 0x7f2570152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25700cfbe0 0x7f2570152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60784/0 (socket says 172.21.15.3:60784) 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 -- 172.21.15.3:0/3465279668 learned_addr learned my addr 172.21.15.3:0/3465279668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256effd640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700d5050 0x7f257006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256ffff640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257015e5d0 0x7f2570159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 -- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700d5050 msgr2=0x7f257006e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:43.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700d5050 0x7f257006e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 -- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257015e5d0 msgr2=0x7f2570159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257015e5d0 0x7f2570159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 -- 172.21.15.3:0/3465279668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257015b5f0 con 0x7f25700cfbe0 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256ffff640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257015e5d0 0x7f2570159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f256f7fe640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25700cfbe0 0x7f2570152b20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f256801aac0 tx=0x7f25680191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2567fff640 1 -- 172.21.15.3:0/3465279668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680bd480 con 0x7f25700cfbe0 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2567fff640 1 -- 172.21.15.3:0/3465279668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25680bd660 con 0x7f25700cfbe0 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2567fff640 1 -- 172.21.15.3:0/3465279668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680bda50 con 0x7f25700cfbe0 2024-09-12T12:21:43.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257015b8d0 con 0x7f25700cfbe0 2024-09-12T12:21:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.325+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2570159090 con 0x7f25700cfbe0 2024-09-12T12:21:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.329+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25700b14f0 con 0x7f25700cfbe0 2024-09-12T12:21:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.329+0000 7f2567fff640 1 -- 172.21.15.3:0/3465279668 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f25680b43f0 con 0x7f25700cfbe0 2024-09-12T12:21:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.329+0000 7f2567fff640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25500678f0 0x7f2550069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.329+0000 7f2567fff640 1 -- 172.21.15.3:0/3465279668 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f25681200c0 con 0x7f25700cfbe0 2024-09-12T12:21:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.329+0000 7f256effd640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25500678f0 0x7f2550069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:43.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.329+0000 7f256effd640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25500678f0 0x7f2550069db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2570157a90 tx=0x7f2558009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:43.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.333+0000 7f2567fff640 1 -- 172.21.15.3:0/3465279668 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f25680c4030 con 0x7f25700cfbe0 2024-09-12T12:21:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.529+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f25700ba490 con 0x7f25700cfbe0 2024-09-12T12:21:43.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2567fff640 1 -- 172.21.15.3:0/3465279668 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f2568016090 con 0x7f25700cfbe0 2024-09-12T12:21:43.534 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25500678f0 msgr2=0x7f2550069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25500678f0 0x7f2550069db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2570157a90 tx=0x7f2558009040 comp rx=0 tx=0).stop 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25700cfbe0 msgr2=0x7f2570152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25700cfbe0 0x7f2570152b20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f256801aac0 tx=0x7f25680191b0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 shutdown_connections 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25500678f0 0x7f2550069db0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f257015e5d0 0x7f2570159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25700d5050 0x7f257006e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 --2- 172.21.15.3:0/3465279668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25700cfbe0 0x7f2570152b20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:43.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 >> 172.21.15.3:0/3465279668 conn(0x7f257008c000 msgr2=0x7f2570158240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:43.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 shutdown_connections 2024-09-12T12:21:43.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:43.533+0000 7f2575cc2640 1 -- 172.21.15.3:0/3465279668 wait complete. 2024-09-12T12:21:43.553 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:21:43.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:21:43.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:21:43.554 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:21:43.554 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:21:43.554 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:21:43.554 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:21:43.554 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:21:43.554 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:21:43.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:43.555 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:21:43.562 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:21:43.562 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:21:43.562 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:21:43.562 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.562 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:21:43.562 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42381sec preferred_lft 42381sec 2024-09-12T12:21:43.563 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:21:43.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:21:43.565 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.565 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:21:43.565 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:21:43.565 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:21:43.565 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:21:43.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:43.565 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:21:43.616 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:21:43.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:43.618 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:21:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:21:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:21:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:21:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:21:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:21:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:21:43.670 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:21:43.671 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:21:43.671 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:21:43.671 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:21:43.671 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:21:43.671 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:21:43.672 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:21:43.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:21:43.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:21:43.672 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:21:43.672 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:21:43.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:21:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:21:43.788 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:21:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:21:43.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:43.795 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:21:43.801 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:21:43.801 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:21:43.801 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:21:43.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:43.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:21:43.862 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:21:43.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:21:43.862 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:21:43.909 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:21:43.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:21:43.909 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:21:43.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:21:44.027 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:44.097 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:21:44.098 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:44.099 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:21:44.100 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:21:44.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:44.101 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:21:44.159 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:21:44.160 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:21:44.212 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:21:44.227 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:21:44.281 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:21:44.282 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:21:44.283 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:21:44.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:44.284 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:21:44.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:21:44.289+0000 7f93674ab580 -1 init, newargv = 0x5592fa739de0 newargc=15 2024-09-12T12:21:44.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35682]: starting ceph client 2024-09-12T12:21:44.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35682]: starting fuse 2024-09-12T12:21:45.398 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:21:45.415 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:21:45.463 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:21:45.463 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:21:45.463 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:21:45.463 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:21:45.464 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:21:45.465 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:21:45.466 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:21:45.466 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:21:45.466 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:21:45.466 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:21:45.466 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:21:45.466 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:21:45.466 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:21:45.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:45.468 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:21:45.521 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:21:45.522 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:21:45.522 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:21:45.522 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:21:45.522 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:21:45.522 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:21:45.522 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:21:45.523 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:21:45.524 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:21:45.525 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:21:45.525 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:21:45.525 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:21:45.525 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:21:45.525 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:21:45.526 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:21:45.643 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.35682.asok 2024-09-12T12:21:45.647 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:21:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:21:45.655 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35682.asok 2024-09-12T12:21:45.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:45.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35682.asok status 2024-09-12T12:21:45.804 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:21:45.805 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:21:45.805 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:21:45.805 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:21:45.805 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:21:45.805 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:21:45.805 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:21:45.805 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "35682", 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 5059, 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:21:45.806 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:21:45.807 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 5059 2024-09-12T12:21:45.807 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:21:45.807 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:21:45.807 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:21:45.807 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:21:45.807 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 134331345 2024-09-12T12:21:45.808 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:21:45.808 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:21:45.808 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:21:45.808 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:21:45.808 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:21:45.808 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 134331345 2024-09-12T12:21:45.809 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:21:45.809 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.5059 192.168.0.1:0/134331345", 2024-09-12T12:21:45.809 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/134331345", 2024-09-12T12:21:45.809 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:21:45.809 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 23, 2024-09-12T12:21:45.809 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 43, 2024-09-12T12:21:45.809 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:21:45.810 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:21:45.810 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:21:45.810 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:21:45.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:45.819 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:21:45.819 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:21:45.825 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:21:45.825 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:21:45.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:45.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:21:45.880 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-12T12:21:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3458934640 1 Processor -- start 2024-09-12T12:21:46.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3458934640 1 -- start start 2024-09-12T12:21:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3458934640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3454159dd0 0x7f345415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3458934640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 0x7f34540ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3458934640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f345415d830 con 0x7f345406f5b0 2024-09-12T12:21:46.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3458934640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f345406e0b0 con 0x7f3454159dd0 2024-09-12T12:21:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3458934640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f345406e390 con 0x7f345406f150 2024-09-12T12:21:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3452d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f345406f5b0 0x7f34540b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35088/0 (socket says 172.21.15.3:35088) 2024-09-12T12:21:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3452d9d640 1 -- 172.21.15.3:0/3800624789 learned_addr learned my addr 172.21.15.3:0/3800624789 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345259c640 1 --2- 172.21.15.3:0/3800624789 >> v2:172.21.15.142:3300/0 conn(0x7f3454159dd0 0x7f345415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3451d9b640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 0x7f34540ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042148966 0 0) 0x7f344c0014a0 con 0x7f345406f5b0 2024-09-12T12:21:46.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f343c003530 con 0x7f345406f5b0 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 897890075 0 0) 0x7f343c003530 con 0x7f345406f5b0 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 msgr2=0x7f34540ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 0x7f34540ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 >> v2:172.21.15.142:3300/0 conn(0x7f3454159dd0 msgr2=0x7f345415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 --2- 172.21.15.3:0/3800624789 >> v2:172.21.15.142:3300/0 conn(0x7f3454159dd0 0x7f345415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345406e670 con 0x7f345406f5b0 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f344c001bd0 con 0x7f345406f5b0 2024-09-12T12:21:46.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 >> v1:172.21.15.3:6789/0 conn(0x7f345406f5b0 legacy=0x7f34540b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f343c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 0x7f343c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f343c004d50 0x7f343c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f345406e0b0 con 0x7f343c0039c0 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f345406e390 con 0x7f343c004d50 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f343c00b9d0 con 0x7f345406f150 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345259c640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f343c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3451d9b640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 0x7f343c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3452d9d640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f343c004d50 0x7f343c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345259c640 1 -- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 msgr2=0x7f343c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345259c640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f345406f150 0x7f343c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345259c640 1 -- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f343c004d50 msgr2=0x7f343c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345259c640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f343c004d50 0x7f343c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f345259c640 1 -- 172.21.15.3:0/3800624789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c003530 con 0x7f343c0039c0 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.021+0000 7f3452d9d640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f343c004d50 0x7f343c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f345259c640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f343c003de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ae0 tx=0x7f344401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3181ca1139ebd1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:46.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440b2070 con 0x7f343c0039c0 2024-09-12T12:21:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34440a5650 con 0x7f343c0039c0 2024-09-12T12:21:46.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f345159a640 1 -- 172.21.15.3:0/3800624789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440096b0 con 0x7f343c0039c0 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 -- 172.21.15.3:0/3800624789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 msgr2=0x7f343c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 --2- 172.21.15.3:0/3800624789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f343c003de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ae0 tx=0x7f344401ca80 comp rx=0 tx=0).stop 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3452d9d640 1 -- 172.21.15.3:0/3800624789 reap_dead start 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 -- 172.21.15.3:0/3800624789 shutdown_connections 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 -- 172.21.15.3:0/3800624789 >> 172.21.15.3:0/3800624789 conn(0x7f34540b1d80 msgr2=0x7f34540b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 -- 172.21.15.3:0/3800624789 shutdown_connections 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 -- 172.21.15.3:0/3800624789 wait complete. 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 Processor -- start 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.025+0000 7f3458934640 1 -- start start 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f34541f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3454153840 0x7f3454153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34541544c0 0x7f3454151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34540b2a40 con 0x7f343c0039c0 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f345415a1b0 con 0x7f34541544c0 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34540b2bc0 con 0x7f3454153840 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f34541f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f34541f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60804/0 (socket says 172.21.15.3:60804) 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 -- 172.21.15.3:0/2459062097 learned_addr learned my addr 172.21.15.3:0/2459062097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3452d9d640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34541544c0 0x7f3454151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3451d9b640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3454153840 0x7f3454153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 -- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3454153840 msgr2=0x7f3454153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3454153840 0x7f3454153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 -- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34541544c0 msgr2=0x7f3454151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34541544c0 0x7f3454151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 -- 172.21.15.3:0/2459062097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3454154960 con 0x7f343c0039c0 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3451d9b640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3454153840 0x7f3454153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f345259c640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f34541f84d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ae0 tx=0x7f343c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3442ffd640 1 -- 172.21.15.3:0/2459062097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440bc030 con 0x7f343c0039c0 2024-09-12T12:21:46.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3442ffd640 1 -- 172.21.15.3:0/2459062097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344400f000 con 0x7f343c0039c0 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3454154c40 con 0x7f343c0039c0 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3442ffd640 1 -- 172.21.15.3:0/2459062097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440b2040 con 0x7f343c0039c0 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3454151430 con 0x7f343c0039c0 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345415d2f0 con 0x7f343c0039c0 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3442ffd640 1 -- 172.21.15.3:0/2459062097 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34440bf070 con 0x7f343c0039c0 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3442ffd640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34340678c0 0x7f3434069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3442ffd640 1 -- 172.21.15.3:0/2459062097 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f344411fda0 con 0x7f343c0039c0 2024-09-12T12:21:46.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.029+0000 7f3451d9b640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34340678c0 0x7f3434069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.033+0000 7f3451d9b640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34340678c0 0x7f3434069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3454153c40 tx=0x7f3448008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:46.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.033+0000 7f3442ffd640 1 -- 172.21.15.3:0/2459062097 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f34440167c0 con 0x7f343c0039c0 2024-09-12T12:21:46.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.241+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f345406fae0 con 0x7f343c0039c0 2024-09-12T12:21:46.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:21:46.591 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:21:46.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.785+0000 7f3442ffd640 1 -- 172.21.15.3:0/2459062097 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v24) ==== 105+0+0 (secure 0 0 0) 0x7f34440f0a00 con 0x7f343c0039c0 2024-09-12T12:21:46.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34340678c0 msgr2=0x7f3434069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34340678c0 0x7f3434069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3454153c40 tx=0x7f3448008040 comp rx=0 tx=0).stop 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 msgr2=0x7f34541f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f34541f84d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ae0 tx=0x7f343c005840 comp rx=0 tx=0).stop 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 shutdown_connections 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34340678c0 0x7f3434069d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34541544c0 0x7f3454151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3454153840 0x7f3454153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 --2- 172.21.15.3:0/2459062097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f343c0039c0 0x7f34541f84d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 >> 172.21.15.3:0/2459062097 conn(0x7f34540b1d80 msgr2=0x7f34540b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 shutdown_connections 2024-09-12T12:21:46.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.789+0000 7f3458934640 1 -- 172.21.15.3:0/2459062097 wait complete. 2024-09-12T12:21:46.818 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:46.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd83d0640 1 Processor -- start 2024-09-12T12:21:46.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd83d0640 1 -- start start 2024-09-12T12:21:46.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd83d0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbd015e5d0 0x7fbbd00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd83d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00d5050 0x7fbbd00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd83d0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbd00d31a0 con 0x7fbbd00cfa10 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd83d0640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd00b1060 con 0x7fbbd015e5d0 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd83d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd00cfe40 con 0x7fbbd00d5050 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd616c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbd015e5d0 0x7fbbd00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd616c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbd015e5d0 0x7fbbd00ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60816/0 (socket says 172.21.15.3:60816) 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd616c640 1 -- 172.21.15.3:0/148933928 learned_addr learned my addr 172.21.15.3:0/148933928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd616c640 1 -- 172.21.15.3:0/148933928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00d5050 msgr2=0x7fbbd00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd596b640 1 --2- 172.21.15.3:0/148933928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00d5050 0x7fbbd00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd616c640 1 --2- 172.21.15.3:0/148933928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00d5050 0x7fbbd00cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd616c640 1 -- 172.21.15.3:0/148933928 >> v1:172.21.15.142:6789/0 conn(0x7fbbd00cfa10 legacy=0x7fbbd0162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:46.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.937+0000 7fbbd616c640 1 -- 172.21.15.3:0/148933928 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd00d3480 con 0x7fbbd015e5d0 2024-09-12T12:21:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd616c640 1 --2- 172.21.15.3:0/148933928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd00ceb60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009e50 tx=0x7fbbc801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f5eaafa62e5eda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd516a640 1 -- 172.21.15.3:0/148933928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80a5850 con 0x7fbbd015e5d0 2024-09-12T12:21:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd516a640 1 -- 172.21.15.3:0/148933928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc80a59f0 con 0x7fbbd015e5d0 2024-09-12T12:21:46.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd516a640 1 -- 172.21.15.3:0/148933928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80a5dc0 con 0x7fbbd015e5d0 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- 172.21.15.3:0/148933928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 msgr2=0x7fbbd00ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/148933928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd00ceb60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009e50 tx=0x7fbbc801c9f0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- 172.21.15.3:0/148933928 shutdown_connections 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/148933928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00d5050 0x7fbbd00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/148933928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd00ceb60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- 172.21.15.3:0/148933928 >> 172.21.15.3:0/148933928 conn(0x7fbbd008c000 msgr2=0x7fbbd00d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- 172.21.15.3:0/148933928 shutdown_connections 2024-09-12T12:21:46.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- 172.21.15.3:0/148933928 wait complete. 2024-09-12T12:21:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 Processor -- start 2024-09-12T12:21:46.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- start start 2024-09-12T12:21:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00cfa10 0x7fbbd015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd00d5050 0x7fbbd0156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd016b5c0 con 0x7fbbd015e5d0 2024-09-12T12:21:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd015ec00 con 0x7fbbd00d5050 2024-09-12T12:21:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd83d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd016b740 con 0x7fbbd00cfa10 2024-09-12T12:21:46.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60822/0 (socket says 172.21.15.3:60822) 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 -- 172.21.15.3:0/1925447937 learned_addr learned my addr 172.21.15.3:0/1925447937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd616c640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00cfa10 0x7fbbd015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd596b640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd00d5050 0x7fbbd0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 -- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00cfa10 msgr2=0x7fbbd015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00cfa10 0x7fbbd015aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 -- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd00d5050 msgr2=0x7fbbd0156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd00d5050 0x7fbbd0156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 -- 172.21.15.3:0/1925447937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd00c9f20 con 0x7fbbd015e5d0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbd696d640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd015c5f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b440 tx=0x7fbbc000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.941+0000 7fbbceffd640 1 -- 172.21.15.3:0/1925447937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0015020 con 0x7fbbd015e5d0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbceffd640 1 -- 172.21.15.3:0/1925447937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc0011830 con 0x7fbbd015e5d0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd00ca1e0 con 0x7fbbd015e5d0 2024-09-12T12:21:46.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbceffd640 1 -- 172.21.15.3:0/1925447937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0011c20 con 0x7fbbd015e5d0 2024-09-12T12:21:46.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd00ca7a0 con 0x7fbbd015e5d0 2024-09-12T12:21:46.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd00cf4d0 con 0x7fbbd015e5d0 2024-09-12T12:21:46.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbceffd640 1 -- 172.21.15.3:0/1925447937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbc0018020 con 0x7fbbd015e5d0 2024-09-12T12:21:46.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbceffd640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678c0 0x7fbbb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:46.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbceffd640 1 -- 172.21.15.3:0/1925447937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fbbc0085c90 con 0x7fbbd015e5d0 2024-09-12T12:21:46.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbd616c640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678c0 0x7fbbb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:46.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.945+0000 7fbbd616c640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678c0 0x7fbbb8069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80ae000 tx=0x7fbbc80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:46.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:46.949+0000 7fbbceffd640 1 -- 172.21.15.3:0/1925447937 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbbc001f030 con 0x7fbbd015e5d0 2024-09-12T12:21:47.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.149+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbd00ba490 con 0x7fbbd015e5d0 2024-09-12T12:21:47.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.149+0000 7fbbceffd640 1 -- 172.21.15.3:0/1925447937 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+5236 (secure 0 0 0) 0x7fbbc00172c0 con 0x7fbbd015e5d0 2024-09-12T12:21:47.152 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:47.152 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":25,"btime":"2024-09-12T12:21:46:792652+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:46.792619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5262,"mds_1":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":1,"incarnation":25,"state":"up:creating","state_seq":1,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5262]},"id":3}]} 2024-09-12T12:21:47.152 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 25 2024-09-12T12:21:47.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678c0 msgr2=0x7fbbb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:47.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678c0 0x7fbbb8069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80ae000 tx=0x7fbbc80a7070 comp rx=0 tx=0).stop 2024-09-12T12:21:47.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 msgr2=0x7fbbd015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd015c5f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b440 tx=0x7fbbc000b900 comp rx=0 tx=0).stop 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 shutdown_connections 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678c0 0x7fbbb8069d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd015e5d0 0x7fbbd015c5f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd00d5050 0x7fbbd0156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 --2- 172.21.15.3:0/1925447937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd00cfa10 0x7fbbd015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 >> 172.21.15.3:0/1925447937 conn(0x7fbbd008c000 msgr2=0x7fbbd00b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 shutdown_connections 2024-09-12T12:21:47.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:47.153+0000 7fbbd83d0640 1 -- 172.21.15.3:0/1925447937 wait complete. 2024-09-12T12:21:47.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:21:29.563260+0000', 'modified': '2024-09-12T12:21:46.792619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5262, 'mds_1': 5256}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5256': {'gid': 5256, 'name': 'a', 'rank': 1, 'incarnation': 25, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.3:6835/653673671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 653673671}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 653673671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5262': {'gid': 5262, 'name': 'd', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6837/2796023141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6835', 'nonce': 2796023141}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2796023141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5262, 'qdb_cluster': [5262]} 2024-09-12T12:21:47.574 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5256:up:creating 2024-09-12T12:21:48.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:21:48.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40ee231640 1 Processor -- start 2024-09-12T12:21:48.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40ee231640 1 -- start start 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40ee231640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f40e80b42b0 0x7f40e8071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40ee231640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 0x7f40e80722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40ee231640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e806dee0 con 0x7f40e815e4b0 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40ee231640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f40e806e810 con 0x7f40e80b42b0 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40ee231640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40e806e1c0 con 0x7f40e80b4bd0 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e77fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40e815e4b0 0x7f40e80ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54422/0 (socket says 172.21.15.3:54422) 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e77fe640 1 -- 172.21.15.3:0/429365001 learned_addr learned my addr 172.21.15.3:0/429365001 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717939328 0 0) 0x7f40e806dee0 con 0x7f40e815e4b0 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e7fff640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 0x7f40e80722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e6ffd640 1 --2- 172.21.15.3:0/429365001 >> v2:172.21.15.142:3300/0 conn(0x7f40e80b42b0 0x7f40e8071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40d4003530 con 0x7f40e815e4b0 2024-09-12T12:21:48.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3056468624 0 0) 0x7f40d4003530 con 0x7f40e815e4b0 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 msgr2=0x7f40e80722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 0x7f40e80722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 >> v2:172.21.15.142:3300/0 conn(0x7f40e80b42b0 msgr2=0x7f40e8071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 --2- 172.21.15.3:0/429365001 >> v2:172.21.15.142:3300/0 conn(0x7f40e80b42b0 0x7f40e8071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e806e4a0 con 0x7f40e815e4b0 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f40d8001a00 con 0x7f40e815e4b0 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 >> v1:172.21.15.3:6789/0 conn(0x7f40e815e4b0 legacy=0x7f40e80ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d40039c0 0x7f40d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 0x7f40d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d4004d50 0x7f40d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40e806e810 con 0x7f40d4004d50 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40e806e1c0 con 0x7f40d40039c0 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40d400ba00 con 0x7f40e80b4bd0 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e7fff640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d4004d50 0x7f40d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e6ffd640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 0x7f40d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e77fe640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d40039c0 0x7f40d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e7fff640 1 -- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 msgr2=0x7f40d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e7fff640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80b4bd0 0x7f40d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e7fff640 1 -- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d40039c0 msgr2=0x7f40d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e7fff640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d40039c0 0x7f40d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e7fff640 1 -- 172.21.15.3:0/429365001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d4003530 con 0x7f40d4004d50 2024-09-12T12:21:48.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.709+0000 7f40e77fe640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d40039c0 0x7f40d4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:48.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40e7fff640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d4004d50 0x7f40d40051d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009ab0 tx=0x7f40e001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929270064daa1745 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:48.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00b2070 con 0x7f40d4004d50 2024-09-12T12:21:48.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e00a5640 con 0x7f40d4004d50 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40e67fc640 1 -- 172.21.15.3:0/429365001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00096b0 con 0x7f40d4004d50 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40ee231640 1 -- 172.21.15.3:0/429365001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d4004d50 msgr2=0x7f40d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40ee231640 1 --2- 172.21.15.3:0/429365001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d4004d50 0x7f40d40051d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009ab0 tx=0x7f40e001ca50 comp rx=0 tx=0).stop 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40e7fff640 1 -- 172.21.15.3:0/429365001 reap_dead start 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40ee231640 1 -- 172.21.15.3:0/429365001 shutdown_connections 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40ee231640 1 -- 172.21.15.3:0/429365001 >> 172.21.15.3:0/429365001 conn(0x7f40e80b1d80 msgr2=0x7f40e80b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40ee231640 1 -- 172.21.15.3:0/429365001 shutdown_connections 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40ee231640 1 -- 172.21.15.3:0/429365001 wait complete. 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.713+0000 7f40ee231640 1 Processor -- start 2024-09-12T12:21:48.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 -- start start 2024-09-12T12:21:48.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e815bad0 0x7f40e81f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80d04d0 0x7f40e809d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e815beb0 0x7f40e80ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40e80b3390 con 0x7f40e815beb0 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8072a20 con 0x7f40e815bad0 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40e80722e0 con 0x7f40e80d04d0 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e815beb0 0x7f40e80ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e815beb0 0x7f40e80ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41096/0 (socket says 172.21.15.3:41096) 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 -- 172.21.15.3:0/770321296 learned_addr learned my addr 172.21.15.3:0/770321296 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e6ffd640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80d04d0 0x7f40e809d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e77fe640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e815bad0 0x7f40e81f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 -- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80d04d0 msgr2=0x7f40e809d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80d04d0 0x7f40e809d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 -- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e815bad0 msgr2=0x7f40e81f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e815bad0 0x7f40e81f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 -- 172.21.15.3:0/770321296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e809a280 con 0x7f40e815beb0 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e77fe640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e815bad0 0x7f40e81f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e6ffd640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80d04d0 0x7f40e809d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40e7fff640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e815beb0 0x7f40e80ca500 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009ab0 tx=0x7f40d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40dffff640 1 -- 172.21.15.3:0/770321296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00b2040 con 0x7f40e815beb0 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40dffff640 1 -- 172.21.15.3:0/770321296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e0000e50 con 0x7f40e815beb0 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e809e020 con 0x7f40e815beb0 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40dffff640 1 -- 172.21.15.3:0/770321296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00096b0 con 0x7f40e815beb0 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e809a800 con 0x7f40e815beb0 2024-09-12T12:21:48.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e8071da0 con 0x7f40e815beb0 2024-09-12T12:21:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40dffff640 1 -- 172.21.15.3:0/770321296 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40e00bd030 con 0x7f40e815beb0 2024-09-12T12:21:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.717+0000 7f40dffff640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c80678f0 0x7f40c8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.721+0000 7f40dffff640 1 -- 172.21.15.3:0/770321296 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f40e011eba0 con 0x7f40e815beb0 2024-09-12T12:21:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.721+0000 7f40e77fe640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c80678f0 0x7f40c8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:48.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.721+0000 7f40e77fe640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c80678f0 0x7f40c8069db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40d80013e0 tx=0x7f40d8000f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:48.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.721+0000 7f40dffff640 1 -- 172.21.15.3:0/770321296 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f40e011ee20 con 0x7f40e815beb0 2024-09-12T12:21:48.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.921+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40e80b47a0 con 0x7f40e815beb0 2024-09-12T12:21:48.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.921+0000 7f40dffff640 1 -- 172.21.15.3:0/770321296 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5239 (secure 0 0 0) 0x7f40e0016760 con 0x7f40e815beb0 2024-09-12T12:21:48.925 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:21:48.925 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":26,"btime":"2024-09-12T12:21:47:796066+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:47.796057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5262,"mds_1":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5256,5262]},"id":3}]} 2024-09-12T12:21:48.925 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 26 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c80678f0 msgr2=0x7f40c8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c80678f0 0x7f40c8069db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40d80013e0 tx=0x7f40d8000f50 comp rx=0 tx=0).stop 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e815beb0 msgr2=0x7f40e80ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e815beb0 0x7f40e80ca500 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009ab0 tx=0x7f40d4005840 comp rx=0 tx=0).stop 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 shutdown_connections 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c80678f0 0x7f40c8069db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40e815beb0 0x7f40e80ca500 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40e80d04d0 0x7f40e809d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 --2- 172.21.15.3:0/770321296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40e815bad0 0x7f40e81f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:48.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 >> 172.21.15.3:0/770321296 conn(0x7f40e80b1d80 msgr2=0x7f40e80adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:48.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 shutdown_connections 2024-09-12T12:21:48.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:48.925+0000 7f40ee231640 1 -- 172.21.15.3:0/770321296 wait complete. 2024-09-12T12:21:49.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:21:29.563260+0000', 'modified': '2024-09-12T12:21:47.796057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5262, 'mds_1': 5256}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5256': {'gid': 5256, 'name': 'a', 'rank': 1, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.3:6835/653673671', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 653673671}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 653673671}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5262': {'gid': 5262, 'name': 'd', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.142:6837/2796023141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6835', 'nonce': 2796023141}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2796023141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5262, 'qdb_cluster': [5256, 5262]} 2024-09-12T12:21:49.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-12T12:21:49.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T12:21:49.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 Processor -- start 2024-09-12T12:21:49.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 -- start start 2024-09-12T12:21:49.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7cd01568f0 0x7f7cd00aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cd00b47a0 0x7f7cd00b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00d0380 0x7f7cd00d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd00d0d30 con 0x7f7cd00b47a0 2024-09-12T12:21:49.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd00afb00 con 0x7f7cd01568f0 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd7546640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd00b31b0 con 0x7f7cd00d0380 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cd00b47a0 0x7f7cd00b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7cd00b47a0 0x7f7cd00b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41106/0 (socket says 172.21.15.3:41106) 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/552027771 learned_addr learned my addr 172.21.15.3:0/552027771 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd5ae3640 1 --2- 172.21.15.3:0/552027771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00d0380 0x7f7cd00d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/552027771 >> v2:172.21.15.142:3300/0 conn(0x7f7cd01568f0 0x7f7cd00aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/552027771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00d0380 msgr2=0x7f7cd00d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/552027771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00d0380 0x7f7cd00d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/552027771 >> v2:172.21.15.142:3300/0 conn(0x7f7cd01568f0 msgr2=0x7f7cd00aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/552027771 >> v2:172.21.15.142:3300/0 conn(0x7f7cd01568f0 0x7f7cd00aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.477+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/552027771 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd00af540 con 0x7f7cd00b47a0 2024-09-12T12:21:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/552027771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00b47a0 0x7f7cd00b3970 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006cc60 tx=0x7f7cc001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f512e4ad82886ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cc7fff640 1 -- 172.21.15.3:0/552027771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00b0070 con 0x7f7cd00b47a0 2024-09-12T12:21:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cc7fff640 1 -- 172.21.15.3:0/552027771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc00ab600 con 0x7f7cd00b47a0 2024-09-12T12:21:49.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cc7fff640 1 -- 172.21.15.3:0/552027771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00ab9d0 con 0x7f7cd00b47a0 2024-09-12T12:21:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 -- 172.21.15.3:0/552027771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00b47a0 msgr2=0x7f7cd00b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 --2- 172.21.15.3:0/552027771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00b47a0 0x7f7cd00b3970 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7cd006cc60 tx=0x7f7cc001c490 comp rx=0 tx=0).stop 2024-09-12T12:21:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 -- 172.21.15.3:0/552027771 shutdown_connections 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 --2- 172.21.15.3:0/552027771 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00d0380 0x7f7cd00d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 --2- 172.21.15.3:0/552027771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00b47a0 0x7f7cd00b3970 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 --2- 172.21.15.3:0/552027771 >> v2:172.21.15.142:3300/0 conn(0x7f7cd01568f0 0x7f7cd00aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 -- 172.21.15.3:0/552027771 >> 172.21.15.3:0/552027771 conn(0x7f7cd006be90 msgr2=0x7f7cd006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 -- 172.21.15.3:0/552027771 shutdown_connections 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 -- 172.21.15.3:0/552027771 wait complete. 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 Processor -- start 2024-09-12T12:21:49.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.481+0000 7f7cd7546640 1 -- start start 2024-09-12T12:21:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00b47a0 0x7f7cd0149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00d0380 0x7f7cd015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd01568f0 0x7f7cd015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd015d0a0 con 0x7f7cd00d0380 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd00b3eb0 con 0x7f7cd01568f0 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd015d220 con 0x7f7cd00b47a0 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00d0380 0x7f7cd015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00d0380 0x7f7cd015f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41120/0 (socket says 172.21.15.3:41120) 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/2172084572 learned_addr learned my addr 172.21.15.3:0/2172084572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00b47a0 0x7f7cd0149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd5ae3640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd01568f0 0x7f7cd015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00b47a0 msgr2=0x7f7cd0149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00b47a0 0x7f7cd0149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd01568f0 msgr2=0x7f7cd015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd01568f0 0x7f7cd015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/2172084572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd014f5c0 con 0x7f7cd00d0380 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00b47a0 0x7f7cd0149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00d0380 0x7f7cd015f3d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7cc001ab40 tx=0x7f7cc0019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cc5ffb640 1 -- 172.21.15.3:0/2172084572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00b0040 con 0x7f7cd00d0380 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cc5ffb640 1 -- 172.21.15.3:0/2172084572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc00b4620 con 0x7f7cd00d0380 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd01642c0 con 0x7f7cd00d0380 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cc5ffb640 1 -- 172.21.15.3:0/2172084572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00b4a10 con 0x7f7cd00d0380 2024-09-12T12:21:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd014fb10 con 0x7f7cd00d0380 2024-09-12T12:21:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.485+0000 7f7cc5ffb640 1 -- 172.21.15.3:0/2172084572 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7cc00ab4c0 con 0x7f7cd00d0380 2024-09-12T12:21:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cc5ffb640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7cb40678f0 0x7f7cb4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cc5ffb640 1 -- 172.21.15.3:0/2172084572 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7cc011f7b0 con 0x7f7cd00d0380 2024-09-12T12:21:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7cb40678f0 0x7f7cb4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 Processor -- start 2024-09-12T12:21:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- start start 2024-09-12T12:21:49.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7cb40678f0 0x7f7cb4069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7cd015f1c0 tx=0x7f7ccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853869bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x558538651d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538652290 0x558538652670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558538539b60 con 0x55853869bb20 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585384a0c70 con 0x55853869c440 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x558538446ed0 con 0x558538652290 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853869bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853869bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41124/0 (socket says 172.21.15.3:41124) 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 -- 172.21.15.3:0/425280405 learned_addr learned my addr 172.21.15.3:0/425280405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x558538651d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd5ae3640 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538652290 0x558538652670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 -- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538652290 msgr2=0x558538652670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538652290 0x558538652670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 -- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 msgr2=0x558538651d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x558538651d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 -- 172.21.15.3:0/425280405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558538458e50 con 0x55853869bb20 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd5ae3640 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538652290 0x558538652670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853869bf00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00e920 tx=0x7f7ccc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145423c7eea20651 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cbaffd640 1 -- 172.21.15.3:0/425280405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc025070 con 0x55853869bb20 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cbaffd640 1 -- 172.21.15.3:0/425280405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ccc015700 con 0x55853869bb20 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cbaffd640 1 -- 172.21.15.3:0/425280405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00f040 con 0x55853869bb20 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- 172.21.15.3:0/425280405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 msgr2=0x55853869bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853869bf00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00e920 tx=0x7f7ccc011000 comp rx=0 tx=0).stop 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- 172.21.15.3:0/425280405 shutdown_connections 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538652290 0x558538652670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x558538651d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 --2- 172.21.15.3:0/425280405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853869bf00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- 172.21.15.3:0/425280405 >> 172.21.15.3:0/425280405 conn(0x558538658600 msgr2=0x558538650630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- 172.21.15.3:0/425280405 shutdown_connections 2024-09-12T12:21:49.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- 172.21.15.3:0/425280405 wait complete. 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 -- start start 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.489+0000 7f7cda602000 1 Processor -- start 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869bb20 0x558538656df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869c440 0x55853865ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558538657330 0x55853865d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x558538502b00 con 0x55853869c440 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55853847b360 con 0x558538657330 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5585384782d0 con 0x55853869bb20 2024-09-12T12:21:49.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 10 client.0 _ll_register_callbacks cb 0x55853863d820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869c440 0x55853865ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869c440 0x55853865ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41134/0 (socket says 172.21.15.3:41134) 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/82972658 learned_addr learned my addr 172.21.15.3:0/82972658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/82972658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869bb20 msgr2=0x558538656df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869bb20 0x558538656df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/82972658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558538657330 msgr2=0x55853865d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558538657330 0x55853865d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/82972658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558538447120 con 0x55853869c440 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869c440 0x55853865ccf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7cc011dd00 tx=0x7f7cc00f0410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a80de47baff8610 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 1 -- 172.21.15.3:0/82972658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00f88f0 con 0x55853869c440 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 1 -- 172.21.15.3:0/82972658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7cc00f8460 con 0x55853869c440 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ccc015700 con 0x55853869c440 2024-09-12T12:21:49.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 1 -- 172.21.15.3:0/82972658 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7cc00f8660 con 0x55853869c440 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 10 client.5466 fetch_fsmap learned FSMap version 26 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x558538440450 con 0x55853869c440 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 1 -- 172.21.15.3:0/82972658 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ccc015700 con 0x55853869c440 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 1 -- 172.21.15.3:0/82972658 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 3522+0+0 (secure 0 0 0) 0x7f7cc00b4bb0 con 0x55853869c440 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 10 client.5466 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 10 client.5466 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 20 client.5466 populate_metadata read hostname 'smithi003' 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a1f50 0x5585386a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 4 client.5466 mds_command: new command op to 5256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 --> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55853843ac00 con 0x5585386a1f50 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a1f50 0x5585386a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a1f50 0x5585386a2330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d70cd182b8ee05f2 server_cookie=eea4ca4e4f2c6d1e in_seq=0 out_seq=0 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 10 client.5466 ms_handle_connect on v2:172.21.15.3:6834/653673671 2024-09-12T12:21:49.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 1 -- 172.21.15.3:0/82972658 <== mds.1 v2:172.21.15.3:6834/653673671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55853843ac00 con 0x5585386a1f50 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 10 client.5466 handle_command_reply: tid=0 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 client.5466 shutdown 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869c440 msgr2=0x55853865ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869c440 0x55853865ccf0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7cc011dd00 tx=0x7f7cc00f0410 comp rx=0 tx=0).stop 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 shutdown_connections 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558538657330 0x55853865d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a1f50 0x5585386a2330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869c440 0x55853865ccf0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 --2- 172.21.15.3:0/82972658 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869bb20 0x558538656df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7caffff640 0 client.5466 ms_handle_reset on v2:172.21.15.3:6834/653673671 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 >> 172.21.15.3:0/82972658 conn(0x558538652290 msgr2=0x558538650d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 shutdown_connections 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 1 -- 172.21.15.3:0/82972658 wait complete. 2024-09-12T12:21:49.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.493+0000 7f7cda602000 20 client.5466 trim_cache size 0 max 16384 2024-09-12T12:21:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 Processor -- start 2024-09-12T12:21:49.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 -- start start 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558538657330 0x5585386b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869bb20 0x55853869ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869c440 0x55853869b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0203a00 con 0x558538657330 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd0165370 con 0x55853869bb20 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd01659c0 con 0x55853869c440 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558538657330 0x5585386b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558538657330 0x5585386b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41142/0 (socket says 172.21.15.3:41142) 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 -- 172.21.15.3:0/485037059 learned_addr learned my addr 172.21.15.3:0/485037059 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869bb20 0x55853869ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd5ae3640 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869c440 0x55853869b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 -- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869c440 msgr2=0x55853869b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869c440 0x55853869b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 -- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869bb20 msgr2=0x55853869ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869bb20 0x55853869ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 -- 172.21.15.3:0/485037059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558538427fd0 con 0x558538657330 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558538657330 0x5585386b2850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc023040 tx=0x7f7ccc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b7d440aec1051b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cba7fc640 1 -- 172.21.15.3:0/485037059 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc025040 con 0x558538657330 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cba7fc640 1 -- 172.21.15.3:0/485037059 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ccc031040 con 0x558538657330 2024-09-12T12:21:49.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.513+0000 7f7cba7fc640 1 -- 172.21.15.3:0/485037059 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc015dd0 con 0x558538657330 2024-09-12T12:21:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/485037059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558538657330 msgr2=0x5585386b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558538657330 0x5585386b2850 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc023040 tx=0x7f7ccc013000 comp rx=0 tx=0).stop 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/485037059 shutdown_connections 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55853869c440 0x55853869b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869bb20 0x55853869ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/485037059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x558538657330 0x5585386b2850 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/485037059 >> 172.21.15.3:0/485037059 conn(0x558538652290 msgr2=0x558538656840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/485037059 shutdown_connections 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/485037059 wait complete. 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- start start 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 Processor -- start 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538657330 0x55853870ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853870f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x5585386a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55853849ff80 con 0x55853869bb20 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558538458ca0 con 0x55853869c440 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5585384a52a0 con 0x558538657330 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 10 client.0 _ll_register_callbacks cb 0x7f7cc00a4ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853870f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853870f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41150/0 (socket says 172.21.15.3:41150) 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/3144344979 learned_addr learned my addr 172.21.15.3:0/3144344979 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538657330 0x55853870ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd5ae3640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x5585386a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538657330 msgr2=0x55853870ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538657330 0x55853870ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 msgr2=0x5585386a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x5585386a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 -- 172.21.15.3:0/3144344979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558538523750 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538657330 0x55853870ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd5ae3640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x5585386a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd4ae1640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853870f320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7cc001c970 tx=0x7f7cc011dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef8dcf14d00da53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 1 -- 172.21.15.3:0/3144344979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00f88f0 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 1 -- 172.21.15.3:0/3144344979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7cc00ef370 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ccc031040 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 1 -- 172.21.15.3:0/3144344979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7cc00ef850 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 10 client.5472 fetch_fsmap learned FSMap version 26 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55853842ea30 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 1 -- 172.21.15.3:0/3144344979 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ccc031040 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 1 -- 172.21.15.3:0/3144344979 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 3522+0+0 (secure 0 0 0) 0x7f7cc00f62c0 con 0x55853869bb20 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 10 client.5472 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T12:21:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 10 client.5472 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 20 client.5472 populate_metadata read hostname 'smithi003' 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a5d30 0x5585386a6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 4 client.5472 mds_command: new command op to 5256 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 --> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] -- command(tid 0: {"prefix": "status"}) -- 0x5585384ae3a0 con 0x5585386a5d30 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a5d30 0x5585386a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cd52e2640 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a5d30 0x5585386a6110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fffd1d18854c44 server_cookie=17fdfacee6c2c719 in_seq=0 out_seq=0 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 10 client.5472 ms_handle_connect on v2:172.21.15.3:6834/653673671 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 1 -- 172.21.15.3:0/3144344979 <== mds.1 v2:172.21.15.3:6834/653673671 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5585384ae3a0 con 0x5585386a5d30 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 10 client.5472 handle_command_reply: tid=0 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 client.5472 shutdown 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 msgr2=0x55853870f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853870f320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7cc001c970 tx=0x7f7cc011dbb0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 shutdown_connections 2024-09-12T12:21:49.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55853869c440 0x5585386a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:6834/653673671,v1:172.21.15.3:6835/653673671] conn(0x5585386a5d30 0x5585386a6110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55853869bb20 0x55853870f320 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7caffff640 0 client.5472 ms_handle_reset on v2:172.21.15.3:6834/653673671 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 --2- 172.21.15.3:0/3144344979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x558538657330 0x55853870ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 >> 172.21.15.3:0/3144344979 conn(0x558538652290 msgr2=0x558538699c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 shutdown_connections 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 1 -- 172.21.15.3:0/3144344979 wait complete. 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.517+0000 7f7cda602000 20 client.5472 trim_cache size 0 max 16384 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:21:49.527 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5256, 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 2.724200427, 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 26, 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 43, 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 43, 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 24.324796972000001 2024-09-12T12:21:49.528 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7cb40678f0 msgr2=0x7f7cb4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7cb40678f0 0x7f7cb4069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7cd015f1c0 tx=0x7f7ccc008040 comp rx=0 tx=0).stop 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00d0380 msgr2=0x7f7cd015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00d0380 0x7f7cd015f3d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7cc001ab40 tx=0x7f7cc0019230 comp rx=0 tx=0).stop 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 shutdown_connections 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7cb40678f0 0x7f7cb4069db0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7cd01568f0 0x7f7cd015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd00d0380 0x7f7cd015f3d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 --2- 172.21.15.3:0/2172084572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7cd00b47a0 0x7f7cd0149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 >> 172.21.15.3:0/2172084572 conn(0x7f7cd006be90 msgr2=0x7f7cd006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 shutdown_connections 2024-09-12T12:21:49.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.521+0000 7f7cd7546640 1 -- 172.21.15.3:0/2172084572 wait complete. 2024-09-12T12:21:49.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-12T12:21:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.649+0000 7f4f25cbf640 1 Processor -- start 2024-09-12T12:21:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.649+0000 7f4f25cbf640 1 -- start start 2024-09-12T12:21:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.649+0000 7f4f25cbf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4f2015c2c0 0x7f4f2009b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.649+0000 7f4f25cbf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f200ce900 0x7f4f2009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.649+0000 7f4f25cbf640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f2009bc30 con 0x7f4f2009afb0 2024-09-12T12:21:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.649+0000 7f4f25cbf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4f200d09f0 con 0x7f4f2015c2c0 2024-09-12T12:21:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.649+0000 7f4f25cbf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2009b3e0 con 0x7f4f200ce900 2024-09-12T12:21:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4f2009afb0 0x7f4f2015ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54424/0 (socket says 172.21.15.3:54424) 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1ffff640 1 -- 172.21.15.3:0/256480177 learned_addr learned my addr 172.21.15.3:0/256480177 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1effd640 1 --2- 172.21.15.3:0/256480177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f200ce900 0x7f4f2009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/256480177 >> v2:172.21.15.142:3300/0 conn(0x7f4f2015c2c0 0x7f4f2009b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1e7fc640 1 -- 172.21.15.3:0/256480177 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948287505 0 0) 0x7f4f2009bc30 con 0x7f4f2009afb0 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1e7fc640 1 -- 172.21.15.3:0/256480177 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f08003530 con 0x7f4f2009afb0 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1effd640 1 -- 172.21.15.3:0/256480177 >> v2:172.21.15.142:3300/0 conn(0x7f4f2015c2c0 msgr2=0x7f4f2009b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1effd640 1 --2- 172.21.15.3:0/256480177 >> v2:172.21.15.142:3300/0 conn(0x7f4f2015c2c0 0x7f4f2009b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1e7fc640 1 -- 172.21.15.3:0/256480177 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1621059148 0 0) 0x7f4f08003530 con 0x7f4f2009afb0 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1effd640 1 -- 172.21.15.3:0/256480177 >> v1:172.21.15.3:6789/0 conn(0x7f4f2009afb0 legacy=0x7f4f2015ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1effd640 1 -- 172.21.15.3:0/256480177 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f200d0c40 con 0x7f4f200ce900 2024-09-12T12:21:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1effd640 1 --2- 172.21.15.3:0/256480177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f200ce900 0x7f4f2009aa70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f140097c0 tx=0x7f4f1401c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba7221724c1e0f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1e7fc640 1 -- 172.21.15.3:0/256480177 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140b0070 con 0x7f4f200ce900 2024-09-12T12:21:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1e7fc640 1 -- 172.21.15.3:0/256480177 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f140ab400 con 0x7f4f200ce900 2024-09-12T12:21:49.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f1e7fc640 1 -- 172.21.15.3:0/256480177 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18001680 con 0x7f4f200ce900 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.653+0000 7f4f25cbf640 1 -- 172.21.15.3:0/256480177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f200ce900 msgr2=0x7f4f2009aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/256480177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f200ce900 0x7f4f2009aa70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f140097c0 tx=0x7f4f1401c270 comp rx=0 tx=0).stop 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- 172.21.15.3:0/256480177 shutdown_connections 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/256480177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f200ce900 0x7f4f2009aa70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/256480177 >> v2:172.21.15.142:3300/0 conn(0x7f4f2015c2c0 0x7f4f2009b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- 172.21.15.3:0/256480177 >> 172.21.15.3:0/256480177 conn(0x7f4f200b3f00 msgr2=0x7f4f200b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- 172.21.15.3:0/256480177 shutdown_connections 2024-09-12T12:21:49.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- 172.21.15.3:0/256480177 wait complete. 2024-09-12T12:21:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 Processor -- start 2024-09-12T12:21:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- start start 2024-09-12T12:21:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f2009afb0 0x7f4f200c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200ce900 0x7f4f200c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f2015c2c0 0x7f4f20152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f200d1380 con 0x7f4f2009afb0 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4f200d1200 con 0x7f4f200ce900 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f200b1d70 con 0x7f4f2015c2c0 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f2009afb0 0x7f4f200c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f2009afb0 0x7f4f200c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41158/0 (socket says 172.21.15.3:41158) 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/49647307 learned_addr learned my addr 172.21.15.3:0/49647307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f2015c2c0 0x7f4f20152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f2015c2c0 msgr2=0x7f4f20152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1effd640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200ce900 0x7f4f200c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f2015c2c0 0x7f4f20152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200ce900 msgr2=0x7f4f200c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200ce900 0x7f4f200c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/49647307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f200c5fe0 con 0x7f4f2009afb0 2024-09-12T12:21:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f2015c2c0 0x7f4f20152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f2009afb0 0x7f4f200c8c50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000efc0 tx=0x7f4f1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f07fff640 1 -- 172.21.15.3:0/49647307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10018020 con 0x7f4f2009afb0 2024-09-12T12:21:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f07fff640 1 -- 172.21.15.3:0/49647307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f10014480 con 0x7f4f2009afb0 2024-09-12T12:21:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f07fff640 1 -- 172.21.15.3:0/49647307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10014850 con 0x7f4f2009afb0 2024-09-12T12:21:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f200c62c0 con 0x7f4f2009afb0 2024-09-12T12:21:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.657+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f200c6880 con 0x7f4f2009afb0 2024-09-12T12:21:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f07fff640 1 -- 172.21.15.3:0/49647307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4f1001b020 con 0x7f4f2009afb0 2024-09-12T12:21:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f07fff640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4f0c0678c0 0x7f4f0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f07fff640 1 -- 172.21.15.3:0/49647307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f4f10086a90 con 0x7f4f2009afb0 2024-09-12T12:21:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4f0c0678c0 0x7f4f0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 Processor -- start 2024-09-12T12:21:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 -- start start 2024-09-12T12:21:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4f0c0678c0 0x7f4f0c069d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4f140ad000 tx=0x7f4f140a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 0x557a92028f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fded50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a91fdf290 0x557a91fdf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557a91ec6b60 con 0x557a92029440 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557a91e2dc70 con 0x557a91fdf290 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557a91dd3ed0 con 0x557a92028b20 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fded50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41170/0 (socket says 172.21.15.3:41170) 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 -- 172.21.15.3:0/3593718655 learned_addr learned my addr 172.21.15.3:0/3593718655 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 -- 172.21.15.3:0/3593718655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 msgr2=0x557a92028f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 --2- 172.21.15.3:0/3593718655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 0x557a92028f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 -- 172.21.15.3:0/3593718655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a91fdf290 msgr2=0x557a91fdf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 --2- 172.21.15.3:0/3593718655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a91fdf290 0x557a91fdf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 -- 172.21.15.3:0/3593718655 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a91de5e50 con 0x557a92029440 2024-09-12T12:21:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f1effd640 1 --2- 172.21.15.3:0/3593718655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fded50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f14097c20 tx=0x7f4f14004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14300ed26d7ce74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f057fa640 1 -- 172.21.15.3:0/3593718655 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140c6070 con 0x557a92029440 2024-09-12T12:21:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f057fa640 1 -- 172.21.15.3:0/3593718655 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f140044e0 con 0x557a92029440 2024-09-12T12:21:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.661+0000 7f4f057fa640 1 -- 172.21.15.3:0/3593718655 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140b0040 con 0x557a92029440 2024-09-12T12:21:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/3593718655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 msgr2=0x557a91fded50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/3593718655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fded50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4f14097c20 tx=0x7f4f14004740 comp rx=0 tx=0).stop 2024-09-12T12:21:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/3593718655 shutdown_connections 2024-09-12T12:21:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/3593718655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a91fdf290 0x557a91fdf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/3593718655 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fded50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/3593718655 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 0x557a92028f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/3593718655 >> 172.21.15.3:0/3593718655 conn(0x557a91fe5600 msgr2=0x557a91fdd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/3593718655 shutdown_connections 2024-09-12T12:21:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/3593718655 wait complete. 2024-09-12T12:21:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- start start 2024-09-12T12:21:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 Processor -- start 2024-09-12T12:21:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 0x557a91fe3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fe9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a91fea230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557a91e8fb00 con 0x557a92029440 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557a91e08360 con 0x557a92028b20 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557a91e052d0 con 0x557a91fe4390 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 10 client.0 _ll_register_callbacks cb 0x557a91fca820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fe9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fe9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41174/0 (socket says 172.21.15.3:41174) 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 -- 172.21.15.3:0/1981613078 learned_addr learned my addr 172.21.15.3:0/1981613078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a91fea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 0x557a91fe3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 -- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 msgr2=0x557a91fea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a91fea230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 -- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 msgr2=0x557a91fe3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 0x557a91fe3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 -- 172.21.15.3:0/1981613078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a91dd4120 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a91fea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1effd640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fe9cf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4f1401cb10 tx=0x7f4f1401aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d20bb9b2b956f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4efe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1981613078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140d1060 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1981613078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f4f140c1530 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f140044e0 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1981613078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f140b0050 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 10 client.5484 fetch_fsmap learned FSMap version 26 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x557a91dcd450 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1981613078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f140044e0 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1981613078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 3522+0+0 (secure 0 0 0) 0x7f4f140c1770 con 0x557a92029440 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 10 client.5484 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 10 client.5484 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T12:21:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 20 client.5484 populate_metadata read hostname 'smithi003' 2024-09-12T12:21:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92037280 0x557a92039670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 4 client.5484 mds_command: new command op to 5262 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 --> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a91dc7c00 con 0x557a92037280 2024-09-12T12:21:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.665+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92037280 0x557a92039670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92037280 0x557a92039670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582934069189be68 server_cookie=756695b4ee44a5fd in_seq=0 out_seq=0 2024-09-12T12:21:49.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4efe7fc640 10 client.5484 ms_handle_connect on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1981613078 <== mds.0 v2:172.21.15.142:6835/2796023141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a91dc7c00 con 0x557a92037280 2024-09-12T12:21:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4efe7fc640 10 client.5484 handle_command_reply: tid=0 2024-09-12T12:21:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 client.5484 shutdown 2024-09-12T12:21:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 msgr2=0x557a91fe9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fe9cf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4f1401cb10 tx=0x7f4f1401aab0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 shutdown_connections 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92037280 0x557a92039670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a91fea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a91fe9cf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1981613078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 0x557a91fe3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4efe7fc640 0 client.5484 ms_handle_reset on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 >> 172.21.15.3:0/1981613078 conn(0x557a91fdf290 msgr2=0x557a91fddd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 shutdown_connections 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1981613078 wait complete. 2024-09-12T12:21:49.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.669+0000 7f4f28d7b000 20 client.5484 trim_cache size 0 max 16384 2024-09-12T12:21:49.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 Processor -- start 2024-09-12T12:21:49.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- start start 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a92027cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 0x557a92028200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a92039670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2006efb0 con 0x557a92029440 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4f200b5290 con 0x557a92028b20 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4f200b58e0 con 0x557a91fe4390 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a92039670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a92039670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41184/0 (socket says 172.21.15.3:41184) 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 -- 172.21.15.3:0/4153236982 learned_addr learned my addr 172.21.15.3:0/4153236982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 -- 172.21.15.3:0/4153236982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 msgr2=0x557a92027cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/4153236982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a92027cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 -- 172.21.15.3:0/4153236982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 msgr2=0x557a92028200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/4153236982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 0x557a92028200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 -- 172.21.15.3:0/4153236982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a91db4fd0 con 0x557a92029440 2024-09-12T12:21:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/4153236982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a92039670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4f18001e40 tx=0x7f4f18007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7effd831a197def8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f04ff9640 1 -- 172.21.15.3:0/4153236982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18005cf0 con 0x557a92029440 2024-09-12T12:21:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f04ff9640 1 -- 172.21.15.3:0/4153236982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f180070d0 con 0x557a92029440 2024-09-12T12:21:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f04ff9640 1 -- 172.21.15.3:0/4153236982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f180074c0 con 0x557a92029440 2024-09-12T12:21:49.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- 172.21.15.3:0/4153236982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 msgr2=0x557a92039670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/4153236982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a92039670 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4f18001e40 tx=0x7f4f18007d90 comp rx=0 tx=0).stop 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- 172.21.15.3:0/4153236982 shutdown_connections 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/4153236982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a92029440 0x557a92039670 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/4153236982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92028b20 0x557a92028200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/4153236982 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a91fe4390 0x557a92027cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- 172.21.15.3:0/4153236982 >> 172.21.15.3:0/4153236982 conn(0x557a91fdf290 msgr2=0x557a91fe3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- 172.21.15.3:0/4153236982 shutdown_connections 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- 172.21.15.3:0/4153236982 wait complete. 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 -- start start 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.689+0000 7f4f28d7b000 1 Processor -- start 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a91fe4390 0x557a9203c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 0x557a92060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92029440 0x557a920606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557a91e2cf80 con 0x557a91fe4390 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557a91de5ca0 con 0x557a92029440 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x557a91e322a0 con 0x557a92028b20 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 10 client.0 _ll_register_callbacks cb 0x7f4f140c4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a91fe4390 0x557a9203c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a91fe4390 0x557a9203c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41198/0 (socket says 172.21.15.3:41198) 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/1914685342 learned_addr learned my addr 172.21.15.3:0/1914685342 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92029440 0x557a920606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1effd640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 0x557a92060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 msgr2=0x557a92060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 0x557a92060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92029440 msgr2=0x557a920606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92029440 0x557a920606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 -- 172.21.15.3:0/1914685342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a91eb0750 con 0x557a91fe4390 2024-09-12T12:21:49.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1ffff640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92029440 0x557a920606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1f7fe640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a91fe4390 0x557a9203c590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f10056bd0 tx=0x7f4f10057030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0009f9f31df42af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1914685342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100551d0 con 0x557a91fe4390 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1914685342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f4f10024de0 con 0x557a91fe4390 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f180070d0 con 0x557a91fe4390 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1914685342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f10075400 con 0x557a91fe4390 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 10 client.5490 fetch_fsmap learned FSMap version 26 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x557a91dbba30 con 0x557a91fe4390 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1914685342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f180070d0 con 0x557a91fe4390 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1914685342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 26) ==== 3522+0+0 (secure 0 0 0) 0x7f4f10053540 con 0x557a91fe4390 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 10 client.5490 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 10 client.5490 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 20 client.5490 populate_metadata read hostname 'smithi003' 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92060bf0 0x557a920a4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 4 client.5490 mds_command: new command op to 5262 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:21:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 --> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] -- command(tid 0: {"prefix": "status"}) -- 0x557a91e3b3a0 con 0x557a92060bf0 2024-09-12T12:21:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1effd640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92060bf0 0x557a920a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 10 client.5490 ms_handle_connect on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f1effd640 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92060bf0 0x557a920a4410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ba705d5167bc2c server_cookie=df6e1a01ed260040 in_seq=0 out_seq=0 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 1 -- 172.21.15.3:0/1914685342 <== mds.0 v2:172.21.15.142:6835/2796023141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x557a91e3b3a0 con 0x557a92060bf0 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 10 client.5490 handle_command_reply: tid=0 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 client.5490 shutdown 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a91fe4390 msgr2=0x557a9203c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a91fe4390 0x557a9203c590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4f10056bd0 tx=0x7f4f10057030 comp rx=0 tx=0).stop 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 shutdown_connections 2024-09-12T12:21:49.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:6835/2796023141,v1:172.21.15.142:6837/2796023141] conn(0x557a92060bf0 0x557a920a4410 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a92029440 0x557a920606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x557a92028b20 0x557a92060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 --2- 172.21.15.3:0/1914685342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557a91fe4390 0x557a9203c590 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4efe7fc640 0 client.5490 ms_handle_reset on v2:172.21.15.142:6835/2796023141 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 >> 172.21.15.3:0/1914685342 conn(0x557a91fdf290 msgr2=0x557a92026cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 shutdown_connections 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 1 -- 172.21.15.3:0/1914685342 wait complete. 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f28d7b000 20 client.5490 trim_cache size 0 max 16384 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:21:49.701 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5262, 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 19.493879624000002, 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 26, 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 43, 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 43, 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 24.497078252000001 2024-09-12T12:21:49.702 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4f0c0678c0 msgr2=0x7f4f0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4f0c0678c0 0x7f4f0c069d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4f140ad000 tx=0x7f4f140a6000 comp rx=0 tx=0).stop 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f2009afb0 msgr2=0x7f4f200c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f2009afb0 0x7f4f200c8c50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000efc0 tx=0x7f4f1000c490 comp rx=0 tx=0).stop 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 shutdown_connections 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4f0c0678c0 0x7f4f0c069d80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f2015c2c0 0x7f4f20152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200ce900 0x7f4f200c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 --2- 172.21.15.3:0/49647307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f2009afb0 0x7f4f200c8c50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.693+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 >> 172.21.15.3:0/49647307 conn(0x7f4f200b3f00 msgr2=0x7f4f20071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.697+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 shutdown_connections 2024-09-12T12:21:49.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49.697+0000 7f4f25cbf640 1 -- 172.21.15.3:0/49647307 wait complete. 2024-09-12T12:21:49.713 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir {d0,d1} && touch {d0,d1}/file') 2024-09-12T12:21:49.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T12:21:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:21:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:21:49.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:49.784 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d0) 2024-09-12T12:21:49.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:21:49.848 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2024-09-12T12:21:49.902 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-12T12:21:50.048 INFO:teuthology.orchestra.run.smithi142.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"9","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-12T12:21:50.049 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "9" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-12T12:21:50.049 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-12T12:21:52.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-12T12:21:52.195 INFO:teuthology.orchestra.run.smithi142.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-12T12:21:52.196 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-12T12:21:52.196 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-12T12:21:54.197 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-12T12:21:54.344 INFO:teuthology.orchestra.run.smithi142.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-12T12:21:54.345 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-12T12:21:54.345 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d0', 0), ('/d1', 1)] 2024-09-12T12:21:56.346 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-12T12:21:56.492 INFO:teuthology.orchestra.run.smithi142.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"18","version":"14","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"5","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":1},"nref":4}}] 2024-09-12T12:21:56.493 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 5, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "18", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0", "pins": { "authpin": 1, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-12T12:21:56.493 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0)] =?= [('/d0', 0), ('/d1', 1)] 2024-09-12T12:21:58.494 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-12T12:21:58.640 INFO:teuthology.orchestra.run.smithi142.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"18","version":"14","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":1},"nref":5}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"5","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":1},"nref":4}}] 2024-09-12T12:21:58.640 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 2, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 5, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "18", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d0", "pins": { "authpin": 1, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-12T12:21:58.641 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0)] =?= [('/d0', 0), ('/d1', 1)] 2024-09-12T12:22:00.642 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-12T12:22:00.792 INFO:teuthology.orchestra.run.smithi142.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"18","version":"18","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d0","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-12T12:22:00.793 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "18", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "18" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d0", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "5", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000001", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "5", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-12T12:22:00.793 INFO:tasks.cephfs.cephfs_test_case:[('/d0', 0), ('/d1', 1)] =?= [('/d0', 0), ('/d1', 1)] 2024-09-12T12:22:00.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:00.793 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d0/f0) 2024-09-12T12:22:00.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:00.806 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch d1/f0) 2024-09-12T12:22:00.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:00.871 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch d0/f1) 2024-09-12T12:22:00.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:00.890 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch d1/f1) 2024-09-12T12:22:00.960 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-12T12:22:01.153 INFO:teuthology.orchestra.run.smithi142.stdout:[{"id":5059,"entity":{"name":{"type":"client","num":5059},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":134331345}},"state":"open","num_leases":1,"num_caps":4,"request_load_avg":6,"uptime":16.843822065000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":7.9071480267186605,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":9,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5059 192.168.0.1:0/134331345","completed_requests":[],"prealloc_inos":[{"start":"0x100000005e1","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"1","hostname":"smithi142","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"35682","root":"/"}},{"id":5424,"entity":{"name":{"type":"client","num":5424},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2872188260}},"state":"open","num_leases":3,"num_caps":5,"request_load_avg":11,"uptime":19.583791165000001,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":2,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.563416647061072,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":12,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000003","length":500}],"inst":"client.5424 192.168.0.1:0/2872188260","completed_requests":[{"tid":12,"created_ino":"0x100000001f8"}],"prealloc_inos":[{"start":"0x10000000003","length":500},{"start":"0x100000001f9","length":999}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"38387","root":"/"}}] 2024-09-12T12:22:01.154 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi142", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "35682", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 134331345, "type": "any" }, "name": { "num": 5059, "type": "client" } }, "id": 5059, "inst": "client.5059 192.168.0.1:0/134331345", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 9, "num_caps": 4, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x100000005e1" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 7.9071480267186605 }, "state": "open", "uptime": 16.843822065 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "38387", "root": "/" }, "completed_requests": [ { "created_ino": "0x100000001f8", "tid": 12 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000003" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2872188260, "type": "any" }, "name": { "num": 5424, "type": "client" } }, "id": 5424, "inst": "client.5424 192.168.0.1:0/2872188260", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 12, "num_caps": 5, "num_completed_flushes": 2, "num_completed_requests": 1, "num_leases": 3, "prealloc_inos": [ { "length": 500, "start": "0x10000000003" }, { "length": 999, "start": "0x100000001f9" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.56341664706107 }, "state": "open", "uptime": 19.583791165 } ] 2024-09-12T12:22:01.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T12:22:01.312 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":5059,"entity":{"name":{"type":"client","num":5059},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":134331345}},"state":"open","num_leases":1,"num_caps":2,"request_load_avg":2,"uptime":0.355997287,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":3.9968405214386959,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":9,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5059 192.168.0.1:0/134331345","completed_requests":[{"tid":10,"created_ino":"0x200000003e9"}],"prealloc_inos":[{"start":"0x200000003ea","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"1","hostname":"smithi142","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"35682","root":"/"}},{"id":5424,"entity":{"name":{"type":"client","num":5424},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":2872188260}},"state":"open","num_leases":1,"num_caps":3,"request_load_avg":2,"uptime":1.0919916810000001,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":4.993480391567096,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":12,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5424 192.168.0.1:0/2872188260","completed_requests":[{"tid":16,"created_ino":"0x20000000000"}],"prealloc_inos":[{"start":"0x20000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"38387","root":"/"}}] 2024-09-12T12:22:01.313 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi142", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "35682", "root": "/" }, "completed_requests": [ { "created_ino": "0x200000003e9", "tid": 10 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 134331345, "type": "any" }, "name": { "num": 5059, "type": "client" } }, "id": 5059, "inst": "client.5059 192.168.0.1:0/134331345", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 9, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x200000003ea" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 3.996840521438696 }, "state": "open", "uptime": 0.355997287 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "38387", "root": "/" }, "completed_requests": [ { "created_ino": "0x20000000000", "tid": 16 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2872188260, "type": "any" }, "name": { "num": 5424, "type": "client" } }, "id": 5424, "inst": "client.5424 192.168.0.1:0/2872188260", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 12, "num_caps": 3, "num_completed_flushes": 0, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x20000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 4.993480391567096 }, "state": "open", "uptime": 1.091991681 } ] 2024-09-12T12:22:01.313 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:22:01.313 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:22:01.313 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:22:01.313 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:22:01.313 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:22:01.313 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:22:01.314 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:22:01.315 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:22:01.315 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:22:01.315 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:22:01.315 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:01.315 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:22:01.315 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:22:01.399 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.38387.asok 2024-09-12T12:22:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:01.406 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38387.asok 2024-09-12T12:22:01.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:01.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38387.asok mds_sessions 2024-09-12T12:22:01.567 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:22:01.567 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5424, 2024-09-12T12:22:01.567 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5424 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2872188260 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5424 v1:192.168.0.1:0/2872188260", 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "v1:192.168.0.1:0/2872188260", 2024-09-12T12:22:01.568 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 0, 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.142:6835", 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2796023141 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.142:6837", 2024-09-12T12:22:01.569 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2796023141 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 11, 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2024-09-12T12:22:42.551514+0000", 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2024-09-12T12:21:42.551514+0000", 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 2, 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 5, 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:01.570 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": 1, 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "addrs": { 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6834", 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 653673671 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6835", 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 653673671 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:22:01.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "seq": 1, 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_gen": 0, 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_ttl": "2024-09-12T12:23:00.218306+0000", 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "last_cap_renew_request": "2024-09-12T12:22:00.218306+0000", 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_renew_seq": 1, 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 3, 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open" 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 27 2024-09-12T12:22:01.572 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:22:01.581 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session evict 5424 2024-09-12T12:22:01.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:22:01.865+0000 7fa4927fc640 -1 client.5424 I was blocklisted at osd epoch 44 2024-09-12T12:22:01.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:22:01.865+0000 7fa4927fc640 -1 client.5424 remove_session_caps still has dirty|flushing caps on 0x100000001f8.head(faked_ino=0 nref=5 ll_ref=1 cap_refs={} open={2=0} mode=100664 size=0/4194304 nlink=1 btime=2024-09-12T12:22:00.801268+0000 mtime=2024-09-12T12:22:00.801150+0000 ctime=2024-09-12T12:22:00.804515+0000 change_attr=1 caps=- dirty_caps=AxFx objectset[0x100000001f8 ts 0/0 objects 0 dirty_or_tx 0] parents=0x10000000000.head["f0"] 0x7fa48c014b90) 2024-09-12T12:22:01.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:22:01.865+0000 7fa4927fc640 -1 client.5424 remove_session_caps still has dirty|flushing caps on 0x20000000000.head(faked_ino=0 nref=5 ll_ref=1 cap_refs={} open={2=0} mode=100664 size=0/4194304 nlink=1 btime=2024-09-12T12:22:00.861356+0000 mtime=2024-09-12T12:22:00.865149+0000 ctime=2024-09-12T12:22:00.869654+0000 change_attr=1 caps=- dirty_caps=AxFx objectset[0x20000000000 ts 0/0 objects 0 dirty_or_tx 0] parents=0x10000000001.head["f0"] 0x7fa48c015170) 2024-09-12T12:22:01.973 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-09-12T12:22:01.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:22:02.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc64fac640 1 Processor -- start 2024-09-12T12:22:02.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc64fac640 1 -- start start 2024-09-12T12:22:02.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc64fac640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b42b0 0x7fdc60071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc64fac640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdc600b4bd0 0x7fdc600722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc64fac640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc6006dee0 con 0x7fdc6015e4b0 2024-09-12T12:22:02.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc64fac640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdc6006e810 con 0x7fdc600b4bd0 2024-09-12T12:22:02.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc64fac640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6006e1c0 con 0x7fdc600b42b0 2024-09-12T12:22:02.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc55d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b42b0 0x7fdc60071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc5ed9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdc600b4bd0 0x7fdc600722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc55d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b42b0 0x7fdc60071da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:33908/0 (socket says 172.21.15.3:33908) 2024-09-12T12:22:02.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc55d9b640 1 -- 172.21.15.3:0/3639808127 learned_addr learned my addr 172.21.15.3:0/3639808127 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:02.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc5e59c640 1 --1- 172.21.15.3:0/3639808127 >> v1:172.21.15.142:6789/0 conn(0x7fdc6015e4b0 0x7fdc600ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:58542/0 (socket says 172.21.15.3:58542) 2024-09-12T12:22:02.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc5ed9d640 1 -- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b42b0 msgr2=0x7fdc60071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc5ed9d640 1 --2- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b42b0 0x7fdc60071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc5ed9d640 1 -- 172.21.15.3:0/3639808127 >> v1:172.21.15.142:6789/0 conn(0x7fdc6015e4b0 legacy=0x7fdc600ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc5ed9d640 1 -- 172.21.15.3:0/3639808127 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6006e4a0 con 0x7fdc600b4bd0 2024-09-12T12:22:02.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.101+0000 7fdc55d9b640 1 --2- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b42b0 0x7fdc60071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:02.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc5ed9d640 1 --2- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc600b4bd0 0x7fdc600722e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009d90 tx=0x7fdc5801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61120d6c4c4ffa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc5dd9b640 1 -- 172.21.15.3:0/3639808127 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580b2070 con 0x7fdc600b4bd0 2024-09-12T12:22:02.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc5dd9b640 1 -- 172.21.15.3:0/3639808127 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc580a5820 con 0x7fdc600b4bd0 2024-09-12T12:22:02.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc5dd9b640 1 -- 172.21.15.3:0/3639808127 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580a5bf0 con 0x7fdc600b4bd0 2024-09-12T12:22:02.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc600b4bd0 msgr2=0x7fdc600722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 --2- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc600b4bd0 0x7fdc600722e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009d90 tx=0x7fdc5801c8e0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- 172.21.15.3:0/3639808127 shutdown_connections 2024-09-12T12:22:02.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 --2- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b42b0 0x7fdc60071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 --2- 172.21.15.3:0/3639808127 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc600b4bd0 0x7fdc600722e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- 172.21.15.3:0/3639808127 >> 172.21.15.3:0/3639808127 conn(0x7fdc600b1d80 msgr2=0x7fdc600b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- 172.21.15.3:0/3639808127 shutdown_connections 2024-09-12T12:22:02.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- 172.21.15.3:0/3639808127 wait complete. 2024-09-12T12:22:02.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 Processor -- start 2024-09-12T12:22:02.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- start start 2024-09-12T12:22:02.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b4bd0 0x7fdc601520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc6015e4b0 0x7fdc60155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc60157db0 0x7fdc6015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6006f610 con 0x7fdc60157db0 2024-09-12T12:22:02.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6006f490 con 0x7fdc6015e4b0 2024-09-12T12:22:02.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.105+0000 7fdc64fac640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6006f790 con 0x7fdc600b4bd0 2024-09-12T12:22:02.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc60157db0 0x7fdc6015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc60157db0 0x7fdc6015c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34200/0 (socket says 172.21.15.3:34200) 2024-09-12T12:22:02.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 -- 172.21.15.3:0/420596678 learned_addr learned my addr 172.21.15.3:0/420596678 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:02.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc55d9b640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc6015e4b0 0x7fdc60155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5e59c640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b4bd0 0x7fdc601520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 -- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b4bd0 msgr2=0x7fdc601520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b4bd0 0x7fdc601520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 -- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc6015e4b0 msgr2=0x7fdc60155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc6015e4b0 0x7fdc60155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 -- 172.21.15.3:0/420596678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601529b0 con 0x7fdc60157db0 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5e59c640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b4bd0 0x7fdc601520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5ed9d640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc60157db0 0x7fdc6015c240 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdc580b0040 tx=0x7fdc58019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc577fe640 1 -- 172.21.15.3:0/420596678 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580b2050 con 0x7fdc60157db0 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc577fe640 1 -- 172.21.15.3:0/420596678 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc580be070 con 0x7fdc60157db0 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc60156d10 con 0x7fdc60157db0 2024-09-12T12:22:02.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc577fe640 1 -- 172.21.15.3:0/420596678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580ad850 con 0x7fdc60157db0 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601572d0 con 0x7fdc60157db0 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc600ad060 con 0x7fdc60157db0 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc577fe640 1 -- 172.21.15.3:0/420596678 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc580ad9f0 con 0x7fdc60157db0 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc577fe640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc400678c0 0x7fdc40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc577fe640 1 -- 172.21.15.3:0/420596678 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7fdc58120e10 con 0x7fdc60157db0 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5e59c640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc400678c0 0x7fdc40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.109+0000 7fdc5e59c640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc400678c0 0x7fdc40069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000f70 tx=0x7fdc50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.113+0000 7fdc577fe640 1 -- 172.21.15.3:0/420596678 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdc58016560 con 0x7fdc60157db0 2024-09-12T12:22:02.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.285+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc600b47a0 con 0x7fdc60157db0 2024-09-12T12:22:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.285+0000 7fdc577fe640 1 -- 172.21.15.3:0/420596678 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16673 (secure 0 0 0) 0x7fdc580f1930 con 0x7fdc60157db0 2024-09-12T12:22:02.288 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:02.288 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":44,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:01.841006+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:21:27.233763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T12:21:28.546504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2872188260":"2024-09-12T13:22:01.717396+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc400678c0 msgr2=0x7fdc40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc400678c0 0x7fdc40069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000f70 tx=0x7fdc50009040 comp rx=0 tx=0).stop 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc60157db0 msgr2=0x7fdc6015c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc60157db0 0x7fdc6015c240 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdc580b0040 tx=0x7fdc58019680 comp rx=0 tx=0).stop 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 shutdown_connections 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc400678c0 0x7fdc40069d80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc60157db0 0x7fdc6015c240 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc6015e4b0 0x7fdc60155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 --2- 172.21.15.3:0/420596678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc600b4bd0 0x7fdc601520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 >> 172.21.15.3:0/420596678 conn(0x7fdc600b1d80 msgr2=0x7fdc6015eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 shutdown_connections 2024-09-12T12:22:02.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:02.289+0000 7fdc64fac640 1 -- 172.21.15.3:0/420596678 wait complete. 2024-09-12T12:22:02.308 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-12T12:22:12.309 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-12T12:22:12.458 INFO:teuthology.orchestra.run.smithi142.stdout:[{"id":5059,"entity":{"name":{"type":"client","num":5059},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":134331345}},"state":"open","num_leases":1,"num_caps":4,"request_load_avg":5,"uptime":28.151703523999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":8.6891309685423526,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":9,"last_trim_completed_flushes_tid":2,"delegated_inos":[],"inst":"client.5059 192.168.0.1:0/134331345","completed_requests":[],"prealloc_inos":[{"start":"0x100000005e1","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"1","hostname":"smithi142","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"35682","root":"/"}}] 2024-09-12T12:22:12.459 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi142", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "35682", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 134331345, "type": "any" }, "name": { "num": 5059, "type": "client" } }, "id": 5059, "inst": "client.5059 192.168.0.1:0/134331345", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 9, "num_caps": 4, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x100000005e1" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 8.689130968542353 }, "state": "open", "uptime": 28.151703524 } ] 2024-09-12T12:22:12.459 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T12:22:12.613 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":5059,"entity":{"name":{"type":"client","num":5059},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":134331345}},"state":"open","num_leases":1,"num_caps":2,"request_load_avg":2,"uptime":11.655910889999999,"requests_in_flight":0,"num_completed_requests":1,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":4.8793895166219219,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":9,"last_trim_completed_flushes_tid":3,"delegated_inos":[],"inst":"client.5059 192.168.0.1:0/134331345","completed_requests":[{"tid":10,"created_ino":"0x200000003e9"}],"prealloc_inos":[{"start":"0x200000003ea","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"1","hostname":"smithi142","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"35682","root":"/"}}] 2024-09-12T12:22:12.613 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi142", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "35682", "root": "/" }, "completed_requests": [ { "created_ino": "0x200000003e9", "tid": 10 } ], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 134331345, "type": "any" }, "name": { "num": 5059, "type": "client" } }, "id": 5059, "inst": "client.5059 192.168.0.1:0/134331345", "last_trim_completed_flushes_tid": 3, "last_trim_completed_requests_tid": 9, "num_caps": 2, "num_completed_flushes": 1, "num_completed_requests": 1, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x200000003ea" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 4.879389516621922 }, "state": "open", "uptime": 11.65591089 } ] 2024-09-12T12:22:12.613 INFO:tasks.cephfs.mount:Cleaning up killed connection on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:22:12.613 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:22:12.614 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:12.619 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28bd709640 1 Processor -- start 2024-09-12T12:22:12.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28bd709640 1 -- start start 2024-09-12T12:22:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28bd709640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28b81568f0 0x7f28b80aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28bd709640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f28b80b47a0 0x7f28b80b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28bd709640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28b80afb00 con 0x7f28b80d0380 2024-09-12T12:22:12.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28bd709640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f28b80b31b0 con 0x7f28b81568f0 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28bd709640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f28b80af540 con 0x7f28b80b47a0 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b6ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28b81568f0 0x7f28b80aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b6ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f28b81568f0 0x7f28b80aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34410/0 (socket says 172.21.15.3:34410) 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b6ffd640 1 -- 172.21.15.3:0/276195636 learned_addr learned my addr 172.21.15.3:0/276195636 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b67fc640 1 --2- 172.21.15.3:0/276195636 >> v2:172.21.15.142:3300/0 conn(0x7f28b80b47a0 0x7f28b80b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b6ffd640 1 -- 172.21.15.3:0/276195636 >> v2:172.21.15.142:3300/0 conn(0x7f28b80b47a0 msgr2=0x7f28b80b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b6ffd640 1 --2- 172.21.15.3:0/276195636 >> v2:172.21.15.142:3300/0 conn(0x7f28b80b47a0 0x7f28b80b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b6ffd640 1 -- 172.21.15.3:0/276195636 >> v1:172.21.15.142:6789/0 conn(0x7f28b80d0380 legacy=0x7f28b815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b6ffd640 1 -- 172.21.15.3:0/276195636 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b80af820 con 0x7f28b81568f0 2024-09-12T12:22:12.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.785+0000 7f28b67fc640 1 --2- 172.21.15.3:0/276195636 >> v2:172.21.15.142:3300/0 conn(0x7f28b80b47a0 0x7f28b80b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28b6ffd640 1 --2- 172.21.15.3:0/276195636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80aedf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28b80cfba0 tx=0x7f28b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461ad69cff95aee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28b5ffb640 1 -- 172.21.15.3:0/276195636 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00b0070 con 0x7f28b81568f0 2024-09-12T12:22:12.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28b5ffb640 1 -- 172.21.15.3:0/276195636 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b00ab600 con 0x7f28b81568f0 2024-09-12T12:22:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28b5ffb640 1 -- 172.21.15.3:0/276195636 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00ab9d0 con 0x7f28b81568f0 2024-09-12T12:22:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 -- 172.21.15.3:0/276195636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 msgr2=0x7f28b80aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 --2- 172.21.15.3:0/276195636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80aedf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28b80cfba0 tx=0x7f28b001c450 comp rx=0 tx=0).stop 2024-09-12T12:22:12.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 -- 172.21.15.3:0/276195636 shutdown_connections 2024-09-12T12:22:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 --2- 172.21.15.3:0/276195636 >> v2:172.21.15.142:3300/0 conn(0x7f28b80b47a0 0x7f28b80b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 --2- 172.21.15.3:0/276195636 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80aedf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 -- 172.21.15.3:0/276195636 >> 172.21.15.3:0/276195636 conn(0x7f28b806be90 msgr2=0x7f28b806c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 -- 172.21.15.3:0/276195636 shutdown_connections 2024-09-12T12:22:12.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 -- 172.21.15.3:0/276195636 wait complete. 2024-09-12T12:22:12.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 Processor -- start 2024-09-12T12:22:12.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.789+0000 7f28bd709640 1 -- start start 2024-09-12T12:22:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80b47a0 0x7f28b81599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80d0380 0x7f28b8165ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28b80d0bc0 con 0x7f28b81568f0 2024-09-12T12:22:12.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f28b815d3b0 con 0x7f28b80d0380 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f28b81696e0 con 0x7f28b80b47a0 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34420/0 (socket says 172.21.15.3:34420) 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 -- 172.21.15.3:0/1515635830 learned_addr learned my addr 172.21.15.3:0/1515635830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b67fc640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80d0380 0x7f28b8165ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b6ffd640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80b47a0 0x7f28b81599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 -- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80b47a0 msgr2=0x7f28b81599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80b47a0 0x7f28b81599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 -- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80d0380 msgr2=0x7f28b8165ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80d0380 0x7f28b8165ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 -- 172.21.15.3:0/1515635830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b80b1280 con 0x7f28b81568f0 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b67fc640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80d0380 0x7f28b8165ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:12.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b77fe640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80b0d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28a800b410 tx=0x7f28a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28af7fe640 1 -- 172.21.15.3:0/1515635830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8015020 con 0x7f28b81568f0 2024-09-12T12:22:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28af7fe640 1 -- 172.21.15.3:0/1515635830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a8011870 con 0x7f28b81568f0 2024-09-12T12:22:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b80b1520 con 0x7f28b81568f0 2024-09-12T12:22:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28af7fe640 1 -- 172.21.15.3:0/1515635830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8011c60 con 0x7f28b81568f0 2024-09-12T12:22:12.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b8072f50 con 0x7f28b81568f0 2024-09-12T12:22:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b80b4400 con 0x7f28b81568f0 2024-09-12T12:22:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28af7fe640 1 -- 172.21.15.3:0/1515635830 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f28a8018020 con 0x7f28b81568f0 2024-09-12T12:22:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28af7fe640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28af7fe640 1 -- 172.21.15.3:0/1515635830 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f28a8085b90 con 0x7f28b81568f0 2024-09-12T12:22:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.793+0000 7f28b6ffd640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:12.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.797+0000 7f28b6ffd640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28b00ad000 tx=0x7f28b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:12.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.797+0000 7f28af7fe640 1 -- 172.21.15.3:0/1515635830 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f28a801f020 con 0x7f28b81568f0 2024-09-12T12:22:12.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28b80b3950 con 0x7f28b81568f0 2024-09-12T12:22:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28af7fe640 1 -- 172.21.15.3:0/1515635830 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v44) ==== 80+0+57 (secure 0 0 0) 0x7f28a80172c0 con 0x7f28b81568f0 2024-09-12T12:22:12.979 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.1:0/2872188260 2024-09-12T13:22:01.717396+0000 2024-09-12T12:22:12.979 INFO:teuthology.orchestra.run.smithi003.stderr:listed 1 entries 2024-09-12T12:22:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 msgr2=0x7f2898069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28b00ad000 tx=0x7f28b0008040 comp rx=0 tx=0).stop 2024-09-12T12:22:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 msgr2=0x7f28b80b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80b0d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28a800b410 tx=0x7f28a800b8d0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 shutdown_connections 2024-09-12T12:22:12.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f28980678c0 0x7f2898069d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b81568f0 0x7f28b80b0d40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28b80d0380 0x7f28b8165ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 --2- 172.21.15.3:0/1515635830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28b80b47a0 0x7f28b81599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 >> 172.21.15.3:0/1515635830 conn(0x7f28b806be90 msgr2=0x7f28b8152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 shutdown_connections 2024-09-12T12:22:12.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:12.977+0000 7f28bd709640 1 -- 172.21.15.3:0/1515635830 wait complete. 2024-09-12T12:22:12.997 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2024-09-12T12:22:12.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:12.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:13.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:22:13.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:13.022 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:22:13.074 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:22:13.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 Processor -- start 2024-09-12T12:22:13.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 -- start start 2024-09-12T12:22:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0b1015b2d0 0x7f0b100b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0b1006f6e0 0x7f0b10072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:13.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b100cc300 0x7f0b10073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100ae920 con 0x7f0b1015b2d0 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1015f150 con 0x7f0b1006f6e0 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b15e79640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1015b6b0 con 0x7f0b100cc300 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0b1015b2d0 0x7f0b100b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0b1015b2d0 0x7f0b100b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34436/0 (socket says 172.21.15.3:34436) 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 -- 172.21.15.3:0/2467089711 learned_addr learned my addr 172.21.15.3:0/2467089711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0effd640 1 --2- 172.21.15.3:0/2467089711 >> v2:172.21.15.142:3300/0 conn(0x7f0b1006f6e0 0x7f0b10072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0ffff640 1 --2- 172.21.15.3:0/2467089711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b100cc300 0x7f0b10073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 -- 172.21.15.3:0/2467089711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b100cc300 msgr2=0x7f0b10073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 --2- 172.21.15.3:0/2467089711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b100cc300 0x7f0b10073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 -- 172.21.15.3:0/2467089711 >> v2:172.21.15.142:3300/0 conn(0x7f0b1006f6e0 msgr2=0x7f0b10072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 --2- 172.21.15.3:0/2467089711 >> v2:172.21.15.142:3300/0 conn(0x7f0b1006f6e0 0x7f0b10072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0f7fe640 1 -- 172.21.15.3:0/2467089711 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1015ee70 con 0x7f0b1015b2d0 2024-09-12T12:22:13.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0effd640 1 --2- 172.21.15.3:0/2467089711 >> v2:172.21.15.142:3300/0 conn(0x7f0b1006f6e0 0x7f0b10072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:13.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.237+0000 7f0b0ffff640 1 --2- 172.21.15.3:0/2467089711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b100cc300 0x7f0b10073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:13.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b0f7fe640 1 --2- 172.21.15.3:0/2467089711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b100b3080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0b100b52c0 tx=0x7f0b0001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20489c687d3cecb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:13.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b0e7fc640 1 -- 172.21.15.3:0/2467089711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000aa610 con 0x7f0b1015b2d0 2024-09-12T12:22:13.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b0e7fc640 1 -- 172.21.15.3:0/2467089711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b000af070 con 0x7f0b1015b2d0 2024-09-12T12:22:13.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b0e7fc640 1 -- 172.21.15.3:0/2467089711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000aa9d0 con 0x7f0b1015b2d0 2024-09-12T12:22:13.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 -- 172.21.15.3:0/2467089711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 msgr2=0x7f0b100b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:13.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 --2- 172.21.15.3:0/2467089711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b100b3080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0b100b52c0 tx=0x7f0b0001c4f0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 -- 172.21.15.3:0/2467089711 shutdown_connections 2024-09-12T12:22:13.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 --2- 172.21.15.3:0/2467089711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b100cc300 0x7f0b10073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 --2- 172.21.15.3:0/2467089711 >> v2:172.21.15.142:3300/0 conn(0x7f0b1006f6e0 0x7f0b10072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 --2- 172.21.15.3:0/2467089711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b100b3080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 -- 172.21.15.3:0/2467089711 >> 172.21.15.3:0/2467089711 conn(0x7f0b100b1d80 msgr2=0x7f0b100b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:13.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 -- 172.21.15.3:0/2467089711 shutdown_connections 2024-09-12T12:22:13.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.241+0000 7f0b15e79640 1 -- 172.21.15.3:0/2467089711 wait complete. 2024-09-12T12:22:13.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 Processor -- start 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 -- start start 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1006f6e0 0x7f0b10140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b100cc300 0x7f0b101563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b10141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100b35c0 con 0x7f0b1015b2d0 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1006d8d0 con 0x7f0b100cc300 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100b3740 con 0x7f0b1006f6e0 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b10141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b10141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34452/0 (socket says 172.21.15.3:34452) 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 -- 172.21.15.3:0/1449485236 learned_addr learned my addr 172.21.15.3:0/1449485236 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:13.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0effd640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b100cc300 0x7f0b101563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0f7fe640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1006f6e0 0x7f0b10140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 -- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1006f6e0 msgr2=0x7f0b10140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1006f6e0 0x7f0b10140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 -- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b100cc300 msgr2=0x7f0b101563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b100cc300 0x7f0b101563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 -- 172.21.15.3:0/1449485236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10140990 con 0x7f0b1015b2d0 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0effd640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b100cc300 0x7f0b101563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b0ffff640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b10141a30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800cf20 tx=0x7f0b0800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0af7fff640 1 -- 172.21.15.3:0/1449485236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0801a070 con 0x7f0b1015b2d0 2024-09-12T12:22:13.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0af7fff640 1 -- 172.21.15.3:0/1449485236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0800d840 con 0x7f0b1015b2d0 2024-09-12T12:22:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0af7fff640 1 -- 172.21.15.3:0/1449485236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0800dc30 con 0x7f0b1015b2d0 2024-09-12T12:22:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b10140c70 con 0x7f0b1015b2d0 2024-09-12T12:22:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b10141510 con 0x7f0b1015b2d0 2024-09-12T12:22:13.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b10141710 con 0x7f0b1015b2d0 2024-09-12T12:22:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.245+0000 7f0af7fff640 1 -- 172.21.15.3:0/1449485236 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0b0801d070 con 0x7f0b1015b2d0 2024-09-12T12:22:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.249+0000 7f0af7fff640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0afc0678c0 0x7f0afc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.249+0000 7f0af7fff640 1 -- 172.21.15.3:0/1449485236 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f0b08086570 con 0x7f0b1015b2d0 2024-09-12T12:22:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.249+0000 7f0b0f7fe640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0afc0678c0 0x7f0afc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:13.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.249+0000 7f0b0f7fe640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0afc0678c0 0x7f0afc069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0b000ae000 tx=0x7f0b00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.253+0000 7f0af7fff640 1 -- 172.21.15.3:0/1449485236 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0b10141710 con 0x7f0b1015b2d0 2024-09-12T12:22:13.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.449+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0b100b3bd0 con 0x7f0b1015b2d0 2024-09-12T12:22:13.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.449+0000 7f0af7fff640 1 -- 172.21.15.3:0/1449485236 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f0b08057080 con 0x7f0b1015b2d0 2024-09-12T12:22:13.452 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:22:13.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0afc0678c0 msgr2=0x7f0afc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:13.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0afc0678c0 0x7f0afc069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0b000ae000 tx=0x7f0b00008040 comp rx=0 tx=0).stop 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 msgr2=0x7f0b10141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b10141a30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800cf20 tx=0x7f0b0800a7d0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 shutdown_connections 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0afc0678c0 0x7f0afc069d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0b1015b2d0 0x7f0b10141a30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b100cc300 0x7f0b101563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 --2- 172.21.15.3:0/1449485236 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0b1006f6e0 0x7f0b10140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 >> 172.21.15.3:0/1449485236 conn(0x7f0b100b1d80 msgr2=0x7f0b10154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 shutdown_connections 2024-09-12T12:22:13.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13.453+0000 7f0b15e79640 1 -- 172.21.15.3:0/1449485236 wait complete. 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:22:13.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:22:13.471 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:22:13.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:13.471 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:22:13.477 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:22:13.477 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:22:13.477 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:22:13.477 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.477 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:22:13.477 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42353sec preferred_lft 42353sec 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:22:13.478 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.479 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:13.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:13.481 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:22:13.534 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:22:13.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:13.536 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:22:13.584 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:22:13.584 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:22:13.584 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:22:13.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:22:13.584 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:22:13.584 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:22:13.585 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:22:13.585 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:22:13.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:22:13.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:22:13.585 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:22:13.585 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:22:13.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:22:13.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:13.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:13.707 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:22:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:22:13.712 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:22:13.712 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:22:13.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:13.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:22:13.766 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:22:13.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:13.767 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:13.818 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:22:13.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:13.819 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:13.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:22:13.928 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:13.975 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:22:13.975 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:13.976 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:22:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:13.978 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:13.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:13.979 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:22:14.026 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:22:14.026 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:22:14.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:22:14.087 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:14.135 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:14.136 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:22:14.137 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:22:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:22:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:22:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:14.138 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:14.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:14.139 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:22:14.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:22:14.153+0000 7f332b401580 -1 init, newargv = 0x5628007d9fe0 newargc=15 2024-09-12T12:22:14.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[38798]: starting ceph client 2024-09-12T12:22:14.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[38798]: starting fuse 2024-09-12T12:22:15.259 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:22:15.277 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:15.323 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:22:15.323 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:22:15.324 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:22:15.325 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:15.326 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:15.326 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:15.326 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:15.326 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:15.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:15.327 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:22:15.375 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:22:15.375 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:22:15.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:22:15.375 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:22:15.375 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:22:15.375 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:22:15.375 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:22:15.375 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:15.375 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:22:15.376 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:22:15.377 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:22:15.377 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:22:15.377 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:15.377 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:22:15.377 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:22:15.502 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.38798.asok 2024-09-12T12:22:15.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:15.515 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38798.asok 2024-09-12T12:22:15.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:15.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38798.asok status 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "38798", 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:22:15.675 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5511, 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5511 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3907197934 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:15.676 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3907197934 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5511 192.168.0.1:0/3907197934", 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3907197934", 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 27, 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 44, 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:22:15.677 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:22:15.678 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:22:15.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:15.689 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:22:15.689 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:15.695 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:22:15.696 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:15.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:15.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:15.752 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:22:15.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:22:15.827 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:22:15.874 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:22:15.874 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:22:15.874 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:22:15.874 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:15.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:16.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f956074f640 1 Processor -- start 2024-09-12T12:22:16.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f956074f640 1 -- start start 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f956074f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f955815c2c0 0x7f955809b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f956074f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f955809b180 0x7f95580cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f956074f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955809bc30 con 0x7f95580ce900 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f956074f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f95580d09f0 con 0x7f955815c2c0 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f956074f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f95580d0c40 con 0x7f955809b180 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f955e4eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f955815c2c0 0x7f955809b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f955dcea640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f95580ce900 0x7f955809aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40752/0 (socket says 172.21.15.3:40752) 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f955e4eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f955815c2c0 0x7f955809b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34468/0 (socket says 172.21.15.3:34468) 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f955dcea640 1 -- 172.21.15.3:0/2182469535 learned_addr learned my addr 172.21.15.3:0/2182469535 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:16.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.053+0000 7f955ecec640 1 --2- 172.21.15.3:0/2182469535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f955809b180 0x7f95580cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955e4eb640 1 -- 172.21.15.3:0/2182469535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f955809b180 msgr2=0x7f95580cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955e4eb640 1 --2- 172.21.15.3:0/2182469535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f955809b180 0x7f95580cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955e4eb640 1 -- 172.21.15.3:0/2182469535 >> v1:172.21.15.3:6789/0 conn(0x7f95580ce900 legacy=0x7f955809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955d4e9640 1 -- 172.21.15.3:0/2182469535 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947514740 0 0) 0x7f955809bc30 con 0x7f95580ce900 2024-09-12T12:22:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955e4eb640 1 -- 172.21.15.3:0/2182469535 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95580d0f20 con 0x7f955815c2c0 2024-09-12T12:22:16.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955ecec640 1 --2- 172.21.15.3:0/2182469535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f955809b180 0x7f95580cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955e4eb640 1 --2- 172.21.15.3:0/2182469535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955815c2c0 0x7f955809b6f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9550009d90 tx=0x7f955001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4127cc8e35e271 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955d4e9640 1 -- 172.21.15.3:0/2182469535 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500b2070 con 0x7f955815c2c0 2024-09-12T12:22:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955d4e9640 1 -- 172.21.15.3:0/2182469535 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95500a5900 con 0x7f955815c2c0 2024-09-12T12:22:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f955d4e9640 1 -- 172.21.15.3:0/2182469535 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500a5cd0 con 0x7f955815c2c0 2024-09-12T12:22:16.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 -- 172.21.15.3:0/2182469535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955815c2c0 msgr2=0x7f955809b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 --2- 172.21.15.3:0/2182469535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955815c2c0 0x7f955809b6f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9550009d90 tx=0x7f955001c820 comp rx=0 tx=0).stop 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 -- 172.21.15.3:0/2182469535 shutdown_connections 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 --2- 172.21.15.3:0/2182469535 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f955809b180 0x7f95580cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 --2- 172.21.15.3:0/2182469535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955815c2c0 0x7f955809b6f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 -- 172.21.15.3:0/2182469535 >> 172.21.15.3:0/2182469535 conn(0x7f95580b3f00 msgr2=0x7f95580b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 -- 172.21.15.3:0/2182469535 shutdown_connections 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.057+0000 7f956074f640 1 -- 172.21.15.3:0/2182469535 wait complete. 2024-09-12T12:22:16.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 Processor -- start 2024-09-12T12:22:16.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 -- start start 2024-09-12T12:22:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955809b180 0x7f955806e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95580ce900 0x7f95581667e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f955815c2c0 0x7f95580b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f95580b1ef0 con 0x7f955809b180 2024-09-12T12:22:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95580b1d70 con 0x7f955815c2c0 2024-09-12T12:22:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9558169240 con 0x7f95580ce900 2024-09-12T12:22:16.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955809b180 0x7f955806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955809b180 0x7f955806e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34470/0 (socket says 172.21.15.3:34470) 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 -- 172.21.15.3:0/1506606109 learned_addr learned my addr 172.21.15.3:0/1506606109 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955dcea640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95580ce900 0x7f95581667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955ecec640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f955815c2c0 0x7f95580b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 -- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95580ce900 msgr2=0x7f95581667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95580ce900 0x7f95581667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 -- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f955815c2c0 msgr2=0x7f95580b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f955815c2c0 0x7f95580b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 -- 172.21.15.3:0/1506606109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955815cc50 con 0x7f955809b180 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955ecec640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f955815c2c0 0x7f95580b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f955e4eb640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955809b180 0x7f955806e980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f95500b0040 tx=0x7f95500195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f954effd640 1 -- 172.21.15.3:0/1506606109 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500b2050 con 0x7f955809b180 2024-09-12T12:22:16.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f954effd640 1 -- 172.21.15.3:0/1506606109 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95500be070 con 0x7f955809b180 2024-09-12T12:22:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955815cf30 con 0x7f955809b180 2024-09-12T12:22:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f954effd640 1 -- 172.21.15.3:0/1506606109 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500adb10 con 0x7f955809b180 2024-09-12T12:22:16.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9558166d20 con 0x7f955809b180 2024-09-12T12:22:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95580b8390 con 0x7f955809b180 2024-09-12T12:22:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.061+0000 7f954effd640 1 -- 172.21.15.3:0/1506606109 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95500adcb0 con 0x7f955809b180 2024-09-12T12:22:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.065+0000 7f954effd640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f953c0678c0 0x7f953c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.065+0000 7f954effd640 1 -- 172.21.15.3:0/1506606109 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f9550121040 con 0x7f955809b180 2024-09-12T12:22:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.065+0000 7f955dcea640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f953c0678c0 0x7f953c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.065+0000 7f955dcea640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f953c0678c0 0x7f953c069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f95580b38a0 tx=0x7f9548007d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.069+0000 7f954effd640 1 -- 172.21.15.3:0/1506606109 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9550016560 con 0x7f955809b180 2024-09-12T12:22:16.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f95580af310 con 0x7f955809b180 2024-09-12T12:22:16.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f954effd640 1 -- 172.21.15.3:0/1506606109 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v44) ==== 80+0+57 (secure 0 0 0) 0x7f95500f1b60 con 0x7f955809b180 2024-09-12T12:22:16.247 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.1:0/2872188260 2024-09-12T13:22:01.717396+0000 2024-09-12T12:22:16.247 INFO:teuthology.orchestra.run.smithi003.stderr:listed 1 entries 2024-09-12T12:22:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f953c0678c0 msgr2=0x7f953c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f953c0678c0 0x7f953c069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f95580b38a0 tx=0x7f9548007d60 comp rx=0 tx=0).stop 2024-09-12T12:22:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955809b180 msgr2=0x7f955806e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955809b180 0x7f955806e980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f95500b0040 tx=0x7f95500195c0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 shutdown_connections 2024-09-12T12:22:16.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f953c0678c0 0x7f953c069d80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f955815c2c0 0x7f95580b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95580ce900 0x7f95581667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 --2- 172.21.15.3:0/1506606109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f955809b180 0x7f955806e980 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 >> 172.21.15.3:0/1506606109 conn(0x7f95580b3f00 msgr2=0x7f95580cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 shutdown_connections 2024-09-12T12:22:16.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.245+0000 7f956074f640 1 -- 172.21.15.3:0/1506606109 wait complete. 2024-09-12T12:22:16.266 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:22:16.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:16.267 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:16.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[38798]: fuse finished with error 0 and tester_r 0 2024-09-12T12:22:16.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:22:16.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:16.280 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:22:16.346 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:22:16.346 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:16.346 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:16.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:16.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd5010a640 1 Processor -- start 2024-09-12T12:22:16.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd5010a640 1 -- start start 2024-09-12T12:22:16.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd5010a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd480b5fc0 0x7efd480d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd5010a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd4815f620 0x7efd480b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd5010a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd480b68e0 con 0x7efd48162110 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd5010a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7efd480d2d90 con 0x7efd4815f620 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd5010a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efd480d3070 con 0x7efd480b5fc0 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4d6a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd4815f620 0x7efd480b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4d6a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efd4815f620 0x7efd480b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34474/0 (socket says 172.21.15.3:34474) 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4d6a5640 1 -- 172.21.15.3:0/2721330718 learned_addr learned my addr 172.21.15.3:0/2721330718 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4dea6640 1 --2- 172.21.15.3:0/2721330718 >> v2:172.21.15.142:3300/0 conn(0x7efd480b5fc0 0x7efd480d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4d6a5640 1 -- 172.21.15.3:0/2721330718 >> v2:172.21.15.142:3300/0 conn(0x7efd480b5fc0 msgr2=0x7efd480d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4d6a5640 1 --2- 172.21.15.3:0/2721330718 >> v2:172.21.15.142:3300/0 conn(0x7efd480b5fc0 0x7efd480d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4d6a5640 1 -- 172.21.15.3:0/2721330718 >> v1:172.21.15.142:6789/0 conn(0x7efd48162110 legacy=0x7efd48162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.517+0000 7efd4d6a5640 1 -- 172.21.15.3:0/2721330718 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd480d3350 con 0x7efd4815f620 2024-09-12T12:22:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd4d6a5640 1 --2- 172.21.15.3:0/2721330718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd480b63a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efd4808de20 tx=0x7efd4401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522f771e0bae23a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd4cea4640 1 -- 172.21.15.3:0/2721330718 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440a5840 con 0x7efd4815f620 2024-09-12T12:22:16.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd4cea4640 1 -- 172.21.15.3:0/2721330718 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd440a59e0 con 0x7efd4815f620 2024-09-12T12:22:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd4cea4640 1 -- 172.21.15.3:0/2721330718 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440a5db0 con 0x7efd4815f620 2024-09-12T12:22:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 -- 172.21.15.3:0/2721330718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 msgr2=0x7efd480b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 --2- 172.21.15.3:0/2721330718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd480b63a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efd4808de20 tx=0x7efd4401c980 comp rx=0 tx=0).stop 2024-09-12T12:22:16.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 -- 172.21.15.3:0/2721330718 shutdown_connections 2024-09-12T12:22:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 --2- 172.21.15.3:0/2721330718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd480b63a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 --2- 172.21.15.3:0/2721330718 >> v2:172.21.15.142:3300/0 conn(0x7efd480b5fc0 0x7efd480d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 -- 172.21.15.3:0/2721330718 >> 172.21.15.3:0/2721330718 conn(0x7efd4808c000 msgr2=0x7efd480d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 -- 172.21.15.3:0/2721330718 shutdown_connections 2024-09-12T12:22:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 -- 172.21.15.3:0/2721330718 wait complete. 2024-09-12T12:22:16.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 Processor -- start 2024-09-12T12:22:16.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 -- start start 2024-09-12T12:22:16.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd480b5fc0 0x7efd480b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd481fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.521+0000 7efd5010a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd48162110 0x7efd481fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd5010a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efd4815ea30 con 0x7efd4815f620 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd5010a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd4815e8b0 con 0x7efd48162110 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd5010a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efd4816bc60 con 0x7efd480b5fc0 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd481fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd481fb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34484/0 (socket says 172.21.15.3:34484) 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 -- 172.21.15.3:0/3258015094 learned_addr learned my addr 172.21.15.3:0/3258015094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4dea6640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd480b5fc0 0x7efd480b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4e6a7640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd48162110 0x7efd481fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 -- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd480b5fc0 msgr2=0x7efd480b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd480b5fc0 0x7efd480b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 -- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd48162110 msgr2=0x7efd481fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd48162110 0x7efd481fbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 -- 172.21.15.3:0/3258015094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd480b76c0 con 0x7efd4815f620 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4e6a7640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd48162110 0x7efd481fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4d6a5640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd481fb780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efd4401b030 tx=0x7efd44019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd3a7fc640 1 -- 172.21.15.3:0/3258015094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440b8670 con 0x7efd4815f620 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd3a7fc640 1 -- 172.21.15.3:0/3258015094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd440be070 con 0x7efd4815f620 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd480b79a0 con 0x7efd4815f620 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd3a7fc640 1 -- 172.21.15.3:0/3258015094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440b8a90 con 0x7efd4815f620 2024-09-12T12:22:16.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd480b7f60 con 0x7efd4815f620 2024-09-12T12:22:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd480d26f0 con 0x7efd4815f620 2024-09-12T12:22:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd3a7fc640 1 -- 172.21.15.3:0/3258015094 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efd440b8c30 con 0x7efd4815f620 2024-09-12T12:22:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd3a7fc640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd2c0678c0 0x7efd2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd3a7fc640 1 -- 172.21.15.3:0/3258015094 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7efd44121b10 con 0x7efd4815f620 2024-09-12T12:22:16.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.525+0000 7efd4dea6640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd2c0678c0 0x7efd2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.529+0000 7efd4dea6640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd2c0678c0 0x7efd2c069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00b9d0 tx=0x7efd3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.529+0000 7efd3a7fc640 1 -- 172.21.15.3:0/3258015094 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efd440165a0 con 0x7efd4815f620 2024-09-12T12:22:16.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:22:16.639 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:22:16.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.709+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd480d42b0 con 0x7efd4815f620 2024-09-12T12:22:16.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.709+0000 7efd3a7fc640 1 -- 172.21.15.3:0/3258015094 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v44) ==== 80+0+57 (secure 0 0 0) 0x7efd440f2820 con 0x7efd4815f620 2024-09-12T12:22:16.713 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.1:0/2872188260 2024-09-12T13:22:01.717396+0000 2024-09-12T12:22:16.714 INFO:teuthology.orchestra.run.smithi003.stderr:listed 1 entries 2024-09-12T12:22:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd2c0678c0 msgr2=0x7efd2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd2c0678c0 0x7efd2c069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00b9d0 tx=0x7efd3c008040 comp rx=0 tx=0).stop 2024-09-12T12:22:16.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 msgr2=0x7efd481fb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd481fb780 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7efd4401b030 tx=0x7efd44019720 comp rx=0 tx=0).stop 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 shutdown_connections 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd2c0678c0 0x7efd2c069d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd48162110 0x7efd481fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd4815f620 0x7efd481fb780 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 --2- 172.21.15.3:0/3258015094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd480b5fc0 0x7efd480b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 >> 172.21.15.3:0/3258015094 conn(0x7efd4808c000 msgr2=0x7efd480b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 shutdown_connections 2024-09-12T12:22:16.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:16.713+0000 7efd5010a640 1 -- 172.21.15.3:0/3258015094 wait complete. 2024-09-12T12:22:16.732 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:22:16.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:16.732 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:16.744 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35682]: fuse finished with error 0 and tester_r 0 2024-09-12T12:22:16.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:22:16.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:16.746 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:22:16.914 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:22:16.914 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:22:16.915 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:16.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:22:16.922 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:22:16.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:16.922 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T12:22:16.979 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-12T12:22:16.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:22:16.980 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:22:16.980 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:16.980 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:16.987 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:22:16.987 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:22:16.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:16.987 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T12:22:17.043 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-12T12:22:17.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:22:17.045 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:22:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e96d50640 1 Processor -- start 2024-09-12T12:22:17.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e96d50640 1 -- start start 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e96d50640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e90159fb0 0x7f0e9006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e96d50640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9006fbb0 con 0x7f0e9015d2d0 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e96d50640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9015d760 con 0x7f0e90159bd0 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e96d50640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9006f010 con 0x7f0e90159fb0 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e8ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e90159fb0 0x7f0e9006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e8ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0e90159fb0 0x7f0e9006dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34494/0 (socket says 172.21.15.3:34494) 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2846558716 learned_addr learned my addr 172.21.15.3:0/2846558716 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:17.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e952ed640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0e9015d2d0 0x7f0e900ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40762/0 (socket says 172.21.15.3:40762) 2024-09-12T12:22:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2846558716 >> v1:172.21.15.142:6789/0 conn(0x7f0e90159bd0 legacy=0x7f0e9006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2846558716 >> v1:172.21.15.3:6789/0 conn(0x7f0e9015d2d0 legacy=0x7f0e900ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.181+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2846558716 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9006e400 con 0x7f0e90159fb0 2024-09-12T12:22:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e8ffff640 1 --2- 172.21.15.3:0/2846558716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e9006dd40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006f190 tx=0x7f0e7801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7713ff13ab0a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e8f7fe640 1 -- 172.21.15.3:0/2846558716 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780a5860 con 0x7f0e90159fb0 2024-09-12T12:22:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e8f7fe640 1 -- 172.21.15.3:0/2846558716 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780a5a00 con 0x7f0e90159fb0 2024-09-12T12:22:17.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e8f7fe640 1 -- 172.21.15.3:0/2846558716 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780a5dd0 con 0x7f0e90159fb0 2024-09-12T12:22:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e96d50640 1 -- 172.21.15.3:0/2846558716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 msgr2=0x7f0e9006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2846558716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e9006dd40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006f190 tx=0x7f0e7801c9a0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e96d50640 1 -- 172.21.15.3:0/2846558716 shutdown_connections 2024-09-12T12:22:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2846558716 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e9006dd40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e96d50640 1 -- 172.21.15.3:0/2846558716 >> 172.21.15.3:0/2846558716 conn(0x7f0e900b1d80 msgr2=0x7f0e900b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e96d50640 1 -- 172.21.15.3:0/2846558716 shutdown_connections 2024-09-12T12:22:17.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.185+0000 7f0e96d50640 1 -- 172.21.15.3:0/2846558716 wait complete. 2024-09-12T12:22:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 Processor -- start 2024-09-12T12:22:17.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 -- start start 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e90159bd0 0x7f0e901507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e90073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e9015d2d0 0x7f0e90165360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e900b3a10 con 0x7f0e90159fb0 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0e900b3890 con 0x7f0e90159bd0 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0e900ad770 con 0x7f0e9015d2d0 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e90073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e90073610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34502/0 (socket says 172.21.15.3:34502) 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2516390361 learned_addr learned my addr 172.21.15.3:0/2516390361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e94aec640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e90159bd0 0x7f0e901507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e9015d2d0 msgr2=0x7f0e90165360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e9015d2d0 0x7f0e90165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e90159bd0 msgr2=0x7f0e901507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e90159bd0 0x7f0e901507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 -- 172.21.15.3:0/2516390361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e90167f10 con 0x7f0e90159fb0 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8ffff640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e90073610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e780af040 tx=0x7f0e78019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8d7fa640 1 -- 172.21.15.3:0/2516390361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780b8880 con 0x7f0e90159fb0 2024-09-12T12:22:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8d7fa640 1 -- 172.21.15.3:0/2516390361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780be070 con 0x7f0e90159fb0 2024-09-12T12:22:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8d7fa640 1 -- 172.21.15.3:0/2516390361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780b8ca0 con 0x7f0e90159fb0 2024-09-12T12:22:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e90072850 con 0x7f0e90159fb0 2024-09-12T12:22:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e90072e10 con 0x7f0e90159fb0 2024-09-12T12:22:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9006dd40 con 0x7f0e90159fb0 2024-09-12T12:22:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.189+0000 7f0e8d7fa640 1 -- 172.21.15.3:0/2516390361 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0e780c3460 con 0x7f0e90159fb0 2024-09-12T12:22:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.193+0000 7f0e8d7fa640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e700678c0 0x7f0e70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.193+0000 7f0e8d7fa640 1 -- 172.21.15.3:0/2516390361 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f0e781211d0 con 0x7f0e90159fb0 2024-09-12T12:22:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.193+0000 7f0e94aec640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e700678c0 0x7f0e70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.193+0000 7f0e94aec640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e700678c0 0x7f0e70069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e80004e30 tx=0x7f0e80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.197+0000 7f0e8d7fa640 1 -- 172.21.15.3:0/2516390361 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0e780165c0 con 0x7f0e90159fb0 2024-09-12T12:22:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.393+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e900b2b00 con 0x7f0e90159fb0 2024-09-12T12:22:17.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.393+0000 7f0e8d7fa640 1 -- 172.21.15.3:0/2516390361 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5240 (secure 0 0 0) 0x7f0e780f1e00 con 0x7f0e90159fb0 2024-09-12T12:22:17.396 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:17.396 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-12T12:21:55:820828+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:55.203279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5262,"mds_1":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5262,5256]},"id":3}]} 2024-09-12T12:22:17.396 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-12T12:22:17.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e700678c0 msgr2=0x7f0e70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e700678c0 0x7f0e70069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0e80004e30 tx=0x7f0e80009040 comp rx=0 tx=0).stop 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 msgr2=0x7f0e90073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e90073610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0e780af040 tx=0x7f0e78019740 comp rx=0 tx=0).stop 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 shutdown_connections 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0e700678c0 0x7f0e70069d80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0e9015d2d0 0x7f0e90165360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e90159fb0 0x7f0e90073610 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 --2- 172.21.15.3:0/2516390361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e90159bd0 0x7f0e901507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 >> 172.21.15.3:0/2516390361 conn(0x7f0e900b1d80 msgr2=0x7f0e9015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 shutdown_connections 2024-09-12T12:22:17.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.397+0000 7f0e96d50640 1 -- 172.21.15.3:0/2516390361 wait complete. 2024-09-12T12:22:17.415 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:22:17.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a673e8640 1 Processor -- start 2024-09-12T12:22:17.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a673e8640 1 -- start start 2024-09-12T12:22:17.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a673e8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a60159fb0 0x7f5a6006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a673e8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a6015d2d0 0x7f5a600ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a673e8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6006fbb0 con 0x7f5a60159bd0 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a673e8640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6006f010 con 0x7f5a6015d2d0 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a673e8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5a6015d760 con 0x7f5a60159fb0 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a65985640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a6015d2d0 0x7f5a600ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a64983640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a60159fb0 0x7f5a6006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a65985640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5a6015d2d0 0x7f5a600ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34506/0 (socket says 172.21.15.3:34506) 2024-09-12T12:22:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a65985640 1 -- 172.21.15.3:0/1990197445 learned_addr learned my addr 172.21.15.3:0/1990197445 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a65985640 1 -- 172.21.15.3:0/1990197445 >> v2:172.21.15.142:3300/0 conn(0x7f5a60159fb0 msgr2=0x7f5a6006dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a65985640 1 --2- 172.21.15.3:0/1990197445 >> v2:172.21.15.142:3300/0 conn(0x7f5a60159fb0 0x7f5a6006dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a65985640 1 -- 172.21.15.3:0/1990197445 >> v1:172.21.15.142:6789/0 conn(0x7f5a60159bd0 legacy=0x7f5a6006f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.541+0000 7f5a65985640 1 -- 172.21.15.3:0/1990197445 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6006e400 con 0x7f5a6015d2d0 2024-09-12T12:22:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a65985640 1 --2- 172.21.15.3:0/1990197445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6015d2d0 0x7f5a600ad060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5a6015c850 tx=0x7f5a5801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a42e712f3963324 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a57fff640 1 -- 172.21.15.3:0/1990197445 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a580a5790 con 0x7f5a6015d2d0 2024-09-12T12:22:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a57fff640 1 -- 172.21.15.3:0/1990197445 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a580a5930 con 0x7f5a6015d2d0 2024-09-12T12:22:17.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a57fff640 1 -- 172.21.15.3:0/1990197445 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a580a5d00 con 0x7f5a6015d2d0 2024-09-12T12:22:17.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 -- 172.21.15.3:0/1990197445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6015d2d0 msgr2=0x7f5a600ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 --2- 172.21.15.3:0/1990197445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6015d2d0 0x7f5a600ad060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5a6015c850 tx=0x7f5a5801c9c0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 -- 172.21.15.3:0/1990197445 shutdown_connections 2024-09-12T12:22:17.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 --2- 172.21.15.3:0/1990197445 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6015d2d0 0x7f5a600ad060 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 --2- 172.21.15.3:0/1990197445 >> v2:172.21.15.142:3300/0 conn(0x7f5a60159fb0 0x7f5a6006dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 -- 172.21.15.3:0/1990197445 >> 172.21.15.3:0/1990197445 conn(0x7f5a600b1d80 msgr2=0x7f5a600b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:17.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 -- 172.21.15.3:0/1990197445 shutdown_connections 2024-09-12T12:22:17.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 -- 172.21.15.3:0/1990197445 wait complete. 2024-09-12T12:22:17.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 Processor -- start 2024-09-12T12:22:17.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.545+0000 7f5a673e8640 1 -- start start 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a60159bd0 0x7f5a60158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a60159fb0 0x7f5a60151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a6015d2d0 0x7f5a60072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5a60169360 con 0x7f5a60159bd0 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a601691e0 con 0x7f5a60159fb0 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5a601694e0 con 0x7f5a6015d2d0 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a60159bd0 0x7f5a60158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a60159bd0 0x7f5a60158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34520/0 (socket says 172.21.15.3:34520) 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 -- 172.21.15.3:0/321577681 learned_addr learned my addr 172.21.15.3:0/321577681 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a64983640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a60159fb0 0x7f5a60151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65985640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a6015d2d0 0x7f5a60072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 -- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a6015d2d0 msgr2=0x7f5a60072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a6015d2d0 0x7f5a60072860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 -- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a60159fb0 msgr2=0x7f5a60151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a60159fb0 0x7f5a60151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 -- 172.21.15.3:0/321577681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a60151d00 con 0x7f5a60159bd0 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a64983640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a60159fb0 0x7f5a60151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a65184640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a60159bd0 0x7f5a60158cd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c450 tx=0x7f5a5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a55ffb640 1 -- 172.21.15.3:0/321577681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c016020 con 0x7f5a60159bd0 2024-09-12T12:22:17.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a55ffb640 1 -- 172.21.15.3:0/321577681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0052d0 con 0x7f5a60159bd0 2024-09-12T12:22:17.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a60151fe0 con 0x7f5a60159bd0 2024-09-12T12:22:17.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a55ffb640 1 -- 172.21.15.3:0/321577681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c007260 con 0x7f5a60159bd0 2024-09-12T12:22:17.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a601525a0 con 0x7f5a60159bd0 2024-09-12T12:22:17.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a600b2b00 con 0x7f5a60159bd0 2024-09-12T12:22:17.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a55ffb640 1 -- 172.21.15.3:0/321577681 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a5c007400 con 0x7f5a60159bd0 2024-09-12T12:22:17.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a55ffb640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a440678c0 0x7f5a44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a55ffb640 1 -- 172.21.15.3:0/321577681 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f5a5c0855f0 con 0x7f5a60159bd0 2024-09-12T12:22:17.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.549+0000 7f5a64983640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a440678c0 0x7f5a44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.553+0000 7f5a64983640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a440678c0 0x7f5a44069d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5a6006f830 tx=0x7f5a50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.553+0000 7f5a55ffb640 1 -- 172.21.15.3:0/321577681 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5a5c020020 con 0x7f5a60159bd0 2024-09-12T12:22:17.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.749+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a60156f40 con 0x7f5a60159bd0 2024-09-12T12:22:17.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a55ffb640 1 -- 172.21.15.3:0/321577681 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5240 (secure 0 0 0) 0x7f5a5c005480 con 0x7f5a60159bd0 2024-09-12T12:22:17.755 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:17.755 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-12T12:21:55:820828+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:55.203279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5262,"mds_1":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5262,5256]},"id":3}]} 2024-09-12T12:22:17.755 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-12T12:22:17.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a440678c0 msgr2=0x7f5a44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a440678c0 0x7f5a44069d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5a6006f830 tx=0x7f5a50008040 comp rx=0 tx=0).stop 2024-09-12T12:22:17.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a60159bd0 msgr2=0x7f5a60158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a60159bd0 0x7f5a60158cd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00c450 tx=0x7f5a5c00c910 comp rx=0 tx=0).stop 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 shutdown_connections 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5a440678c0 0x7f5a44069d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a6015d2d0 0x7f5a60072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a60159fb0 0x7f5a60151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 --2- 172.21.15.3:0/321577681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a60159bd0 0x7f5a60158cd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 >> 172.21.15.3:0/321577681 conn(0x7f5a600b1d80 msgr2=0x7f5a6015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 shutdown_connections 2024-09-12T12:22:17.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.753+0000 7f5a673e8640 1 -- 172.21.15.3:0/321577681 wait complete. 2024-09-12T12:22:17.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:22:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc63c21c640 1 Processor -- start 2024-09-12T12:22:17.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc63c21c640 1 -- start start 2024-09-12T12:22:17.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc63c21c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc634144110 0x7fc63415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc63c21c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 0x7fc6340727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc63c21c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc634071ed0 con 0x7fc63406f6b0 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc63c21c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc634072490 con 0x7fc634144110 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc63c21c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6340b45a0 con 0x7fc63406dd40 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc639fb8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc634144110 0x7fc63415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc639fb8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc634144110 0x7fc63415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34530/0 (socket says 172.21.15.3:34530) 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc639fb8640 1 -- 172.21.15.3:0/472236292 learned_addr learned my addr 172.21.15.3:0/472236292 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc639fb8640 1 -- 172.21.15.3:0/472236292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 msgr2=0x7fc6340727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc639fb8640 1 --2- 172.21.15.3:0/472236292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 0x7fc6340727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc639fb8640 1 -- 172.21.15.3:0/472236292 >> v1:172.21.15.142:6789/0 conn(0x7fc63406f6b0 legacy=0x7fc6340b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:17.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.881+0000 7fc639fb8640 1 -- 172.21.15.3:0/472236292 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6340721b0 con 0x7fc634144110 2024-09-12T12:22:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc639fb8640 1 --2- 172.21.15.3:0/472236292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63415ac30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc630009870 tx=0x7fc63001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3eba5a177ff767 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc638fb6640 1 -- 172.21.15.3:0/472236292 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6300b2070 con 0x7fc634144110 2024-09-12T12:22:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc638fb6640 1 -- 172.21.15.3:0/472236292 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6300ad640 con 0x7fc634144110 2024-09-12T12:22:17.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc638fb6640 1 -- 172.21.15.3:0/472236292 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6300ada10 con 0x7fc634144110 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- 172.21.15.3:0/472236292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 msgr2=0x7fc63415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 --2- 172.21.15.3:0/472236292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63415ac30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc630009870 tx=0x7fc63001c3d0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- 172.21.15.3:0/472236292 shutdown_connections 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 --2- 172.21.15.3:0/472236292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 0x7fc6340727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 --2- 172.21.15.3:0/472236292 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63415ac30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- 172.21.15.3:0/472236292 >> 172.21.15.3:0/472236292 conn(0x7fc6340b1d80 msgr2=0x7fc6340b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- 172.21.15.3:0/472236292 shutdown_connections 2024-09-12T12:22:17.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- 172.21.15.3:0/472236292 wait complete. 2024-09-12T12:22:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 Processor -- start 2024-09-12T12:22:17.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- start start 2024-09-12T12:22:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 0x7fc634158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc63406f6b0 0x7fc63414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340ae0e0 con 0x7fc634144110 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340adf60 con 0x7fc63406f6b0 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63c21c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc634073100 con 0x7fc63406dd40 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63414e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34536/0 (socket says 172.21.15.3:34536) 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 -- 172.21.15.3:0/2654000198 learned_addr learned my addr 172.21.15.3:0/2654000198 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc639fb8640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 0x7fc634158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc6397b7640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc63406f6b0 0x7fc63414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 -- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 msgr2=0x7fc634158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 0x7fc634158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 -- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc63406f6b0 msgr2=0x7fc63414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc63406f6b0 0x7fc63414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 -- 172.21.15.3:0/2654000198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63414eea0 con 0x7fc634144110 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc6397b7640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc63406f6b0 0x7fc63414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.885+0000 7fc63a7b9640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63414e810 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b970 tx=0x7fc62c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc62a7fc640 1 -- 172.21.15.3:0/2654000198 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c019070 con 0x7fc634144110 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc62a7fc640 1 -- 172.21.15.3:0/2654000198 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62c00cb60 con 0x7fc634144110 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63414f180 con 0x7fc634144110 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc62a7fc640 1 -- 172.21.15.3:0/2654000198 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c014440 con 0x7fc634144110 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63415b5d0 con 0x7fc634144110 2024-09-12T12:22:17.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63415b7d0 con 0x7fc634144110 2024-09-12T12:22:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc62a7fc640 1 -- 172.21.15.3:0/2654000198 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc62c01c070 con 0x7fc634144110 2024-09-12T12:22:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc62a7fc640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6180678c0 0x7fc618069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:17.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc62a7fc640 1 -- 172.21.15.3:0/2654000198 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7fc62c085ca0 con 0x7fc634144110 2024-09-12T12:22:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc639fb8640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6180678c0 0x7fc618069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:17.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.889+0000 7fc639fb8640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6180678c0 0x7fc618069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6300af000 tx=0x7fc630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:17.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:17.893+0000 7fc62a7fc640 1 -- 172.21.15.3:0/2654000198 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc63415b7d0 con 0x7fc634144110 2024-09-12T12:22:18.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.053+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6341502f0 con 0x7fc634144110 2024-09-12T12:22:18.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.057+0000 7fc62a7fc640 1 -- 172.21.15.3:0/2654000198 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16673 (secure 0 0 0) 0x7fc62c023020 con 0x7fc634144110 2024-09-12T12:22:18.060 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:18.060 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":44,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:01.841006+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:21:27.233763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T12:21:28.546504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2872188260":"2024-09-12T13:22:01.717396+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:22:18.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6180678c0 msgr2=0x7fc618069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6180678c0 0x7fc618069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6300af000 tx=0x7fc630008040 comp rx=0 tx=0).stop 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 msgr2=0x7fc63414e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63414e810 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b970 tx=0x7fc62c00be30 comp rx=0 tx=0).stop 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 shutdown_connections 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6180678c0 0x7fc618069d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc634144110 0x7fc63414e810 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc63406f6b0 0x7fc63414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 --2- 172.21.15.3:0/2654000198 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc63406dd40 0x7fc634158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 >> 172.21.15.3:0/2654000198 conn(0x7fc6340b1d80 msgr2=0x7fc6340b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 shutdown_connections 2024-09-12T12:22:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.061+0000 7fc63c21c640 1 -- 172.21.15.3:0/2654000198 wait complete. 2024-09-12T12:22:18.081 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:22:18.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:22:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f584b2ac640 1 Processor -- start 2024-09-12T12:22:18.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f584b2ac640 1 -- start start 2024-09-12T12:22:18.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f584b2ac640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f584415e5d0 0x7f58440ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f584b2ac640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f58440d5050 0x7f58440cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f584b2ac640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58440cfe20 con 0x7f58440cfa10 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f584b2ac640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f58440b1060 con 0x7f58440d5050 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f584b2ac640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f58440d31a0 con 0x7f584415e5d0 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5849849640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f58440cfa10 0x7f5844162280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39222/0 (socket says 172.21.15.3:39222) 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5849849640 1 -- 172.21.15.3:0/2563615011 learned_addr learned my addr 172.21.15.3:0/2563615011 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5848847640 1 --2- 172.21.15.3:0/2563615011 >> v2:172.21.15.3:3300/0 conn(0x7f58440d5050 0x7f58440cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5849048640 1 --2- 172.21.15.3:0/2563615011 >> v2:172.21.15.142:3300/0 conn(0x7f584415e5d0 0x7f58440ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5848847640 1 -- 172.21.15.3:0/2563615011 >> v2:172.21.15.142:3300/0 conn(0x7f584415e5d0 msgr2=0x7f58440ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5848847640 1 --2- 172.21.15.3:0/2563615011 >> v2:172.21.15.142:3300/0 conn(0x7f584415e5d0 0x7f58440ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5848847640 1 -- 172.21.15.3:0/2563615011 >> v1:172.21.15.3:6789/0 conn(0x7f58440cfa10 legacy=0x7f5844162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.201+0000 7f5848847640 1 -- 172.21.15.3:0/2563615011 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58440d3480 con 0x7f58440d5050 2024-09-12T12:22:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f5848847640 1 --2- 172.21.15.3:0/2563615011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440d5050 0x7f58440cf4d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5844163340 tx=0x7f583401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417e5baa176fc6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f583bfff640 1 -- 172.21.15.3:0/2563615011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58340b1070 con 0x7f58440d5050 2024-09-12T12:22:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f583bfff640 1 -- 172.21.15.3:0/2563615011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58340ac630 con 0x7f58440d5050 2024-09-12T12:22:18.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f583bfff640 1 -- 172.21.15.3:0/2563615011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58340a8050 con 0x7f58440d5050 2024-09-12T12:22:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- 172.21.15.3:0/2563615011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440d5050 msgr2=0x7f58440cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2563615011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440d5050 0x7f58440cf4d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5844163340 tx=0x7f583401c3f0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- 172.21.15.3:0/2563615011 shutdown_connections 2024-09-12T12:22:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2563615011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440d5050 0x7f58440cf4d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2563615011 >> v2:172.21.15.142:3300/0 conn(0x7f584415e5d0 0x7f58440ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- 172.21.15.3:0/2563615011 >> 172.21.15.3:0/2563615011 conn(0x7f584408c000 msgr2=0x7f58440d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- 172.21.15.3:0/2563615011 shutdown_connections 2024-09-12T12:22:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- 172.21.15.3:0/2563615011 wait complete. 2024-09-12T12:22:18.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 Processor -- start 2024-09-12T12:22:18.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- start start 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440cfa10 0x7f58440cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58440d5050 0x7f5844151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584415e5d0 0x7f584415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f584416b5c0 con 0x7f58440cfa10 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f584415ebc0 con 0x7f58440d5050 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f584b2ac640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f584416b740 con 0x7f584415e5d0 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f5849048640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440cfa10 0x7f58440cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f5849048640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440cfa10 0x7f58440cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47806/0 (socket says 172.21.15.3:47806) 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f5849048640 1 -- 172.21.15.3:0/2274228372 learned_addr learned my addr 172.21.15.3:0/2274228372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.205+0000 7f5848847640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58440d5050 0x7f5844151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849849640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584415e5d0 0x7f584415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849048640 1 -- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584415e5d0 msgr2=0x7f584415ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849048640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584415e5d0 0x7f584415ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849048640 1 -- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58440d5050 msgr2=0x7f5844151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849048640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58440d5050 0x7f5844151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849048640 1 -- 172.21.15.3:0/2274228372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584415c2a0 con 0x7f58440cfa10 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5848847640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58440d5050 0x7f5844151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849849640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584415e5d0 0x7f584415ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5849048640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440cfa10 0x7f58440cacf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f584000d830 tx=0x7f584000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5839ffb640 1 -- 172.21.15.3:0/2274228372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584000b6c0 con 0x7f58440cfa10 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5839ffb640 1 -- 172.21.15.3:0/2274228372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5840019070 con 0x7f58440cfa10 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584415c540 con 0x7f58440cfa10 2024-09-12T12:22:18.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5839ffb640 1 -- 172.21.15.3:0/2274228372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584000ba60 con 0x7f58440cfa10 2024-09-12T12:22:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584415cde0 con 0x7f58440cfa10 2024-09-12T12:22:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58440cf4d0 con 0x7f58440cfa10 2024-09-12T12:22:18.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5839ffb640 1 -- 172.21.15.3:0/2274228372 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f584001e460 con 0x7f58440cfa10 2024-09-12T12:22:18.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5839ffb640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f58280678f0 0x7f5828069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5839ffb640 1 -- 172.21.15.3:0/2274228372 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f58400862d0 con 0x7f58440cfa10 2024-09-12T12:22:18.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5848847640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f58280678f0 0x7f5828069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.209+0000 7f5848847640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f58280678f0 0x7f5828069db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58340ae000 tx=0x7f5834008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.213+0000 7f5839ffb640 1 -- 172.21.15.3:0/2274228372 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5840056e70 con 0x7f58440cfa10 2024-09-12T12:22:18.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.413+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58440ba490 con 0x7f58440cfa10 2024-09-12T12:22:18.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.413+0000 7f5839ffb640 1 -- 172.21.15.3:0/2274228372 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5240 (secure 0 0 0) 0x7f5840056770 con 0x7f58440cfa10 2024-09-12T12:22:18.416 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:18.416 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-12T12:21:55:820828+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:55.203279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5262,"mds_1":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5262,5256]},"id":3}]} 2024-09-12T12:22:18.417 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f58280678f0 msgr2=0x7f5828069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f58280678f0 0x7f5828069db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f58340ae000 tx=0x7f5834008040 comp rx=0 tx=0).stop 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440cfa10 msgr2=0x7f58440cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440cfa10 0x7f58440cacf0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f584000d830 tx=0x7f584000dcf0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 shutdown_connections 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f58280678f0 0x7f5828069db0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f584415e5d0 0x7f584415ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58440d5050 0x7f5844151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 --2- 172.21.15.3:0/2274228372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58440cfa10 0x7f58440cacf0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 >> 172.21.15.3:0/2274228372 conn(0x7f584408c000 msgr2=0x7f58440b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 shutdown_connections 2024-09-12T12:22:18.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.417+0000 7f584b2ac640 1 -- 172.21.15.3:0/2274228372 wait complete. 2024-09-12T12:22:18.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:22:18.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.557+0000 7f08e0b12640 1 Processor -- start 2024-09-12T12:22:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.557+0000 7f08e0b12640 1 -- start start 2024-09-12T12:22:18.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.557+0000 7f08e0b12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.557+0000 7f08e0b12640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f08dc0ce980 0x7f08dc0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.557+0000 7f08e0b12640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08dc0b4a00 con 0x7f08dc070220 2024-09-12T12:22:18.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.557+0000 7f08e0b12640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f08dc06ed00 con 0x7f08dc0ce980 2024-09-12T12:22:18.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.557+0000 7f08e0b12640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f08dc15b800 con 0x7f08dc0b4540 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d1d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08dad9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f08dc0ce980 0x7f08dc0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08da59c640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f08dc070220 0x7f08dc070600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:50268/0 (socket says 172.21.15.3:50268) 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08da59c640 1 -- 172.21.15.3:0/2967974567 learned_addr learned my addr 172.21.15.3:0/2967974567 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d1d9b640 1 -- 172.21.15.3:0/2967974567 >> v2:172.21.15.142:3300/0 conn(0x7f08dc0ce980 msgr2=0x7f08dc0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d1d9b640 1 --2- 172.21.15.3:0/2967974567 >> v2:172.21.15.142:3300/0 conn(0x7f08dc0ce980 0x7f08dc0cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d1d9b640 1 -- 172.21.15.3:0/2967974567 >> v1:172.21.15.142:6789/0 conn(0x7f08dc070220 legacy=0x7f08dc070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d1d9b640 1 -- 172.21.15.3:0/2967974567 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc15bae0 con 0x7f08dc0b4540 2024-09-12T12:22:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d1d9b640 1 --2- 172.21.15.3:0/2967974567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc06e7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08c40097c0 tx=0x7f08c401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4544a8a6a8274147 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d9d9b640 1 -- 172.21.15.3:0/2967974567 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c40b0070 con 0x7f08dc0b4540 2024-09-12T12:22:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d9d9b640 1 -- 172.21.15.3:0/2967974567 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c40ab600 con 0x7f08dc0b4540 2024-09-12T12:22:18.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.561+0000 7f08d9d9b640 1 -- 172.21.15.3:0/2967974567 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c40ab9d0 con 0x7f08dc0b4540 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- 172.21.15.3:0/2967974567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 msgr2=0x7f08dc06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 --2- 172.21.15.3:0/2967974567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc06e7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f08c40097c0 tx=0x7f08c401c380 comp rx=0 tx=0).stop 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- 172.21.15.3:0/2967974567 shutdown_connections 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 --2- 172.21.15.3:0/2967974567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc06e7c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 --2- 172.21.15.3:0/2967974567 >> v2:172.21.15.142:3300/0 conn(0x7f08dc0ce980 0x7f08dc0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- 172.21.15.3:0/2967974567 >> 172.21.15.3:0/2967974567 conn(0x7f08dc0cd750 msgr2=0x7f08dc0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- 172.21.15.3:0/2967974567 shutdown_connections 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- 172.21.15.3:0/2967974567 wait complete. 2024-09-12T12:22:18.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 Processor -- start 2024-09-12T12:22:18.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- start start 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08dc070220 0x7f08dc14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc14a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08dc0ce980 0x7f08dc154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc0b0020 con 0x7f08dc070220 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc0afea0 con 0x7f08dc0ce980 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f08dc167d30 con 0x7f08dc0b4540 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08dc070220 0x7f08dc14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08dc070220 0x7f08dc14f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47808/0 (socket says 172.21.15.3:47808) 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 -- 172.21.15.3:0/1498692906 learned_addr learned my addr 172.21.15.3:0/1498692906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08d1d9b640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc14a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08dad9d640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08dc0ce980 0x7f08dc154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 -- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 msgr2=0x7f08dc14a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc14a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 -- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08dc0ce980 msgr2=0x7f08dc154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08dc0ce980 0x7f08dc154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 -- 172.21.15.3:0/1498692906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc14aa60 con 0x7f08dc070220 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08d1d9b640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc14a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08da59c640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08dc070220 0x7f08dc14f6e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c370 tx=0x7f08cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08d37fe640 1 -- 172.21.15.3:0/1498692906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc017070 con 0x7f08dc070220 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08d37fe640 1 -- 172.21.15.3:0/1498692906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc005cf0 con 0x7f08dc070220 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc14fe70 con 0x7f08dc070220 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08d37fe640 1 -- 172.21.15.3:0/1498692906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0072f0 con 0x7f08dc070220 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.565+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc0ad980 con 0x7f08dc070220 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.569+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc06e7c0 con 0x7f08dc070220 2024-09-12T12:22:18.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.569+0000 7f08d37fe640 1 -- 172.21.15.3:0/1498692906 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f08cc01a070 con 0x7f08dc070220 2024-09-12T12:22:18.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.569+0000 7f08d37fe640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f08bc0678f0 0x7f08bc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.569+0000 7f08d37fe640 1 -- 172.21.15.3:0/1498692906 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f08cc084fd0 con 0x7f08dc070220 2024-09-12T12:22:18.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.569+0000 7f08d1d9b640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f08bc0678f0 0x7f08bc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.569+0000 7f08d1d9b640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f08bc0678f0 0x7f08bc069db0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08c40ad000 tx=0x7f08c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.573+0000 7f08d37fe640 1 -- 172.21.15.3:0/1498692906 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f08dc14aa60 con 0x7f08dc070220 2024-09-12T12:22:18.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.769+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08dc0b5ad0 con 0x7f08dc070220 2024-09-12T12:22:18.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08d37fe640 1 -- 172.21.15.3:0/1498692906 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5240 (secure 0 0 0) 0x7f08cc055ae0 con 0x7f08dc070220 2024-09-12T12:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:18.775 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"btime":"2024-09-12T12:21:55:820828+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:21:29.563260+0000","modified":"2024-09-12T12:21:55.203279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5262,"mds_1":5256},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5256":{"gid":5256,"name":"a","rank":1,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6835/653673671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":653673671},{"type":"v1","addr":"172.21.15.3:6835","nonce":653673671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5262":{"gid":5262,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.142:6837/2796023141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2796023141},{"type":"v1","addr":"172.21.15.142:6837","nonce":2796023141}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5262,"qdb_cluster":[5262,5256]},"id":3}]} 2024-09-12T12:22:18.775 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 27 2024-09-12T12:22:18.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f08bc0678f0 msgr2=0x7f08bc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f08bc0678f0 0x7f08bc069db0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08c40ad000 tx=0x7f08c4008040 comp rx=0 tx=0).stop 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08dc070220 msgr2=0x7f08dc14f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08dc070220 0x7f08dc14f6e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c370 tx=0x7f08cc00c830 comp rx=0 tx=0).stop 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 shutdown_connections 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f08bc0678f0 0x7f08bc069db0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08dc0ce980 0x7f08dc154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f08dc0b4540 0x7f08dc14a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 --2- 172.21.15.3:0/1498692906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08dc070220 0x7f08dc14f6e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 >> 172.21.15.3:0/1498692906 conn(0x7f08dc0cd750 msgr2=0x7f08dc15ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 shutdown_connections 2024-09-12T12:22:18.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.773+0000 7f08e0b12640 1 -- 172.21.15.3:0/1498692906 wait complete. 2024-09-12T12:22:18.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:22:18.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.917+0000 7fea56cc0640 1 Processor -- start 2024-09-12T12:22:18.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.917+0000 7fea56cc0640 1 -- start start 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea56cc0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fea5015e5d0 0x7fea500ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea56cc0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 0x7fea500b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea56cc0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea500d31a0 con 0x7fea500d5050 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea56cc0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fea500b1060 con 0x7fea5015e5d0 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea56cc0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea500d3480 con 0x7fea500cfbe0 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fea500d5050 0x7fea500cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39232/0 (socket says 172.21.15.3:39232) 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4ffff640 1 -- 172.21.15.3:0/3740971588 learned_addr learned my addr 172.21.15.3:0/3740971588 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375056648 0 0) 0x7fea500d31a0 con 0x7fea500d5050 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea38003530 con 0x7fea500d5050 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 --2- 172.21.15.3:0/3740971588 >> v2:172.21.15.142:3300/0 conn(0x7fea5015e5d0 0x7fea500ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea5525d640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 0x7fea500b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4045046302 0 0) 0x7fea38003530 con 0x7fea500d5050 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 msgr2=0x7fea500b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 0x7fea500b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 >> v2:172.21.15.142:3300/0 conn(0x7fea5015e5d0 msgr2=0x7fea500ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 --2- 172.21.15.3:0/3740971588 >> v2:172.21.15.142:3300/0 conn(0x7fea5015e5d0 0x7fea500ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea500d3760 con 0x7fea500d5050 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fea44001680 con 0x7fea500d5050 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea5525d640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 0x7fea500b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 --2- 172.21.15.3:0/3740971588 >> v2:172.21.15.142:3300/0 conn(0x7fea5015e5d0 0x7fea500ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 >> v1:172.21.15.3:6789/0 conn(0x7fea500d5050 legacy=0x7fea500cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea380039c0 0x7fea38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 0x7fea38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea38004d50 0x7fea380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea500b1060 con 0x7fea380039c0 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea500d3480 con 0x7fea38004d50 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea3800bb00 con 0x7fea500cfbe0 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea380039c0 0x7fea38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea4ffff640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 0x7fea38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea5525d640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea38004d50 0x7fea380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 -- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 msgr2=0x7fea38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea500cfbe0 0x7fea38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 -- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea38004d50 msgr2=0x7fea380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea38004d50 0x7fea380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea54a5c640 1 -- 172.21.15.3:0/3740971588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea38003530 con 0x7fea380039c0 2024-09-12T12:22:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.921+0000 7fea5525d640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea38004d50 0x7fea380051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea54a5c640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea380039c0 0x7fea38003de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea400098d0 tx=0x7fea4001c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9e7cd77db63521 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea400ac070 con 0x7fea380039c0 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea400a6660 con 0x7fea380039c0 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4f7fe640 1 -- 172.21.15.3:0/3740971588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4000f040 con 0x7fea380039c0 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- 172.21.15.3:0/3740971588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea380039c0 msgr2=0x7fea38003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 --2- 172.21.15.3:0/3740971588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea380039c0 0x7fea38003de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fea400098d0 tx=0x7fea4001c650 comp rx=0 tx=0).stop 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea5525d640 1 -- 172.21.15.3:0/3740971588 reap_dead start 2024-09-12T12:22:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- 172.21.15.3:0/3740971588 shutdown_connections 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- 172.21.15.3:0/3740971588 >> 172.21.15.3:0/3740971588 conn(0x7fea5008c000 msgr2=0x7fea500d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- 172.21.15.3:0/3740971588 shutdown_connections 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- 172.21.15.3:0/3740971588 wait complete. 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 Processor -- start 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- start start 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea380039c0 0x7fea50157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50158ba0 0x7fea50159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea50150840 0x7fea50150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea500b2090 con 0x7fea50158ba0 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea500b1f10 con 0x7fea380039c0 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea56cc0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea500b2210 con 0x7fea50150840 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50158ba0 0x7fea50159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50158ba0 0x7fea50159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47826/0 (socket says 172.21.15.3:47826) 2024-09-12T12:22:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 -- 172.21.15.3:0/1683558136 learned_addr learned my addr 172.21.15.3:0/1683558136 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea54a5c640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea380039c0 0x7fea50157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 -- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea50150840 msgr2=0x7fea50150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea50150840 0x7fea50150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 -- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea380039c0 msgr2=0x7fea50157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea380039c0 0x7fea50157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 -- 172.21.15.3:0/1683558136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea501537f0 con 0x7fea50158ba0 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.925+0000 7fea4ffff640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50158ba0 0x7fea50159020 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea4400e7d0 tx=0x7fea4400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea4d7fa640 1 -- 172.21.15.3:0/1683558136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4400c7f0 con 0x7fea50158ba0 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea4d7fa640 1 -- 172.21.15.3:0/1683558136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea44018070 con 0x7fea50158ba0 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea4d7fa640 1 -- 172.21.15.3:0/1683558136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4400cbf0 con 0x7fea50158ba0 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea50157a00 con 0x7fea50158ba0 2024-09-12T12:22:18.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea50157fc0 con 0x7fea50158ba0 2024-09-12T12:22:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea500b25d0 con 0x7fea50158ba0 2024-09-12T12:22:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea4d7fa640 1 -- 172.21.15.3:0/1683558136 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fea44013420 con 0x7fea50158ba0 2024-09-12T12:22:18.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea4d7fa640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea3c0678f0 0x7fea3c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea4d7fa640 1 -- 172.21.15.3:0/1683558136 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7fea440857a0 con 0x7fea50158ba0 2024-09-12T12:22:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea54a5c640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea3c0678f0 0x7fea3c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:18.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.929+0000 7fea54a5c640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea3c0678f0 0x7fea3c069db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea400ae000 tx=0x7fea400046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:18.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:18.933+0000 7fea4d7fa640 1 -- 172.21.15.3:0/1683558136 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fea44056340 con 0x7fea50158ba0 2024-09-12T12:22:19.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.097+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea500ba490 con 0x7fea50158ba0 2024-09-12T12:22:19.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea4d7fa640 1 -- 172.21.15.3:0/1683558136 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16673 (secure 0 0 0) 0x7fea44055c40 con 0x7fea50158ba0 2024-09-12T12:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:19.103 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":44,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:01.841006+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:21:27.233763+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-12T12:21:28.546504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"192.168.0.1:0/2872188260":"2024-09-12T13:22:01.717396+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:22:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea3c0678f0 msgr2=0x7fea3c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea3c0678f0 0x7fea3c069db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea400ae000 tx=0x7fea400046d0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50158ba0 msgr2=0x7fea50159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50158ba0 0x7fea50159020 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fea4400e7d0 tx=0x7fea4400ec90 comp rx=0 tx=0).stop 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 shutdown_connections 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea3c0678f0 0x7fea3c069db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea50150840 0x7fea50150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea50158ba0 0x7fea50159020 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 --2- 172.21.15.3:0/1683558136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea380039c0 0x7fea50157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 >> 172.21.15.3:0/1683558136 conn(0x7fea5008c000 msgr2=0x7fea5015a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 shutdown_connections 2024-09-12T12:22:19.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.101+0000 7fea56cc0640 1 -- 172.21.15.3:0/1683558136 wait complete. 2024-09-12T12:22:19.123 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T12:22:19.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d3a0f640 1 Processor -- start 2024-09-12T12:22:19.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d3a0f640 1 -- start start 2024-09-12T12:22:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d3a0f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78cc0aedf0 0x7f78cc0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d3a0f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78cc0afb00 con 0x7f78cc0d0380 2024-09-12T12:22:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d3a0f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78cc0b31b0 con 0x7f78cc1568f0 2024-09-12T12:22:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d3a0f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f78cc0b4ce0 con 0x7f78cc0aedf0 2024-09-12T12:22:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d0faa640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f78cc0aedf0 0x7f78cc0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d1fac640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f78cc0d0380 0x7f78cc15c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39242/0 (socket says 172.21.15.3:39242) 2024-09-12T12:22:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d17ab640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f78cc1568f0 0x7f78cc0b3970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:50276/0 (socket says 172.21.15.3:50276) 2024-09-12T12:22:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d1fac640 1 -- 172.21.15.3:0/1623363474 learned_addr learned my addr 172.21.15.3:0/1623363474 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:19.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d0faa640 1 --2- 172.21.15.3:0/1623363474 >> v2:172.21.15.3:3300/0 conn(0x7f78cc0aedf0 0x7f78cc0b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47828/0 (socket says 172.21.15.3:47828) 2024-09-12T12:22:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d0faa640 1 -- 172.21.15.3:0/1623363474 >> v1:172.21.15.142:6789/0 conn(0x7f78cc1568f0 legacy=0x7f78cc0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d0faa640 1 -- 172.21.15.3:0/1623363474 >> v1:172.21.15.3:6789/0 conn(0x7f78cc0d0380 legacy=0x7f78cc15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78cbfff640 1 -- 172.21.15.3:0/1623363474 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400934991 0 0) 0x7f78cc0afb00 con 0x7f78cc0d0380 2024-09-12T12:22:19.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.245+0000 7f78d0faa640 1 -- 172.21.15.3:0/1623363474 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc0af540 con 0x7f78cc0aedf0 2024-09-12T12:22:19.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d0faa640 1 --2- 172.21.15.3:0/1623363474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc0aedf0 0x7f78cc0b47a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78c00097c0 tx=0x7f78c001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bdeb997875fed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:19.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78cbfff640 1 -- 172.21.15.3:0/1623363474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c00b0070 con 0x7f78cc0aedf0 2024-09-12T12:22:19.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78cbfff640 1 -- 172.21.15.3:0/1623363474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c00ab600 con 0x7f78cc0aedf0 2024-09-12T12:22:19.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78cbfff640 1 -- 172.21.15.3:0/1623363474 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c00ab9d0 con 0x7f78cc0aedf0 2024-09-12T12:22:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 -- 172.21.15.3:0/1623363474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc0aedf0 msgr2=0x7f78cc0b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/1623363474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc0aedf0 0x7f78cc0b47a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f78c00097c0 tx=0x7f78c001c410 comp rx=0 tx=0).stop 2024-09-12T12:22:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 -- 172.21.15.3:0/1623363474 shutdown_connections 2024-09-12T12:22:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/1623363474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc0aedf0 0x7f78cc0b47a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 -- 172.21.15.3:0/1623363474 >> 172.21.15.3:0/1623363474 conn(0x7f78cc06be90 msgr2=0x7f78cc06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 -- 172.21.15.3:0/1623363474 shutdown_connections 2024-09-12T12:22:19.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 -- 172.21.15.3:0/1623363474 wait complete. 2024-09-12T12:22:19.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 Processor -- start 2024-09-12T12:22:19.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.249+0000 7f78d3a0f640 1 -- start start 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0aedf0 0x7f78cc155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc0d0380 0x7f78cc071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc1568f0 0x7f78cc073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0d0ba0 con 0x7f78cc1568f0 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc15d3b0 con 0x7f78cc0aedf0 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc0730b0 con 0x7f78cc0d0380 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc1568f0 0x7f78cc073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc1568f0 0x7f78cc073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47840/0 (socket says 172.21.15.3:47840) 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 -- 172.21.15.3:0/596542005 learned_addr learned my addr 172.21.15.3:0/596542005 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:19.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d17ab640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0aedf0 0x7f78cc155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d0faa640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc0d0380 0x7f78cc071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 -- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc0d0380 msgr2=0x7f78cc071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc0d0380 0x7f78cc071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 -- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0aedf0 msgr2=0x7f78cc155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0aedf0 0x7f78cc155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 -- 172.21.15.3:0/596542005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc0b0590 con 0x7f78cc1568f0 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d17ab640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0aedf0 0x7f78cc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d0faa640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc0d0380 0x7f78cc071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d1fac640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc1568f0 0x7f78cc073620 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c480 tx=0x7f78c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78c9ffb640 1 -- 172.21.15.3:0/596542005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4018070 con 0x7f78cc1568f0 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78c9ffb640 1 -- 172.21.15.3:0/596542005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c40074e0 con 0x7f78cc1568f0 2024-09-12T12:22:19.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78c9ffb640 1 -- 172.21.15.3:0/596542005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4004a30 con 0x7f78cc1568f0 2024-09-12T12:22:19.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc073370 con 0x7f78cc1568f0 2024-09-12T12:22:19.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc167aa0 con 0x7f78cc1568f0 2024-09-12T12:22:19.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78cc0b47a0 con 0x7f78cc1568f0 2024-09-12T12:22:19.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78c9ffb640 1 -- 172.21.15.3:0/596542005 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f78c401b070 con 0x7f78cc1568f0 2024-09-12T12:22:19.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78c9ffb640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78b80678c0 0x7f78b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78c9ffb640 1 -- 172.21.15.3:0/596542005 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5752+0+0 (secure 0 0 0) 0x7f78c4084f80 con 0x7f78cc1568f0 2024-09-12T12:22:19.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.253+0000 7f78d17ab640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78b80678c0 0x7f78b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.257+0000 7f78d17ab640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78b80678c0 0x7f78b8069d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0012f0 tx=0x7f78bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:19.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.257+0000 7f78c9ffb640 1 -- 172.21.15.3:0/596542005 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f78c4055b20 con 0x7f78cc1568f0 2024-09-12T12:22:19.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.457+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f78cc0b3e30 con 0x7f78cc1568f0 2024-09-12T12:22:19.461 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:22:19.457+0000 7f0a5d435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:22:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.461+0000 7f78c9ffb640 1 -- 172.21.15.3:0/596542005 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f78c4055420 con 0x7f78cc1568f0 2024-09-12T12:22:19.464 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:22:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78b80678c0 msgr2=0x7f78b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78b80678c0 0x7f78b8069d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0012f0 tx=0x7f78bc009040 comp rx=0 tx=0).stop 2024-09-12T12:22:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc1568f0 msgr2=0x7f78cc073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc1568f0 0x7f78cc073620 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c480 tx=0x7f78c400c940 comp rx=0 tx=0).stop 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 shutdown_connections 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78b80678c0 0x7f78b8069d80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78cc1568f0 0x7f78cc073620 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78cc0d0380 0x7f78cc071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 --2- 172.21.15.3:0/596542005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78cc0aedf0 0x7f78cc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 >> 172.21.15.3:0/596542005 conn(0x7f78cc06be90 msgr2=0x7f78cc06c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 shutdown_connections 2024-09-12T12:22:19.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.465+0000 7f78d3a0f640 1 -- 172.21.15.3:0/596542005 wait complete. 2024-09-12T12:22:19.481 INFO:tasks.ceph.mds.d.smithi142.stderr: -8772> 2024-09-12T12:21:32.161+0000 7f44b59e2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:22:19.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:22:19.509 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:22:19.542 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:22:19.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.605+0000 7ff82ad54640 1 Processor -- start 2024-09-12T12:22:19.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.605+0000 7ff82ad54640 1 -- start start 2024-09-12T12:22:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff82ad54640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8240ad770 0x7ff8240b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff82ad54640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff82406e0b0 con 0x7ff824159dd0 2024-09-12T12:22:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff82ad54640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff82406e390 con 0x7ff82406f150 2024-09-12T12:22:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff82ad54640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff82406e670 con 0x7ff8240ad770 2024-09-12T12:22:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8292f1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8240ad770 0x7ff8240b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff828af0640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff824159dd0 0x7ff82415d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39268/0 (socket says 172.21.15.3:39268) 2024-09-12T12:22:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff828af0640 1 -- 172.21.15.3:0/438094139 learned_addr learned my addr 172.21.15.3:0/438094139 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8292f1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff8240ad770 0x7ff8240b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47862/0 (socket says 172.21.15.3:47862) 2024-09-12T12:22:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8292f1640 1 -- 172.21.15.3:0/438094139 >> v1:172.21.15.142:6789/0 conn(0x7ff82406f150 legacy=0x7ff8240ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8292f1640 1 -- 172.21.15.3:0/438094139 >> v1:172.21.15.3:6789/0 conn(0x7ff824159dd0 legacy=0x7ff82415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8237fe640 1 -- 172.21.15.3:0/438094139 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540473330 0 0) 0x7ff82406e0b0 con 0x7ff824159dd0 2024-09-12T12:22:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8292f1640 1 -- 172.21.15.3:0/438094139 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82406e950 con 0x7ff8240ad770 2024-09-12T12:22:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8292f1640 1 --2- 172.21.15.3:0/438094139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff8240b3080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009d90 tx=0x7ff81c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428b34e5f8504757 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8237fe640 1 -- 172.21.15.3:0/438094139 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0b2070 con 0x7ff8240ad770 2024-09-12T12:22:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8237fe640 1 -- 172.21.15.3:0/438094139 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c0a5910 con 0x7ff8240ad770 2024-09-12T12:22:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.609+0000 7ff8237fe640 1 -- 172.21.15.3:0/438094139 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0a5ce0 con 0x7ff8240ad770 2024-09-12T12:22:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- 172.21.15.3:0/438094139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 msgr2=0x7ff8240b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 --2- 172.21.15.3:0/438094139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff8240b3080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009d90 tx=0x7ff81c01c940 comp rx=0 tx=0).stop 2024-09-12T12:22:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- 172.21.15.3:0/438094139 shutdown_connections 2024-09-12T12:22:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 --2- 172.21.15.3:0/438094139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff8240b3080 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- 172.21.15.3:0/438094139 >> 172.21.15.3:0/438094139 conn(0x7ff8240b1d80 msgr2=0x7ff8240b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- 172.21.15.3:0/438094139 shutdown_connections 2024-09-12T12:22:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- 172.21.15.3:0/438094139 wait complete. 2024-09-12T12:22:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 Processor -- start 2024-09-12T12:22:19.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- start start 2024-09-12T12:22:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82406f150 0x7ff8241456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff824099860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824159dd0 0x7ff82414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff8240b2a40 con 0x7ff8240ad770 2024-09-12T12:22:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff82415a1b0 con 0x7ff824159dd0 2024-09-12T12:22:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff8240b2bc0 con 0x7ff82406f150 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff824099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff824099860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47874/0 (socket says 172.21.15.3:47874) 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 -- 172.21.15.3:0/2733368859 learned_addr learned my addr 172.21.15.3:0/2733368859 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff828af0640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82406f150 0x7ff8241456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff8292f1640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824159dd0 0x7ff82414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 -- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82406f150 msgr2=0x7ff8241456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82406f150 0x7ff8241456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 -- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824159dd0 msgr2=0x7ff82414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824159dd0 0x7ff82414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 -- 172.21.15.3:0/2733368859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8240b4d20 con 0x7ff8240ad770 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff828af0640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82406f150 0x7ff8241456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff8292f1640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824159dd0 0x7ff82414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff823fff640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff824099860 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c500 tx=0x7ff81400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff8217fa640 1 -- 172.21.15.3:0/2733368859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814016020 con 0x7ff8240ad770 2024-09-12T12:22:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff8217fa640 1 -- 172.21.15.3:0/2733368859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff814005220 con 0x7ff8240ad770 2024-09-12T12:22:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff8217fa640 1 -- 172.21.15.3:0/2733368859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814007030 con 0x7ff8240ad770 2024-09-12T12:22:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8240b5000 con 0x7ff8240ad770 2024-09-12T12:22:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.613+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff824099060 con 0x7ff8240ad770 2024-09-12T12:22:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.617+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8240b3080 con 0x7ff8240ad770 2024-09-12T12:22:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.617+0000 7ff8217fa640 1 -- 172.21.15.3:0/2733368859 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff8140071f0 con 0x7ff8240ad770 2024-09-12T12:22:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.617+0000 7ff8217fa640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8100678c0 0x7ff810069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.617+0000 7ff8217fa640 1 -- 172.21.15.3:0/2733368859 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5924+0+0 (secure 0 0 0) 0x7ff814085780 con 0x7ff8240ad770 2024-09-12T12:22:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.617+0000 7ff828af0640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8100678c0 0x7ff810069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.617+0000 7ff828af0640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8100678c0 0x7ff810069d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff818000f70 tx=0x7ff818009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:19.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.621+0000 7ff8217fa640 1 -- 172.21.15.3:0/2733368859 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff814020020 con 0x7ff8240ad770 2024-09-12T12:22:19.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:19.813+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff82406fae0 con 0x7ff8240ad770 2024-09-12T12:22:20.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.464+0000 7ff8217fa640 1 -- 172.21.15.3:0/2733368859 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7ff8140561f0 con 0x7ff8240ad770 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8100678c0 msgr2=0x7ff810069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8100678c0 0x7ff810069d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff818000f70 tx=0x7ff818009040 comp rx=0 tx=0).stop 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 msgr2=0x7ff824099860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff824099860 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c500 tx=0x7ff81400c9c0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 shutdown_connections 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff8100678c0 0x7ff810069d80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824159dd0 0x7ff82414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8240ad770 0x7ff824099860 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 --2- 172.21.15.3:0/2733368859 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff82406f150 0x7ff8241456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 >> 172.21.15.3:0/2733368859 conn(0x7ff8240b1d80 msgr2=0x7ff8240aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:20.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 shutdown_connections 2024-09-12T12:22:20.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.468+0000 7ff82ad54640 1 -- 172.21.15.3:0/2733368859 wait complete. 2024-09-12T12:22:20.496 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:22:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3bed018640 1 Processor -- start 2024-09-12T12:22:20.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3bed018640 1 -- start start 2024-09-12T12:22:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3bed018640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3be806f150 0x7f3be80ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3bed018640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f5b0 0x7f3be80ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3bed018640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be806e0b0 con 0x7f3be8159dd0 2024-09-12T12:22:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3bed018640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3be806e390 con 0x7f3be806f150 2024-09-12T12:22:20.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3bed018640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3be806e670 con 0x7f3be806f5b0 2024-09-12T12:22:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3be6d9d640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f3be8159dd0 0x7f3be815d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:50302/0 (socket says 172.21.15.3:50302) 2024-09-12T12:22:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.672+0000 7f3be6d9d640 1 -- 172.21.15.3:0/2527432785 learned_addr learned my addr 172.21.15.3:0/2527432785 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be759e640 1 --2- 172.21.15.3:0/2527432785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f5b0 0x7f3be80ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be659c640 1 --2- 172.21.15.3:0/2527432785 >> v2:172.21.15.142:3300/0 conn(0x7f3be806f150 0x7f3be80ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:20.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be5d9b640 1 -- 172.21.15.3:0/2527432785 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262764058 0 0) 0x7f3be806e0b0 con 0x7f3be8159dd0 2024-09-12T12:22:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be5d9b640 1 -- 172.21.15.3:0/2527432785 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bd4003530 con 0x7f3be8159dd0 2024-09-12T12:22:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be759e640 1 -- 172.21.15.3:0/2527432785 >> v2:172.21.15.142:3300/0 conn(0x7f3be806f150 msgr2=0x7f3be80ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be759e640 1 --2- 172.21.15.3:0/2527432785 >> v2:172.21.15.142:3300/0 conn(0x7f3be806f150 0x7f3be80ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be759e640 1 -- 172.21.15.3:0/2527432785 >> v1:172.21.15.142:6789/0 conn(0x7f3be8159dd0 legacy=0x7f3be815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be759e640 1 -- 172.21.15.3:0/2527432785 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be806e950 con 0x7f3be806f5b0 2024-09-12T12:22:20.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be659c640 1 --2- 172.21.15.3:0/2527432785 >> v2:172.21.15.142:3300/0 conn(0x7f3be806f150 0x7f3be80ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:20.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be759e640 1 --2- 172.21.15.3:0/2527432785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f5b0 0x7f3be80ad5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3be00097c0 tx=0x7f3be001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391ace7a39691cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be5d9b640 1 -- 172.21.15.3:0/2527432785 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be00b1070 con 0x7f3be806f5b0 2024-09-12T12:22:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be5d9b640 1 -- 172.21.15.3:0/2527432785 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be00ac630 con 0x7f3be806f5b0 2024-09-12T12:22:20.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.676+0000 7f3be5d9b640 1 -- 172.21.15.3:0/2527432785 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be00a8050 con 0x7f3be806f5b0 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- 172.21.15.3:0/2527432785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f5b0 msgr2=0x7f3be80ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 --2- 172.21.15.3:0/2527432785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f5b0 0x7f3be80ad5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3be00097c0 tx=0x7f3be001c470 comp rx=0 tx=0).stop 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- 172.21.15.3:0/2527432785 shutdown_connections 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 --2- 172.21.15.3:0/2527432785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f5b0 0x7f3be80ad5a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 --2- 172.21.15.3:0/2527432785 >> v2:172.21.15.142:3300/0 conn(0x7f3be806f150 0x7f3be80ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- 172.21.15.3:0/2527432785 >> 172.21.15.3:0/2527432785 conn(0x7f3be80b1d80 msgr2=0x7f3be80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- 172.21.15.3:0/2527432785 shutdown_connections 2024-09-12T12:22:20.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- 172.21.15.3:0/2527432785 wait complete. 2024-09-12T12:22:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 Processor -- start 2024-09-12T12:22:20.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- start start 2024-09-12T12:22:20.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f150 0x7f3be8099920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:20.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be806f5b0 0x7f3be81445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3be8159dd0 0x7f3be815e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3be80b2bc0 con 0x7f3be8159dd0 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3be80b2a40 con 0x7f3be806f5b0 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3bed018640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3be8169200 con 0x7f3be806f150 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3be8159dd0 0x7f3be815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3be8159dd0 0x7f3be815e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47888/0 (socket says 172.21.15.3:47888) 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 -- 172.21.15.3:0/3273960375 learned_addr learned my addr 172.21.15.3:0/3273960375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be659c640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be806f5b0 0x7f3be81445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be6d9d640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f150 0x7f3be8099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 -- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f150 msgr2=0x7f3be8099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f150 0x7f3be8099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 -- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be806f5b0 msgr2=0x7f3be81445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be806f5b0 0x7f3be81445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:20.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 -- 172.21.15.3:0/3273960375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be815d8c0 con 0x7f3be8159dd0 2024-09-12T12:22:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be6d9d640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f150 0x7f3be8099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.680+0000 7f3be759e640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3be8159dd0 0x7f3be815e960 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3be00af040 tx=0x7f3be0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bdf7fe640 1 -- 172.21.15.3:0/3273960375 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be00b9020 con 0x7f3be8159dd0 2024-09-12T12:22:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bdf7fe640 1 -- 172.21.15.3:0/3273960375 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be00acdd0 con 0x7f3be8159dd0 2024-09-12T12:22:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bdf7fe640 1 -- 172.21.15.3:0/3273960375 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be00b1040 con 0x7f3be8159dd0 2024-09-12T12:22:20.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be815dba0 con 0x7f3be8159dd0 2024-09-12T12:22:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be815e160 con 0x7f3be8159dd0 2024-09-12T12:22:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be806fae0 con 0x7f3be8159dd0 2024-09-12T12:22:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bdf7fe640 1 -- 172.21.15.3:0/3273960375 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3be00bc020 con 0x7f3be8159dd0 2024-09-12T12:22:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bdf7fe640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3bc80678c0 0x7f3bc8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:20.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3bdf7fe640 1 -- 172.21.15.3:0/3273960375 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5924+0+0 (secure 0 0 0) 0x7f3be011f700 con 0x7f3be8159dd0 2024-09-12T12:22:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3be6d9d640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3bc80678c0 0x7f3bc8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:20.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.684+0000 7f3be6d9d640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3bc80678c0 0x7f3bc8069d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8005c10 tx=0x7f3bd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.688+0000 7f3bdf7fe640 1 -- 172.21.15.3:0/3273960375 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3be00c3030 con 0x7f3be8159dd0 2024-09-12T12:22:20.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:20.848+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3be8153170 con 0x7f3be8159dd0 2024-09-12T12:22:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.472+0000 7f3bdf7fe640 1 -- 172.21.15.3:0/3273960375 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) ==== 181+0+0 (secure 0 0 0) 0x7f3be0016090 con 0x7f3be8159dd0 2024-09-12T12:22:21.477 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:22:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3bc80678c0 msgr2=0x7f3bc8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3bc80678c0 0x7f3bc8069d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8005c10 tx=0x7f3bd8009040 comp rx=0 tx=0).stop 2024-09-12T12:22:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3be8159dd0 msgr2=0x7f3be815e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:21.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3be8159dd0 0x7f3be815e960 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3be00af040 tx=0x7f3be0019210 comp rx=0 tx=0).stop 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 shutdown_connections 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3bc80678c0 0x7f3bc8069d80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3be8159dd0 0x7f3be815e960 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3be806f5b0 0x7f3be81445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 --2- 172.21.15.3:0/3273960375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3be806f150 0x7f3be8099920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 >> 172.21.15.3:0/3273960375 conn(0x7f3be80b1d80 msgr2=0x7f3be815a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 shutdown_connections 2024-09-12T12:22:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.476+0000 7f3bed018640 1 -- 172.21.15.3:0/3273960375 wait complete. 2024-09-12T12:22:21.506 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:22:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1466930640 1 Processor -- start 2024-09-12T12:22:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1466930640 1 -- start start 2024-09-12T12:22:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1466930640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 0x7f146006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1466930640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14600d1690 con 0x7f14600b6350 2024-09-12T12:22:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1466930640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14600d1970 con 0x7f14600b6b10 2024-09-12T12:22:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1466930640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14600d23f0 con 0x7f14600b5f70 2024-09-12T12:22:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1464ecd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f14600b6350 0x7f14600cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39274/0 (socket says 172.21.15.3:39274) 2024-09-12T12:22:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1464ecd640 1 -- 172.21.15.3:0/3425617309 learned_addr learned my addr 172.21.15.3:0/3425617309 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:21.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145ffff640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 0x7f146006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977386760 0 0) 0x7f14600d1690 con 0x7f14600b6350 2024-09-12T12:22:21.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f144c003530 con 0x7f14600b6350 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 145025970 0 0) 0x7f144c003530 con 0x7f14600b6350 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 msgr2=0x7f146006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 0x7f146006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 >> v1:172.21.15.142:6789/0 conn(0x7f14600b6b10 legacy=0x7f14600d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14600d26d0 con 0x7f14600b6350 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1458001a00 con 0x7f14600b6350 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 >> v1:172.21.15.3:6789/0 conn(0x7f14600b6350 legacy=0x7f14600cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 0x7f144c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14600d23f0 con 0x7f144c0039f0 2024-09-12T12:22:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f144c00ba80 con 0x7f144c004d50 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f144c00bc80 con 0x7f14600b5f70 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145ffff640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1464ecd640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f14577fe640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 0x7f144c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145ffff640 1 -- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 msgr2=0x7f144c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145ffff640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 0x7f144c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145ffff640 1 -- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f144c004d50 msgr2=0x7f144c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145ffff640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f145ffff640 1 -- 172.21.15.3:0/3425617309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c003530 con 0x7f144c0039f0 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f1464ecd640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:21.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.640+0000 7f14577fe640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14600b5f70 0x7f144c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f145ffff640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1450009ab0 tx=0x7f145001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85eb7b1e3648d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500096b0 con 0x7f144c0039f0 2024-09-12T12:22:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14500a6410 con 0x7f144c0039f0 2024-09-12T12:22:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f145f7fe640 1 -- 172.21.15.3:0/3425617309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14500a6800 con 0x7f144c0039f0 2024-09-12T12:22:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 -- 172.21.15.3:0/3425617309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0039f0 msgr2=0x7f144c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 --2- 172.21.15.3:0/3425617309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1450009ab0 tx=0x7f145001ca50 comp rx=0 tx=0).stop 2024-09-12T12:22:21.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1464ecd640 1 -- 172.21.15.3:0/3425617309 reap_dead start 2024-09-12T12:22:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 -- 172.21.15.3:0/3425617309 shutdown_connections 2024-09-12T12:22:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 -- 172.21.15.3:0/3425617309 >> 172.21.15.3:0/3425617309 conn(0x7f146009d880 msgr2=0x7f146008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 -- 172.21.15.3:0/3425617309 shutdown_connections 2024-09-12T12:22:21.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 -- 172.21.15.3:0/3425617309 wait complete. 2024-09-12T12:22:21.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 Processor -- start 2024-09-12T12:22:21.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.644+0000 7f1466930640 1 -- start start 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144c0039f0 0x7f14600c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f146015cc50 0x7f146015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14600cb5c0 0x7f146015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f146016a720 con 0x7f146015cc50 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f146016a5a0 con 0x7f14600cb5c0 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f146016a8a0 con 0x7f144c0039f0 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f146015cc50 0x7f146015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f146015cc50 0x7f146015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47902/0 (socket says 172.21.15.3:47902) 2024-09-12T12:22:21.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 -- 172.21.15.3:0/4091714812 learned_addr learned my addr 172.21.15.3:0/4091714812 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1464ecd640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14600cb5c0 0x7f146015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145ffff640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144c0039f0 0x7f14600c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 -- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144c0039f0 msgr2=0x7f14600c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144c0039f0 0x7f14600c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 -- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14600cb5c0 msgr2=0x7f146015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14600cb5c0 0x7f146015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 -- 172.21.15.3:0/4091714812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1460160650 con 0x7f146015cc50 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145ffff640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144c0039f0 0x7f14600c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f14577fe640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f146015cc50 0x7f146015c2e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f144800d770 tx=0x7f144800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145d7fa640 1 -- 172.21.15.3:0/4091714812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448019070 con 0x7f146015cc50 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145d7fa640 1 -- 172.21.15.3:0/4091714812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1448005270 con 0x7f146015cc50 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14600cbf30 con 0x7f146015cc50 2024-09-12T12:22:21.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145d7fa640 1 -- 172.21.15.3:0/4091714812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448007030 con 0x7f146015cc50 2024-09-12T12:22:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14600cc4f0 con 0x7f146015cc50 2024-09-12T12:22:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14600cf000 con 0x7f146015cc50 2024-09-12T12:22:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145d7fa640 1 -- 172.21.15.3:0/4091714812 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f144801c070 con 0x7f146015cc50 2024-09-12T12:22:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145d7fa640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14400678c0 0x7f1440069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145d7fa640 1 -- 172.21.15.3:0/4091714812 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5486+0+0 (secure 0 0 0) 0x7f1448085400 con 0x7f146015cc50 2024-09-12T12:22:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.648+0000 7f145ffff640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14400678c0 0x7f1440069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:21.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.652+0000 7f145ffff640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14400678c0 0x7f1440069d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f14500ad000 tx=0x7f14500046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:21.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.656+0000 7f145d7fa640 1 -- 172.21.15.3:0/4091714812 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1448056020 con 0x7f146015cc50 2024-09-12T12:22:21.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:21.828+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f146006e560 con 0x7f146015cc50 2024-09-12T12:22:22.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.484+0000 7f145d7fa640 1 -- 172.21.15.3:0/4091714812 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) ==== 169+0+0 (secure 0 0 0) 0x7f1448055920 con 0x7f146015cc50 2024-09-12T12:22:22.487 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T12:22:22.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14400678c0 msgr2=0x7f1440069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14400678c0 0x7f1440069d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f14500ad000 tx=0x7f14500046d0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f146015cc50 msgr2=0x7f146015c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f146015cc50 0x7f146015c2e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f144800d770 tx=0x7f144800dc30 comp rx=0 tx=0).stop 2024-09-12T12:22:22.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 shutdown_connections 2024-09-12T12:22:22.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f14400678c0 0x7f1440069d80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14600cb5c0 0x7f146015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f146015cc50 0x7f146015c2e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 --2- 172.21.15.3:0/4091714812 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144c0039f0 0x7f14600c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 >> 172.21.15.3:0/4091714812 conn(0x7f146009d880 msgr2=0x7f14600b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:22.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 shutdown_connections 2024-09-12T12:22:22.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.488+0000 7f1466930640 1 -- 172.21.15.3:0/4091714812 wait complete. 2024-09-12T12:22:22.511 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T12:22:22.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.640+0000 7fa9067b7640 1 Processor -- start 2024-09-12T12:22:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.640+0000 7fa9067b7640 1 -- start start 2024-09-12T12:22:22.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa9067b7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9000b6700 0x7fa900070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa9067b7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa90015f640 0x7fa9000addb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa9067b7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9000af540 con 0x7fa9000b0300 2024-09-12T12:22:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa9067b7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9000af820 con 0x7fa9000b6700 2024-09-12T12:22:22.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa9067b7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9000afb00 con 0x7fa90015f640 2024-09-12T12:22:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8fffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9000b6700 0x7fa900070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa904d54640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fa9000b0300 0x7fa90006e880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:50318/0 (socket says 172.21.15.3:50318) 2024-09-12T12:22:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa904d54640 1 -- 172.21.15.3:0/1655597646 learned_addr learned my addr 172.21.15.3:0/1655597646 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8ff7fe640 1 --2- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa90015f640 0x7fa9000addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8fffff640 1 -- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa90015f640 msgr2=0x7fa9000addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:22.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8fffff640 1 --2- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa90015f640 0x7fa9000addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8fffff640 1 -- 172.21.15.3:0/1655597646 >> v1:172.21.15.142:6789/0 conn(0x7fa9000b0300 legacy=0x7fa90006e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8fffff640 1 -- 172.21.15.3:0/1655597646 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9000afde0 con 0x7fa9000b6700 2024-09-12T12:22:22.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8fffff640 1 --2- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 0x7fa900070900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009d90 tx=0x7fa90006f7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c08cb96e2b9439cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:22.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8feffd640 1 -- 172.21.15.3:0/1655597646 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00b2070 con 0x7fa9000b6700 2024-09-12T12:22:22.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8feffd640 1 -- 172.21.15.3:0/1655597646 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f00a5830 con 0x7fa9000b6700 2024-09-12T12:22:22.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.644+0000 7fa8feffd640 1 -- 172.21.15.3:0/1655597646 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00a5c00 con 0x7fa9000b6700 2024-09-12T12:22:22.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 msgr2=0x7fa900070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 --2- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 0x7fa900070900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009d90 tx=0x7fa90006f7a0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- 172.21.15.3:0/1655597646 shutdown_connections 2024-09-12T12:22:22.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 --2- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa90015f640 0x7fa9000addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 --2- 172.21.15.3:0/1655597646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 0x7fa900070900 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- 172.21.15.3:0/1655597646 >> 172.21.15.3:0/1655597646 conn(0x7fa900073550 msgr2=0x7fa90006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:22.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- 172.21.15.3:0/1655597646 shutdown_connections 2024-09-12T12:22:22.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- 172.21.15.3:0/1655597646 wait complete. 2024-09-12T12:22:22.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 Processor -- start 2024-09-12T12:22:22.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- start start 2024-09-12T12:22:22.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9000b0300 0x7fa9000cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 0x7fa90014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90015f640 0x7fa900157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa90016b320 con 0x7fa90015f640 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9000b6cf0 con 0x7fa9000b6700 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa90016b4a0 con 0x7fa9000b0300 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90015f640 0x7fa900157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90015f640 0x7fa900157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47914/0 (socket says 172.21.15.3:47914) 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 -- 172.21.15.3:0/376057334 learned_addr learned my addr 172.21.15.3:0/376057334 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa8ff7fe640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 0x7fa90014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa8fffff640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9000b0300 0x7fa9000cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 -- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9000b0300 msgr2=0x7fa9000cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9000b0300 0x7fa9000cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 -- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 msgr2=0x7fa90014ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 0x7fa90014ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 -- 172.21.15.3:0/376057334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9000cdf90 con 0x7fa90015f640 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa8fffff640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9000b0300 0x7fa9000cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa904d54640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90015f640 0x7fa900157830 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c3f0 tx=0x7fa8f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa8fcff9640 1 -- 172.21.15.3:0/376057334 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8017070 con 0x7fa90015f640 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa8fcff9640 1 -- 172.21.15.3:0/376057334 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f8005cf0 con 0x7fa90015f640 2024-09-12T12:22:22.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9001f85d0 con 0x7fa90015f640 2024-09-12T12:22:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa8fcff9640 1 -- 172.21.15.3:0/376057334 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80072f0 con 0x7fa90015f640 2024-09-12T12:22:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.648+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9001f8b90 con 0x7fa90015f640 2024-09-12T12:22:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.652+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa900070900 con 0x7fa90015f640 2024-09-12T12:22:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.652+0000 7fa8fcff9640 1 -- 172.21.15.3:0/376057334 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa8f801a070 con 0x7fa90015f640 2024-09-12T12:22:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.652+0000 7fa8fcff9640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa8ec0678f0 0x7fa8ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.652+0000 7fa8fcff9640 1 -- 172.21.15.3:0/376057334 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7fa8f8084c70 con 0x7fa90015f640 2024-09-12T12:22:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.652+0000 7fa8fffff640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa8ec0678f0 0x7fa8ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.652+0000 7fa8fffff640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa8ec0678f0 0x7fa8ec069db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00af000 tx=0x7fa8f0004ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.656+0000 7fa8fcff9640 1 -- 172.21.15.3:0/376057334 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa8f8055470 con 0x7fa90015f640 2024-09-12T12:22:22.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.828+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa9000b0d90 con 0x7fa90015f640 2024-09-12T12:22:22.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.832+0000 7fa8fcff9640 1 -- 172.21.15.3:0/376057334 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7fa8f8059770 con 0x7fa90015f640 2024-09-12T12:22:22.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa8ec0678f0 msgr2=0x7fa8ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa8ec0678f0 0x7fa8ec069db0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00af000 tx=0x7fa8f0004ac0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90015f640 msgr2=0x7fa900157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90015f640 0x7fa900157830 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c3f0 tx=0x7fa8f800c8b0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 shutdown_connections 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa8ec0678f0 0x7fa8ec069db0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa90015f640 0x7fa900157830 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9000b6700 0x7fa90014ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 --2- 172.21.15.3:0/376057334 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9000b0300 0x7fa9000cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 >> 172.21.15.3:0/376057334 conn(0x7fa900073550 msgr2=0x7fa90015fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 shutdown_connections 2024-09-12T12:22:22.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.836+0000 7fa9067b7640 1 -- 172.21.15.3:0/376057334 wait complete. 2024-09-12T12:22:22.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted' 2024-09-12T12:22:22.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c5d873640 1 Processor -- start 2024-09-12T12:22:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c5d873640 1 -- start start 2024-09-12T12:22:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c5d873640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c5815e5d0 0x7f5c580ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c5d873640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c580d5050 0x7f5c580cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c5d873640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c580d31a0 con 0x7f5c580cfa10 2024-09-12T12:22:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c5d873640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5c580b1060 con 0x7f5c580d5050 2024-09-12T12:22:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c5d873640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5c580cfe40 con 0x7f5c5815e5d0 2024-09-12T12:22:22.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c5815e5d0 0x7f5c580ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c567fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5c580d5050 0x7f5c580cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c56ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c5815e5d0 0x7f5c580ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:52988/0 (socket says 172.21.15.3:52988) 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c577fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f5c580cfa10 0x7f5c58162280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:50322/0 (socket says 172.21.15.3:50322) 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c56ffd640 1 -- 172.21.15.3:0/4183940682 learned_addr learned my addr 172.21.15.3:0/4183940682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c567fc640 1 -- 172.21.15.3:0/4183940682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c5815e5d0 msgr2=0x7f5c580ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c567fc640 1 --2- 172.21.15.3:0/4183940682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c5815e5d0 0x7f5c580ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c567fc640 1 -- 172.21.15.3:0/4183940682 >> v1:172.21.15.142:6789/0 conn(0x7f5c580cfa10 legacy=0x7f5c58162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.988+0000 7f5c567fc640 1 -- 172.21.15.3:0/4183940682 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c580d3480 con 0x7f5c580d5050 2024-09-12T12:22:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c567fc640 1 --2- 172.21.15.3:0/4183940682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c580cf4d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0097c0 tx=0x7f5c4c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65d4853f2fcac88 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c55ffb640 1 -- 172.21.15.3:0/4183940682 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0ab680 con 0x7f5c580d5050 2024-09-12T12:22:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c55ffb640 1 -- 172.21.15.3:0/4183940682 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c0b0070 con 0x7f5c580d5050 2024-09-12T12:22:22.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c55ffb640 1 -- 172.21.15.3:0/4183940682 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0aba40 con 0x7f5c580d5050 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- 172.21.15.3:0/4183940682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 msgr2=0x7f5c580cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 --2- 172.21.15.3:0/4183940682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c580cf4d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0097c0 tx=0x7f5c4c01c4a0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- 172.21.15.3:0/4183940682 shutdown_connections 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 --2- 172.21.15.3:0/4183940682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c580cf4d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 --2- 172.21.15.3:0/4183940682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c5815e5d0 0x7f5c580ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- 172.21.15.3:0/4183940682 >> 172.21.15.3:0/4183940682 conn(0x7f5c5808c000 msgr2=0x7f5c580d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- 172.21.15.3:0/4183940682 shutdown_connections 2024-09-12T12:22:22.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- 172.21.15.3:0/4183940682 wait complete. 2024-09-12T12:22:22.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 Processor -- start 2024-09-12T12:22:22.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- start start 2024-09-12T12:22:22.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c580cfa10 0x7f5c580c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c5815f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5815e5d0 0x7f5c581548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5816b5c0 con 0x7f5c5815e5d0 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5815ebc0 con 0x7f5c580d5050 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c5d873640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5816b740 con 0x7f5c580cfa10 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5815e5d0 0x7f5c581548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5815e5d0 0x7f5c581548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47918/0 (socket says 172.21.15.3:47918) 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.992+0000 7f5c577fe640 1 -- 172.21.15.3:0/3722528101 learned_addr learned my addr 172.21.15.3:0/3722528101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c56ffd640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c580cfa10 0x7f5c580c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c567fc640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c5815f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c577fe640 1 -- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c580cfa10 msgr2=0x7f5c580c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c577fe640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c580cfa10 0x7f5c580c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c577fe640 1 -- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 msgr2=0x7f5c5815f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c577fe640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c5815f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c577fe640 1 -- 172.21.15.3:0/3722528101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c58160210 con 0x7f5c5815e5d0 2024-09-12T12:22:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c567fc640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c5815f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:22.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c577fe640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5815e5d0 0x7f5c581548e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000c760 tx=0x7f5c5000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c3ffff640 1 -- 172.21.15.3:0/3722528101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50017070 con 0x7f5c5815e5d0 2024-09-12T12:22:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c3ffff640 1 -- 172.21.15.3:0/3722528101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5000a640 con 0x7f5c5815e5d0 2024-09-12T12:22:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5815a340 con 0x7f5c5815e5d0 2024-09-12T12:22:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c3ffff640 1 -- 172.21.15.3:0/3722528101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000a9f0 con 0x7f5c5815e5d0 2024-09-12T12:22:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5815a900 con 0x7f5c5815e5d0 2024-09-12T12:22:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c580cf4d0 con 0x7f5c5815e5d0 2024-09-12T12:22:23.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c3ffff640 1 -- 172.21.15.3:0/3722528101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5c5001a070 con 0x7f5c5815e5d0 2024-09-12T12:22:23.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c3ffff640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5c440678c0 0x7f5c44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c3ffff640 1 -- 172.21.15.3:0/3722528101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7f5c50084e70 con 0x7f5c5815e5d0 2024-09-12T12:22:23.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:22.996+0000 7f5c56ffd640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5c440678c0 0x7f5c44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.000+0000 7f5c56ffd640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5c440678c0 0x7f5c44069d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009a40 tx=0x7f5c48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:23.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.000+0000 7f5c3ffff640 1 -- 172.21.15.3:0/3722528101 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5c50055670 con 0x7f5c5815e5d0 2024-09-12T12:22:23.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.228+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]} v 0) -- 0x7f5c580b6bc0 con 0x7f5c5815e5d0 2024-09-12T12:22:23.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.488+0000 7f5c3ffff640 1 -- 172.21.15.3:0/3722528101 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_evict_blocklisted"]}]=0 v145) ==== 153+0+0 (secure 0 0 0) 0x7f5c50059970 con 0x7f5c5815e5d0 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5c440678c0 msgr2=0x7f5c44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5c440678c0 0x7f5c44069d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009a40 tx=0x7f5c48008040 comp rx=0 tx=0).stop 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5815e5d0 msgr2=0x7f5c581548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5815e5d0 0x7f5c581548e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000c760 tx=0x7f5c5000cc20 comp rx=0 tx=0).stop 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 shutdown_connections 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5c440678c0 0x7f5c44069d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c5815e5d0 0x7f5c581548e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c580d5050 0x7f5c5815f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 --2- 172.21.15.3:0/3722528101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5c580cfa10 0x7f5c580c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 >> 172.21.15.3:0/3722528101 conn(0x7f5c5808c000 msgr2=0x7f5c580d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 shutdown_connections 2024-09-12T12:22:23.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.492+0000 7f5c5d873640 1 -- 172.21.15.3:0/3722528101 wait complete. 2024-09-12T12:22:23.517 INFO:tasks.cephfs_test_runner:Check that mds evicts blocklisted client ... ok 2024-09-12T12:22:23.517 INFO:tasks.cephfs_test_runner:Starting test: test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-12T12:22:23.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:22:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc67bca640 1 Processor -- start 2024-09-12T12:22:23.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc67bca640 1 -- start start 2024-09-12T12:22:23.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc67bca640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efc6015b2d0 0x7efc600b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc67bca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc60073640 0x7efc600aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc67bca640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc6015f150 con 0x7efc6006f6e0 2024-09-12T12:22:23.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc67bca640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7efc600ae920 con 0x7efc6015b2d0 2024-09-12T12:22:23.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc67bca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc6015b6b0 con 0x7efc60073640 2024-09-12T12:22:23.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc65966640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efc6015b2d0 0x7efc600b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc65966640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efc6015b2d0 0x7efc600b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47930/0 (socket says 172.21.15.3:47930) 2024-09-12T12:22:23.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc65966640 1 -- 172.21.15.3:0/4231753296 learned_addr learned my addr 172.21.15.3:0/4231753296 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc65966640 1 -- 172.21.15.3:0/4231753296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc60073640 msgr2=0x7efc600aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc66167640 1 --2- 172.21.15.3:0/4231753296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc60073640 0x7efc600aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc65966640 1 --2- 172.21.15.3:0/4231753296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc60073640 0x7efc600aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc65966640 1 -- 172.21.15.3:0/4231753296 >> v1:172.21.15.142:6789/0 conn(0x7efc6006f6e0 legacy=0x7efc60072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.644+0000 7efc65966640 1 -- 172.21.15.3:0/4231753296 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6015ee70 con 0x7efc6015b2d0 2024-09-12T12:22:23.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc65966640 1 --2- 172.21.15.3:0/4231753296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6015b2d0 0x7efc600b3080 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d150 tx=0x7efc5801ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfd1e5207af6b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc64964640 1 -- 172.21.15.3:0/4231753296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5800b2d0 con 0x7efc6015b2d0 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc64964640 1 -- 172.21.15.3:0/4231753296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5800b470 con 0x7efc6015b2d0 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc64964640 1 -- 172.21.15.3:0/4231753296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580a7b70 con 0x7efc6015b2d0 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 -- 172.21.15.3:0/4231753296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6015b2d0 msgr2=0x7efc600b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 --2- 172.21.15.3:0/4231753296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6015b2d0 0x7efc600b3080 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d150 tx=0x7efc5801ebc0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 -- 172.21.15.3:0/4231753296 shutdown_connections 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 --2- 172.21.15.3:0/4231753296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc60073640 0x7efc600aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 --2- 172.21.15.3:0/4231753296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc6015b2d0 0x7efc600b3080 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 -- 172.21.15.3:0/4231753296 >> 172.21.15.3:0/4231753296 conn(0x7efc600b1d80 msgr2=0x7efc600b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 -- 172.21.15.3:0/4231753296 shutdown_connections 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.648+0000 7efc67bca640 1 -- 172.21.15.3:0/4231753296 wait complete. 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 Processor -- start 2024-09-12T12:22:23.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 -- start start 2024-09-12T12:22:23.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc6006f6e0 0x7efc60099a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc60073640 0x7efc600d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc6015b2d0 0x7efc600b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc600af700 con 0x7efc60073640 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc6006d8d0 con 0x7efc6015b2d0 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc600af880 con 0x7efc6006f6e0 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc60073640 0x7efc600d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc60073640 0x7efc600d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47944/0 (socket says 172.21.15.3:47944) 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 -- 172.21.15.3:0/874364820 learned_addr learned my addr 172.21.15.3:0/874364820 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65966640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc6006f6e0 0x7efc60099a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc66167640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc6015b2d0 0x7efc600b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 -- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc6006f6e0 msgr2=0x7efc60099a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc6006f6e0 0x7efc60099a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 -- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc6015b2d0 msgr2=0x7efc600b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc6015b2d0 0x7efc600b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 -- 172.21.15.3:0/874364820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc60156dc0 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc65165640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc60073640 0x7efc600d02f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efc5400b410 tx=0x7efc5400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc5e7fc640 1 -- 172.21.15.3:0/874364820 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54015030 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc5e7fc640 1 -- 172.21.15.3:0/874364820 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc54004020 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc60157060 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc5e7fc640 1 -- 172.21.15.3:0/874364820 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54004410 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc60157620 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc600aed90 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.652+0000 7efc5e7fc640 1 -- 172.21.15.3:0/874364820 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc54018070 con 0x7efc60073640 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.656+0000 7efc5e7fc640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efc4c0678c0 0x7efc4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.656+0000 7efc5e7fc640 1 -- 172.21.15.3:0/874364820 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7efc54085600 con 0x7efc60073640 2024-09-12T12:22:23.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.656+0000 7efc65966640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efc4c0678c0 0x7efc4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.656+0000 7efc65966640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efc4c0678c0 0x7efc4c069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc580b0000 tx=0x7efc580a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:23.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.660+0000 7efc5e7fc640 1 -- 172.21.15.3:0/874364820 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efc540563c0 con 0x7efc60073640 2024-09-12T12:22:23.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.824+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc600d0830 con 0x7efc60073640 2024-09-12T12:22:23.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.824+0000 7efc5e7fc640 1 -- 172.21.15.3:0/874364820 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+13161 (secure 0 0 0) 0x7efc54055cc0 con 0x7efc60073640 2024-09-12T12:22:23.829 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:23.830 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":47,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:22.476856+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/653673671":"2024-09-13T12:22:19.461262+0000","172.21.15.3:6834/653673671":"2024-09-13T12:22:19.461262+0000","172.21.15.142:6837/2796023141":"2024-09-13T12:22:19.461277+0000","172.21.15.142:6835/2796023141":"2024-09-13T12:22:19.461277+0000","192.168.0.1:0/2872188260":"2024-09-12T13:22:01.717396+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efc4c0678c0 msgr2=0x7efc4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efc4c0678c0 0x7efc4c069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc580b0000 tx=0x7efc580a9070 comp rx=0 tx=0).stop 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc60073640 msgr2=0x7efc600d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc60073640 0x7efc600d02f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efc5400b410 tx=0x7efc5400b8d0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 shutdown_connections 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efc4c0678c0 0x7efc4c069d80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc6015b2d0 0x7efc600b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc60073640 0x7efc600d02f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 --2- 172.21.15.3:0/874364820 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc6006f6e0 0x7efc60099a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 >> 172.21.15.3:0/874364820 conn(0x7efc600b1d80 msgr2=0x7efc6015e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:23.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 shutdown_connections 2024-09-12T12:22:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.828+0000 7efc67bca640 1 -- 172.21.15.3:0/874364820 wait complete. 2024-09-12T12:22:23.849 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:19:37.735209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:22:23.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T12:22:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 Processor -- start 2024-09-12T12:22:23.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 -- start start 2024-09-12T12:22:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd58015c580 0x7fd58006bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b3820 0x7fd5800b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd5800b26b0 0x7fd5800ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd5800b46e0 con 0x7fd58015c580 2024-09-12T12:22:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd58006c630 con 0x7fd5800b26b0 2024-09-12T12:22:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd5886f2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd58006c910 con 0x7fd5800b3820 2024-09-12T12:22:23.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd58015c580 0x7fd58006bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd58015c580 0x7fd58006bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47956/0 (socket says 172.21.15.3:47956) 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 -- 172.21.15.3:0/247785036 learned_addr learned my addr 172.21.15.3:0/247785036 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd586c8f640 1 --2- 172.21.15.3:0/247785036 >> v2:172.21.15.142:3300/0 conn(0x7fd5800b26b0 0x7fd5800ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd585c8d640 1 --2- 172.21.15.3:0/247785036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b3820 0x7fd5800b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 -- 172.21.15.3:0/247785036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b3820 msgr2=0x7fd5800b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 --2- 172.21.15.3:0/247785036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b3820 0x7fd5800b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 -- 172.21.15.3:0/247785036 >> v2:172.21.15.142:3300/0 conn(0x7fd5800b26b0 msgr2=0x7fd5800ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 --2- 172.21.15.3:0/247785036 >> v2:172.21.15.142:3300/0 conn(0x7fd5800b26b0 0x7fd5800ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd58648e640 1 -- 172.21.15.3:0/247785036 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5800b4090 con 0x7fd58015c580 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd586c8f640 1 --2- 172.21.15.3:0/247785036 >> v2:172.21.15.142:3300/0 conn(0x7fd5800b26b0 0x7fd5800ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:23.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.980+0000 7fd585c8d640 1 --2- 172.21.15.3:0/247785036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b3820 0x7fd5800b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:23.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd58648e640 1 --2- 172.21.15.3:0/247785036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd58015c580 0x7fd58006bcc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009870 tx=0x7fd57c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369fa1cb79f26e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:23.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd58548c640 1 -- 172.21.15.3:0/247785036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0ab5f0 con 0x7fd58015c580 2024-09-12T12:22:23.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd58548c640 1 -- 172.21.15.3:0/247785036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd57c0b0070 con 0x7fd58015c580 2024-09-12T12:22:23.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd58548c640 1 -- 172.21.15.3:0/247785036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0ab9b0 con 0x7fd58015c580 2024-09-12T12:22:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 -- 172.21.15.3:0/247785036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd58015c580 msgr2=0x7fd58006bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 --2- 172.21.15.3:0/247785036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd58015c580 0x7fd58006bcc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009870 tx=0x7fd57c01c520 comp rx=0 tx=0).stop 2024-09-12T12:22:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 -- 172.21.15.3:0/247785036 shutdown_connections 2024-09-12T12:22:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 --2- 172.21.15.3:0/247785036 >> v2:172.21.15.142:3300/0 conn(0x7fd5800b26b0 0x7fd5800ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 --2- 172.21.15.3:0/247785036 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b3820 0x7fd5800b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 --2- 172.21.15.3:0/247785036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd58015c580 0x7fd58006bcc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 -- 172.21.15.3:0/247785036 >> 172.21.15.3:0/247785036 conn(0x7fd58009aa70 msgr2=0x7fd5800b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:23.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 -- 172.21.15.3:0/247785036 shutdown_connections 2024-09-12T12:22:23.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.984+0000 7fd5886f2640 1 -- 172.21.15.3:0/247785036 wait complete. 2024-09-12T12:22:23.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 Processor -- start 2024-09-12T12:22:23.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 -- start start 2024-09-12T12:22:23.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b26b0 0x7fd58014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5800b3820 0x7fd5800c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd58015c580 0x7fd58015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800b4f00 con 0x7fd5800b3820 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800b4d80 con 0x7fd58015c580 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd5800b5080 con 0x7fd5800b26b0 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5800b3820 0x7fd5800c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5800b3820 0x7fd5800c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47964/0 (socket says 172.21.15.3:47964) 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 -- 172.21.15.3:0/2970857662 learned_addr learned my addr 172.21.15.3:0/2970857662 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd58648e640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b26b0 0x7fd58014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd586c8f640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd58015c580 0x7fd58015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 -- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b26b0 msgr2=0x7fd58014f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b26b0 0x7fd58014f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 -- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd58015c580 msgr2=0x7fd58015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:23.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd58015c580 0x7fd58015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:23.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 -- 172.21.15.3:0/2970857662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58014fa50 con 0x7fd5800b3820 2024-09-12T12:22:23.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd585c8d640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5800b3820 0x7fd5800c6bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c7f0 tx=0x7fd57000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:23.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd576ffd640 1 -- 172.21.15.3:0/2970857662 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570015070 con 0x7fd5800b3820 2024-09-12T12:22:23.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd576ffd640 1 -- 172.21.15.3:0/2970857662 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570007a30 con 0x7fd5800b3820 2024-09-12T12:22:23.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58015d510 con 0x7fd5800b3820 2024-09-12T12:22:23.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd576ffd640 1 -- 172.21.15.3:0/2970857662 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570007e20 con 0x7fd5800b3820 2024-09-12T12:22:23.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5800c8b70 con 0x7fd5800b3820 2024-09-12T12:22:23.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.988+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5800c4ad0 con 0x7fd5800b3820 2024-09-12T12:22:23.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.992+0000 7fd576ffd640 1 -- 172.21.15.3:0/2970857662 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd570018070 con 0x7fd5800b3820 2024-09-12T12:22:23.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.992+0000 7fd576ffd640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5640678c0 0x7fd564069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:23.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.992+0000 7fd576ffd640 1 -- 172.21.15.3:0/2970857662 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7fd570085650 con 0x7fd5800b3820 2024-09-12T12:22:23.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.992+0000 7fd58648e640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5640678c0 0x7fd564069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:23.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.992+0000 7fd58648e640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5640678c0 0x7fd564069d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0af000 tx=0x7fd57c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:24.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:23.996+0000 7fd576ffd640 1 -- 172.21.15.3:0/2970857662 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd570056410 con 0x7fd5800b3820 2024-09-12T12:22:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.160+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd5800cb850 con 0x7fd5800b3820 2024-09-12T12:22:24.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.160+0000 7fd576ffd640 1 -- 172.21.15.3:0/2970857662 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7fd570055d10 con 0x7fd5800b3820 2024-09-12T12:22:24.166 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T12:22:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5640678c0 msgr2=0x7fd564069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5640678c0 0x7fd564069d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd57c0af000 tx=0x7fd57c0a7000 comp rx=0 tx=0).stop 2024-09-12T12:22:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5800b3820 msgr2=0x7fd5800c6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5800b3820 0x7fd5800c6bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c7f0 tx=0x7fd57000ccb0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 shutdown_connections 2024-09-12T12:22:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5640678c0 0x7fd564069d80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd58015c580 0x7fd58015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5800b3820 0x7fd5800c6bf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 --2- 172.21.15.3:0/2970857662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5800b26b0 0x7fd58014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 >> 172.21.15.3:0/2970857662 conn(0x7fd58009aa70 msgr2=0x7fd5800b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 shutdown_connections 2024-09-12T12:22:24.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.164+0000 7fd5886f2640 1 -- 172.21.15.3:0/2970857662 wait complete. 2024-09-12T12:22:24.185 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-09-12T12:22:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f2224844640 1 Processor -- start 2024-09-12T12:22:24.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f2224844640 1 -- start start 2024-09-12T12:22:24.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f2224844640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 0x7f22200cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f2224844640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f22200cfa10 0x7f22200b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f2224844640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22200d31a0 con 0x7f222015e5d0 2024-09-12T12:22:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f2224844640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f22200b1060 con 0x7f22200cfa10 2024-09-12T12:22:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f2224844640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f22200cfe40 con 0x7f22200d5050 2024-09-12T12:22:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f221e59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f222015e5d0 0x7f22200ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39290/0 (socket says 172.21.15.3:39290) 2024-09-12T12:22:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f221e59c640 1 -- 172.21.15.3:0/826600376 learned_addr learned my addr 172.21.15.3:0/826600376 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f221ed9d640 1 --2- 172.21.15.3:0/826600376 >> v2:172.21.15.142:3300/0 conn(0x7f22200cfa10 0x7f22200b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f221dd9b640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 0x7f22200cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 96893928 0 0) 0x7f22200d31a0 con 0x7f222015e5d0 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.308+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2208003530 con 0x7f222015e5d0 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2537396451 0 0) 0x7f2208003530 con 0x7f222015e5d0 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 msgr2=0x7f22200cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 0x7f22200cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 >> v2:172.21.15.142:3300/0 conn(0x7f22200cfa10 msgr2=0x7f22200b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 --2- 172.21.15.3:0/826600376 >> v2:172.21.15.142:3300/0 conn(0x7f22200cfa10 0x7f22200b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22200d3480 con 0x7f222015e5d0 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f2210001680 con 0x7f222015e5d0 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 >> v1:172.21.15.3:6789/0 conn(0x7f222015e5d0 legacy=0x7f22200ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 0x7f22080039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2208003f90 0x7f2208004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208004d50 0x7f22080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f22200b1060 con 0x7f2208004d50 2024-09-12T12:22:24.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f22200cfe40 con 0x7f2208003f90 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f220800b9d0 con 0x7f22200d5050 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208004d50 0x7f22080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221e59c640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 0x7f22080039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221dd9b640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2208003f90 0x7f2208004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 -- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 msgr2=0x7f22080039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f22200d5050 0x7f22080039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 -- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2208003f90 msgr2=0x7f2208004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2208003f90 0x7f2208004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 -- 172.21.15.3:0/826600376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2208003530 con 0x7f2208004d50 2024-09-12T12:22:24.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208004d50 0x7f22080051d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f221800e170 tx=0x7f221801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba77018acc59d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:24.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22180b2070 con 0x7f2208004d50 2024-09-12T12:22:24.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22180a5cb0 con 0x7f2208004d50 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221d59a640 1 -- 172.21.15.3:0/826600376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218009c10 con 0x7f2208004d50 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f2224844640 1 -- 172.21.15.3:0/826600376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208004d50 msgr2=0x7f22080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f2224844640 1 --2- 172.21.15.3:0/826600376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2208004d50 0x7f22080051d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f221800e170 tx=0x7f221801cca0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f221ed9d640 1 -- 172.21.15.3:0/826600376 reap_dead start 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.312+0000 7f2224844640 1 -- 172.21.15.3:0/826600376 shutdown_connections 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- 172.21.15.3:0/826600376 >> 172.21.15.3:0/826600376 conn(0x7f222008c000 msgr2=0x7f22200d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- 172.21.15.3:0/826600376 shutdown_connections 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- 172.21.15.3:0/826600376 wait complete. 2024-09-12T12:22:24.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 Processor -- start 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- start start 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2208004d50 0x7f22200cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222015ecb0 0x7f2220158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2220159a60 0x7f22200b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f222015eb30 con 0x7f2220159a60 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f222015e9b0 con 0x7f222015ecb0 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f22200cf4d0 con 0x7f2208004d50 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2220159a60 0x7f22200b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2220159a60 0x7f22200b8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47986/0 (socket says 172.21.15.3:47986) 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 -- 172.21.15.3:0/3459910603 learned_addr learned my addr 172.21.15.3:0/3459910603 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221dd9b640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222015ecb0 0x7f2220158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221e59c640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2208004d50 0x7f22200cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 -- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2208004d50 msgr2=0x7f22200cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:24.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2208004d50 0x7f22200cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 -- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222015ecb0 msgr2=0x7f2220158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222015ecb0 0x7f2220158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 -- 172.21.15.3:0/3459910603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222006e870 con 0x7f2220159a60 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f221ed9d640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2220159a60 0x7f22200b8cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f221800e170 tx=0x7f2218009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f220effd640 1 -- 172.21.15.3:0/3459910603 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22180bc070 con 0x7f2220159a60 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f220effd640 1 -- 172.21.15.3:0/3459910603 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22180adb50 con 0x7f2220159a60 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22200758f0 con 0x7f2220159a60 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f220effd640 1 -- 172.21.15.3:0/3459910603 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22180b2040 con 0x7f2220159a60 2024-09-12T12:22:24.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.316+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22200b4ab0 con 0x7f2220159a60 2024-09-12T12:22:24.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.320+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22200ceb60 con 0x7f2220159a60 2024-09-12T12:22:24.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.320+0000 7f220effd640 1 -- 172.21.15.3:0/3459910603 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f22180bf070 con 0x7f2220159a60 2024-09-12T12:22:24.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.320+0000 7f220effd640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22000678f0 0x7f2200069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:24.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.320+0000 7f220effd640 1 -- 172.21.15.3:0/3459910603 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7f221811fad0 con 0x7f2220159a60 2024-09-12T12:22:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.320+0000 7f221e59c640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22000678f0 0x7f2200069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:24.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.320+0000 7f221e59c640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22000678f0 0x7f2200069db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2210001060 tx=0x7f2210009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:24.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.324+0000 7f220effd640 1 -- 172.21.15.3:0/3459910603 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2218016a30 con 0x7f2220159a60 2024-09-12T12:22:24.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:24.552+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7f22200ba490 con 0x7f2220159a60 2024-09-12T12:22:25.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.496+0000 7f220effd640 1 -- 172.21.15.3:0/3459910603 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v147) ==== 145+0+0 (secure 0 0 0) 0x7f22180f02d0 con 0x7f2220159a60 2024-09-12T12:22:25.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22000678f0 msgr2=0x7f2200069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22000678f0 0x7f2200069db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2210001060 tx=0x7f2210009040 comp rx=0 tx=0).stop 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2220159a60 msgr2=0x7f22200b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2220159a60 0x7f22200b8cc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f221800e170 tx=0x7f2218009750 comp rx=0 tx=0).stop 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 shutdown_connections 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f22000678f0 0x7f2200069db0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2220159a60 0x7f22200b8cc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f222015ecb0 0x7f2220158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 --2- 172.21.15.3:0/3459910603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2208004d50 0x7f22200cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 >> 172.21.15.3:0/3459910603 conn(0x7f222008c000 msgr2=0x7f22200d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 shutdown_connections 2024-09-12T12:22:25.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.500+0000 7f2224844640 1 -- 172.21.15.3:0/3459910603 wait complete. 2024-09-12T12:22:25.525 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T12:22:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a6626640 1 Processor -- start 2024-09-12T12:22:25.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a6626640 1 -- start start 2024-09-12T12:22:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a6626640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 0x7f41a006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a6626640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a00d1690 con 0x7f41a00b6b10 2024-09-12T12:22:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a6626640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41a00d1970 con 0x7f41a00b6350 2024-09-12T12:22:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a6626640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41a00d23f0 con 0x7f41a00b5f70 2024-09-12T12:22:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f41a00b6b10 0x7f41a00d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39294/0 (socket says 172.21.15.3:39294) 2024-09-12T12:22:25.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419f7fe640 1 -- 172.21.15.3:0/1754389308 learned_addr learned my addr 172.21.15.3:0/1754389308 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419ffff640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 0x7f41a006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851289389 0 0) 0x7f41a00d1690 con 0x7f41a00b6b10 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4188003530 con 0x7f41a00b6b10 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3180731462 0 0) 0x7f4188003530 con 0x7f41a00b6b10 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 msgr2=0x7f41a006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 0x7f41a006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 >> v1:172.21.15.142:6789/0 conn(0x7f41a00b6350 legacy=0x7f41a00cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a00d26d0 con 0x7f41a00b6b10 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f4194001680 con 0x7f41a00b6b10 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 >> v1:172.21.15.3:6789/0 conn(0x7f41a00b6b10 legacy=0x7f41a00d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41880039f0 0x7f4188003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 0x7f4188004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4188004d50 0x7f41880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f41a00d23f0 con 0x7f4188004d50 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f418800ba80 con 0x7f41880039f0 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f418800bc80 con 0x7f41a00b5f70 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a4bc3640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4188004d50 0x7f41880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419f7fe640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 0x7f4188004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419ffff640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41880039f0 0x7f4188003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a4bc3640 1 -- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 msgr2=0x7f4188004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a4bc3640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 0x7f4188004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a4bc3640 1 -- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41880039f0 msgr2=0x7f4188003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a4bc3640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41880039f0 0x7f4188003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f41a4bc3640 1 -- 172.21.15.3:0/1754389308 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4188003530 con 0x7f4188004d50 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419f7fe640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a00b5f70 0x7f4188004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.656+0000 7f419ffff640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41880039f0 0x7f4188003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a4bc3640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4188004d50 0x7f41880051d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4198001e40 tx=0x7f419801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbf5cbb9bc82335 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198001a90 con 0x7f4188004d50 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4198003b80 con 0x7f4188004d50 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f419effd640 1 -- 172.21.15.3:0/1754389308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198005080 con 0x7f4188004d50 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a6626640 1 -- 172.21.15.3:0/1754389308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4188004d50 msgr2=0x7f41880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a6626640 1 --2- 172.21.15.3:0/1754389308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4188004d50 0x7f41880051d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4198001e40 tx=0x7f419801c760 comp rx=0 tx=0).stop 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a4bc3640 1 -- 172.21.15.3:0/1754389308 reap_dead start 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a6626640 1 -- 172.21.15.3:0/1754389308 shutdown_connections 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a6626640 1 -- 172.21.15.3:0/1754389308 >> 172.21.15.3:0/1754389308 conn(0x7f41a009d880 msgr2=0x7f41a008c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a6626640 1 -- 172.21.15.3:0/1754389308 shutdown_connections 2024-09-12T12:22:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.660+0000 7f41a6626640 1 -- 172.21.15.3:0/1754389308 wait complete. 2024-09-12T12:22:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 Processor -- start 2024-09-12T12:22:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 -- start start 2024-09-12T12:22:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41a00b7940 0x7f41a0147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a01f7d20 0x7f41a01f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41a00cdc00 0x7f41a00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f41a015ea90 con 0x7f41a00cdc00 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f41a00b6800 con 0x7f41a00b7940 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f41a006e780 con 0x7f41a01f7d20 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41a00cdc00 0x7f41a00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41a00cdc00 0x7f41a00ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48010/0 (socket says 172.21.15.3:48010) 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 -- 172.21.15.3:0/3850930933 learned_addr learned my addr 172.21.15.3:0/3850930933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419f7fe640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a01f7d20 0x7f41a01f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419ffff640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41a00b7940 0x7f41a0147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 -- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a01f7d20 msgr2=0x7f41a01f81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a01f7d20 0x7f41a01f81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 -- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41a00b7940 msgr2=0x7f41a0147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41a00b7940 0x7f41a0147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 -- 172.21.15.3:0/3850930933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a00ba260 con 0x7f41a00cdc00 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419f7fe640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a01f7d20 0x7f41a01f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419ffff640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41a00b7940 0x7f41a0147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a4bc3640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41a00cdc00 0x7f41a00ce0b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f419801ccd0 tx=0x7f4188005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419cff9640 1 -- 172.21.15.3:0/3850930933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41980b7070 con 0x7f41a00cdc00 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419cff9640 1 -- 172.21.15.3:0/3850930933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4198006b60 con 0x7f41a00cdc00 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a015fc60 con 0x7f41a00cdc00 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419cff9640 1 -- 172.21.15.3:0/3850930933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198007dc0 con 0x7f41a00cdc00 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a0160220 con 0x7f41a00cdc00 2024-09-12T12:22:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a00d1510 con 0x7f41a00cdc00 2024-09-12T12:22:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419cff9640 1 -- 172.21.15.3:0/3850930933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f41980ba070 con 0x7f41a00cdc00 2024-09-12T12:22:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419cff9640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f418c0678f0 0x7f418c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.664+0000 7f419cff9640 1 -- 172.21.15.3:0/3850930933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7f419811ec30 con 0x7f41a00cdc00 2024-09-12T12:22:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.668+0000 7f419ffff640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f418c0678f0 0x7f418c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.668+0000 7f419ffff640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f418c0678f0 0x7f418c069db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f41a0147950 tx=0x7f4190008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:25.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.672+0000 7f419cff9640 1 -- 172.21.15.3:0/3850930933 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f41980ac020 con 0x7f41a00cdc00 2024-09-12T12:22:25.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.836+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f41a006e560 con 0x7f41a00cdc00 2024-09-12T12:22:25.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.840+0000 7f419cff9640 1 -- 172.21.15.3:0/3850930933 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f4198016470 con 0x7f41a00cdc00 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f418c0678f0 msgr2=0x7f418c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f418c0678f0 0x7f418c069db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f41a0147950 tx=0x7f4190008040 comp rx=0 tx=0).stop 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41a00cdc00 msgr2=0x7f41a00ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41a00cdc00 0x7f41a00ce0b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f419801ccd0 tx=0x7f4188005840 comp rx=0 tx=0).stop 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 shutdown_connections 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f418c0678f0 0x7f418c069db0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f41a00cdc00 0x7f41a00ce0b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f41a01f7d20 0x7f41a01f81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 --2- 172.21.15.3:0/3850930933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f41a00b7940 0x7f41a0147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 >> 172.21.15.3:0/3850930933 conn(0x7f41a009d880 msgr2=0x7f41a00d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:25.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 shutdown_connections 2024-09-12T12:22:25.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:25.844+0000 7f41a6626640 1 -- 172.21.15.3:0/3850930933 wait complete. 2024-09-12T12:22:25.864 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:22:25.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:22:25.899 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:22:25.957 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:22:25.958 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:22:25.958 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:25.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:22:25.963 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:25.963 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:25.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:22:25.970 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:22:25.970 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:26.011 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:22:26.011 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:26.011 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:26.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:22:26.017 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T12:22:26.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:22:26.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb74d650640 1 Processor -- start 2024-09-12T12:22:26.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb74d650640 1 -- start start 2024-09-12T12:22:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb74d650640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb74815b2d0 0x7fb7480b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb74d650640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb74806f6e0 0x7fb748072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb74d650640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb74815f150 con 0x7fb7480cc300 2024-09-12T12:22:26.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb74d650640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb7480ae920 con 0x7fb74806f6e0 2024-09-12T12:22:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb74d650640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb74815b6b0 con 0x7fb74815b2d0 2024-09-12T12:22:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7467fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb74806f6e0 0x7fb748072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7467fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb74806f6e0 0x7fb748072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:38266/0 (socket says 172.21.15.3:38266) 2024-09-12T12:22:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7477fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fb7480cc300 0x7fb7480aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:50344/0 (socket says 172.21.15.3:50344) 2024-09-12T12:22:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7467fc640 1 -- 172.21.15.3:0/1353984409 learned_addr learned my addr 172.21.15.3:0/1353984409 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:26.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb746ffd640 1 --2- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb74815b2d0 0x7fb7480b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7467fc640 1 -- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb74815b2d0 msgr2=0x7fb7480b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7467fc640 1 --2- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb74815b2d0 0x7fb7480b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7467fc640 1 -- 172.21.15.3:0/1353984409 >> v1:172.21.15.142:6789/0 conn(0x7fb7480cc300 legacy=0x7fb7480aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.188+0000 7fb7467fc640 1 -- 172.21.15.3:0/1353984409 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74815ee70 con 0x7fb74806f6e0 2024-09-12T12:22:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb7467fc640 1 --2- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 0x7fb748072f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009870 tx=0x7fb73c01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61267c58c6ed78a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb745ffb640 1 -- 172.21.15.3:0/1353984409 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0ab640 con 0x7fb74806f6e0 2024-09-12T12:22:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb745ffb640 1 -- 172.21.15.3:0/1353984409 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c0b0070 con 0x7fb74806f6e0 2024-09-12T12:22:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb745ffb640 1 -- 172.21.15.3:0/1353984409 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c0aba00 con 0x7fb74806f6e0 2024-09-12T12:22:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 msgr2=0x7fb748072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 --2- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 0x7fb748072f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009870 tx=0x7fb73c01c530 comp rx=0 tx=0).stop 2024-09-12T12:22:26.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- 172.21.15.3:0/1353984409 shutdown_connections 2024-09-12T12:22:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 --2- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 0x7fb748072f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 --2- 172.21.15.3:0/1353984409 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb74815b2d0 0x7fb7480b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- 172.21.15.3:0/1353984409 >> 172.21.15.3:0/1353984409 conn(0x7fb7480b1d80 msgr2=0x7fb7480b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- 172.21.15.3:0/1353984409 shutdown_connections 2024-09-12T12:22:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- 172.21.15.3:0/1353984409 wait complete. 2024-09-12T12:22:26.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 Processor -- start 2024-09-12T12:22:26.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- start start 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 0x7fb748155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480cc300 0x7fb7481faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb74815b2d0 0x7fb7481f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb74806d8b0 con 0x7fb74815b2d0 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb748073880 con 0x7fb74806f6e0 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb74d650640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb7480b35c0 con 0x7fb7480cc300 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb7477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb74815b2d0 0x7fb7481f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb7477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb74815b2d0 0x7fb7481f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48026/0 (socket says 172.21.15.3:48026) 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.192+0000 7fb7477fe640 1 -- 172.21.15.3:0/378243978 learned_addr learned my addr 172.21.15.3:0/378243978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb746ffd640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 0x7fb748155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb7467fc640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480cc300 0x7fb7481faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb7477fe640 1 -- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480cc300 msgr2=0x7fb7481faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb7477fe640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480cc300 0x7fb7481faa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb7477fe640 1 -- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 msgr2=0x7fb748155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb7477fe640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 0x7fb748155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb7477fe640 1 -- 172.21.15.3:0/378243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7481fb2d0 con 0x7fb74815b2d0 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb7477fe640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb74815b2d0 0x7fb7481f7da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c3f0 tx=0x7fb74000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb72f7fe640 1 -- 172.21.15.3:0/378243978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7400074e0 con 0x7fb74815b2d0 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb72f7fe640 1 -- 172.21.15.3:0/378243978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb740007680 con 0x7fb74815b2d0 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7481fb5b0 con 0x7fb74815b2d0 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb72f7fe640 1 -- 172.21.15.3:0/378243978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740004a10 con 0x7fb74815b2d0 2024-09-12T12:22:26.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7481f75a0 con 0x7fb74815b2d0 2024-09-12T12:22:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7480b0030 con 0x7fb74815b2d0 2024-09-12T12:22:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb72f7fe640 1 -- 172.21.15.3:0/378243978 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb74001a020 con 0x7fb74815b2d0 2024-09-12T12:22:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb72f7fe640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb7340678c0 0x7fb734069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb72f7fe640 1 -- 172.21.15.3:0/378243978 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7fb740085200 con 0x7fb74815b2d0 2024-09-12T12:22:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.196+0000 7fb746ffd640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb7340678c0 0x7fb734069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.200+0000 7fb746ffd640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb7340678c0 0x7fb734069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb7380074a0 tx=0x7fb73800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:26.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.200+0000 7fb72f7fe640 1 -- 172.21.15.3:0/378243978 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb740055f40 con 0x7fb74815b2d0 2024-09-12T12:22:26.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7481fafc0 con 0x7fb74815b2d0 2024-09-12T12:22:26.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb72f7fe640 1 -- 172.21.15.3:0/378243978 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3911 (secure 0 0 0) 0x7fb740055840 con 0x7fb74815b2d0 2024-09-12T12:22:26.403 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:26.403 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T12:22:20:463544+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4945,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/1730233184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1730233184},{"type":"v1","addr":"172.21.15.142:6836","nonce":1730233184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3978467517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":3978467517},{"type":"v1","addr":"172.21.15.142:6837","nonce":3978467517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/4080842654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":4080842654},{"type":"v1","addr":"172.21.15.3:6833","nonce":4080842654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2019053569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2019053569},{"type":"v1","addr":"172.21.15.3:6835","nonce":2019053569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-12T12:22:26.404 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb7340678c0 msgr2=0x7fb734069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb7340678c0 0x7fb734069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb7380074a0 tx=0x7fb73800b040 comp rx=0 tx=0).stop 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb74815b2d0 msgr2=0x7fb7481f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb74815b2d0 0x7fb7481f7da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb74000c3f0 tx=0x7fb74000c8b0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 shutdown_connections 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb7340678c0 0x7fb734069d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb74815b2d0 0x7fb7481f7da0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480cc300 0x7fb7481faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 --2- 172.21.15.3:0/378243978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb74806f6e0 0x7fb748155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 >> 172.21.15.3:0/378243978 conn(0x7fb7480b1d80 msgr2=0x7fb7480750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:26.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.400+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 shutdown_connections 2024-09-12T12:22:26.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.404+0000 7fb74d650640 1 -- 172.21.15.3:0/378243978 wait complete. 2024-09-12T12:22:26.422 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:22:26.422 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T12:22:26.422 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:22:26.423 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:22:26.423 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:22:26.423 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T12:22:26.423 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:22:26.424 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:22:26.424 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:22:26.424 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T12:22:26.424 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:22:26.424 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:22:26.424 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:22:26.425 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T12:22:26.425 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:22:26.425 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:22:26.432 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:22:26.432 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:22:26.435 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:22:26.435 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:22:26.438 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:22:26.438 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:22:26.439 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:22:26.440 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:22:26.478 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:22:26.479 DEBUG:teuthology.parallel:result is None 2024-09-12T12:22:26.480 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:22:26.480 DEBUG:teuthology.parallel:result is None 2024-09-12T12:22:26.485 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:22:26.485 DEBUG:teuthology.parallel:result is None 2024-09-12T12:22:26.486 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:22:26.487 DEBUG:teuthology.parallel:result is None 2024-09-12T12:22:26.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T12:22:26.539 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:22:26.550 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:22:26.556 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:22:26.561 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:22:26.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f126b094640 1 Processor -- start 2024-09-12T12:22:26.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f126b094640 1 -- start start 2024-09-12T12:22:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f126b094640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f12641568f0 0x7f12640aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f126b094640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640b47a0 0x7f12640b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f126b094640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12640afb00 con 0x7f12640d0380 2024-09-12T12:22:26.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f126b094640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f12640b31b0 con 0x7f12641568f0 2024-09-12T12:22:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f126b094640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f12640af540 con 0x7f12640b47a0 2024-09-12T12:22:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1263fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640b47a0 0x7f12640b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1269631640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f12640d0380 0x7f126415c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:50350/0 (socket says 172.21.15.3:50350) 2024-09-12T12:22:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1269631640 1 -- 172.21.15.3:0/2858323738 learned_addr learned my addr 172.21.15.3:0/2858323738 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1268e30640 1 --2- 172.21.15.3:0/2858323738 >> v2:172.21.15.142:3300/0 conn(0x7f12641568f0 0x7f12640aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1263fff640 1 -- 172.21.15.3:0/2858323738 >> v2:172.21.15.142:3300/0 conn(0x7f12641568f0 msgr2=0x7f12640aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:26.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1263fff640 1 --2- 172.21.15.3:0/2858323738 >> v2:172.21.15.142:3300/0 conn(0x7f12641568f0 0x7f12640aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1263fff640 1 -- 172.21.15.3:0/2858323738 >> v1:172.21.15.142:6789/0 conn(0x7f12640d0380 legacy=0x7f126415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f1263fff640 1 -- 172.21.15.3:0/2858323738 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12640af820 con 0x7f12640b47a0 2024-09-12T12:22:26.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.596+0000 7f12637fe640 1 -- 172.21.15.3:0/2858323738 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199897293 0 0) 0x7f12640afb00 con 0x7f12640d0380 2024-09-12T12:22:26.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f1263fff640 1 --2- 172.21.15.3:0/2858323738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640b47a0 0x7f12640b3970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12580097c0 tx=0x7f125801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6112b394a40272 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f12637fe640 1 -- 172.21.15.3:0/2858323738 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12580b0070 con 0x7f12640b47a0 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f12637fe640 1 -- 172.21.15.3:0/2858323738 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12580ab600 con 0x7f12640b47a0 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f12637fe640 1 -- 172.21.15.3:0/2858323738 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12580ab9d0 con 0x7f12640b47a0 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- 172.21.15.3:0/2858323738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640b47a0 msgr2=0x7f12640b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 --2- 172.21.15.3:0/2858323738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640b47a0 0x7f12640b3970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12580097c0 tx=0x7f125801c410 comp rx=0 tx=0).stop 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- 172.21.15.3:0/2858323738 shutdown_connections 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 --2- 172.21.15.3:0/2858323738 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640b47a0 0x7f12640b3970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 --2- 172.21.15.3:0/2858323738 >> v2:172.21.15.142:3300/0 conn(0x7f12641568f0 0x7f12640aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- 172.21.15.3:0/2858323738 >> 172.21.15.3:0/2858323738 conn(0x7f126406be90 msgr2=0x7f126406c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:26.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- 172.21.15.3:0/2858323738 shutdown_connections 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- 172.21.15.3:0/2858323738 wait complete. 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 Processor -- start 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- start start 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12640b47a0 0x7f1264155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640d0380 0x7f126415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12641568f0 0x7f126415f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12640b3eb0 con 0x7f12640b47a0 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f126415d3b0 con 0x7f12641568f0 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.600+0000 7f126b094640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f12641696e0 con 0x7f12640d0380 2024-09-12T12:22:26.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12640b47a0 0x7f1264155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12640b47a0 0x7f1264155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48080/0 (socket says 172.21.15.3:48080) 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 -- 172.21.15.3:0/3602496580 learned_addr learned my addr 172.21.15.3:0/3602496580 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1263fff640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640d0380 0x7f126415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1269631640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12641568f0 0x7f126415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 -- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640d0380 msgr2=0x7f126415ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640d0380 0x7f126415ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 -- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12641568f0 msgr2=0x7f126415f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12641568f0 0x7f126415f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 -- 172.21.15.3:0/3602496580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641560c0 con 0x7f12640b47a0 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1269631640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12641568f0 0x7f126415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1268e30640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12640b47a0 0x7f1264155b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f125400e9b0 tx=0x7f125400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f12617fa640 1 -- 172.21.15.3:0/3602496580 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254017070 con 0x7f12640b47a0 2024-09-12T12:22:26.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f12617fa640 1 -- 172.21.15.3:0/3602496580 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125400c9a0 con 0x7f12640b47a0 2024-09-12T12:22:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126409c530 con 0x7f12640b47a0 2024-09-12T12:22:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f12617fa640 1 -- 172.21.15.3:0/3602496580 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125400cd70 con 0x7f12640b47a0 2024-09-12T12:22:26.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1264073580 con 0x7f12640b47a0 2024-09-12T12:22:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12640b3970 con 0x7f12640b47a0 2024-09-12T12:22:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f12617fa640 1 -- 172.21.15.3:0/3602496580 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f125401a070 con 0x7f12640b47a0 2024-09-12T12:22:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f12617fa640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f12500678c0 0x7f1250069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f12617fa640 1 -- 172.21.15.3:0/3602496580 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5072+0+0 (secure 0 0 0) 0x7f1254085a10 con 0x7f12640b47a0 2024-09-12T12:22:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1263fff640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f12500678c0 0x7f1250069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:26.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.604+0000 7f1263fff640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f12500678c0 0x7f1250069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12580ad000 tx=0x7f1258008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:26.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.608+0000 7f12617fa640 1 -- 172.21.15.3:0/3602496580 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1254056190 con 0x7f12640b47a0 2024-09-12T12:22:26.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:26.784+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f126409c700 con 0x7f12640b47a0 2024-09-12T12:22:27.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.520+0000 7f12617fa640 1 -- 172.21.15.3:0/3602496580 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) ==== 97+0+0 (secure 0 0 0) 0x7f125405a490 con 0x7f12640b47a0 2024-09-12T12:22:27.523 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-12T12:22:27.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f12500678c0 msgr2=0x7f1250069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:27.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f12500678c0 0x7f1250069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12580ad000 tx=0x7f1258008040 comp rx=0 tx=0).stop 2024-09-12T12:22:27.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12640b47a0 msgr2=0x7f1264155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:27.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12640b47a0 0x7f1264155b80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f125400e9b0 tx=0x7f125400ee70 comp rx=0 tx=0).stop 2024-09-12T12:22:27.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 shutdown_connections 2024-09-12T12:22:27.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f12500678c0 0x7f1250069d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12641568f0 0x7f126415f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f12640d0380 0x7f126415ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 --2- 172.21.15.3:0/3602496580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12640b47a0 0x7f1264155b80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 >> 172.21.15.3:0/3602496580 conn(0x7f126406be90 msgr2=0x7f126415dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:27.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 shutdown_connections 2024-09-12T12:22:27.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.524+0000 7f126b094640 1 -- 172.21.15.3:0/3602496580 wait complete. 2024-09-12T12:22:27.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T12:22:27.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 Processor -- start 2024-09-12T12:22:27.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 -- start start 2024-09-12T12:22:27.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0c14159bd0 0x7f0c1406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:27.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159fb0 0x7f0c1415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:27.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c1406f4a0 0x7f0c140ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:27.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1406f010 con 0x7f0c1406f4a0 2024-09-12T12:22:27.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1406e400 con 0x7f0c14159bd0 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1c0d4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1406e6e0 con 0x7f0c14159fb0 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c1406f4a0 0x7f0c140ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0c1406f4a0 0x7f0c140ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48082/0 (socket says 172.21.15.3:48082) 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 -- 172.21.15.3:0/1402384010 learned_addr learned my addr 172.21.15.3:0/1402384010 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c19e70640 1 --2- 172.21.15.3:0/1402384010 >> v2:172.21.15.142:3300/0 conn(0x7f0c14159bd0 0x7f0c1406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1966f640 1 --2- 172.21.15.3:0/1402384010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159fb0 0x7f0c1415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 -- 172.21.15.3:0/1402384010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159fb0 msgr2=0x7f0c1415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 --2- 172.21.15.3:0/1402384010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159fb0 0x7f0c1415d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 -- 172.21.15.3:0/1402384010 >> v2:172.21.15.142:3300/0 conn(0x7f0c14159bd0 msgr2=0x7f0c1406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 --2- 172.21.15.3:0/1402384010 >> v2:172.21.15.142:3300/0 conn(0x7f0c14159bd0 0x7f0c1406dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.672+0000 7f0c1a671640 1 -- 172.21.15.3:0/1402384010 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1406e9c0 con 0x7f0c1406f4a0 2024-09-12T12:22:27.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 --2- 172.21.15.3:0/1402384010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1406f4a0 0x7f0c140ad060 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0c14071320 tx=0x7f0c0c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14378de2c5aa8302 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:27.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c18e6e640 1 -- 172.21.15.3:0/1402384010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0a5780 con 0x7f0c1406f4a0 2024-09-12T12:22:27.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c18e6e640 1 -- 172.21.15.3:0/1402384010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0a5920 con 0x7f0c1406f4a0 2024-09-12T12:22:27.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c18e6e640 1 -- 172.21.15.3:0/1402384010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0a5cf0 con 0x7f0c1406f4a0 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/1402384010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1406f4a0 msgr2=0x7f0c140ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/1402384010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1406f4a0 0x7f0c140ad060 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0c14071320 tx=0x7f0c0c01c9f0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/1402384010 shutdown_connections 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/1402384010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c1406f4a0 0x7f0c140ad060 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/1402384010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159fb0 0x7f0c1415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/1402384010 >> v2:172.21.15.142:3300/0 conn(0x7f0c14159bd0 0x7f0c1406dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/1402384010 >> 172.21.15.3:0/1402384010 conn(0x7f0c140b1d80 msgr2=0x7f0c140b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/1402384010 shutdown_connections 2024-09-12T12:22:27.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/1402384010 wait complete. 2024-09-12T12:22:27.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 Processor -- start 2024-09-12T12:22:27.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- start start 2024-09-12T12:22:27.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c1406f4a0 0x7f0c14075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:27.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159bd0 0x7f0c140b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c14159fb0 0x7f0c14072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14160210 con 0x7f0c14159fb0 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14160090 con 0x7f0c1406f4a0 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1c0d4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0c14160390 con 0x7f0c14159bd0 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c14159fb0 0x7f0c14072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c14159fb0 0x7f0c14072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48096/0 (socket says 172.21.15.3:48096) 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 -- 172.21.15.3:0/3018404578 learned_addr learned my addr 172.21.15.3:0/3018404578 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c19e70640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c1406f4a0 0x7f0c14075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1966f640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159bd0 0x7f0c140b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 -- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159bd0 msgr2=0x7f0c140b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:27.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159bd0 0x7f0c140b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 -- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c1406f4a0 msgr2=0x7f0c14075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c1406f4a0 0x7f0c14075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 -- 172.21.15.3:0/3018404578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c14073bd0 con 0x7f0c14159fb0 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c1a671640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c14159fb0 0x7f0c14072000 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0af040 tx=0x7f0c0c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.676+0000 7f0c0a7fc640 1 -- 172.21.15.3:0/3018404578 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0b8670 con 0x7f0c14159fb0 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c0a7fc640 1 -- 172.21.15.3:0/3018404578 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0be070 con 0x7f0c14159fb0 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c140af7c0 con 0x7f0c14159fb0 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c0a7fc640 1 -- 172.21.15.3:0/3018404578 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0b8a50 con 0x7f0c14159fb0 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1415fa00 con 0x7f0c14159fb0 2024-09-12T12:22:27.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c140ad060 con 0x7f0c14159fb0 2024-09-12T12:22:27.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c0a7fc640 1 -- 172.21.15.3:0/3018404578 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0c0c0b8bf0 con 0x7f0c14159fb0 2024-09-12T12:22:27.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c0a7fc640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0bf80678c0 0x7f0bf8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:27.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c0a7fc640 1 -- 172.21.15.3:0/3018404578 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4857+0+0 (secure 0 0 0) 0x7f0c0c120b40 con 0x7f0c14159fb0 2024-09-12T12:22:27.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c19e70640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0bf80678c0 0x7f0bf8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:27.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.680+0000 7f0c19e70640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0bf80678c0 0x7f0bf8069d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000b9d0 tx=0x7f0c10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:27.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.684+0000 7f0c0a7fc640 1 -- 172.21.15.3:0/3018404578 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0c0c0165e0 con 0x7f0c14159fb0 2024-09-12T12:22:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0c141575a0 con 0x7f0c14159fb0 2024-09-12T12:22:27.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c0a7fc640 1 -- 172.21.15.3:0/3018404578 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f0c0c0ac3a0 con 0x7f0c14159fb0 2024-09-12T12:22:27.916 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:27.916 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:22:27.916 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-12T12:22:27.916 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.917 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-12T12:22:27.917 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg==", 2024-09-12T12:22:27.917 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.917 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.918 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:22:27.918 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:22:27.918 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.918 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.919 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.919 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-12T12:22:27.919 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA==", 2024-09-12T12:22:27.919 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.919 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.920 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:22:27.920 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:22:27.921 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.921 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.921 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.921 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-12T12:22:27.922 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg==", 2024-09-12T12:22:27.922 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.922 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.922 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:22:27.923 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:22:27.923 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.923 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.924 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.924 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-12T12:22:27.925 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw==", 2024-09-12T12:22:27.925 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.925 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.925 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:22:27.925 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:22:27.926 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.926 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.926 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.926 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w==", 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA==", 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.927 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw==", 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg==", 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA==", 2024-09-12T12:22:27.929 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ==", 2024-09-12T12:22:27.930 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.931 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.931 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.931 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.931 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.931 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.931 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.932 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-12T12:22:27.932 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ==", 2024-09-12T12:22:27.932 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.932 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.932 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.932 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA==", 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.933 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.934 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.937 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-12T12:22:27.937 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA==", 2024-09-12T12:22:27.937 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.937 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.937 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-12T12:22:27.937 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T12:22:27.938 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.938 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.938 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-12T12:22:27.938 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw==", 2024-09-12T12:22:27.938 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.939 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.939 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:22:27.939 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:22:27.939 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:22:27.939 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.939 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.940 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-12T12:22:27.940 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw==", 2024-09-12T12:22:27.940 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.940 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.940 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:22:27.941 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:22:27.941 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:22:27.941 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.941 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.942 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-12T12:22:27.942 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA==", 2024-09-12T12:22:27.942 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.942 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:22:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:22:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:22:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:22:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.943 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.944 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.944 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-12T12:22:27.944 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKNVOAxAAMmejHiBys/ZYsGdB0oiroQ==", 2024-09-12T12:22:27.944 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.944 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:22:27.945 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:22:27.945 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:22:27.945 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.945 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.945 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.945 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-12T12:22:27.946 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA==", 2024-09-12T12:22:27.946 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.946 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:22:27.946 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:22:27.946 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.946 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:27.947 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:27.947 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-12T12:22:27.947 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg==", 2024-09-12T12:22:27.948 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:22:27.948 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:22:27.948 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:22:27.948 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:22:27.948 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.948 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:27.949 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:22:27.949 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:22:27.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0bf80678c0 msgr2=0x7f0bf8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:27.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0bf80678c0 0x7f0bf8069d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000b9d0 tx=0x7f0c10008040 comp rx=0 tx=0).stop 2024-09-12T12:22:27.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c14159fb0 msgr2=0x7f0c14072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c14159fb0 0x7f0c14072000 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0af040 tx=0x7f0c0c019790 comp rx=0 tx=0).stop 2024-09-12T12:22:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 shutdown_connections 2024-09-12T12:22:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0bf80678c0 0x7f0bf8069d80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c14159fb0 0x7f0c14072000 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0c14159bd0 0x7f0c140b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 --2- 172.21.15.3:0/3018404578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c1406f4a0 0x7f0c14075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:27.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 >> 172.21.15.3:0/3018404578 conn(0x7f0c140b1d80 msgr2=0x7f0c1415d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:27.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 shutdown_connections 2024-09-12T12:22:27.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:27.912+0000 7f0c1c0d4640 1 -- 172.21.15.3:0/3018404578 wait complete. 2024-09-12T12:22:27.954 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T12:22:28.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f0365bdb640 1 Processor -- start 2024-09-12T12:22:28.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f0365bdb640 1 -- start start 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f0365bdb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360144110 0x7f036015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f0365bdb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f036006dd40 0x7f03600727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f0365bdb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0360071ed0 con 0x7f036006f6b0 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f0365bdb640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0360072490 con 0x7f036006dd40 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f0365bdb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03600b45a0 con 0x7f0360144110 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f036006dd40 0x7f03600727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f036006dd40 0x7f03600727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37522/0 (socket says 172.21.15.3:37522) 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035ffff640 1 -- 172.21.15.3:0/1611471284 learned_addr learned my addr 172.21.15.3:0/1611471284 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035f7fe640 1 --2- 172.21.15.3:0/1611471284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360144110 0x7f036015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035ffff640 1 -- 172.21.15.3:0/1611471284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360144110 msgr2=0x7f036015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:28.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035ffff640 1 --2- 172.21.15.3:0/1611471284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360144110 0x7f036015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035ffff640 1 -- 172.21.15.3:0/1611471284 >> v1:172.21.15.142:6789/0 conn(0x7f036006f6b0 legacy=0x7f03600b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:28.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.064+0000 7f035ffff640 1 -- 172.21.15.3:0/1611471284 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03600721b0 con 0x7f036006dd40 2024-09-12T12:22:28.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035ffff640 1 --2- 172.21.15.3:0/1611471284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006dd40 0x7f03600727b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0358009e50 tx=0x7f035801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea86cda9c638f232 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:28.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035e7fc640 1 -- 172.21.15.3:0/1611471284 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03580a56e0 con 0x7f036006dd40 2024-09-12T12:22:28.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035e7fc640 1 -- 172.21.15.3:0/1611471284 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03580a5880 con 0x7f036006dd40 2024-09-12T12:22:28.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035e7fc640 1 -- 172.21.15.3:0/1611471284 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03580a5c50 con 0x7f036006dd40 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- 172.21.15.3:0/1611471284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006dd40 msgr2=0x7f03600727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 --2- 172.21.15.3:0/1611471284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006dd40 0x7f03600727b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0358009e50 tx=0x7f035801c8a0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- 172.21.15.3:0/1611471284 shutdown_connections 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 --2- 172.21.15.3:0/1611471284 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006dd40 0x7f03600727b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 --2- 172.21.15.3:0/1611471284 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0360144110 0x7f036015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- 172.21.15.3:0/1611471284 >> 172.21.15.3:0/1611471284 conn(0x7f03600b1d80 msgr2=0x7f03600b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- 172.21.15.3:0/1611471284 shutdown_connections 2024-09-12T12:22:28.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- 172.21.15.3:0/1611471284 wait complete. 2024-09-12T12:22:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 Processor -- start 2024-09-12T12:22:28.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- start start 2024-09-12T12:22:28.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f036006dd40 0x7f03600cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006f6b0 0x7f03600cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0360144110 0x7f0360098e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0360073100 con 0x7f036006f6b0 2024-09-12T12:22:28.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03600adfe0 con 0x7f0360144110 2024-09-12T12:22:28.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0365bdb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0360168720 con 0x7f036006dd40 2024-09-12T12:22:28.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006f6b0 0x7f03600cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006f6b0 0x7f03600cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37534/0 (socket says 172.21.15.3:37534) 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 -- 172.21.15.3:0/2670725029 learned_addr learned my addr 172.21.15.3:0/2670725029 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035f7fe640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f036006dd40 0x7f03600cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035ffff640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0360144110 0x7f0360098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 -- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f036006dd40 msgr2=0x7f03600cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f036006dd40 0x7f03600cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 -- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0360144110 msgr2=0x7f0360098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0360144110 0x7f0360098e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 -- 172.21.15.3:0/2670725029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03600ce0f0 con 0x7f036006f6b0 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035f7fe640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f036006dd40 0x7f03600cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035ffff640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0360144110 0x7f0360098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:28.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f035effd640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006f6b0 0x7f03600cd710 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f035400c450 tx=0x7f035400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:28.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.068+0000 7f0347fff640 1 -- 172.21.15.3:0/2670725029 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354012830 con 0x7f036006f6b0 2024-09-12T12:22:28.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0347fff640 1 -- 172.21.15.3:0/2670725029 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0354017070 con 0x7f036006f6b0 2024-09-12T12:22:28.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0347fff640 1 -- 172.21.15.3:0/2670725029 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354012c30 con 0x7f036006f6b0 2024-09-12T12:22:28.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03600ce3d0 con 0x7f036006f6b0 2024-09-12T12:22:28.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03600ce990 con 0x7f036006f6b0 2024-09-12T12:22:28.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0360158eb0 con 0x7f036006f6b0 2024-09-12T12:22:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0347fff640 1 -- 172.21.15.3:0/2670725029 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f035401c460 con 0x7f036006f6b0 2024-09-12T12:22:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0347fff640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f034c0678c0 0x7f034c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f0347fff640 1 -- 172.21.15.3:0/2670725029 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4857+0+0 (secure 0 0 0) 0x7f03540856d0 con 0x7f036006f6b0 2024-09-12T12:22:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f035f7fe640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f034c0678c0 0x7f034c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.072+0000 7f035f7fe640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f034c0678c0 0x7f034c069d80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0350009840 tx=0x7f0350008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:28.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.076+0000 7f0347fff640 1 -- 172.21.15.3:0/2670725029 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0354056560 con 0x7f036006f6b0 2024-09-12T12:22:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.244+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f036015a0b0 con 0x7f036006f6b0 2024-09-12T12:22:28.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.244+0000 7f0347fff640 1 -- 172.21.15.3:0/2670725029 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) ==== 84+0+65 (secure 0 0 0) 0x7f0354055e60 con 0x7f036006f6b0 2024-09-12T12:22:28.247 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:28.248 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T12:22:28.248 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:22:28.248 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T12:22:28.248 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T12:22:28.248 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:28.248 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T12:22:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f034c0678c0 msgr2=0x7f034c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f034c0678c0 0x7f034c069d80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0350009840 tx=0x7f0350008040 comp rx=0 tx=0).stop 2024-09-12T12:22:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006f6b0 msgr2=0x7f03600cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006f6b0 0x7f03600cd710 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f035400c450 tx=0x7f035400c910 comp rx=0 tx=0).stop 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 shutdown_connections 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f034c0678c0 0x7f034c069d80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0360144110 0x7f0360098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f036006f6b0 0x7f03600cd710 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 --2- 172.21.15.3:0/2670725029 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f036006dd40 0x7f03600cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 >> 172.21.15.3:0/2670725029 conn(0x7f03600b1d80 msgr2=0x7f03600b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 shutdown_connections 2024-09-12T12:22:28.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.248+0000 7f0365bdb640 1 -- 172.21.15.3:0/2670725029 wait complete. 2024-09-12T12:22:28.268 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:22:28.268 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:22:28.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7faca1b85640 1 Processor -- start 2024-09-12T12:22:28.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7faca1b85640 1 -- start start 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7faca1b85640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac9c15c580 0x7fac9c06bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7faca1b85640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c06c630 con 0x7fac9c0b3820 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7faca1b85640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c06c910 con 0x7fac9c0b2880 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7faca1b85640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c0b46e0 con 0x7fac9c15c580 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fac9c15c580 0x7fac9c06bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9affd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fac9c0b3820 0x7fac9c0b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48936/0 (socket says 172.21.15.3:48936) 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9affd640 1 -- 172.21.15.3:0/715499552 learned_addr learned my addr 172.21.15.3:0/715499552 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:28.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9a7fc640 1 -- 172.21.15.3:0/715499552 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588974177 0 0) 0x7fac9c06c630 con 0x7fac9c0b3820 2024-09-12T12:22:28.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9a7fc640 1 -- 172.21.15.3:0/715499552 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac84003530 con 0x7fac9c0b3820 2024-09-12T12:22:28.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9b7fe640 1 -- 172.21.15.3:0/715499552 >> v1:172.21.15.142:6789/0 conn(0x7fac9c0b2880 legacy=0x7fac9c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9b7fe640 1 -- 172.21.15.3:0/715499552 >> v1:172.21.15.3:6789/0 conn(0x7fac9c0b3820 legacy=0x7fac9c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.388+0000 7fac9b7fe640 1 -- 172.21.15.3:0/715499552 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0b4090 con 0x7fac9c15c580 2024-09-12T12:22:28.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7fac9b7fe640 1 --2- 172.21.15.3:0/715499552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c15c580 0x7fac9c06bcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0097c0 tx=0x7fac8c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2d404018f6a52f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:28.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7fac9a7fc640 1 -- 172.21.15.3:0/715499552 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c0aa5f0 con 0x7fac9c15c580 2024-09-12T12:22:28.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7fac9a7fc640 1 -- 172.21.15.3:0/715499552 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8c0af070 con 0x7fac9c15c580 2024-09-12T12:22:28.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7fac9a7fc640 1 -- 172.21.15.3:0/715499552 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c0aa9b0 con 0x7fac9c15c580 2024-09-12T12:22:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7faca1b85640 1 -- 172.21.15.3:0/715499552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c15c580 msgr2=0x7fac9c06bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7faca1b85640 1 --2- 172.21.15.3:0/715499552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c15c580 0x7fac9c06bcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0097c0 tx=0x7fac8c01c470 comp rx=0 tx=0).stop 2024-09-12T12:22:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7faca1b85640 1 -- 172.21.15.3:0/715499552 shutdown_connections 2024-09-12T12:22:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7faca1b85640 1 --2- 172.21.15.3:0/715499552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c15c580 0x7fac9c06bcc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7faca1b85640 1 -- 172.21.15.3:0/715499552 >> 172.21.15.3:0/715499552 conn(0x7fac9c09aa70 msgr2=0x7fac9c0b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7faca1b85640 1 -- 172.21.15.3:0/715499552 shutdown_connections 2024-09-12T12:22:28.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.392+0000 7faca1b85640 1 -- 172.21.15.3:0/715499552 wait complete. 2024-09-12T12:22:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 Processor -- start 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 -- start start 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac9c0b2880 0x7fac9c0c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c0b3820 0x7fac9c14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac9c15c580 0x7fac9c14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0b5e20 con 0x7fac9c0b3820 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c0b5ca0 con 0x7fac9c15c580 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c169fb0 con 0x7fac9c0b2880 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c0b3820 0x7fac9c14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c0b3820 0x7fac9c14baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37538/0 (socket says 172.21.15.3:37538) 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 -- 172.21.15.3:0/494188979 learned_addr learned my addr 172.21.15.3:0/494188979 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9b7fe640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac9c0b2880 0x7fac9c0c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9bfff640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac9c15c580 0x7fac9c14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 -- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac9c0b2880 msgr2=0x7fac9c0c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac9c0b2880 0x7fac9c0c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 -- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac9c15c580 msgr2=0x7fac9c14d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac9c15c580 0x7fac9c14d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 -- 172.21.15.3:0/494188979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0cb2e0 con 0x7fac9c0b3820 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac9affd640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c0b3820 0x7fac9c14baa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c420 tx=0x7fac9000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac83fff640 1 -- 172.21.15.3:0/494188979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90017070 con 0x7fac9c0b3820 2024-09-12T12:22:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac83fff640 1 -- 172.21.15.3:0/494188979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac90005cf0 con 0x7fac9c0b3820 2024-09-12T12:22:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac83fff640 1 -- 172.21.15.3:0/494188979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900072d0 con 0x7fac9c0b3820 2024-09-12T12:22:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c1466a0 con 0x7fac9c0b3820 2024-09-12T12:22:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c146c60 con 0x7fac9c0b3820 2024-09-12T12:22:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c06bcc0 con 0x7fac9c0b3820 2024-09-12T12:22:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.396+0000 7fac83fff640 1 -- 172.21.15.3:0/494188979 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac9001a070 con 0x7fac9c0b3820 2024-09-12T12:22:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.400+0000 7fac83fff640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac880678f0 0x7fac88069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.400+0000 7fac83fff640 1 -- 172.21.15.3:0/494188979 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 4857+0+0 (secure 0 0 0) 0x7fac90084bc0 con 0x7fac9c0b3820 2024-09-12T12:22:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.400+0000 7fac9b7fe640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac880678f0 0x7fac88069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.400+0000 7fac9b7fe640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac880678f0 0x7fac88069db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0ae000 tx=0x7fac8c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:28.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.404+0000 7fac83fff640 1 -- 172.21.15.3:0/494188979 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fac90055400 con 0x7fac9c0b3820 2024-09-12T12:22:28.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:28.568+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fac9c0cb850 con 0x7fac9c0b3820 2024-09-12T12:22:29.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.540+0000 7fac83fff640 1 -- 172.21.15.3:0/494188979 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) ==== 138+0+0 (secure 0 0 0) 0x7fac90059700 con 0x7fac9c0b3820 2024-09-12T12:22:29.543 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T12:22:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac880678f0 msgr2=0x7fac88069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac880678f0 0x7fac88069db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0ae000 tx=0x7fac8c0a7000 comp rx=0 tx=0).stop 2024-09-12T12:22:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c0b3820 msgr2=0x7fac9c14baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c0b3820 0x7fac9c14baa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fac9000c420 tx=0x7fac9000c8e0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 shutdown_connections 2024-09-12T12:22:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac880678f0 0x7fac88069db0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac9c15c580 0x7fac9c14d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac9c0b3820 0x7fac9c14baa0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 --2- 172.21.15.3:0/494188979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac9c0b2880 0x7fac9c0c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 >> 172.21.15.3:0/494188979 conn(0x7fac9c09aa70 msgr2=0x7fac9c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 shutdown_connections 2024-09-12T12:22:29.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.544+0000 7faca1b85640 1 -- 172.21.15.3:0/494188979 wait complete. 2024-09-12T12:22:29.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:22:29.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.704+0000 7fc982349640 1 Processor -- start 2024-09-12T12:22:29.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc982349640 1 -- start start 2024-09-12T12:22:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc982349640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc97c15b130 0x7fc97c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc982349640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 0x7fc97c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc982349640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97c15f0b0 con 0x7fc97c06f6e0 2024-09-12T12:22:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc982349640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc97c0ae920 con 0x7fc97c15b130 2024-09-12T12:22:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc982349640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c15edd0 con 0x7fc97c073640 2024-09-12T12:22:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc97c06f6e0 0x7fc97c072f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48944/0 (socket says 172.21.15.3:48944) 2024-09-12T12:22:29.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 -- 172.21.15.3:0/2261027083 learned_addr learned my addr 172.21.15.3:0/2261027083 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:29.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc9808e6640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 0x7fc97c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97bfff640 1 --2- 172.21.15.3:0/2261027083 >> v2:172.21.15.142:3300/0 conn(0x7fc97c15b130 0x7fc97c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1365928420 0 0) 0x7fc97c15f0b0 con 0x7fc97c06f6e0 2024-09-12T12:22:29.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc964005740 con 0x7fc97c06f6e0 2024-09-12T12:22:29.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2242109295 0 0) 0x7fc964005740 con 0x7fc97c06f6e0 2024-09-12T12:22:29.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 msgr2=0x7fc97c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 0x7fc97c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 >> v2:172.21.15.142:3300/0 conn(0x7fc97c15b130 msgr2=0x7fc97c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 --2- 172.21.15.3:0/2261027083 >> v2:172.21.15.142:3300/0 conn(0x7fc97c15b130 0x7fc97c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c0adff0 con 0x7fc97c06f6e0 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fc970001680 con 0x7fc97c06f6e0 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 >> v1:172.21.15.3:6789/0 conn(0x7fc97c06f6e0 legacy=0x7fc97c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 0x7fc964005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc964006150 0x7fc964006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006f60 0x7fc9640073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0ae920 con 0x7fc964006150 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c15edd0 con 0x7fc964006f60 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc96400dbe0 con 0x7fc97c073640 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc964006150 0x7fc964006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97bfff640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 0x7fc964005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc9808e6640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006f60 0x7fc9640073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 -- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 msgr2=0x7fc964005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c073640 0x7fc964005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 -- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006f60 msgr2=0x7fc9640073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006f60 0x7fc9640073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.708+0000 7fc97b7fe640 1 -- 172.21.15.3:0/2261027083 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc964005740 con 0x7fc964006150 2024-09-12T12:22:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc97b7fe640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc964006150 0x7fc964006a20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc97000c210 tx=0x7fc97001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e34ec49105b58a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970007d60 con 0x7fc964006150 2024-09-12T12:22:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc970005b00 con 0x7fc964006150 2024-09-12T12:22:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc97affd640 1 -- 172.21.15.3:0/2261027083 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970007070 con 0x7fc964006150 2024-09-12T12:22:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 -- 172.21.15.3:0/2261027083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc964006150 msgr2=0x7fc964006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 --2- 172.21.15.3:0/2261027083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc964006150 0x7fc964006a20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc97000c210 tx=0x7fc97001f390 comp rx=0 tx=0).stop 2024-09-12T12:22:29.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc9808e6640 1 -- 172.21.15.3:0/2261027083 reap_dead start 2024-09-12T12:22:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 -- 172.21.15.3:0/2261027083 shutdown_connections 2024-09-12T12:22:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 -- 172.21.15.3:0/2261027083 >> 172.21.15.3:0/2261027083 conn(0x7fc97c0b1d80 msgr2=0x7fc97c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 -- 172.21.15.3:0/2261027083 shutdown_connections 2024-09-12T12:22:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 -- 172.21.15.3:0/2261027083 wait complete. 2024-09-12T12:22:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 Processor -- start 2024-09-12T12:22:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.712+0000 7fc982349640 1 -- start start 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006150 0x7fc97c15a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c1ef720 0x7fc97c1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc97c1efb60 0x7fc97c153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c06d860 con 0x7fc97c1efb60 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0ae5b0 con 0x7fc964006150 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c15b510 con 0x7fc97c1ef720 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc97c1efb60 0x7fc97c153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc97c1efb60 0x7fc97c153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37552/0 (socket says 172.21.15.3:37552) 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 -- 172.21.15.3:0/2998602444 learned_addr learned my addr 172.21.15.3:0/2998602444 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc97b7fe640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c1ef720 0x7fc97c1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc97bfff640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006150 0x7fc97c15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 -- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c1ef720 msgr2=0x7fc97c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c1ef720 0x7fc97c1536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 -- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006150 msgr2=0x7fc97c15a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006150 0x7fc97c15a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 -- 172.21.15.3:0/2998602444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c150b70 con 0x7fc97c1efb60 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc97bfff640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006150 0x7fc97c15a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc9808e6640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc97c1efb60 0x7fc97c153c30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0afc40 tx=0x7fc974011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc978ff9640 1 -- 172.21.15.3:0/2998602444 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974019020 con 0x7fc97c1efb60 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc978ff9640 1 -- 172.21.15.3:0/2998602444 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc974004440 con 0x7fc97c1efb60 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c150e50 con 0x7fc97c1efb60 2024-09-12T12:22:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc978ff9640 1 -- 172.21.15.3:0/2998602444 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974003860 con 0x7fc97c1efb60 2024-09-12T12:22:29.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c15a8c0 con 0x7fc97c1efb60 2024-09-12T12:22:29.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c0cc270 con 0x7fc97c1efb60 2024-09-12T12:22:29.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.716+0000 7fc978ff9640 1 -- 172.21.15.3:0/2998602444 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc97401c020 con 0x7fc97c1efb60 2024-09-12T12:22:29.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.720+0000 7fc978ff9640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9680678f0 0x7fc968069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.720+0000 7fc978ff9640 1 -- 172.21.15.3:0/2998602444 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5227+0+0 (secure 0 0 0) 0x7fc974085fa0 con 0x7fc97c1efb60 2024-09-12T12:22:29.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.720+0000 7fc97bfff640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9680678f0 0x7fc968069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.720+0000 7fc97bfff640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9680678f0 0x7fc968069db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007440 tx=0x7fc96c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.724+0000 7fc978ff9640 1 -- 172.21.15.3:0/2998602444 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc974023030 con 0x7fc97c1efb60 2024-09-12T12:22:29.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:29.880+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc97c073180 con 0x7fc97c1efb60 2024-09-12T12:22:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.552+0000 7fc978ff9640 1 -- 172.21.15.3:0/2998602444 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) ==== 170+0+0 (secure 0 0 0) 0x7fc974056d30 con 0x7fc97c1efb60 2024-09-12T12:22:30.554 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T12:22:30.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9680678f0 msgr2=0x7fc968069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:30.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9680678f0 0x7fc968069db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007440 tx=0x7fc96c00b040 comp rx=0 tx=0).stop 2024-09-12T12:22:30.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc97c1efb60 msgr2=0x7fc97c153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:30.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc97c1efb60 0x7fc97c153c30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0afc40 tx=0x7fc974011920 comp rx=0 tx=0).stop 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 shutdown_connections 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc9680678f0 0x7fc968069db0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc97c1efb60 0x7fc97c153c30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc97c1ef720 0x7fc97c1536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 --2- 172.21.15.3:0/2998602444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc964006150 0x7fc97c15a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 >> 172.21.15.3:0/2998602444 conn(0x7fc97c0b1d80 msgr2=0x7fc97c06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 shutdown_connections 2024-09-12T12:22:30.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.556+0000 7fc982349640 1 -- 172.21.15.3:0/2998602444 wait complete. 2024-09-12T12:22:30.593 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:22:30.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 Processor -- start 2024-09-12T12:22:30.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 -- start start 2024-09-12T12:22:30.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6015b130 0x7f3c600b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:30.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3c6006f6e0 0x7f3c60072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3c600cc270 0x7f3c60073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3c600ae920 con 0x7f3c6006f6e0 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3c6015f0b0 con 0x7f3c600cc270 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c66a02640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6015edd0 con 0x7f3c6015b130 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3c6006f6e0 0x7f3c60072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3c6006f6e0 0x7f3c60072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37566/0 (socket says 172.21.15.3:37566) 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 -- 172.21.15.3:0/2757358363 learned_addr learned my addr 172.21.15.3:0/2757358363 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5ffff640 1 --2- 172.21.15.3:0/2757358363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6015b130 0x7f3c600b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:30.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c64f9f640 1 --2- 172.21.15.3:0/2757358363 >> v2:172.21.15.142:3300/0 conn(0x7f3c600cc270 0x7f3c60073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:30.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 -- 172.21.15.3:0/2757358363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6015b130 msgr2=0x7f3c600b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:30.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 --2- 172.21.15.3:0/2757358363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6015b130 0x7f3c600b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 -- 172.21.15.3:0/2757358363 >> v2:172.21.15.142:3300/0 conn(0x7f3c600cc270 msgr2=0x7f3c60073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:30.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 --2- 172.21.15.3:0/2757358363 >> v2:172.21.15.142:3300/0 conn(0x7f3c600cc270 0x7f3c60073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5f7fe640 1 -- 172.21.15.3:0/2757358363 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c600adff0 con 0x7f3c6006f6e0 2024-09-12T12:22:30.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c5ffff640 1 --2- 172.21.15.3:0/2757358363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6015b130 0x7f3c600b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:30.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.720+0000 7f3c64f9f640 1 --2- 172.21.15.3:0/2757358363 >> v2:172.21.15.142:3300/0 conn(0x7f3c600cc270 0x7f3c60073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c5f7fe640 1 --2- 172.21.15.3:0/2757358363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c60072f30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bb10 tx=0x7f3c5401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d35616de24a2578 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c5effd640 1 -- 172.21.15.3:0/2757358363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c540b4070 con 0x7f3c6006f6e0 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c5effd640 1 -- 172.21.15.3:0/2757358363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c540a7c50 con 0x7f3c6006f6e0 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c5effd640 1 -- 172.21.15.3:0/2757358363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c540af440 con 0x7f3c6006f6e0 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 -- 172.21.15.3:0/2757358363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 msgr2=0x7f3c60072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 --2- 172.21.15.3:0/2757358363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c60072f30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bb10 tx=0x7f3c5401ebc0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 -- 172.21.15.3:0/2757358363 shutdown_connections 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 --2- 172.21.15.3:0/2757358363 >> v2:172.21.15.142:3300/0 conn(0x7f3c600cc270 0x7f3c60073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 --2- 172.21.15.3:0/2757358363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c60072f30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 --2- 172.21.15.3:0/2757358363 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c6015b130 0x7f3c600b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 -- 172.21.15.3:0/2757358363 >> 172.21.15.3:0/2757358363 conn(0x7f3c600b1d80 msgr2=0x7f3c600b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:30.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 -- 172.21.15.3:0/2757358363 shutdown_connections 2024-09-12T12:22:30.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 -- 172.21.15.3:0/2757358363 wait complete. 2024-09-12T12:22:30.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 Processor -- start 2024-09-12T12:22:30.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 -- start start 2024-09-12T12:22:30.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c6014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.724+0000 7f3c66a02640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c600cc270 0x7f3c600cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c66a02640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015b130 0x7f3c600afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c66a02640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6015b690 con 0x7f3c6006f6e0 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c66a02640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6015b510 con 0x7f3c6015b130 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c66a02640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3c600b35c0 con 0x7f3c600cc270 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c6014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c6014fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37568/0 (socket says 172.21.15.3:37568) 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 -- 172.21.15.3:0/508828668 learned_addr learned my addr 172.21.15.3:0/508828668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5f7fe640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c600cc270 0x7f3c600cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c64f9f640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015b130 0x7f3c600afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 -- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c600cc270 msgr2=0x7f3c600cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:30.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c600cc270 0x7f3c600cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 -- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015b130 msgr2=0x7f3c600afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015b130 0x7f3c600afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 -- 172.21.15.3:0/508828668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c600ce440 con 0x7f3c6006f6e0 2024-09-12T12:22:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5ffff640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c6014fbc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c7f0 tx=0x7f3c5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5cff9640 1 -- 172.21.15.3:0/508828668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50015070 con 0x7f3c6006f6e0 2024-09-12T12:22:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5cff9640 1 -- 172.21.15.3:0/508828668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c50007820 con 0x7f3c6006f6e0 2024-09-12T12:22:30.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c66a02640 1 -- 172.21.15.3:0/508828668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6009b2d0 con 0x7f3c6006f6e0 2024-09-12T12:22:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5cff9640 1 -- 172.21.15.3:0/508828668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50007c10 con 0x7f3c6006f6e0 2024-09-12T12:22:30.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c66a02640 1 -- 172.21.15.3:0/508828668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6009bb70 con 0x7f3c6006f6e0 2024-09-12T12:22:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c66a02640 1 -- 172.21.15.3:0/508828668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6009b4a0 con 0x7f3c6006f6e0 2024-09-12T12:22:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5cff9640 1 -- 172.21.15.3:0/508828668 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3c50018070 con 0x7f3c6006f6e0 2024-09-12T12:22:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5cff9640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3c4c0678c0 0x7f3c4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5cff9640 1 -- 172.21.15.3:0/508828668 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5609+0+0 (secure 0 0 0) 0x7f3c50086150 con 0x7f3c6006f6e0 2024-09-12T12:22:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.728+0000 7f3c5f7fe640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3c4c0678c0 0x7f3c4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:30.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.732+0000 7f3c5f7fe640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3c4c0678c0 0x7f3c4c069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3c540b1000 tx=0x7f3c540a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:30.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.732+0000 7f3c5cff9640 1 -- 172.21.15.3:0/508828668 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3c50056dd0 con 0x7f3c6006f6e0 2024-09-12T12:22:30.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:30.932+0000 7f3c66a02640 1 -- 172.21.15.3:0/508828668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3c600af5a0 con 0x7f3c6006f6e0 2024-09-12T12:22:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.588+0000 7f3c5cff9640 1 -- 172.21.15.3:0/508828668 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f3c500566d0 con 0x7f3c6006f6e0 2024-09-12T12:22:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 8 and data pool 9 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 -- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3c4c0678c0 msgr2=0x7f3c4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3c4c0678c0 0x7f3c4c069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3c540b1000 tx=0x7f3c540a9070 comp rx=0 tx=0).stop 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 -- 172.21.15.3:0/508828668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 msgr2=0x7f3c6014fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c6014fbc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c7f0 tx=0x7f3c5000ccb0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 -- 172.21.15.3:0/508828668 shutdown_connections 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3c4c0678c0 0x7f3c4c069d80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c6015b130 0x7f3c600afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c600cc270 0x7f3c600cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 --2- 172.21.15.3:0/508828668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6006f6e0 0x7f3c6014fbc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.600+0000 7f3c467fc640 1 -- 172.21.15.3:0/508828668 >> 172.21.15.3:0/508828668 conn(0x7f3c600b1d80 msgr2=0x7f3c600b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.604+0000 7f3c467fc640 1 -- 172.21.15.3:0/508828668 shutdown_connections 2024-09-12T12:22:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.604+0000 7f3c467fc640 1 -- 172.21.15.3:0/508828668 wait complete. 2024-09-12T12:22:31.633 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:22:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f314173d640 1 Processor -- start 2024-09-12T12:22:31.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f314173d640 1 -- start start 2024-09-12T12:22:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f314173d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f313c144110 0x7f313c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f314173d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06f6b0 0x7f313c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f314173d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313c071ed0 con 0x7f313c06dd40 2024-09-12T12:22:31.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f314173d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f313c072490 con 0x7f313c144110 2024-09-12T12:22:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f314173d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f313c0b45a0 con 0x7f313c06f6b0 2024-09-12T12:22:31.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f31327fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06f6b0 0x7f313c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f313affd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f313c144110 0x7f313c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f313b7fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f313c06dd40 0x7f313c0727b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:46484/0 (socket says 172.21.15.3:46484) 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f31327fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06f6b0 0x7f313c0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:49386/0 (socket says 172.21.15.3:49386) 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f313b7fe640 1 -- 172.21.15.3:0/399285884 learned_addr learned my addr 172.21.15.3:0/399285884 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f31327fc640 1 -- 172.21.15.3:0/399285884 >> v2:172.21.15.142:3300/0 conn(0x7f313c144110 msgr2=0x7f313c15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.748+0000 7f31327fc640 1 --2- 172.21.15.3:0/399285884 >> v2:172.21.15.142:3300/0 conn(0x7f313c144110 0x7f313c15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f31327fc640 1 -- 172.21.15.3:0/399285884 >> v1:172.21.15.142:6789/0 conn(0x7f313c06dd40 legacy=0x7f313c0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f31327fc640 1 -- 172.21.15.3:0/399285884 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c0721b0 con 0x7f313c06f6b0 2024-09-12T12:22:31.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f31327fc640 1 --2- 172.21.15.3:0/399285884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06f6b0 0x7f313c0b4a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f31240097c0 tx=0x7f312401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515bced1ff6098c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f313a7fc640 1 -- 172.21.15.3:0/399285884 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240b1070 con 0x7f313c06f6b0 2024-09-12T12:22:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f313a7fc640 1 -- 172.21.15.3:0/399285884 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31240ac640 con 0x7f313c06f6b0 2024-09-12T12:22:31.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f313a7fc640 1 -- 172.21.15.3:0/399285884 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240aca10 con 0x7f313c06f6b0 2024-09-12T12:22:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 -- 172.21.15.3:0/399285884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06f6b0 msgr2=0x7f313c0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 --2- 172.21.15.3:0/399285884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06f6b0 0x7f313c0b4a30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f31240097c0 tx=0x7f312401c350 comp rx=0 tx=0).stop 2024-09-12T12:22:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 -- 172.21.15.3:0/399285884 shutdown_connections 2024-09-12T12:22:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 --2- 172.21.15.3:0/399285884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06f6b0 0x7f313c0b4a30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 --2- 172.21.15.3:0/399285884 >> v2:172.21.15.142:3300/0 conn(0x7f313c144110 0x7f313c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 -- 172.21.15.3:0/399285884 >> 172.21.15.3:0/399285884 conn(0x7f313c0b1d80 msgr2=0x7f313c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 -- 172.21.15.3:0/399285884 shutdown_connections 2024-09-12T12:22:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.752+0000 7f314173d640 1 -- 172.21.15.3:0/399285884 wait complete. 2024-09-12T12:22:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 Processor -- start 2024-09-12T12:22:31.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 -- start start 2024-09-12T12:22:31.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06dd40 0x7f313c1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:31.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f313c06f6b0 0x7f313c0ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c144110 0x7f313c0cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f313c073040 con 0x7f313c06f6b0 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f313c072ec0 con 0x7f313c144110 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f313c06e150 con 0x7f313c06dd40 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f313c06f6b0 0x7f313c0ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f313c06f6b0 0x7f313c0ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37576/0 (socket says 172.21.15.3:37576) 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 -- 172.21.15.3:0/3685275472 learned_addr learned my addr 172.21.15.3:0/3685275472 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:31.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f313affd640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06dd40 0x7f313c1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f313b7fe640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c144110 0x7f313c0cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 -- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06dd40 msgr2=0x7f313c1435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06dd40 0x7f313c1435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 -- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c144110 msgr2=0x7f313c0cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c144110 0x7f313c0cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 -- 172.21.15.3:0/3685275472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c152b70 con 0x7f313c06f6b0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f313b7fe640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c144110 0x7f313c0cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f31327fc640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f313c06f6b0 0x7f313c0ced90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f31240af040 tx=0x7f31240190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f3133fff640 1 -- 172.21.15.3:0/3685275472 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240b9020 con 0x7f313c06f6b0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f3133fff640 1 -- 172.21.15.3:0/3685275472 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31240b5410 con 0x7f313c06f6b0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c152e50 con 0x7f313c06f6b0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f3133fff640 1 -- 172.21.15.3:0/3685275472 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240b1040 con 0x7f313c06f6b0 2024-09-12T12:22:31.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c153410 con 0x7f313c06f6b0 2024-09-12T12:22:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.756+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c0b4a30 con 0x7f313c06f6b0 2024-09-12T12:22:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.760+0000 7f3133fff640 1 -- 172.21.15.3:0/3685275472 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f31240bc020 con 0x7f313c06f6b0 2024-09-12T12:22:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.760+0000 7f3133fff640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f311c0678c0 0x7f311c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:31.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.760+0000 7f3133fff640 1 -- 172.21.15.3:0/3685275472 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f312411f8c0 con 0x7f313c06f6b0 2024-09-12T12:22:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.760+0000 7f313affd640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f311c0678c0 0x7f311c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:31.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.760+0000 7f313affd640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f311c0678c0 0x7f311c069d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f312c009ac0 tx=0x7f312c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:31.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.764+0000 7f3133fff640 1 -- 172.21.15.3:0/3685275472 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f31240c3030 con 0x7f313c06f6b0 2024-09-12T12:22:31.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.928+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f313c0b15e0 con 0x7f313c06f6b0 2024-09-12T12:22:31.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.928+0000 7f3133fff640 1 -- 172.21.15.3:0/3685275472 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16678 (secure 0 0 0) 0x7f3124016090 con 0x7f313c06f6b0 2024-09-12T12:22:31.934 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:31.934 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:30.937020+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:22:28.609722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T12:22:29.925008+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f311c0678c0 msgr2=0x7f311c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f311c0678c0 0x7f311c069d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f312c009ac0 tx=0x7f312c009190 comp rx=0 tx=0).stop 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f313c06f6b0 msgr2=0x7f313c0ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f313c06f6b0 0x7f313c0ced90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f31240af040 tx=0x7f31240190f0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 shutdown_connections 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f311c0678c0 0x7f311c069d80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c144110 0x7f313c0cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f313c06f6b0 0x7f313c0ced90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 --2- 172.21.15.3:0/3685275472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f313c06dd40 0x7f313c1435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:31.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 >> 172.21.15.3:0/3685275472 conn(0x7f313c0b1d80 msgr2=0x7f313c14fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 shutdown_connections 2024-09-12T12:22:31.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:31.932+0000 7f314173d640 1 -- 172.21.15.3:0/3685275472 wait complete. 2024-09-12T12:22:31.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:22:32.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.072+0000 7f34f4ce1640 1 Processor -- start 2024-09-12T12:22:32.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.072+0000 7f34f4ce1640 1 -- start start 2024-09-12T12:22:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.072+0000 7f34f4ce1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f00cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.072+0000 7f34f4ce1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f00ce0f0 con 0x7f34f009ba60 2024-09-12T12:22:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.072+0000 7f34f4ce1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f00721e0 con 0x7f34f00703d0 2024-09-12T12:22:32.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.072+0000 7f34f4ce1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34f00ccd50 con 0x7f34f009aa70 2024-09-12T12:22:32.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34edd9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f34f009ba60 0x7f34f00cd4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48946/0 (socket says 172.21.15.3:48946) 2024-09-12T12:22:32.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34edd9b640 1 -- 172.21.15.3:0/515357564 learned_addr learned my addr 172.21.15.3:0/515357564 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:32.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34eed9d640 1 --2- 172.21.15.3:0/515357564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f00cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34ed59a640 1 -- 172.21.15.3:0/515357564 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1709996951 0 0) 0x7f34e4001120 con 0x7f34f009ba60 2024-09-12T12:22:32.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34ed59a640 1 -- 172.21.15.3:0/515357564 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34d8003530 con 0x7f34f009ba60 2024-09-12T12:22:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34eed9d640 1 -- 172.21.15.3:0/515357564 >> v1:172.21.15.142:6789/0 conn(0x7f34f00703d0 legacy=0x7f34f009b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34eed9d640 1 -- 172.21.15.3:0/515357564 >> v1:172.21.15.3:6789/0 conn(0x7f34f009ba60 legacy=0x7f34f00cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34eed9d640 1 -- 172.21.15.3:0/515357564 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f00ce7c0 con 0x7f34f009aa70 2024-09-12T12:22:32.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34eed9d640 1 --2- 172.21.15.3:0/515357564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f00cdbb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009dc0 tx=0x7f34e801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3626abd3711ba07b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34ed59a640 1 -- 172.21.15.3:0/515357564 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e80b2070 con 0x7f34f009aa70 2024-09-12T12:22:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34ed59a640 1 -- 172.21.15.3:0/515357564 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e80a5850 con 0x7f34f009aa70 2024-09-12T12:22:32.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34ed59a640 1 -- 172.21.15.3:0/515357564 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e80a5c00 con 0x7f34f009aa70 2024-09-12T12:22:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34f4ce1640 1 -- 172.21.15.3:0/515357564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 msgr2=0x7f34f00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/515357564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f00cdbb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34e8009dc0 tx=0x7f34e801c990 comp rx=0 tx=0).stop 2024-09-12T12:22:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34f4ce1640 1 -- 172.21.15.3:0/515357564 shutdown_connections 2024-09-12T12:22:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/515357564 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f00cdbb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.076+0000 7f34f4ce1640 1 -- 172.21.15.3:0/515357564 >> 172.21.15.3:0/515357564 conn(0x7f34f00b3e60 msgr2=0x7f34f00b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- 172.21.15.3:0/515357564 shutdown_connections 2024-09-12T12:22:32.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- 172.21.15.3:0/515357564 wait complete. 2024-09-12T12:22:32.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 Processor -- start 2024-09-12T12:22:32.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- start start 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34f00703d0 0x7f34f0157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f0153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34f009ba60 0x7f34f00b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34f0070fc0 con 0x7f34f009ba60 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34f009ae80 con 0x7f34f00703d0 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f34f0071140 con 0x7f34f009aa70 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34f009ba60 0x7f34f00b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34f009ba60 0x7f34f00b7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37578/0 (socket says 172.21.15.3:37578) 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 -- 172.21.15.3:0/3084349454 learned_addr learned my addr 172.21.15.3:0/3084349454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34ee59c640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34f00703d0 0x7f34f0157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34edd9b640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f0153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 -- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 msgr2=0x7f34f0153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f0153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 -- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34f00703d0 msgr2=0x7f34f0157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34f00703d0 0x7f34f0157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 -- 172.21.15.3:0/3084349454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f0153770 con 0x7f34f009ba60 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34ee59c640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34f00703d0 0x7f34f0157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34eed9d640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34f009ba60 0x7f34f00b7860 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f34e80b0040 tx=0x7f34e8019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34deffd640 1 -- 172.21.15.3:0/3084349454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e80b2050 con 0x7f34f009ba60 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34deffd640 1 -- 172.21.15.3:0/3084349454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e80be070 con 0x7f34f009ba60 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f00b5340 con 0x7f34f009ba60 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34deffd640 1 -- 172.21.15.3:0/3084349454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8003db0 con 0x7f34f009ba60 2024-09-12T12:22:32.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.080+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f00b7060 con 0x7f34f009ba60 2024-09-12T12:22:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.084+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f0068d40 con 0x7f34f009ba60 2024-09-12T12:22:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.084+0000 7f34deffd640 1 -- 172.21.15.3:0/3084349454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34e8004040 con 0x7f34f009ba60 2024-09-12T12:22:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.084+0000 7f34deffd640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34d00678f0 0x7f34d0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.084+0000 7f34deffd640 1 -- 172.21.15.3:0/3084349454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f34e8120d20 con 0x7f34f009ba60 2024-09-12T12:22:32.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.084+0000 7f34ee59c640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34d00678f0 0x7f34d0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.084+0000 7f34ee59c640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34d00678f0 0x7f34d0069db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34e0001710 tx=0x7f34e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:32.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.088+0000 7f34deffd640 1 -- 172.21.15.3:0/3084349454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f34e8016580 con 0x7f34f009ba60 2024-09-12T12:22:32.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.256+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34f00714d0 con 0x7f34f009ba60 2024-09-12T12:22:32.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.256+0000 7f34deffd640 1 -- 172.21.15.3:0/3084349454 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16678 (secure 0 0 0) 0x7f34e80f18f0 con 0x7f34f009ba60 2024-09-12T12:22:32.261 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:32.261 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:30.937020+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:22:28.609722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T12:22:29.925008+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34d00678f0 msgr2=0x7f34d0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34d00678f0 0x7f34d0069db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f34e0001710 tx=0x7f34e0009040 comp rx=0 tx=0).stop 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34f009ba60 msgr2=0x7f34f00b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34f009ba60 0x7f34f00b7860 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f34e80b0040 tx=0x7f34e8019730 comp rx=0 tx=0).stop 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 shutdown_connections 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34d00678f0 0x7f34d0069db0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34f009ba60 0x7f34f00b7860 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34f009aa70 0x7f34f0153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 --2- 172.21.15.3:0/3084349454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34f00703d0 0x7f34f0157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 >> 172.21.15.3:0/3084349454 conn(0x7f34f00b3e60 msgr2=0x7f34f00cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:32.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 shutdown_connections 2024-09-12T12:22:32.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.260+0000 7f34f4ce1640 1 -- 172.21.15.3:0/3084349454 wait complete. 2024-09-12T12:22:32.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:22:32.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff2a0f640 1 Processor -- start 2024-09-12T12:22:32.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff2a0f640 1 -- start start 2024-09-12T12:22:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff2a0f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3fec15c510 0x7f3fec0b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff2a0f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3fec0cd430 0x7f3fec0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff2a0f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fec15c8f0 con 0x7f3fec06c6a0 2024-09-12T12:22:32.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff2a0f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3fec06cb10 con 0x7f3fec15c510 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff2a0f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3fec0b2b70 con 0x7f3fec0cd430 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3febfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3fec15c510 0x7f3fec0b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3febfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3fec15c510 0x7f3fec0b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37590/0 (socket says 172.21.15.3:37590) 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3feb7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3fec06c6a0 0x7f3fec0ccd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48960/0 (socket says 172.21.15.3:48960) 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3febfff640 1 -- 172.21.15.3:0/2230739517 learned_addr learned my addr 172.21.15.3:0/2230739517 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3febfff640 1 -- 172.21.15.3:0/2230739517 >> v2:172.21.15.142:3300/0 conn(0x7f3fec0cd430 msgr2=0x7f3fec0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff0fac640 1 --2- 172.21.15.3:0/2230739517 >> v2:172.21.15.142:3300/0 conn(0x7f3fec0cd430 0x7f3fec0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3feaffd640 1 -- 172.21.15.3:0/2230739517 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675890832 0 0) 0x7f3fec15c8f0 con 0x7f3fec06c6a0 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3febfff640 1 --2- 172.21.15.3:0/2230739517 >> v2:172.21.15.142:3300/0 conn(0x7f3fec0cd430 0x7f3fec0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3febfff640 1 -- 172.21.15.3:0/2230739517 >> v1:172.21.15.3:6789/0 conn(0x7f3fec06c6a0 legacy=0x7f3fec0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3febfff640 1 -- 172.21.15.3:0/2230739517 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec0b25b0 con 0x7f3fec15c510 2024-09-12T12:22:32.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.404+0000 7f3ff0fac640 1 --2- 172.21.15.3:0/2230739517 >> v2:172.21.15.142:3300/0 conn(0x7f3fec0cd430 0x7f3fec0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3febfff640 1 --2- 172.21.15.3:0/2230739517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec0b1f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009e50 tx=0x7f3fdc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1062d1704dbd6e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3feaffd640 1 -- 172.21.15.3:0/2230739517 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0a57a0 con 0x7f3fec15c510 2024-09-12T12:22:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3feaffd640 1 -- 172.21.15.3:0/2230739517 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc0a5940 con 0x7f3fec15c510 2024-09-12T12:22:32.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3feaffd640 1 -- 172.21.15.3:0/2230739517 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0a5d10 con 0x7f3fec15c510 2024-09-12T12:22:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/2230739517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 msgr2=0x7f3fec0b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/2230739517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec0b1f20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc009e50 tx=0x7f3fdc01c9f0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/2230739517 shutdown_connections 2024-09-12T12:22:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/2230739517 >> v2:172.21.15.142:3300/0 conn(0x7f3fec0cd430 0x7f3fec0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/2230739517 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec0b1f20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/2230739517 >> 172.21.15.3:0/2230739517 conn(0x7f3fec09aa70 msgr2=0x7f3fec0b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:32.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/2230739517 shutdown_connections 2024-09-12T12:22:32.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/2230739517 wait complete. 2024-09-12T12:22:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 Processor -- start 2024-09-12T12:22:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- start start 2024-09-12T12:22:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec06c6a0 0x7f3fec15a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fec0cd430 0x7f3fec15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec0b5800 con 0x7f3fec15c510 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec0b5680 con 0x7f3fec06c6a0 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff2a0f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3fec0b5980 con 0x7f3fec0cd430 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff0fac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff0fac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec15de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37604/0 (socket says 172.21.15.3:37604) 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.408+0000 7f3ff0fac640 1 -- 172.21.15.3:0/4171155996 learned_addr learned my addr 172.21.15.3:0/4171155996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3feb7fe640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fec0cd430 0x7f3fec15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3febfff640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec06c6a0 0x7f3fec15a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff0fac640 1 -- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fec0cd430 msgr2=0x7f3fec15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff0fac640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fec0cd430 0x7f3fec15d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff0fac640 1 -- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec06c6a0 msgr2=0x7f3fec15a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff0fac640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec06c6a0 0x7f3fec15a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff0fac640 1 -- 172.21.15.3:0/4171155996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec15a660 con 0x7f3fec15c510 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff0fac640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec15de30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c7f0 tx=0x7f3fe400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3fe8ff9640 1 -- 172.21.15.3:0/4171155996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4015070 con 0x7f3fec15c510 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3fe8ff9640 1 -- 172.21.15.3:0/4171155996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe4007860 con 0x7f3fec15c510 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3fe8ff9640 1 -- 172.21.15.3:0/4171155996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4007c50 con 0x7f3fec15c510 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec1640f0 con 0x7f3fec15c510 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec1646b0 con 0x7f3fec15c510 2024-09-12T12:22:32.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fec0b4930 con 0x7f3fec15c510 2024-09-12T12:22:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3fe8ff9640 1 -- 172.21.15.3:0/4171155996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3fe4018070 con 0x7f3fec15c510 2024-09-12T12:22:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3fe8ff9640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fd80678c0 0x7f3fd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3fe8ff9640 1 -- 172.21.15.3:0/4171155996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f3fe4085d50 con 0x7f3fec15c510 2024-09-12T12:22:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3febfff640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fd80678c0 0x7f3fd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:32.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.412+0000 7f3febfff640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fd80678c0 0x7f3fd8069d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0ae000 tx=0x7f3fdc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:32.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.416+0000 7f3fe8ff9640 1 -- 172.21.15.3:0/4171155996 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3fe4089030 con 0x7f3fec15c510 2024-09-12T12:22:32.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:32.624+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3fec0b22f0 con 0x7f3fec15c510 2024-09-12T12:22:33.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.600+0000 7f3fe8ff9640 1 -- 172.21.15.3:0/4171155996 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f3fe4056920 con 0x7f3fec15c510 2024-09-12T12:22:33.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fd80678c0 msgr2=0x7f3fd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:33.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fd80678c0 0x7f3fd8069d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0ae000 tx=0x7f3fdc0a7070 comp rx=0 tx=0).stop 2024-09-12T12:22:33.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 msgr2=0x7f3fec15de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec15de30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c7f0 tx=0x7f3fe400ccb0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 shutdown_connections 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fd80678c0 0x7f3fd8069d80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fec15c510 0x7f3fec15de30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fec0cd430 0x7f3fec15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 --2- 172.21.15.3:0/4171155996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fec06c6a0 0x7f3fec15a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 >> 172.21.15.3:0/4171155996 conn(0x7f3fec09aa70 msgr2=0x7f3fec15b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 shutdown_connections 2024-09-12T12:22:33.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.604+0000 7f3ff2a0f640 1 -- 172.21.15.3:0/4171155996 wait complete. 2024-09-12T12:22:33.632 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T12:22:33.632 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T12:22:33.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.764+0000 7fd88eff9640 1 Processor -- start 2024-09-12T12:22:33.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.764+0000 7fd88eff9640 1 -- start start 2024-09-12T12:22:33.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.764+0000 7fd88eff9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 0x7fd88806e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.764+0000 7fd88eff9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd8880b6350 0x7fd8880cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.764+0000 7fd88eff9640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8880d1690 con 0x7fd8880b6b10 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.764+0000 7fd88eff9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd8880d1970 con 0x7fd8880b6350 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.764+0000 7fd88eff9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd8880d23f0 con 0x7fd8880b5f70 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd887fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd8880b6b10 0x7fd8880d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48976/0 (socket says 172.21.15.3:48976) 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd887fff640 1 -- 172.21.15.3:0/1759434609 learned_addr learned my addr 172.21.15.3:0/1759434609 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88cd95640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 0x7fd88806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 --2- 172.21.15.3:0/1759434609 >> v2:172.21.15.142:3300/0 conn(0x7fd8880b6350 0x7fd8880cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4086499752 0 0) 0x7fd8880d1690 con 0x7fd8880b6b10 2024-09-12T12:22:33.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd878003530 con 0x7fd8880b6b10 2024-09-12T12:22:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3875578176 0 0) 0x7fd878003530 con 0x7fd8880b6b10 2024-09-12T12:22:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 msgr2=0x7fd88806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 0x7fd88806e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 >> v2:172.21.15.142:3300/0 conn(0x7fd8880b6350 msgr2=0x7fd8880cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 --2- 172.21.15.3:0/1759434609 >> v2:172.21.15.142:3300/0 conn(0x7fd8880b6350 0x7fd8880cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8880d26d0 con 0x7fd8880b6b10 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fd874001680 con 0x7fd8880b6b10 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 >> v1:172.21.15.3:6789/0 conn(0x7fd8880b6b10 legacy=0x7fd8880d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 0x7fd8780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd878003f40 0x7fd878004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd878004d50 0x7fd8780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880d1970 con 0x7fd878004d50 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880d23f0 con 0x7fd878003f40 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd87800b9d0 con 0x7fd8880b5f70 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd878004d50 0x7fd8780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88cd95640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 0x7fd8780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd887fff640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd878003f40 0x7fd878004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 -- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 msgr2=0x7fd8780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b5f70 0x7fd8780039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 -- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd878003f40 msgr2=0x7fd878004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd878003f40 0x7fd878004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 -- 172.21.15.3:0/1759434609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd878003530 con 0x7fd878004d50 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd88d596640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd878004d50 0x7fd8780051d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd880009fc0 tx=0x7fd88001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c528197c845fca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800b2070 con 0x7fd878004d50 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8800a5cd0 con 0x7fd878004d50 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.768+0000 7fd8877fe640 1 -- 172.21.15.3:0/1759434609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880009c10 con 0x7fd878004d50 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/1759434609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd878004d50 msgr2=0x7fd8780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 --2- 172.21.15.3:0/1759434609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd878004d50 0x7fd8780051d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd880009fc0 tx=0x7fd88001cc60 comp rx=0 tx=0).stop 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88d596640 1 -- 172.21.15.3:0/1759434609 reap_dead start 2024-09-12T12:22:33.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/1759434609 shutdown_connections 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/1759434609 >> 172.21.15.3:0/1759434609 conn(0x7fd88809d880 msgr2=0x7fd88808c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/1759434609 shutdown_connections 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/1759434609 wait complete. 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 Processor -- start 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- start start 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd88815b560 0x7fd8880ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8880b8220 0x7fd8880b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b9f90 0x7fd8880cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd88815ea90 con 0x7fd88815b560 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8880b6800 con 0x7fd8880b8220 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd88806e780 con 0x7fd8880b9f90 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd88815b560 0x7fd8880ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd88815b560 0x7fd8880ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37636/0 (socket says 172.21.15.3:37636) 2024-09-12T12:22:33.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 -- 172.21.15.3:0/173742983 learned_addr learned my addr 172.21.15.3:0/173742983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd887fff640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8880b8220 0x7fd8880b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88d596640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b9f90 0x7fd8880cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 -- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b9f90 msgr2=0x7fd8880cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b9f90 0x7fd8880cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 -- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8880b8220 msgr2=0x7fd8880b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8880b8220 0x7fd8880b9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 -- 172.21.15.3:0/173742983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8880b89b0 con 0x7fd88815b560 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88cd95640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd88815b560 0x7fd8880ca8a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0115e0 tx=0x7fd87c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd8857fa640 1 -- 172.21.15.3:0/173742983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00f8f0 con 0x7fd88815b560 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd8857fa640 1 -- 172.21.15.3:0/173742983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87c00fa90 con 0x7fd88815b560 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8880b8c90 con 0x7fd88815b560 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd8857fa640 1 -- 172.21.15.3:0/173742983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00fe40 con 0x7fd88815b560 2024-09-12T12:22:33.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8880b9250 con 0x7fd88815b560 2024-09-12T12:22:33.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.772+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88806eab0 con 0x7fd88815b560 2024-09-12T12:22:33.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.776+0000 7fd8857fa640 1 -- 172.21.15.3:0/173742983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd87c015460 con 0x7fd88815b560 2024-09-12T12:22:33.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.776+0000 7fd8857fa640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd86c0678c0 0x7fd86c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:33.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.776+0000 7fd8857fa640 1 -- 172.21.15.3:0/173742983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fd87c0865e0 con 0x7fd88815b560 2024-09-12T12:22:33.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.776+0000 7fd887fff640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd86c0678c0 0x7fd86c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:33.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.776+0000 7fd887fff640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd86c0678c0 0x7fd86c069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd874001060 tx=0x7fd874009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:33.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.780+0000 7fd8857fa640 1 -- 172.21.15.3:0/173742983 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd87c019030 con 0x7fd88815b560 2024-09-12T12:22:33.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:33.980+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd88806e560 con 0x7fd88815b560 2024-09-12T12:22:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.608+0000 7fd8857fa640 1 -- 172.21.15.3:0/173742983 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fd87c0571b0 con 0x7fd88815b560 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd86c0678c0 msgr2=0x7fd86c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd86c0678c0 0x7fd86c069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd874001060 tx=0x7fd874009040 comp rx=0 tx=0).stop 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd88815b560 msgr2=0x7fd8880ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd88815b560 0x7fd8880ca8a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0115e0 tx=0x7fd87c011aa0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 shutdown_connections 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd86c0678c0 0x7fd86c069d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8880b9f90 0x7fd8880cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8880b8220 0x7fd8880b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 --2- 172.21.15.3:0/173742983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd88815b560 0x7fd8880ca8a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 >> 172.21.15.3:0/173742983 conn(0x7fd88809d880 msgr2=0x7fd8880cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:34.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 shutdown_connections 2024-09-12T12:22:34.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.612+0000 7fd88eff9640 1 -- 172.21.15.3:0/173742983 wait complete. 2024-09-12T12:22:34.635 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T12:22:34.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:22:34.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f1180411640 1 Processor -- start 2024-09-12T12:22:34.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f1180411640 1 -- start start 2024-09-12T12:22:34.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f1180411640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1178159fb0 0x7f117806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:34.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f1180411640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f11780ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:34.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f1180411640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f117806fbb0 con 0x7f1178159bd0 2024-09-12T12:22:34.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f1180411640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f117806f010 con 0x7f1178159fb0 2024-09-12T12:22:34.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f1180411640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f117815d760 con 0x7f117815d2d0 2024-09-12T12:22:34.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117e9ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f11780ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:34.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117d9ac640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1178159fb0 0x7f117806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:34.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117e9ae640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f11780ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:49444/0 (socket says 172.21.15.3:49444) 2024-09-12T12:22:34.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117e9ae640 1 -- 172.21.15.3:0/443298638 learned_addr learned my addr 172.21.15.3:0/443298638 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:34.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117e9ae640 1 -- 172.21.15.3:0/443298638 >> v2:172.21.15.142:3300/0 conn(0x7f1178159fb0 msgr2=0x7f117806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:34.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117e9ae640 1 --2- 172.21.15.3:0/443298638 >> v2:172.21.15.142:3300/0 conn(0x7f1178159fb0 0x7f117806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117e9ae640 1 -- 172.21.15.3:0/443298638 >> v1:172.21.15.142:6789/0 conn(0x7f1178159bd0 legacy=0x7f117806f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.760+0000 7f117e9ae640 1 -- 172.21.15.3:0/443298638 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117806e400 con 0x7f117815d2d0 2024-09-12T12:22:34.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f117e9ae640 1 --2- 172.21.15.3:0/443298638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f11780ad060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1170009870 tx=0x7f117001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf337575b9fe797d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:34.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f117d1ab640 1 -- 172.21.15.3:0/443298638 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11700ab640 con 0x7f117815d2d0 2024-09-12T12:22:34.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f117d1ab640 1 -- 172.21.15.3:0/443298638 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11700b0070 con 0x7f117815d2d0 2024-09-12T12:22:34.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f117d1ab640 1 -- 172.21.15.3:0/443298638 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11700aba00 con 0x7f117815d2d0 2024-09-12T12:22:34.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 -- 172.21.15.3:0/443298638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 msgr2=0x7f11780ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 --2- 172.21.15.3:0/443298638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f11780ad060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1170009870 tx=0x7f117001c530 comp rx=0 tx=0).stop 2024-09-12T12:22:34.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 -- 172.21.15.3:0/443298638 shutdown_connections 2024-09-12T12:22:34.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 --2- 172.21.15.3:0/443298638 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f11780ad060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 --2- 172.21.15.3:0/443298638 >> v2:172.21.15.142:3300/0 conn(0x7f1178159fb0 0x7f117806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 -- 172.21.15.3:0/443298638 >> 172.21.15.3:0/443298638 conn(0x7f11780b1d80 msgr2=0x7f11780b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:34.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 -- 172.21.15.3:0/443298638 shutdown_connections 2024-09-12T12:22:34.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.764+0000 7f1180411640 1 -- 172.21.15.3:0/443298638 wait complete. 2024-09-12T12:22:34.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 Processor -- start 2024-09-12T12:22:34.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 -- start start 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1178159bd0 0x7f1178145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178159fb0 0x7f1178158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f1178158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1178169360 con 0x7f1178159bd0 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f11781691e0 con 0x7f1178159fb0 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f11781694e0 con 0x7f117815d2d0 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1178159bd0 0x7f1178145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:34.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1178159bd0 0x7f1178145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37646/0 (socket says 172.21.15.3:37646) 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 -- 172.21.15.3:0/2945789678 learned_addr learned my addr 172.21.15.3:0/2945789678 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117d9ac640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178159fb0 0x7f1178158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e9ae640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f1178158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 -- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 msgr2=0x7f1178158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f1178158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 -- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178159fb0 msgr2=0x7f1178158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178159fb0 0x7f1178158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 -- 172.21.15.3:0/2945789678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11781556c0 con 0x7f1178159bd0 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f117e1ad640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1178159bd0 0x7f1178145b10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f116800c7d0 tx=0x7f116800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1176ffd640 1 -- 172.21.15.3:0/2945789678 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168003b80 con 0x7f1178159bd0 2024-09-12T12:22:34.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1176ffd640 1 -- 172.21.15.3:0/2945789678 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1168003d20 con 0x7f1178159bd0 2024-09-12T12:22:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1176ffd640 1 -- 172.21.15.3:0/2945789678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800aa90 con 0x7f1178159bd0 2024-09-12T12:22:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11781559a0 con 0x7f1178159bd0 2024-09-12T12:22:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1178155f60 con 0x7f1178159bd0 2024-09-12T12:22:34.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.768+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11780ad060 con 0x7f1178159bd0 2024-09-12T12:22:34.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.772+0000 7f1176ffd640 1 -- 172.21.15.3:0/2945789678 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f116800ac30 con 0x7f1178159bd0 2024-09-12T12:22:34.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.772+0000 7f1176ffd640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11640678c0 0x7f1164069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:34.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.772+0000 7f1176ffd640 1 -- 172.21.15.3:0/2945789678 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f1168085b10 con 0x7f1178159bd0 2024-09-12T12:22:34.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.772+0000 7f117d9ac640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11640678c0 0x7f1164069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:34.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.772+0000 7f117d9ac640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11640678c0 0x7f1164069d80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1178151ec0 tx=0x7f116c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:34.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.776+0000 7f1176ffd640 1 -- 172.21.15.3:0/2945789678 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f11680566e0 con 0x7f1178159bd0 2024-09-12T12:22:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.972+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11780b2b00 con 0x7f1178159bd0 2024-09-12T12:22:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.972+0000 7f1176ffd640 1 -- 172.21.15.3:0/2945789678 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7f1168055fe0 con 0x7f1178159bd0 2024-09-12T12:22:34.976 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:34.977 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"btime":"2024-09-12T12:22:34:602545+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/256387515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":256387515},{"type":"v1","addr":"172.21.15.142:6835","nonce":256387515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2534794519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2534794519},{"type":"v1","addr":"172.21.15.142:6837","nonce":2534794519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2676380928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2676380928},{"type":"v1","addr":"172.21.15.3:6833","nonce":2676380928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:22:30.937780+0000","modified":"2024-09-12T12:22:33.984511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4007447206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4007447206},{"type":"v1","addr":"172.21.15.3:6835","nonce":4007447206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5655,"qdb_cluster":[5655]},"id":4}]} 2024-09-12T12:22:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-12T12:22:34.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11640678c0 msgr2=0x7f1164069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11640678c0 0x7f1164069d80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1178151ec0 tx=0x7f116c008040 comp rx=0 tx=0).stop 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1178159bd0 msgr2=0x7f1178145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1178159bd0 0x7f1178145b10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f116800c7d0 tx=0x7f116800cc90 comp rx=0 tx=0).stop 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 shutdown_connections 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11640678c0 0x7f1164069d80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f117815d2d0 0x7f1178158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1178159fb0 0x7f1178158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 --2- 172.21.15.3:0/2945789678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1178159bd0 0x7f1178145b10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 >> 172.21.15.3:0/2945789678 conn(0x7f11780b1d80 msgr2=0x7f11780750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 shutdown_connections 2024-09-12T12:22:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:34.976+0000 7f1180411640 1 -- 172.21.15.3:0/2945789678 wait complete. 2024-09-12T12:22:34.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:22:35.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f800d47a640 1 Processor -- start 2024-09-12T12:22:35.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f800d47a640 1 -- start start 2024-09-12T12:22:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f800d47a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f800806e810 0x7f800806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f800d47a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80080d0390 con 0x7f80080704f0 2024-09-12T12:22:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f800d47a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800809bc60 con 0x7f800806e2d0 2024-09-12T12:22:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f800d47a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f80080d0630 con 0x7f800806e810 2024-09-12T12:22:35.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f8006ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f80080704f0 0x7f80080708d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48984/0 (socket says 172.21.15.3:48984) 2024-09-12T12:22:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f8006ffd640 1 -- 172.21.15.3:0/3720312982 learned_addr learned my addr 172.21.15.3:0/3720312982 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f80077fe640 1 --2- 172.21.15.3:0/3720312982 >> v2:172.21.15.3:3300/0 conn(0x7f800806e810 0x7f800806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f8005ffb640 1 -- 172.21.15.3:0/3720312982 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034721779 0 0) 0x7f80080d0390 con 0x7f80080704f0 2024-09-12T12:22:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f8005ffb640 1 -- 172.21.15.3:0/3720312982 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ff0003530 con 0x7f80080704f0 2024-09-12T12:22:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f80077fe640 1 -- 172.21.15.3:0/3720312982 >> v1:172.21.15.142:6789/0 conn(0x7f800806e2d0 legacy=0x7f800809b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f80077fe640 1 -- 172.21.15.3:0/3720312982 >> v1:172.21.15.3:6789/0 conn(0x7f80080704f0 legacy=0x7f80080708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.112+0000 7f80077fe640 1 -- 172.21.15.3:0/3720312982 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080d0910 con 0x7f800806e810 2024-09-12T12:22:35.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f80077fe640 1 --2- 172.21.15.3:0/3720312982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f800806cdd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8000009d90 tx=0x7f800001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee8b9ffc6758e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f8005ffb640 1 -- 172.21.15.3:0/3720312982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000b2070 con 0x7f800806e810 2024-09-12T12:22:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f8005ffb640 1 -- 172.21.15.3:0/3720312982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80000a5940 con 0x7f800806e810 2024-09-12T12:22:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f8005ffb640 1 -- 172.21.15.3:0/3720312982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000a5d10 con 0x7f800806e810 2024-09-12T12:22:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- 172.21.15.3:0/3720312982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 msgr2=0x7f800806cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 --2- 172.21.15.3:0/3720312982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f800806cdd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8000009d90 tx=0x7f800001c970 comp rx=0 tx=0).stop 2024-09-12T12:22:35.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- 172.21.15.3:0/3720312982 shutdown_connections 2024-09-12T12:22:35.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 --2- 172.21.15.3:0/3720312982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f800806cdd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- 172.21.15.3:0/3720312982 >> 172.21.15.3:0/3720312982 conn(0x7f80080b3e60 msgr2=0x7f80080b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:35.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- 172.21.15.3:0/3720312982 shutdown_connections 2024-09-12T12:22:35.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- 172.21.15.3:0/3720312982 wait complete. 2024-09-12T12:22:35.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 Processor -- start 2024-09-12T12:22:35.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- start start 2024-09-12T12:22:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800806e2d0 0x7f8008156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f80080b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80080704f0 0x7f80080c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f800816c7e0 con 0x7f800806e810 2024-09-12T12:22:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f800816c660 con 0x7f800806e2d0 2024-09-12T12:22:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.116+0000 7f800d47a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f800816c960 con 0x7f80080704f0 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f80080b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f80080b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37656/0 (socket says 172.21.15.3:37656) 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 -- 172.21.15.3:0/2498938428 learned_addr learned my addr 172.21.15.3:0/2498938428 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f8006ffd640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800806e2d0 0x7f8008156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80077fe640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80080704f0 0x7f80080c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 -- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80080704f0 msgr2=0x7f80080c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80080704f0 0x7f80080c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 -- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800806e2d0 msgr2=0x7f8008156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800806e2d0 0x7f8008156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 -- 172.21.15.3:0/2498938428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80080b56c0 con 0x7f800806e810 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f80067fc640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f80080b68c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c3a0 tx=0x7f7ffc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f7fef7fe640 1 -- 172.21.15.3:0/2498938428 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0052d0 con 0x7f800806e810 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f7fef7fe640 1 -- 172.21.15.3:0/2498938428 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc005470 con 0x7f800806e810 2024-09-12T12:22:35.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f7fef7fe640 1 -- 172.21.15.3:0/2498938428 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc007260 con 0x7f800806e810 2024-09-12T12:22:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80080b59a0 con 0x7f800806e810 2024-09-12T12:22:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80080b63a0 con 0x7f800806e810 2024-09-12T12:22:35.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800806cdd0 con 0x7f800806e810 2024-09-12T12:22:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f7fef7fe640 1 -- 172.21.15.3:0/2498938428 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ffc007400 con 0x7f800806e810 2024-09-12T12:22:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.120+0000 7f7fef7fe640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7ff40678f0 0x7f7ff4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.124+0000 7f7fef7fe640 1 -- 172.21.15.3:0/2498938428 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f7ffc085080 con 0x7f800806e810 2024-09-12T12:22:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.124+0000 7f8006ffd640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7ff40678f0 0x7f7ff4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.124+0000 7f8006ffd640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7ff40678f0 0x7f7ff4069db0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80011f0 tx=0x7f7ff8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.128+0000 7f7fef7fe640 1 -- 172.21.15.3:0/2498938428 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7ffc055c50 con 0x7f800806e810 2024-09-12T12:22:35.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.288+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80080b4a50 con 0x7f800806e810 2024-09-12T12:22:35.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f7fef7fe640 1 -- 172.21.15.3:0/2498938428 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16665 (secure 0 0 0) 0x7f7ffc055550 con 0x7f800806e810 2024-09-12T12:22:35.295 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:35.295 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":55,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:32.556186+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:22:28.609722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T12:22:29.925008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:22:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7ff40678f0 msgr2=0x7f7ff4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7ff40678f0 0x7f7ff4069db0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80011f0 tx=0x7f7ff8009040 comp rx=0 tx=0).stop 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 msgr2=0x7f80080b68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f80080b68c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00c3a0 tx=0x7f7ffc00c860 comp rx=0 tx=0).stop 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 shutdown_connections 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7ff40678f0 0x7f7ff4069db0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f80080704f0 0x7f80080c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f800806e810 0x7f80080b68c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 --2- 172.21.15.3:0/2498938428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800806e2d0 0x7f8008156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 >> 172.21.15.3:0/2498938428 conn(0x7f80080b3e60 msgr2=0x7f80080b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 shutdown_connections 2024-09-12T12:22:35.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.292+0000 7f800d47a640 1 -- 172.21.15.3:0/2498938428 wait complete. 2024-09-12T12:22:35.313 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:22:35.313 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:22:35.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f724a70b640 1 Processor -- start 2024-09-12T12:22:35.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f724a70b640 1 -- start start 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f724a70b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f724415e5d0 0x7f72440ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f724a70b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72440cfbe0 0x7f72440b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f724a70b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72440d31a0 con 0x7f72440d5050 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f724a70b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f72440b1060 con 0x7f724415e5d0 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f724a70b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f72440d3480 con 0x7f72440cfbe0 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f7243fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f724415e5d0 0x7f72440ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f723bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f72440d5050 0x7f72440cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:48994/0 (socket says 172.21.15.3:48994) 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f723bfff640 1 -- 172.21.15.3:0/1885379979 learned_addr learned my addr 172.21.15.3:0/1885379979 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f7248ca8640 1 --2- 172.21.15.3:0/1885379979 >> v2:172.21.15.142:3300/0 conn(0x7f72440cfbe0 0x7f72440b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f7243fff640 1 -- 172.21.15.3:0/1885379979 >> v2:172.21.15.142:3300/0 conn(0x7f72440cfbe0 msgr2=0x7f72440b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f7243fff640 1 --2- 172.21.15.3:0/1885379979 >> v2:172.21.15.142:3300/0 conn(0x7f72440cfbe0 0x7f72440b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f7243fff640 1 -- 172.21.15.3:0/1885379979 >> v1:172.21.15.3:6789/0 conn(0x7f72440d5050 legacy=0x7f72440cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f7243fff640 1 -- 172.21.15.3:0/1885379979 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72440d3760 con 0x7f724415e5d0 2024-09-12T12:22:35.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f72437fe640 1 -- 172.21.15.3:0/1885379979 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098010764 0 0) 0x7f72440d31a0 con 0x7f72440d5050 2024-09-12T12:22:35.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.424+0000 7f7243fff640 1 --2- 172.21.15.3:0/1885379979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f72440ceb60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f72440cb570 tx=0x7f723401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396b4004fac3cea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f72437fe640 1 -- 172.21.15.3:0/1885379979 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340ab610 con 0x7f724415e5d0 2024-09-12T12:22:35.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f72437fe640 1 -- 172.21.15.3:0/1885379979 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72340b0070 con 0x7f724415e5d0 2024-09-12T12:22:35.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f72437fe640 1 -- 172.21.15.3:0/1885379979 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72340ab9d0 con 0x7f724415e5d0 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- 172.21.15.3:0/1885379979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 msgr2=0x7f72440ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 --2- 172.21.15.3:0/1885379979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f72440ceb60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f72440cb570 tx=0x7f723401c3f0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- 172.21.15.3:0/1885379979 shutdown_connections 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 --2- 172.21.15.3:0/1885379979 >> v2:172.21.15.142:3300/0 conn(0x7f72440cfbe0 0x7f72440b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 --2- 172.21.15.3:0/1885379979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f72440ceb60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- 172.21.15.3:0/1885379979 >> 172.21.15.3:0/1885379979 conn(0x7f724408c000 msgr2=0x7f72440d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- 172.21.15.3:0/1885379979 shutdown_connections 2024-09-12T12:22:35.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- 172.21.15.3:0/1885379979 wait complete. 2024-09-12T12:22:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 Processor -- start 2024-09-12T12:22:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- start start 2024-09-12T12:22:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72440cfbe0 0x7f7244155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72440d5050 0x7f7244154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f724415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f72440b25d0 con 0x7f724415e5d0 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72440b2450 con 0x7f72440d5050 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72440b2750 con 0x7f72440cfbe0 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f724415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f724415ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37676/0 (socket says 172.21.15.3:37676) 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 -- 172.21.15.3:0/1962554107 learned_addr learned my addr 172.21.15.3:0/1962554107 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 -- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72440cfbe0 msgr2=0x7f7244155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72440cfbe0 0x7f7244155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 -- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72440d5050 msgr2=0x7f7244154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72440d5050 0x7f7244154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 -- 172.21.15.3:0/1962554107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7244155160 con 0x7f724415e5d0 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f7248ca8640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f724415ad50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f723c007cf0 tx=0x7f723c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f72417fa640 1 -- 172.21.15.3:0/1962554107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c014420 con 0x7f724415e5d0 2024-09-12T12:22:35.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f72417fa640 1 -- 172.21.15.3:0/1962554107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f723c019070 con 0x7f724415e5d0 2024-09-12T12:22:35.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7244169370 con 0x7f724415e5d0 2024-09-12T12:22:35.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f72417fa640 1 -- 172.21.15.3:0/1962554107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c014820 con 0x7f724415e5d0 2024-09-12T12:22:35.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.428+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72440c8450 con 0x7f724415e5d0 2024-09-12T12:22:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.432+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72440b14f0 con 0x7f724415e5d0 2024-09-12T12:22:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.432+0000 7f72417fa640 1 -- 172.21.15.3:0/1962554107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f723c0149c0 con 0x7f724415e5d0 2024-09-12T12:22:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.432+0000 7f72417fa640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72240678c0 0x7f7224069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.432+0000 7f72417fa640 1 -- 172.21.15.3:0/1962554107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f723c086390 con 0x7f724415e5d0 2024-09-12T12:22:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.432+0000 7f7243fff640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72240678c0 0x7f7224069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.432+0000 7f7243fff640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72240678c0 0x7f7224069d80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72340af000 tx=0x7f7234008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.436+0000 7f72417fa640 1 -- 172.21.15.3:0/1962554107 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f723c056f60 con 0x7f724415e5d0 2024-09-12T12:22:35.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.592+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f724406e0a0 con 0x7f72240678c0 2024-09-12T12:22:35.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.612+0000 7f72417fa640 1 -- 172.21.15.3:0/1962554107 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249256 (secure 0 0 0) 0x7f724406e0a0 con 0x7f72240678c0 2024-09-12T12:22:35.619 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:35.620 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:22:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72240678c0 msgr2=0x7f7224069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72240678c0 0x7f7224069d80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f72340af000 tx=0x7f7234008040 comp rx=0 tx=0).stop 2024-09-12T12:22:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 msgr2=0x7f724415ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f724415ad50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f723c007cf0 tx=0x7f723c00e4b0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 shutdown_connections 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72240678c0 0x7f7224069d80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f724415e5d0 0x7f724415ad50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72440d5050 0x7f7244154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 --2- 172.21.15.3:0/1962554107 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72440cfbe0 0x7f7244155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 >> 172.21.15.3:0/1962554107 conn(0x7f724408c000 msgr2=0x7f72440b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 shutdown_connections 2024-09-12T12:22:35.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.616+0000 7f724a70b640 1 -- 172.21.15.3:0/1962554107 wait complete. 2024-09-12T12:22:35.636 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-12T12:22:33.712450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592722,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":156,"acting":156,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":230768,"kb_used_data":6192,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743952,"statfs":{"total":773094113280,"available":772857806848,"internally_reserved":0,"allocated":6340608,"data_stored":3272472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2354,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.168086"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605735+0000","last_change":"2024-09-12T12:22:31.598465+0000","last_active":"2024-09-12T12:22:31.605735+0000","last_peered":"2024-09-12T12:22:31.605735+0000","last_clean":"2024-09-12T12:22:31.605735+0000","last_became_active":"2024-09-12T12:22:31.595054+0000","last_became_peered":"2024-09-12T12:22:31.595054+0000","last_unstale":"2024-09-12T12:22:31.605735+0000","last_undegraded":"2024-09-12T12:22:31.605735+0000","last_fullsized":"2024-09-12T12:22:31.605735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:36:23.938617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603910+0000","last_change":"2024-09-12T12:22:31.597654+0000","last_active":"2024-09-12T12:22:31.603910+0000","last_peered":"2024-09-12T12:22:31.603910+0000","last_clean":"2024-09-12T12:22:31.603910+0000","last_became_active":"2024-09-12T12:22:31.597181+0000","last_became_peered":"2024-09-12T12:22:31.597181+0000","last_unstale":"2024-09-12T12:22:31.603910+0000","last_undegraded":"2024-09-12T12:22:31.603910+0000","last_fullsized":"2024-09-12T12:22:31.603910+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:58.373027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.604562+0000","last_change":"2024-09-12T12:22:31.594786+0000","last_active":"2024-09-12T12:22:31.604562+0000","last_peered":"2024-09-12T12:22:31.604562+0000","last_clean":"2024-09-12T12:22:31.604562+0000","last_became_active":"2024-09-12T12:22:31.593983+0000","last_became_peered":"2024-09-12T12:22:31.593983+0000","last_unstale":"2024-09-12T12:22:31.604562+0000","last_undegraded":"2024-09-12T12:22:31.604562+0000","last_fullsized":"2024-09-12T12:22:31.604562+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:58.373027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594040+0000","last_change":"2024-09-12T12:22:31.592664+0000","last_active":"2024-09-12T12:22:31.594040+0000","last_peered":"2024-09-12T12:22:31.594040+0000","last_clean":"2024-09-12T12:22:31.594040+0000","last_became_active":"2024-09-12T12:22:31.592291+0000","last_became_peered":"2024-09-12T12:22:31.592291+0000","last_unstale":"2024-09-12T12:22:31.594040+0000","last_undegraded":"2024-09-12T12:22:31.594040+0000","last_fullsized":"2024-09-12T12:22:31.594040+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:01:23.913205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597767+0000","last_change":"2024-09-12T12:22:31.599553+0000","last_active":"2024-09-12T12:22:32.597767+0000","last_peered":"2024-09-12T12:22:32.597767+0000","last_clean":"2024-09-12T12:22:32.597767+0000","last_became_active":"2024-09-12T12:22:31.599276+0000","last_became_peered":"2024-09-12T12:22:31.599276+0000","last_unstale":"2024-09-12T12:22:32.597767+0000","last_undegraded":"2024-09-12T12:22:32.597767+0000","last_fullsized":"2024-09-12T12:22:32.597767+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:52:18.582029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609935+0000","last_change":"2024-09-12T12:22:31.597093+0000","last_active":"2024-09-12T12:22:31.609935+0000","last_peered":"2024-09-12T12:22:31.609935+0000","last_clean":"2024-09-12T12:22:31.609935+0000","last_became_active":"2024-09-12T12:22:31.596331+0000","last_became_peered":"2024-09-12T12:22:31.596331+0000","last_unstale":"2024-09-12T12:22:31.609935+0000","last_undegraded":"2024-09-12T12:22:31.609935+0000","last_fullsized":"2024-09-12T12:22:31.609935+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.466723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609977+0000","last_change":"2024-09-12T12:22:31.598663+0000","last_active":"2024-09-12T12:22:31.609977+0000","last_peered":"2024-09-12T12:22:31.609977+0000","last_clean":"2024-09-12T12:22:31.609977+0000","last_became_active":"2024-09-12T12:22:31.598229+0000","last_became_peered":"2024-09-12T12:22:31.598229+0000","last_unstale":"2024-09-12T12:22:31.609977+0000","last_undegraded":"2024-09-12T12:22:31.609977+0000","last_fullsized":"2024-09-12T12:22:31.609977+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:37:37.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603483+0000","last_change":"2024-09-12T12:22:31.597402+0000","last_active":"2024-09-12T12:22:31.603483+0000","last_peered":"2024-09-12T12:22:31.603483+0000","last_clean":"2024-09-12T12:22:31.603483+0000","last_became_active":"2024-09-12T12:22:31.597006+0000","last_became_peered":"2024-09-12T12:22:31.597006+0000","last_unstale":"2024-09-12T12:22:31.603483+0000","last_undegraded":"2024-09-12T12:22:31.603483+0000","last_fullsized":"2024-09-12T12:22:31.603483+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:21.833484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603856+0000","last_change":"2024-09-12T12:22:31.601934+0000","last_active":"2024-09-12T12:22:31.603856+0000","last_peered":"2024-09-12T12:22:31.603856+0000","last_clean":"2024-09-12T12:22:31.603856+0000","last_became_active":"2024-09-12T12:22:31.601580+0000","last_became_peered":"2024-09-12T12:22:31.601580+0000","last_unstale":"2024-09-12T12:22:31.603856+0000","last_undegraded":"2024-09-12T12:22:31.603856+0000","last_fullsized":"2024-09-12T12:22:31.603856+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:28.335911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594489+0000","last_change":"2024-09-12T12:22:31.585564+0000","last_active":"2024-09-12T12:22:31.594489+0000","last_peered":"2024-09-12T12:22:31.594489+0000","last_clean":"2024-09-12T12:22:31.594489+0000","last_became_active":"2024-09-12T12:22:31.584689+0000","last_became_peered":"2024-09-12T12:22:31.584689+0000","last_unstale":"2024-09-12T12:22:31.594489+0000","last_undegraded":"2024-09-12T12:22:31.594489+0000","last_fullsized":"2024-09-12T12:22:31.594489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:04.950897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.568034+0000","last_change":"2024-09-12T12:22:31.599725+0000","last_active":"2024-09-12T12:22:32.568034+0000","last_peered":"2024-09-12T12:22:32.568034+0000","last_clean":"2024-09-12T12:22:32.568034+0000","last_became_active":"2024-09-12T12:22:31.596519+0000","last_became_peered":"2024-09-12T12:22:31.596519+0000","last_unstale":"2024-09-12T12:22:32.568034+0000","last_undegraded":"2024-09-12T12:22:32.568034+0000","last_fullsized":"2024-09-12T12:22:32.568034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:21.833484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.607938+0000","last_change":"2024-09-12T12:22:31.595390+0000","last_active":"2024-09-12T12:22:31.607938+0000","last_peered":"2024-09-12T12:22:31.607938+0000","last_clean":"2024-09-12T12:22:31.607938+0000","last_became_active":"2024-09-12T12:22:31.594931+0000","last_became_peered":"2024-09-12T12:22:31.594931+0000","last_unstale":"2024-09-12T12:22:31.607938+0000","last_undegraded":"2024-09-12T12:22:31.607938+0000","last_fullsized":"2024-09-12T12:22:31.607938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:17:24.315256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.610210+0000","last_change":"2024-09-12T12:22:31.602546+0000","last_active":"2024-09-12T12:22:31.610210+0000","last_peered":"2024-09-12T12:22:31.610210+0000","last_clean":"2024-09-12T12:22:31.610210+0000","last_became_active":"2024-09-12T12:22:31.601858+0000","last_became_peered":"2024-09-12T12:22:31.601858+0000","last_unstale":"2024-09-12T12:22:31.610210+0000","last_undegraded":"2024-09-12T12:22:31.610210+0000","last_fullsized":"2024-09-12T12:22:31.610210+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:17.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.596856+0000","last_change":"2024-09-12T12:22:31.596705+0000","last_active":"2024-09-12T12:22:32.596856+0000","last_peered":"2024-09-12T12:22:32.596856+0000","last_clean":"2024-09-12T12:22:32.596856+0000","last_became_active":"2024-09-12T12:22:31.596379+0000","last_became_peered":"2024-09-12T12:22:31.596379+0000","last_unstale":"2024-09-12T12:22:32.596856+0000","last_undegraded":"2024-09-12T12:22:32.596856+0000","last_fullsized":"2024-09-12T12:22:32.596856+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:40:43.464580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605851+0000","last_change":"2024-09-12T12:22:31.599906+0000","last_active":"2024-09-12T12:22:31.605851+0000","last_peered":"2024-09-12T12:22:31.605851+0000","last_clean":"2024-09-12T12:22:31.605851+0000","last_became_active":"2024-09-12T12:22:31.598008+0000","last_became_peered":"2024-09-12T12:22:31.598008+0000","last_unstale":"2024-09-12T12:22:31.605851+0000","last_undegraded":"2024-09-12T12:22:31.605851+0000","last_fullsized":"2024-09-12T12:22:31.605851+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:18.099595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605985+0000","last_change":"2024-09-12T12:22:31.601928+0000","last_active":"2024-09-12T12:22:31.605985+0000","last_peered":"2024-09-12T12:22:31.605985+0000","last_clean":"2024-09-12T12:22:31.605985+0000","last_became_active":"2024-09-12T12:22:31.601577+0000","last_became_peered":"2024-09-12T12:22:31.601577+0000","last_unstale":"2024-09-12T12:22:31.605985+0000","last_undegraded":"2024-09-12T12:22:31.605985+0000","last_fullsized":"2024-09-12T12:22:31.605985+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:04:32.276759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.592314+0000","last_change":"2024-09-12T12:22:31.586849+0000","last_active":"2024-09-12T12:22:31.592314+0000","last_peered":"2024-09-12T12:22:31.592314+0000","last_clean":"2024-09-12T12:22:31.592314+0000","last_became_active":"2024-09-12T12:22:31.586390+0000","last_became_peered":"2024-09-12T12:22:31.586390+0000","last_unstale":"2024-09-12T12:22:31.592314+0000","last_undegraded":"2024-09-12T12:22:31.592314+0000","last_fullsized":"2024-09-12T12:22:31.592314+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:48:26.666389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.599416+0000","last_change":"2024-09-12T12:22:31.590465+0000","last_active":"2024-09-12T12:22:31.599416+0000","last_peered":"2024-09-12T12:22:31.599416+0000","last_clean":"2024-09-12T12:22:31.599416+0000","last_became_active":"2024-09-12T12:22:31.588823+0000","last_became_peered":"2024-09-12T12:22:31.588823+0000","last_unstale":"2024-09-12T12:22:31.599416+0000","last_undegraded":"2024-09-12T12:22:31.599416+0000","last_fullsized":"2024-09-12T12:22:31.599416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:04.950897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567692+0000","last_change":"2024-09-12T12:22:31.605595+0000","last_active":"2024-09-12T12:22:32.567692+0000","last_peered":"2024-09-12T12:22:32.567692+0000","last_clean":"2024-09-12T12:22:32.567692+0000","last_became_active":"2024-09-12T12:22:31.604813+0000","last_became_peered":"2024-09-12T12:22:31.604813+0000","last_unstale":"2024-09-12T12:22:32.567692+0000","last_undegraded":"2024-09-12T12:22:32.567692+0000","last_fullsized":"2024-09-12T12:22:32.567692+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:10:28.631953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605761+0000","last_change":"2024-09-12T12:22:31.601028+0000","last_active":"2024-09-12T12:22:31.605761+0000","last_peered":"2024-09-12T12:22:31.605761+0000","last_clean":"2024-09-12T12:22:31.605761+0000","last_became_active":"2024-09-12T12:22:31.600689+0000","last_became_peered":"2024-09-12T12:22:31.600689+0000","last_unstale":"2024-09-12T12:22:31.605761+0000","last_undegraded":"2024-09-12T12:22:31.605761+0000","last_fullsized":"2024-09-12T12:22:31.605761+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:26:10.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594528+0000","last_change":"2024-09-12T12:22:31.591387+0000","last_active":"2024-09-12T12:22:31.594528+0000","last_peered":"2024-09-12T12:22:31.594528+0000","last_clean":"2024-09-12T12:22:31.594528+0000","last_became_active":"2024-09-12T12:22:31.590888+0000","last_became_peered":"2024-09-12T12:22:31.590888+0000","last_unstale":"2024-09-12T12:22:31.594528+0000","last_undegraded":"2024-09-12T12:22:31.594528+0000","last_fullsized":"2024-09-12T12:22:31.594528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:09.362130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597540+0000","last_change":"2024-09-12T12:22:31.595631+0000","last_active":"2024-09-12T12:22:32.597540+0000","last_peered":"2024-09-12T12:22:32.597540+0000","last_clean":"2024-09-12T12:22:32.597540+0000","last_became_active":"2024-09-12T12:22:31.593768+0000","last_became_peered":"2024-09-12T12:22:31.593768+0000","last_unstale":"2024-09-12T12:22:32.597540+0000","last_undegraded":"2024-09-12T12:22:32.597540+0000","last_fullsized":"2024-09-12T12:22:32.597540+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:41:02.935897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.598741+0000","last_change":"2024-09-12T12:22:31.593925+0000","last_active":"2024-09-12T12:22:31.598741+0000","last_peered":"2024-09-12T12:22:31.598741+0000","last_clean":"2024-09-12T12:22:31.598741+0000","last_became_active":"2024-09-12T12:22:31.593233+0000","last_became_peered":"2024-09-12T12:22:31.593233+0000","last_unstale":"2024-09-12T12:22:31.598741+0000","last_undegraded":"2024-09-12T12:22:31.598741+0000","last_fullsized":"2024-09-12T12:22:31.598741+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:11:53.433972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.592173+0000","last_change":"2024-09-12T12:22:31.587407+0000","last_active":"2024-09-12T12:22:31.592173+0000","last_peered":"2024-09-12T12:22:31.592173+0000","last_clean":"2024-09-12T12:22:31.592173+0000","last_became_active":"2024-09-12T12:22:31.586683+0000","last_became_peered":"2024-09-12T12:22:31.586683+0000","last_unstale":"2024-09-12T12:22:31.592173+0000","last_undegraded":"2024-09-12T12:22:31.592173+0000","last_fullsized":"2024-09-12T12:22:31.592173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:21:05.594367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605781+0000","last_change":"2024-09-12T12:22:31.600149+0000","last_active":"2024-09-12T12:22:31.605781+0000","last_peered":"2024-09-12T12:22:31.605781+0000","last_clean":"2024-09-12T12:22:31.605781+0000","last_became_active":"2024-09-12T12:22:31.599908+0000","last_became_peered":"2024-09-12T12:22:31.599908+0000","last_unstale":"2024-09-12T12:22:31.605781+0000","last_undegraded":"2024-09-12T12:22:31.605781+0000","last_fullsized":"2024-09-12T12:22:31.605781+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:52.105465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597109+0000","last_change":"2024-09-12T12:22:31.597630+0000","last_active":"2024-09-12T12:22:32.597109+0000","last_peered":"2024-09-12T12:22:32.597109+0000","last_clean":"2024-09-12T12:22:32.597109+0000","last_became_active":"2024-09-12T12:22:31.596358+0000","last_became_peered":"2024-09-12T12:22:31.596358+0000","last_unstale":"2024-09-12T12:22:32.597109+0000","last_undegraded":"2024-09-12T12:22:32.597109+0000","last_fullsized":"2024-09-12T12:22:32.597109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:07.489528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597772+0000","last_change":"2024-09-12T12:22:31.595004+0000","last_active":"2024-09-12T12:22:32.597772+0000","last_peered":"2024-09-12T12:22:32.597772+0000","last_clean":"2024-09-12T12:22:32.597772+0000","last_became_active":"2024-09-12T12:22:31.594630+0000","last_became_peered":"2024-09-12T12:22:31.594630+0000","last_unstale":"2024-09-12T12:22:32.597772+0000","last_undegraded":"2024-09-12T12:22:32.597772+0000","last_fullsized":"2024-09-12T12:22:32.597772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:22.353401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597155+0000","last_change":"2024-09-12T12:22:30.596518+0000","last_active":"2024-09-12T12:22:32.597155+0000","last_peered":"2024-09-12T12:22:32.597155+0000","last_clean":"2024-09-12T12:22:32.597155+0000","last_became_active":"2024-09-12T12:22:30.596074+0000","last_became_peered":"2024-09-12T12:22:30.596074+0000","last_unstale":"2024-09-12T12:22:32.597155+0000","last_undegraded":"2024-09-12T12:22:32.597155+0000","last_fullsized":"2024-09-12T12:22:32.597155+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:56:19.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.599848+0000","last_change":"2024-09-12T12:22:31.597405+0000","last_active":"2024-09-12T12:22:31.599848+0000","last_peered":"2024-09-12T12:22:31.599848+0000","last_clean":"2024-09-12T12:22:31.599848+0000","last_became_active":"2024-09-12T12:22:31.597088+0000","last_became_peered":"2024-09-12T12:22:31.597088+0000","last_unstale":"2024-09-12T12:22:31.599848+0000","last_undegraded":"2024-09-12T12:22:31.599848+0000","last_fullsized":"2024-09-12T12:22:31.599848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:31.693955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:29.538335+0000","last_change":"2024-09-12T12:22:29.538335+0000","last_active":"2024-09-12T12:22:29.538335+0000","last_peered":"2024-09-12T12:22:29.538335+0000","last_clean":"2024-09-12T12:22:29.538335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:29.538335+0000","last_undegraded":"2024-09-12T12:22:29.538335+0000","last_fullsized":"2024-09-12T12:22:29.538335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597127+0000","last_change":"2024-09-12T12:22:31.602435+0000","last_active":"2024-09-12T12:22:32.597127+0000","last_peered":"2024-09-12T12:22:32.597127+0000","last_clean":"2024-09-12T12:22:32.597127+0000","last_became_active":"2024-09-12T12:22:31.602135+0000","last_became_peered":"2024-09-12T12:22:31.602135+0000","last_unstale":"2024-09-12T12:22:32.597127+0000","last_undegraded":"2024-09-12T12:22:32.597127+0000","last_fullsized":"2024-09-12T12:22:32.597127+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:06.996973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.598141+0000","last_change":"2024-09-12T12:22:30.594561+0000","last_active":"2024-09-12T12:22:32.598141+0000","last_peered":"2024-09-12T12:22:32.598141+0000","last_clean":"2024-09-12T12:22:32.598141+0000","last_became_active":"2024-09-12T12:22:30.594226+0000","last_became_peered":"2024-09-12T12:22:30.594226+0000","last_unstale":"2024-09-12T12:22:32.598141+0000","last_undegraded":"2024-09-12T12:22:32.598141+0000","last_fullsized":"2024-09-12T12:22:32.598141+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:36.420998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597881+0000","last_change":"2024-09-12T12:22:31.595458+0000","last_active":"2024-09-12T12:22:32.597881+0000","last_peered":"2024-09-12T12:22:32.597881+0000","last_clean":"2024-09-12T12:22:32.597881+0000","last_became_active":"2024-09-12T12:22:31.593461+0000","last_became_peered":"2024-09-12T12:22:31.593461+0000","last_unstale":"2024-09-12T12:22:32.597881+0000","last_undegraded":"2024-09-12T12:22:32.597881+0000","last_fullsized":"2024-09-12T12:22:32.597881+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:38:31.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.619684+0000","last_change":"2024-09-12T12:22:30.600283+0000","last_active":"2024-09-12T12:22:31.619684+0000","last_peered":"2024-09-12T12:22:31.619684+0000","last_clean":"2024-09-12T12:22:31.619684+0000","last_became_active":"2024-09-12T12:22:30.599503+0000","last_became_peered":"2024-09-12T12:22:30.599503+0000","last_unstale":"2024-09-12T12:22:31.619684+0000","last_undegraded":"2024-09-12T12:22:31.619684+0000","last_fullsized":"2024-09-12T12:22:31.619684+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:49:53.700003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.613406+0000","last_change":"2024-09-12T12:22:30.585642+0000","last_active":"2024-09-12T12:22:31.613406+0000","last_peered":"2024-09-12T12:22:31.613406+0000","last_clean":"2024-09-12T12:22:31.613406+0000","last_became_active":"2024-09-12T12:22:30.584940+0000","last_became_peered":"2024-09-12T12:22:30.584940+0000","last_unstale":"2024-09-12T12:22:31.613406+0000","last_undegraded":"2024-09-12T12:22:31.613406+0000","last_fullsized":"2024-09-12T12:22:31.613406+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:44.353244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597822+0000","last_change":"2024-09-12T12:22:30.586415+0000","last_active":"2024-09-12T12:22:32.597822+0000","last_peered":"2024-09-12T12:22:32.597822+0000","last_clean":"2024-09-12T12:22:32.597822+0000","last_became_active":"2024-09-12T12:22:30.585980+0000","last_became_peered":"2024-09-12T12:22:30.585980+0000","last_unstale":"2024-09-12T12:22:32.597822+0000","last_undegraded":"2024-09-12T12:22:32.597822+0000","last_fullsized":"2024-09-12T12:22:32.597822+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:17:41.342306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567762+0000","last_change":"2024-09-12T12:22:31.580457+0000","last_active":"2024-09-12T12:22:32.567762+0000","last_peered":"2024-09-12T12:22:32.567762+0000","last_clean":"2024-09-12T12:22:32.567762+0000","last_became_active":"2024-09-12T12:22:31.579673+0000","last_became_peered":"2024-09-12T12:22:31.579673+0000","last_unstale":"2024-09-12T12:22:32.567762+0000","last_undegraded":"2024-09-12T12:22:32.567762+0000","last_fullsized":"2024-09-12T12:22:32.567762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.986247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572190+0000","last_change":"2024-09-12T12:22:30.598094+0000","last_active":"2024-09-12T12:22:31.572190+0000","last_peered":"2024-09-12T12:22:31.572190+0000","last_clean":"2024-09-12T12:22:31.572190+0000","last_became_active":"2024-09-12T12:22:30.594331+0000","last_became_peered":"2024-09-12T12:22:30.594331+0000","last_unstale":"2024-09-12T12:22:31.572190+0000","last_undegraded":"2024-09-12T12:22:31.572190+0000","last_fullsized":"2024-09-12T12:22:31.572190+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:48:25.662443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.584879+0000","last_change":"2024-09-12T12:22:30.591999+0000","last_active":"2024-09-12T12:22:31.584879+0000","last_peered":"2024-09-12T12:22:31.584879+0000","last_clean":"2024-09-12T12:22:31.584879+0000","last_became_active":"2024-09-12T12:22:30.591371+0000","last_became_peered":"2024-09-12T12:22:30.591371+0000","last_unstale":"2024-09-12T12:22:31.584879+0000","last_undegraded":"2024-09-12T12:22:31.584879+0000","last_fullsized":"2024-09-12T12:22:31.584879+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:33:05.916427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.610279+0000","last_change":"2024-09-12T12:22:31.604147+0000","last_active":"2024-09-12T12:22:31.610279+0000","last_peered":"2024-09-12T12:22:31.610279+0000","last_clean":"2024-09-12T12:22:31.610279+0000","last_became_active":"2024-09-12T12:22:31.602898+0000","last_became_peered":"2024-09-12T12:22:31.602898+0000","last_unstale":"2024-09-12T12:22:31.610279+0000","last_undegraded":"2024-09-12T12:22:31.610279+0000","last_fullsized":"2024-09-12T12:22:31.610279+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:46.326145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.589560+0000","last_change":"2024-09-12T12:22:30.588745+0000","last_active":"2024-09-12T12:22:31.589560+0000","last_peered":"2024-09-12T12:22:31.589560+0000","last_clean":"2024-09-12T12:22:31.589560+0000","last_became_active":"2024-09-12T12:22:30.588031+0000","last_became_peered":"2024-09-12T12:22:30.588031+0000","last_unstale":"2024-09-12T12:22:31.589560+0000","last_undegraded":"2024-09-12T12:22:31.589560+0000","last_fullsized":"2024-09-12T12:22:31.589560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:09.145112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.611310+0000","last_change":"2024-09-12T12:22:31.604015+0000","last_active":"2024-09-12T12:22:31.611310+0000","last_peered":"2024-09-12T12:22:31.611310+0000","last_clean":"2024-09-12T12:22:31.611310+0000","last_became_active":"2024-09-12T12:22:31.603530+0000","last_became_peered":"2024-09-12T12:22:31.603530+0000","last_unstale":"2024-09-12T12:22:31.611310+0000","last_undegraded":"2024-09-12T12:22:31.611310+0000","last_fullsized":"2024-09-12T12:22:31.611310+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:00.325171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:29.538335+0000","last_change":"2024-09-12T12:22:29.538335+0000","last_active":"2024-09-12T12:22:29.538335+0000","last_peered":"2024-09-12T12:22:29.538335+0000","last_clean":"2024-09-12T12:22:29.538335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:29.538335+0000","last_undegraded":"2024-09-12T12:22:29.538335+0000","last_fullsized":"2024-09-12T12:22:29.538335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603509+0000","last_change":"2024-09-12T12:22:31.598907+0000","last_active":"2024-09-12T12:22:31.603509+0000","last_peered":"2024-09-12T12:22:31.603509+0000","last_clean":"2024-09-12T12:22:31.603509+0000","last_became_active":"2024-09-12T12:22:31.598575+0000","last_became_peered":"2024-09-12T12:22:31.598575+0000","last_unstale":"2024-09-12T12:22:31.603509+0000","last_undegraded":"2024-09-12T12:22:31.603509+0000","last_fullsized":"2024-09-12T12:22:31.603509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.986247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.619656+0000","last_change":"2024-09-12T12:22:30.596808+0000","last_active":"2024-09-12T12:22:31.619656+0000","last_peered":"2024-09-12T12:22:31.619656+0000","last_clean":"2024-09-12T12:22:31.619656+0000","last_became_active":"2024-09-12T12:22:30.595968+0000","last_became_peered":"2024-09-12T12:22:30.595968+0000","last_unstale":"2024-09-12T12:22:31.619656+0000","last_undegraded":"2024-09-12T12:22:31.619656+0000","last_fullsized":"2024-09-12T12:22:31.619656+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:49:53.700003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.611377+0000","last_change":"2024-09-12T12:22:31.604094+0000","last_active":"2024-09-12T12:22:31.611377+0000","last_peered":"2024-09-12T12:22:31.611377+0000","last_clean":"2024-09-12T12:22:31.611377+0000","last_became_active":"2024-09-12T12:22:31.603003+0000","last_became_peered":"2024-09-12T12:22:31.603003+0000","last_unstale":"2024-09-12T12:22:31.611377+0000","last_undegraded":"2024-09-12T12:22:31.611377+0000","last_fullsized":"2024-09-12T12:22:31.611377+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:57.296236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.620352+0000","last_change":"2024-09-12T12:22:30.583697+0000","last_active":"2024-09-12T12:22:31.620352+0000","last_peered":"2024-09-12T12:22:31.620352+0000","last_clean":"2024-09-12T12:22:31.620352+0000","last_became_active":"2024-09-12T12:22:30.582920+0000","last_became_peered":"2024-09-12T12:22:30.582920+0000","last_unstale":"2024-09-12T12:22:31.620352+0000","last_undegraded":"2024-09-12T12:22:31.620352+0000","last_fullsized":"2024-09-12T12:22:31.620352+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:48:20.728320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609619+0000","last_change":"2024-09-12T12:22:31.606102+0000","last_active":"2024-09-12T12:22:31.609619+0000","last_peered":"2024-09-12T12:22:31.609619+0000","last_clean":"2024-09-12T12:22:31.609619+0000","last_became_active":"2024-09-12T12:22:31.605453+0000","last_became_peered":"2024-09-12T12:22:31.605453+0000","last_unstale":"2024-09-12T12:22:31.609619+0000","last_undegraded":"2024-09-12T12:22:31.609619+0000","last_fullsized":"2024-09-12T12:22:31.609619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:00.325171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:29.538335+0000","last_change":"2024-09-12T12:22:29.538335+0000","last_active":"2024-09-12T12:22:29.538335+0000","last_peered":"2024-09-12T12:22:29.538335+0000","last_clean":"2024-09-12T12:22:29.538335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:29.538335+0000","last_undegraded":"2024-09-12T12:22:29.538335+0000","last_fullsized":"2024-09-12T12:22:29.538335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594983+0000","last_change":"2024-09-12T12:22:31.589349+0000","last_active":"2024-09-12T12:22:31.594983+0000","last_peered":"2024-09-12T12:22:31.594983+0000","last_clean":"2024-09-12T12:22:31.594983+0000","last_became_active":"2024-09-12T12:22:31.588607+0000","last_became_peered":"2024-09-12T12:22:31.588607+0000","last_unstale":"2024-09-12T12:22:31.594983+0000","last_undegraded":"2024-09-12T12:22:31.594983+0000","last_fullsized":"2024-09-12T12:22:31.594983+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:11:25.011120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567873+0000","last_change":"2024-09-12T12:22:30.588538+0000","last_active":"2024-09-12T12:22:32.567873+0000","last_peered":"2024-09-12T12:22:32.567873+0000","last_clean":"2024-09-12T12:22:32.567873+0000","last_became_active":"2024-09-12T12:22:30.587825+0000","last_became_peered":"2024-09-12T12:22:30.587825+0000","last_unstale":"2024-09-12T12:22:32.567873+0000","last_undegraded":"2024-09-12T12:22:32.567873+0000","last_fullsized":"2024-09-12T12:22:32.567873+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:23:49.561010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.610118+0000","last_change":"2024-09-12T12:22:31.601216+0000","last_active":"2024-09-12T12:22:31.610118+0000","last_peered":"2024-09-12T12:22:31.610118+0000","last_clean":"2024-09-12T12:22:31.610118+0000","last_became_active":"2024-09-12T12:22:31.595458+0000","last_became_peered":"2024-09-12T12:22:31.595458+0000","last_unstale":"2024-09-12T12:22:31.610118+0000","last_undegraded":"2024-09-12T12:22:31.610118+0000","last_fullsized":"2024-09-12T12:22:31.610118+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:15.576446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572425+0000","last_change":"2024-09-12T12:22:30.596630+0000","last_active":"2024-09-12T12:22:31.572425+0000","last_peered":"2024-09-12T12:22:31.572425+0000","last_clean":"2024-09-12T12:22:31.572425+0000","last_became_active":"2024-09-12T12:22:30.595435+0000","last_became_peered":"2024-09-12T12:22:30.595435+0000","last_unstale":"2024-09-12T12:22:31.572425+0000","last_undegraded":"2024-09-12T12:22:31.572425+0000","last_fullsized":"2024-09-12T12:22:31.572425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:21:04.590421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594566+0000","last_change":"2024-09-12T12:22:31.591794+0000","last_active":"2024-09-12T12:22:31.594566+0000","last_peered":"2024-09-12T12:22:31.594566+0000","last_clean":"2024-09-12T12:22:31.594566+0000","last_became_active":"2024-09-12T12:22:31.589787+0000","last_became_peered":"2024-09-12T12:22:31.589787+0000","last_unstale":"2024-09-12T12:22:31.594566+0000","last_undegraded":"2024-09-12T12:22:31.594566+0000","last_fullsized":"2024-09-12T12:22:31.594566+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:09.362130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.622554+0000","last_change":"2024-09-12T12:22:30.589423+0000","last_active":"2024-09-12T12:22:31.622554+0000","last_peered":"2024-09-12T12:22:31.622554+0000","last_clean":"2024-09-12T12:22:31.622554+0000","last_became_active":"2024-09-12T12:22:30.588637+0000","last_became_peered":"2024-09-12T12:22:30.588637+0000","last_unstale":"2024-09-12T12:22:31.622554+0000","last_undegraded":"2024-09-12T12:22:31.622554+0000","last_fullsized":"2024-09-12T12:22:31.622554+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:03.758234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605887+0000","last_change":"2024-09-12T12:22:31.600493+0000","last_active":"2024-09-12T12:22:31.605887+0000","last_peered":"2024-09-12T12:22:31.605887+0000","last_clean":"2024-09-12T12:22:31.605887+0000","last_became_active":"2024-09-12T12:22:31.599248+0000","last_became_peered":"2024-09-12T12:22:31.599248+0000","last_unstale":"2024-09-12T12:22:31.605887+0000","last_undegraded":"2024-09-12T12:22:31.605887+0000","last_fullsized":"2024-09-12T12:22:31.605887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:04.049615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:29.538335+0000","last_change":"2024-09-12T12:22:29.538335+0000","last_active":"2024-09-12T12:22:29.538335+0000","last_peered":"2024-09-12T12:22:29.538335+0000","last_clean":"2024-09-12T12:22:29.538335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:29.538335+0000","last_undegraded":"2024-09-12T12:22:29.538335+0000","last_fullsized":"2024-09-12T12:22:29.538335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609238+0000","last_change":"2024-09-12T12:22:31.604643+0000","last_active":"2024-09-12T12:22:31.609238+0000","last_peered":"2024-09-12T12:22:31.609238+0000","last_clean":"2024-09-12T12:22:31.609238+0000","last_became_active":"2024-09-12T12:22:31.603937+0000","last_became_peered":"2024-09-12T12:22:31.603937+0000","last_unstale":"2024-09-12T12:22:31.609238+0000","last_undegraded":"2024-09-12T12:22:31.609238+0000","last_fullsized":"2024-09-12T12:22:31.609238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:18.099595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.598058+0000","last_change":"2024-09-12T12:22:30.582323+0000","last_active":"2024-09-12T12:22:32.598058+0000","last_peered":"2024-09-12T12:22:32.598058+0000","last_clean":"2024-09-12T12:22:32.598058+0000","last_became_active":"2024-09-12T12:22:30.582015+0000","last_became_peered":"2024-09-12T12:22:30.582015+0000","last_unstale":"2024-09-12T12:22:32.598058+0000","last_undegraded":"2024-09-12T12:22:32.598058+0000","last_fullsized":"2024-09-12T12:22:32.598058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:13:33.060254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.618084+0000","last_change":"2024-09-12T12:22:30.590249+0000","last_active":"2024-09-12T12:22:31.618084+0000","last_peered":"2024-09-12T12:22:31.618084+0000","last_clean":"2024-09-12T12:22:31.618084+0000","last_became_active":"2024-09-12T12:22:30.589576+0000","last_became_peered":"2024-09-12T12:22:30.589576+0000","last_unstale":"2024-09-12T12:22:31.618084+0000","last_undegraded":"2024-09-12T12:22:31.618084+0000","last_fullsized":"2024-09-12T12:22:31.618084+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:03.758234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.611339+0000","last_change":"2024-09-12T12:22:31.604114+0000","last_active":"2024-09-12T12:22:31.611339+0000","last_peered":"2024-09-12T12:22:31.611339+0000","last_clean":"2024-09-12T12:22:31.611339+0000","last_became_active":"2024-09-12T12:22:31.603548+0000","last_became_peered":"2024-09-12T12:22:31.603548+0000","last_unstale":"2024-09-12T12:22:31.611339+0000","last_undegraded":"2024-09-12T12:22:31.611339+0000","last_fullsized":"2024-09-12T12:22:31.611339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:29:38.122250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597131+0000","last_change":"2024-09-12T12:22:30.582935+0000","last_active":"2024-09-12T12:22:32.597131+0000","last_peered":"2024-09-12T12:22:32.597131+0000","last_clean":"2024-09-12T12:22:32.597131+0000","last_became_active":"2024-09-12T12:22:30.582550+0000","last_became_peered":"2024-09-12T12:22:30.582550+0000","last_unstale":"2024-09-12T12:22:32.597131+0000","last_undegraded":"2024-09-12T12:22:32.597131+0000","last_fullsized":"2024-09-12T12:22:32.597131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:25:11.432036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609329+0000","last_change":"2024-09-12T12:22:31.605555+0000","last_active":"2024-09-12T12:22:31.609329+0000","last_peered":"2024-09-12T12:22:31.609329+0000","last_clean":"2024-09-12T12:22:31.609329+0000","last_became_active":"2024-09-12T12:22:31.604868+0000","last_became_peered":"2024-09-12T12:22:31.604868+0000","last_unstale":"2024-09-12T12:22:31.609329+0000","last_undegraded":"2024-09-12T12:22:31.609329+0000","last_fullsized":"2024-09-12T12:22:31.609329+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:04.049615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:29.538335+0000","last_change":"2024-09-12T12:22:29.538335+0000","last_active":"2024-09-12T12:22:29.538335+0000","last_peered":"2024-09-12T12:22:29.538335+0000","last_clean":"2024-09-12T12:22:29.538335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:29.538335+0000","last_undegraded":"2024-09-12T12:22:29.538335+0000","last_fullsized":"2024-09-12T12:22:29.538335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597336+0000","last_change":"2024-09-12T12:22:31.595271+0000","last_active":"2024-09-12T12:22:32.597336+0000","last_peered":"2024-09-12T12:22:32.597336+0000","last_clean":"2024-09-12T12:22:32.597336+0000","last_became_active":"2024-09-12T12:22:31.594823+0000","last_became_peered":"2024-09-12T12:22:31.594823+0000","last_unstale":"2024-09-12T12:22:32.597336+0000","last_undegraded":"2024-09-12T12:22:32.597336+0000","last_fullsized":"2024-09-12T12:22:32.597336+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:45.881345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.587334+0000","last_change":"2024-09-12T12:22:30.597645+0000","last_active":"2024-09-12T12:22:31.587334+0000","last_peered":"2024-09-12T12:22:31.587334+0000","last_clean":"2024-09-12T12:22:31.587334+0000","last_became_active":"2024-09-12T12:22:30.597231+0000","last_became_peered":"2024-09-12T12:22:30.597231+0000","last_unstale":"2024-09-12T12:22:31.587334+0000","last_undegraded":"2024-09-12T12:22:31.587334+0000","last_fullsized":"2024-09-12T12:22:31.587334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:22.784957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597208+0000","last_change":"2024-09-12T12:22:31.592723+0000","last_active":"2024-09-12T12:22:32.597208+0000","last_peered":"2024-09-12T12:22:32.597208+0000","last_clean":"2024-09-12T12:22:32.597208+0000","last_became_active":"2024-09-12T12:22:31.592438+0000","last_became_peered":"2024-09-12T12:22:31.592438+0000","last_unstale":"2024-09-12T12:22:32.597208+0000","last_undegraded":"2024-09-12T12:22:32.597208+0000","last_fullsized":"2024-09-12T12:22:32.597208+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:33.115619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572009+0000","last_change":"2024-09-12T12:22:30.601472+0000","last_active":"2024-09-12T12:22:31.572009+0000","last_peered":"2024-09-12T12:22:31.572009+0000","last_clean":"2024-09-12T12:22:31.572009+0000","last_became_active":"2024-09-12T12:22:30.600618+0000","last_became_peered":"2024-09-12T12:22:30.600618+0000","last_unstale":"2024-09-12T12:22:31.572009+0000","last_undegraded":"2024-09-12T12:22:31.572009+0000","last_fullsized":"2024-09-12T12:22:31.572009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:45.322199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.604619+0000","last_change":"2024-09-12T12:22:31.594578+0000","last_active":"2024-09-12T12:22:31.604619+0000","last_peered":"2024-09-12T12:22:31.604619+0000","last_clean":"2024-09-12T12:22:31.604619+0000","last_became_active":"2024-09-12T12:22:31.593911+0000","last_became_peered":"2024-09-12T12:22:31.593911+0000","last_unstale":"2024-09-12T12:22:31.604619+0000","last_undegraded":"2024-09-12T12:22:31.604619+0000","last_fullsized":"2024-09-12T12:22:31.604619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:43.978839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.613350+0000","last_change":"2024-09-12T12:22:30.595429+0000","last_active":"2024-09-12T12:22:31.613350+0000","last_peered":"2024-09-12T12:22:31.613350+0000","last_clean":"2024-09-12T12:22:31.613350+0000","last_became_active":"2024-09-12T12:22:30.595014+0000","last_became_peered":"2024-09-12T12:22:30.595014+0000","last_unstale":"2024-09-12T12:22:31.613350+0000","last_undegraded":"2024-09-12T12:22:31.613350+0000","last_fullsized":"2024-09-12T12:22:31.613350+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:23:49.561010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594304+0000","last_change":"2024-09-12T12:22:31.585082+0000","last_active":"2024-09-12T12:22:31.594304+0000","last_peered":"2024-09-12T12:22:31.594304+0000","last_clean":"2024-09-12T12:22:31.594304+0000","last_became_active":"2024-09-12T12:22:31.584621+0000","last_became_peered":"2024-09-12T12:22:31.584621+0000","last_unstale":"2024-09-12T12:22:31.594304+0000","last_undegraded":"2024-09-12T12:22:31.594304+0000","last_fullsized":"2024-09-12T12:22:31.594304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:10.149058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.601047+0000","last_change":"2024-09-12T12:22:30.586241+0000","last_active":"2024-09-12T12:22:32.601047+0000","last_peered":"2024-09-12T12:22:32.601047+0000","last_clean":"2024-09-12T12:22:32.601047+0000","last_became_active":"2024-09-12T12:22:30.585913+0000","last_became_peered":"2024-09-12T12:22:30.585913+0000","last_unstale":"2024-09-12T12:22:32.601047+0000","last_undegraded":"2024-09-12T12:22:32.601047+0000","last_fullsized":"2024-09-12T12:22:32.601047+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:30:15.972093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.602957+0000","last_change":"2024-09-12T12:22:31.598372+0000","last_active":"2024-09-12T12:22:31.602957+0000","last_peered":"2024-09-12T12:22:31.602957+0000","last_clean":"2024-09-12T12:22:31.602957+0000","last_became_active":"2024-09-12T12:22:31.598078+0000","last_became_peered":"2024-09-12T12:22:31.598078+0000","last_unstale":"2024-09-12T12:22:31.602957+0000","last_undegraded":"2024-09-12T12:22:31.602957+0000","last_fullsized":"2024-09-12T12:22:31.602957+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:39.300126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.568059+0000","last_change":"2024-09-12T12:22:30.583044+0000","last_active":"2024-09-12T12:22:32.568059+0000","last_peered":"2024-09-12T12:22:32.568059+0000","last_clean":"2024-09-12T12:22:32.568059+0000","last_became_active":"2024-09-12T12:22:30.582352+0000","last_became_peered":"2024-09-12T12:22:30.582352+0000","last_unstale":"2024-09-12T12:22:32.568059+0000","last_undegraded":"2024-09-12T12:22:32.568059+0000","last_fullsized":"2024-09-12T12:22:32.568059+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:31.490479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597952+0000","last_change":"2024-09-12T12:22:31.594015+0000","last_active":"2024-09-12T12:22:32.597952+0000","last_peered":"2024-09-12T12:22:32.597952+0000","last_clean":"2024-09-12T12:22:32.597952+0000","last_became_active":"2024-09-12T12:22:31.593131+0000","last_became_peered":"2024-09-12T12:22:31.593131+0000","last_unstale":"2024-09-12T12:22:32.597952+0000","last_undegraded":"2024-09-12T12:22:32.597952+0000","last_fullsized":"2024-09-12T12:22:32.597952+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:17.091261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.574351+0000","last_change":"2024-09-12T12:22:30.602111+0000","last_active":"2024-09-12T12:22:31.574351+0000","last_peered":"2024-09-12T12:22:31.574351+0000","last_clean":"2024-09-12T12:22:31.574351+0000","last_became_active":"2024-09-12T12:22:30.601379+0000","last_became_peered":"2024-09-12T12:22:30.601379+0000","last_unstale":"2024-09-12T12:22:31.574351+0000","last_undegraded":"2024-09-12T12:22:31.574351+0000","last_fullsized":"2024-09-12T12:22:31.574351+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:11:24.007173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.582047+0000","last_change":"2024-09-12T12:22:30.594273+0000","last_active":"2024-09-12T12:22:31.582047+0000","last_peered":"2024-09-12T12:22:31.582047+0000","last_clean":"2024-09-12T12:22:31.582047+0000","last_became_active":"2024-09-12T12:22:30.593662+0000","last_became_peered":"2024-09-12T12:22:30.593662+0000","last_unstale":"2024-09-12T12:22:31.582047+0000","last_undegraded":"2024-09-12T12:22:31.582047+0000","last_fullsized":"2024-09-12T12:22:31.582047+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:31.490479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.598043+0000","last_change":"2024-09-12T12:22:31.594190+0000","last_active":"2024-09-12T12:22:31.598043+0000","last_peered":"2024-09-12T12:22:31.598043+0000","last_clean":"2024-09-12T12:22:31.598043+0000","last_became_active":"2024-09-12T12:22:31.593504+0000","last_became_peered":"2024-09-12T12:22:31.593504+0000","last_unstale":"2024-09-12T12:22:31.598043+0000","last_undegraded":"2024-09-12T12:22:31.598043+0000","last_fullsized":"2024-09-12T12:22:31.598043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:43.978839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.580418+0000","last_change":"2024-09-12T12:22:30.583854+0000","last_active":"2024-09-12T12:22:31.580418+0000","last_peered":"2024-09-12T12:22:31.580418+0000","last_clean":"2024-09-12T12:22:31.580418+0000","last_became_active":"2024-09-12T12:22:30.583063+0000","last_became_peered":"2024-09-12T12:22:30.583063+0000","last_unstale":"2024-09-12T12:22:31.580418+0000","last_undegraded":"2024-09-12T12:22:31.580418+0000","last_fullsized":"2024-09-12T12:22:31.580418+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:43.721208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.602853+0000","last_change":"2024-09-12T12:22:31.598459+0000","last_active":"2024-09-12T12:22:31.602853+0000","last_peered":"2024-09-12T12:22:31.602853+0000","last_clean":"2024-09-12T12:22:31.602853+0000","last_became_active":"2024-09-12T12:22:31.598097+0000","last_became_peered":"2024-09-12T12:22:31.598097+0000","last_unstale":"2024-09-12T12:22:31.602853+0000","last_undegraded":"2024-09-12T12:22:31.602853+0000","last_fullsized":"2024-09-12T12:22:31.602853+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:10:28.631953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.574146+0000","last_change":"2024-09-12T12:22:30.593490+0000","last_active":"2024-09-12T12:22:31.574146+0000","last_peered":"2024-09-12T12:22:31.574146+0000","last_clean":"2024-09-12T12:22:31.574146+0000","last_became_active":"2024-09-12T12:22:30.592880+0000","last_became_peered":"2024-09-12T12:22:30.592880+0000","last_unstale":"2024-09-12T12:22:31.574146+0000","last_undegraded":"2024-09-12T12:22:31.574146+0000","last_fullsized":"2024-09-12T12:22:31.574146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:49.609132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:29.538335+0000","last_change":"2024-09-12T12:22:29.538335+0000","last_active":"2024-09-12T12:22:29.538335+0000","last_peered":"2024-09-12T12:22:29.538335+0000","last_clean":"2024-09-12T12:22:29.538335+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:29.538335+0000","last_undegraded":"2024-09-12T12:22:29.538335+0000","last_fullsized":"2024-09-12T12:22:29.538335+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-12T12:22:30.542282+0000","last_change":"2024-09-12T12:22:30.542282+0000","last_active":"2024-09-12T12:22:30.542282+0000","last_peered":"2024-09-12T12:22:30.542282+0000","last_clean":"2024-09-12T12:22:30.542282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:22:30.542282+0000","last_undegraded":"2024-09-12T12:22:30.542282+0000","last_fullsized":"2024-09-12T12:22:30.542282+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567674+0000","last_change":"2024-09-12T12:22:30.594141+0000","last_active":"2024-09-12T12:22:32.567674+0000","last_peered":"2024-09-12T12:22:32.567674+0000","last_clean":"2024-09-12T12:22:32.567674+0000","last_became_active":"2024-09-12T12:22:30.593407+0000","last_became_peered":"2024-09-12T12:22:30.593407+0000","last_unstale":"2024-09-12T12:22:32.567674+0000","last_undegraded":"2024-09-12T12:22:32.567674+0000","last_fullsized":"2024-09-12T12:22:32.567674+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:49.609132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572877+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:22:31.572877+0000","last_peered":"2024-09-12T12:22:31.572877+0000","last_clean":"2024-09-12T12:22:31.572877+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:22:31.572877+0000","last_undegraded":"2024-09-12T12:22:31.572877+0000","last_fullsized":"2024-09-12T12:22:31.572877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2354,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":52,"acting":52,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31772,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340068,"statfs":{"total":96636764160,"available":96604229632,"internally_reserved":0,"allocated":593920,"data_stored":239945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1245184,"data_stored":854389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":643072,"data_stored":263861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":264513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":265473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":263917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":647168,"data_stored":263951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1253376,"data_stored":856423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:22:35.641 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:22:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac856f2640 1 Processor -- start 2024-09-12T12:22:35.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac856f2640 1 -- start start 2024-09-12T12:22:35.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac856f2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015b2d0 0x7fac800b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac856f2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fac8006f6e0 0x7fac80072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac856f2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8015f150 con 0x7fac800cc300 2024-09-12T12:22:35.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac856f2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fac800ae920 con 0x7fac8006f6e0 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac856f2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fac8015b6b0 con 0x7fac8015b2d0 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac7f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fac800cc300 0x7fac800aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49004/0 (socket says 172.21.15.3:49004) 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac7f7fe640 1 -- 172.21.15.3:0/638084417 learned_addr learned my addr 172.21.15.3:0/638084417 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac7effd640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015b2d0 0x7fac800b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac7e7fc640 1 --2- 172.21.15.3:0/638084417 >> v2:172.21.15.142:3300/0 conn(0x7fac8006f6e0 0x7fac80072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1501874816 0 0) 0x7fac8015f150 con 0x7fac800cc300 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.800+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac6c003530 con 0x7fac800cc300 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2525849678 0 0) 0x7fac6c003530 con 0x7fac800cc300 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015b2d0 msgr2=0x7fac800b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015b2d0 0x7fac800b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 >> v2:172.21.15.142:3300/0 conn(0x7fac8006f6e0 msgr2=0x7fac80072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 --2- 172.21.15.3:0/638084417 >> v2:172.21.15.142:3300/0 conn(0x7fac8006f6e0 0x7fac80072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8015ee70 con 0x7fac800cc300 2024-09-12T12:22:35.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fac70001680 con 0x7fac800cc300 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 >> v1:172.21.15.3:6789/0 conn(0x7fac800cc300 legacy=0x7fac800aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c0039c0 0x7fac6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac6c004810 0x7fac6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015b2d0 0x7fac6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac800ae920 con 0x7fac6c004810 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fac8015b6b0 con 0x7fac6c0039c0 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fac6c00b9d0 con 0x7fac8015b2d0 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7e7fc640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac6c004810 0x7fac6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7effd640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c0039c0 0x7fac6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7e7fc640 1 -- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015b2d0 msgr2=0x7fac6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7e7fc640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015b2d0 0x7fac6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7e7fc640 1 -- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c0039c0 msgr2=0x7fac6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:35.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7e7fc640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c0039c0 0x7fac6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7e7fc640 1 -- 172.21.15.3:0/638084417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c003530 con 0x7fac6c004810 2024-09-12T12:22:35.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7e7fc640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac6c004810 0x7fac6c004c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fac80071660 tx=0x7fac6801ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec74053cb83b3daa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac680b4070 con 0x7fac6c004810 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac680a7780 con 0x7fac6c004810 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7dffb640 1 -- 172.21.15.3:0/638084417 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac680096b0 con 0x7fac6c004810 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac856f2640 1 -- 172.21.15.3:0/638084417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac6c004810 msgr2=0x7fac6c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac856f2640 1 --2- 172.21.15.3:0/638084417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac6c004810 0x7fac6c004c90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fac80071660 tx=0x7fac6801ec60 comp rx=0 tx=0).stop 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac7f7fe640 1 -- 172.21.15.3:0/638084417 reap_dead start 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac856f2640 1 -- 172.21.15.3:0/638084417 shutdown_connections 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac856f2640 1 -- 172.21.15.3:0/638084417 >> 172.21.15.3:0/638084417 conn(0x7fac800b1d80 msgr2=0x7fac800b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac856f2640 1 -- 172.21.15.3:0/638084417 shutdown_connections 2024-09-12T12:22:35.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.804+0000 7fac856f2640 1 -- 172.21.15.3:0/638084417 wait complete. 2024-09-12T12:22:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 Processor -- start 2024-09-12T12:22:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 -- start start 2024-09-12T12:22:35.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c004810 0x7fac8014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015dae0 0x7fac8014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac800b3620 0x7fac800b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac800ae5b0 con 0x7fac800b3620 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fac800ae2d0 con 0x7fac6c004810 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fac8006d8d0 con 0x7fac8015dae0 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac800b3620 0x7fac800b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac800b3620 0x7fac800b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37692/0 (socket says 172.21.15.3:37692) 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 -- 172.21.15.3:0/2064339541 learned_addr learned my addr 172.21.15.3:0/2064339541 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7e7fc640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015dae0 0x7fac8014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7effd640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c004810 0x7fac8014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 -- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015dae0 msgr2=0x7fac8014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015dae0 0x7fac8014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 -- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c004810 msgr2=0x7fac8014f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c004810 0x7fac8014f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 -- 172.21.15.3:0/2064339541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac80151be0 con 0x7fac800b3620 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7e7fc640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015dae0 0x7fac8014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7effd640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c004810 0x7fac8014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac7f7fe640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac800b3620 0x7fac800b3aa0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fac70000fc0 tx=0x7fac7000f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac777fe640 1 -- 172.21.15.3:0/2064339541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7000fea0 con 0x7fac800b3620 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac777fe640 1 -- 172.21.15.3:0/2064339541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac70007d40 con 0x7fac800b3620 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8014f6a0 con 0x7fac800b3620 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac777fe640 1 -- 172.21.15.3:0/2064339541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7000a2b0 con 0x7fac800b3620 2024-09-12T12:22:35.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8014fc20 con 0x7fac800b3620 2024-09-12T12:22:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.808+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8006db30 con 0x7fac800b3620 2024-09-12T12:22:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.812+0000 7fac777fe640 1 -- 172.21.15.3:0/2064339541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac7000a450 con 0x7fac800b3620 2024-09-12T12:22:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.812+0000 7fac777fe640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac600678c0 0x7fac60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.812+0000 7fac777fe640 1 -- 172.21.15.3:0/2064339541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fac700882e0 con 0x7fac800b3620 2024-09-12T12:22:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.812+0000 7fac7effd640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac600678c0 0x7fac60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:35.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.812+0000 7fac7effd640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac600678c0 0x7fac60069d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fac78012430 tx=0x7fac7800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:35.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.816+0000 7fac777fe640 1 -- 172.21.15.3:0/2064339541 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fac70058e20 con 0x7fac800b3620 2024-09-12T12:22:35.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.972+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac800b16e0 con 0x7fac600678c0 2024-09-12T12:22:35.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.992+0000 7fac777fe640 1 -- 172.21.15.3:0/2064339541 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251911 (secure 0 0 0) 0x7fac800b16e0 con 0x7fac600678c0 2024-09-12T12:22:35.996 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:35.998 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac600678c0 msgr2=0x7fac60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac600678c0 0x7fac60069d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fac78012430 tx=0x7fac7800b040 comp rx=0 tx=0).stop 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac800b3620 msgr2=0x7fac800b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac800b3620 0x7fac800b3aa0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fac70000fc0 tx=0x7fac7000f820 comp rx=0 tx=0).stop 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 shutdown_connections 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac600678c0 0x7fac60069d80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac800b3620 0x7fac800b3aa0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fac8015dae0 0x7fac8014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 --2- 172.21.15.3:0/2064339541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac6c004810 0x7fac8014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 >> 172.21.15.3:0/2064339541 conn(0x7fac800b1d80 msgr2=0x7fac800750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 shutdown_connections 2024-09-12T12:22:35.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:35.996+0000 7fac856f2640 1 -- 172.21.15.3:0/2064339541 wait complete. 2024-09-12T12:22:36.014 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-12T12:22:35.713247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230836,"kb_used_data":6260,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743884,"statfs":{"total":773094113280,"available":772857737216,"internally_reserved":0,"allocated":6410240,"data_stored":3302744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001927"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605735+0000","last_change":"2024-09-12T12:22:31.598465+0000","last_active":"2024-09-12T12:22:31.605735+0000","last_peered":"2024-09-12T12:22:31.605735+0000","last_clean":"2024-09-12T12:22:31.605735+0000","last_became_active":"2024-09-12T12:22:31.595054+0000","last_became_peered":"2024-09-12T12:22:31.595054+0000","last_unstale":"2024-09-12T12:22:31.605735+0000","last_undegraded":"2024-09-12T12:22:31.605735+0000","last_fullsized":"2024-09-12T12:22:31.605735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:36:23.938617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.566537+0000","last_change":"2024-09-12T12:22:31.600424+0000","last_active":"2024-09-12T12:22:32.566537+0000","last_peered":"2024-09-12T12:22:32.566537+0000","last_clean":"2024-09-12T12:22:32.566537+0000","last_became_active":"2024-09-12T12:22:31.600112+0000","last_became_peered":"2024-09-12T12:22:31.600112+0000","last_unstale":"2024-09-12T12:22:32.566537+0000","last_undegraded":"2024-09-12T12:22:32.566537+0000","last_fullsized":"2024-09-12T12:22:32.566537+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:28:27.624443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603910+0000","last_change":"2024-09-12T12:22:31.597654+0000","last_active":"2024-09-12T12:22:31.603910+0000","last_peered":"2024-09-12T12:22:31.603910+0000","last_clean":"2024-09-12T12:22:31.603910+0000","last_became_active":"2024-09-12T12:22:31.597181+0000","last_became_peered":"2024-09-12T12:22:31.597181+0000","last_unstale":"2024-09-12T12:22:31.603910+0000","last_undegraded":"2024-09-12T12:22:31.603910+0000","last_fullsized":"2024-09-12T12:22:31.603910+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:58.373027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.604562+0000","last_change":"2024-09-12T12:22:31.594786+0000","last_active":"2024-09-12T12:22:31.604562+0000","last_peered":"2024-09-12T12:22:31.604562+0000","last_clean":"2024-09-12T12:22:31.604562+0000","last_became_active":"2024-09-12T12:22:31.593983+0000","last_became_peered":"2024-09-12T12:22:31.593983+0000","last_unstale":"2024-09-12T12:22:31.604562+0000","last_undegraded":"2024-09-12T12:22:31.604562+0000","last_fullsized":"2024-09-12T12:22:31.604562+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:58.373027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594040+0000","last_change":"2024-09-12T12:22:31.592664+0000","last_active":"2024-09-12T12:22:31.594040+0000","last_peered":"2024-09-12T12:22:31.594040+0000","last_clean":"2024-09-12T12:22:31.594040+0000","last_became_active":"2024-09-12T12:22:31.592291+0000","last_became_peered":"2024-09-12T12:22:31.592291+0000","last_unstale":"2024-09-12T12:22:31.594040+0000","last_undegraded":"2024-09-12T12:22:31.594040+0000","last_fullsized":"2024-09-12T12:22:31.594040+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:01:23.913205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597767+0000","last_change":"2024-09-12T12:22:31.599553+0000","last_active":"2024-09-12T12:22:32.597767+0000","last_peered":"2024-09-12T12:22:32.597767+0000","last_clean":"2024-09-12T12:22:32.597767+0000","last_became_active":"2024-09-12T12:22:31.599276+0000","last_became_peered":"2024-09-12T12:22:31.599276+0000","last_unstale":"2024-09-12T12:22:32.597767+0000","last_undegraded":"2024-09-12T12:22:32.597767+0000","last_fullsized":"2024-09-12T12:22:32.597767+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:52:18.582029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609935+0000","last_change":"2024-09-12T12:22:31.597093+0000","last_active":"2024-09-12T12:22:31.609935+0000","last_peered":"2024-09-12T12:22:31.609935+0000","last_clean":"2024-09-12T12:22:31.609935+0000","last_became_active":"2024-09-12T12:22:31.596331+0000","last_became_peered":"2024-09-12T12:22:31.596331+0000","last_unstale":"2024-09-12T12:22:31.609935+0000","last_undegraded":"2024-09-12T12:22:31.609935+0000","last_fullsized":"2024-09-12T12:22:31.609935+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.466723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.564903+0000","last_change":"2024-09-12T12:22:31.596667+0000","last_active":"2024-09-12T12:22:32.564903+0000","last_peered":"2024-09-12T12:22:32.564903+0000","last_clean":"2024-09-12T12:22:32.564903+0000","last_became_active":"2024-09-12T12:22:31.596274+0000","last_became_peered":"2024-09-12T12:22:31.596274+0000","last_unstale":"2024-09-12T12:22:32.564903+0000","last_undegraded":"2024-09-12T12:22:32.564903+0000","last_fullsized":"2024-09-12T12:22:32.564903+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:45.649666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609977+0000","last_change":"2024-09-12T12:22:31.598663+0000","last_active":"2024-09-12T12:22:31.609977+0000","last_peered":"2024-09-12T12:22:31.609977+0000","last_clean":"2024-09-12T12:22:31.609977+0000","last_became_active":"2024-09-12T12:22:31.598229+0000","last_became_peered":"2024-09-12T12:22:31.598229+0000","last_unstale":"2024-09-12T12:22:31.609977+0000","last_undegraded":"2024-09-12T12:22:31.609977+0000","last_fullsized":"2024-09-12T12:22:31.609977+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:37:37.333024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603483+0000","last_change":"2024-09-12T12:22:31.597402+0000","last_active":"2024-09-12T12:22:31.603483+0000","last_peered":"2024-09-12T12:22:31.603483+0000","last_clean":"2024-09-12T12:22:31.603483+0000","last_became_active":"2024-09-12T12:22:31.597006+0000","last_became_peered":"2024-09-12T12:22:31.597006+0000","last_unstale":"2024-09-12T12:22:31.603483+0000","last_undegraded":"2024-09-12T12:22:31.603483+0000","last_fullsized":"2024-09-12T12:22:31.603483+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:21.833484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603856+0000","last_change":"2024-09-12T12:22:31.601934+0000","last_active":"2024-09-12T12:22:31.603856+0000","last_peered":"2024-09-12T12:22:31.603856+0000","last_clean":"2024-09-12T12:22:31.603856+0000","last_became_active":"2024-09-12T12:22:31.601580+0000","last_became_peered":"2024-09-12T12:22:31.601580+0000","last_unstale":"2024-09-12T12:22:31.603856+0000","last_undegraded":"2024-09-12T12:22:31.603856+0000","last_fullsized":"2024-09-12T12:22:31.603856+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:28.335911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594489+0000","last_change":"2024-09-12T12:22:31.585564+0000","last_active":"2024-09-12T12:22:31.594489+0000","last_peered":"2024-09-12T12:22:31.594489+0000","last_clean":"2024-09-12T12:22:31.594489+0000","last_became_active":"2024-09-12T12:22:31.584689+0000","last_became_peered":"2024-09-12T12:22:31.584689+0000","last_unstale":"2024-09-12T12:22:31.594489+0000","last_undegraded":"2024-09-12T12:22:31.594489+0000","last_fullsized":"2024-09-12T12:22:31.594489+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:04.950897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.568034+0000","last_change":"2024-09-12T12:22:31.599725+0000","last_active":"2024-09-12T12:22:32.568034+0000","last_peered":"2024-09-12T12:22:32.568034+0000","last_clean":"2024-09-12T12:22:32.568034+0000","last_became_active":"2024-09-12T12:22:31.596519+0000","last_became_peered":"2024-09-12T12:22:31.596519+0000","last_unstale":"2024-09-12T12:22:32.568034+0000","last_undegraded":"2024-09-12T12:22:32.568034+0000","last_fullsized":"2024-09-12T12:22:32.568034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:21.833484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.607938+0000","last_change":"2024-09-12T12:22:31.595390+0000","last_active":"2024-09-12T12:22:31.607938+0000","last_peered":"2024-09-12T12:22:31.607938+0000","last_clean":"2024-09-12T12:22:31.607938+0000","last_became_active":"2024-09-12T12:22:31.594931+0000","last_became_peered":"2024-09-12T12:22:31.594931+0000","last_unstale":"2024-09-12T12:22:31.607938+0000","last_undegraded":"2024-09-12T12:22:31.607938+0000","last_fullsized":"2024-09-12T12:22:31.607938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:17:24.315256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.610210+0000","last_change":"2024-09-12T12:22:31.602546+0000","last_active":"2024-09-12T12:22:31.610210+0000","last_peered":"2024-09-12T12:22:31.610210+0000","last_clean":"2024-09-12T12:22:31.610210+0000","last_became_active":"2024-09-12T12:22:31.601858+0000","last_became_peered":"2024-09-12T12:22:31.601858+0000","last_unstale":"2024-09-12T12:22:31.610210+0000","last_undegraded":"2024-09-12T12:22:31.610210+0000","last_fullsized":"2024-09-12T12:22:31.610210+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:17.026910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.596856+0000","last_change":"2024-09-12T12:22:31.596705+0000","last_active":"2024-09-12T12:22:32.596856+0000","last_peered":"2024-09-12T12:22:32.596856+0000","last_clean":"2024-09-12T12:22:32.596856+0000","last_became_active":"2024-09-12T12:22:31.596379+0000","last_became_peered":"2024-09-12T12:22:31.596379+0000","last_unstale":"2024-09-12T12:22:32.596856+0000","last_undegraded":"2024-09-12T12:22:32.596856+0000","last_fullsized":"2024-09-12T12:22:32.596856+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:40:43.464580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.566982+0000","last_change":"2024-09-12T12:22:31.569918+0000","last_active":"2024-09-12T12:22:32.566982+0000","last_peered":"2024-09-12T12:22:32.566982+0000","last_clean":"2024-09-12T12:22:32.566982+0000","last_became_active":"2024-09-12T12:22:31.569104+0000","last_became_peered":"2024-09-12T12:22:31.569104+0000","last_unstale":"2024-09-12T12:22:32.566982+0000","last_undegraded":"2024-09-12T12:22:32.566982+0000","last_fullsized":"2024-09-12T12:22:32.566982+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:46:46.242562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605851+0000","last_change":"2024-09-12T12:22:31.599906+0000","last_active":"2024-09-12T12:22:31.605851+0000","last_peered":"2024-09-12T12:22:31.605851+0000","last_clean":"2024-09-12T12:22:31.605851+0000","last_became_active":"2024-09-12T12:22:31.598008+0000","last_became_peered":"2024-09-12T12:22:31.598008+0000","last_unstale":"2024-09-12T12:22:31.605851+0000","last_undegraded":"2024-09-12T12:22:31.605851+0000","last_fullsized":"2024-09-12T12:22:31.605851+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:18.099595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605985+0000","last_change":"2024-09-12T12:22:31.601928+0000","last_active":"2024-09-12T12:22:31.605985+0000","last_peered":"2024-09-12T12:22:31.605985+0000","last_clean":"2024-09-12T12:22:31.605985+0000","last_became_active":"2024-09-12T12:22:31.601577+0000","last_became_peered":"2024-09-12T12:22:31.601577+0000","last_unstale":"2024-09-12T12:22:31.605985+0000","last_undegraded":"2024-09-12T12:22:31.605985+0000","last_fullsized":"2024-09-12T12:22:31.605985+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:04:32.276759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565795+0000","last_change":"2024-09-12T12:22:31.601499+0000","last_active":"2024-09-12T12:22:32.565795+0000","last_peered":"2024-09-12T12:22:32.565795+0000","last_clean":"2024-09-12T12:22:32.565795+0000","last_became_active":"2024-09-12T12:22:31.601193+0000","last_became_peered":"2024-09-12T12:22:31.601193+0000","last_unstale":"2024-09-12T12:22:32.565795+0000","last_undegraded":"2024-09-12T12:22:32.565795+0000","last_fullsized":"2024-09-12T12:22:32.565795+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:50.677196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.592314+0000","last_change":"2024-09-12T12:22:31.586849+0000","last_active":"2024-09-12T12:22:31.592314+0000","last_peered":"2024-09-12T12:22:31.592314+0000","last_clean":"2024-09-12T12:22:31.592314+0000","last_became_active":"2024-09-12T12:22:31.586390+0000","last_became_peered":"2024-09-12T12:22:31.586390+0000","last_unstale":"2024-09-12T12:22:31.592314+0000","last_undegraded":"2024-09-12T12:22:31.592314+0000","last_fullsized":"2024-09-12T12:22:31.592314+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:48:26.666389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565399+0000","last_change":"2024-09-12T12:22:31.595290+0000","last_active":"2024-09-12T12:22:32.565399+0000","last_peered":"2024-09-12T12:22:32.565399+0000","last_clean":"2024-09-12T12:22:32.565399+0000","last_became_active":"2024-09-12T12:22:31.594904+0000","last_became_peered":"2024-09-12T12:22:31.594904+0000","last_unstale":"2024-09-12T12:22:32.565399+0000","last_undegraded":"2024-09-12T12:22:32.565399+0000","last_fullsized":"2024-09-12T12:22:32.565399+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:03:32.344872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.599416+0000","last_change":"2024-09-12T12:22:31.590465+0000","last_active":"2024-09-12T12:22:31.599416+0000","last_peered":"2024-09-12T12:22:31.599416+0000","last_clean":"2024-09-12T12:22:31.599416+0000","last_became_active":"2024-09-12T12:22:31.588823+0000","last_became_peered":"2024-09-12T12:22:31.588823+0000","last_unstale":"2024-09-12T12:22:31.599416+0000","last_undegraded":"2024-09-12T12:22:31.599416+0000","last_fullsized":"2024-09-12T12:22:31.599416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:04.950897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567692+0000","last_change":"2024-09-12T12:22:31.605595+0000","last_active":"2024-09-12T12:22:32.567692+0000","last_peered":"2024-09-12T12:22:32.567692+0000","last_clean":"2024-09-12T12:22:32.567692+0000","last_became_active":"2024-09-12T12:22:31.604813+0000","last_became_peered":"2024-09-12T12:22:31.604813+0000","last_unstale":"2024-09-12T12:22:32.567692+0000","last_undegraded":"2024-09-12T12:22:32.567692+0000","last_fullsized":"2024-09-12T12:22:32.567692+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:10:28.631953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605761+0000","last_change":"2024-09-12T12:22:31.601028+0000","last_active":"2024-09-12T12:22:31.605761+0000","last_peered":"2024-09-12T12:22:31.605761+0000","last_clean":"2024-09-12T12:22:31.605761+0000","last_became_active":"2024-09-12T12:22:31.600689+0000","last_became_peered":"2024-09-12T12:22:31.600689+0000","last_unstale":"2024-09-12T12:22:31.605761+0000","last_undegraded":"2024-09-12T12:22:31.605761+0000","last_fullsized":"2024-09-12T12:22:31.605761+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:26:10.132384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594528+0000","last_change":"2024-09-12T12:22:31.591387+0000","last_active":"2024-09-12T12:22:31.594528+0000","last_peered":"2024-09-12T12:22:31.594528+0000","last_clean":"2024-09-12T12:22:31.594528+0000","last_became_active":"2024-09-12T12:22:31.590888+0000","last_became_peered":"2024-09-12T12:22:31.590888+0000","last_unstale":"2024-09-12T12:22:31.594528+0000","last_undegraded":"2024-09-12T12:22:31.594528+0000","last_fullsized":"2024-09-12T12:22:31.594528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:09.362130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597540+0000","last_change":"2024-09-12T12:22:31.595631+0000","last_active":"2024-09-12T12:22:32.597540+0000","last_peered":"2024-09-12T12:22:32.597540+0000","last_clean":"2024-09-12T12:22:32.597540+0000","last_became_active":"2024-09-12T12:22:31.593768+0000","last_became_peered":"2024-09-12T12:22:31.593768+0000","last_unstale":"2024-09-12T12:22:32.597540+0000","last_undegraded":"2024-09-12T12:22:32.597540+0000","last_fullsized":"2024-09-12T12:22:32.597540+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:41:02.935897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.598741+0000","last_change":"2024-09-12T12:22:31.593925+0000","last_active":"2024-09-12T12:22:31.598741+0000","last_peered":"2024-09-12T12:22:31.598741+0000","last_clean":"2024-09-12T12:22:31.598741+0000","last_became_active":"2024-09-12T12:22:31.593233+0000","last_became_peered":"2024-09-12T12:22:31.593233+0000","last_unstale":"2024-09-12T12:22:31.598741+0000","last_undegraded":"2024-09-12T12:22:31.598741+0000","last_fullsized":"2024-09-12T12:22:31.598741+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:11:53.433972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565630+0000","last_change":"2024-09-12T12:22:31.585031+0000","last_active":"2024-09-12T12:22:32.565630+0000","last_peered":"2024-09-12T12:22:32.565630+0000","last_clean":"2024-09-12T12:22:32.565630+0000","last_became_active":"2024-09-12T12:22:31.584272+0000","last_became_peered":"2024-09-12T12:22:31.584272+0000","last_unstale":"2024-09-12T12:22:32.565630+0000","last_undegraded":"2024-09-12T12:22:32.565630+0000","last_fullsized":"2024-09-12T12:22:32.565630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:12.014971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.592173+0000","last_change":"2024-09-12T12:22:31.587407+0000","last_active":"2024-09-12T12:22:31.592173+0000","last_peered":"2024-09-12T12:22:31.592173+0000","last_clean":"2024-09-12T12:22:31.592173+0000","last_became_active":"2024-09-12T12:22:31.586683+0000","last_became_peered":"2024-09-12T12:22:31.586683+0000","last_unstale":"2024-09-12T12:22:31.592173+0000","last_undegraded":"2024-09-12T12:22:31.592173+0000","last_fullsized":"2024-09-12T12:22:31.592173+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:21:05.594367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605781+0000","last_change":"2024-09-12T12:22:31.600149+0000","last_active":"2024-09-12T12:22:31.605781+0000","last_peered":"2024-09-12T12:22:31.605781+0000","last_clean":"2024-09-12T12:22:31.605781+0000","last_became_active":"2024-09-12T12:22:31.599908+0000","last_became_peered":"2024-09-12T12:22:31.599908+0000","last_unstale":"2024-09-12T12:22:31.605781+0000","last_undegraded":"2024-09-12T12:22:31.605781+0000","last_fullsized":"2024-09-12T12:22:31.605781+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:52.105465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597109+0000","last_change":"2024-09-12T12:22:31.597630+0000","last_active":"2024-09-12T12:22:32.597109+0000","last_peered":"2024-09-12T12:22:32.597109+0000","last_clean":"2024-09-12T12:22:32.597109+0000","last_became_active":"2024-09-12T12:22:31.596358+0000","last_became_peered":"2024-09-12T12:22:31.596358+0000","last_unstale":"2024-09-12T12:22:32.597109+0000","last_undegraded":"2024-09-12T12:22:32.597109+0000","last_fullsized":"2024-09-12T12:22:32.597109+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:07.489528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597772+0000","last_change":"2024-09-12T12:22:31.595004+0000","last_active":"2024-09-12T12:22:32.597772+0000","last_peered":"2024-09-12T12:22:32.597772+0000","last_clean":"2024-09-12T12:22:32.597772+0000","last_became_active":"2024-09-12T12:22:31.594630+0000","last_became_peered":"2024-09-12T12:22:31.594630+0000","last_unstale":"2024-09-12T12:22:32.597772+0000","last_undegraded":"2024-09-12T12:22:32.597772+0000","last_fullsized":"2024-09-12T12:22:32.597772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:22.353401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597155+0000","last_change":"2024-09-12T12:22:30.596518+0000","last_active":"2024-09-12T12:22:32.597155+0000","last_peered":"2024-09-12T12:22:32.597155+0000","last_clean":"2024-09-12T12:22:32.597155+0000","last_became_active":"2024-09-12T12:22:30.596074+0000","last_became_peered":"2024-09-12T12:22:30.596074+0000","last_unstale":"2024-09-12T12:22:32.597155+0000","last_undegraded":"2024-09-12T12:22:32.597155+0000","last_fullsized":"2024-09-12T12:22:32.597155+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:56:19.182389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.599848+0000","last_change":"2024-09-12T12:22:31.597405+0000","last_active":"2024-09-12T12:22:31.599848+0000","last_peered":"2024-09-12T12:22:31.599848+0000","last_clean":"2024-09-12T12:22:31.599848+0000","last_became_active":"2024-09-12T12:22:31.597088+0000","last_became_peered":"2024-09-12T12:22:31.597088+0000","last_unstale":"2024-09-12T12:22:31.599848+0000","last_undegraded":"2024-09-12T12:22:31.599848+0000","last_fullsized":"2024-09-12T12:22:31.599848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:31.693955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.566709+0000","last_change":"2024-09-12T12:22:30.573869+0000","last_active":"2024-09-12T12:22:32.566709+0000","last_peered":"2024-09-12T12:22:32.566709+0000","last_clean":"2024-09-12T12:22:32.566709+0000","last_became_active":"2024-09-12T12:22:30.573057+0000","last_became_peered":"2024-09-12T12:22:30.573057+0000","last_unstale":"2024-09-12T12:22:32.566709+0000","last_undegraded":"2024-09-12T12:22:32.566709+0000","last_fullsized":"2024-09-12T12:22:32.566709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:05.993026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597127+0000","last_change":"2024-09-12T12:22:31.602435+0000","last_active":"2024-09-12T12:22:32.597127+0000","last_peered":"2024-09-12T12:22:32.597127+0000","last_clean":"2024-09-12T12:22:32.597127+0000","last_became_active":"2024-09-12T12:22:31.602135+0000","last_became_peered":"2024-09-12T12:22:31.602135+0000","last_unstale":"2024-09-12T12:22:32.597127+0000","last_undegraded":"2024-09-12T12:22:32.597127+0000","last_fullsized":"2024-09-12T12:22:32.597127+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:39:06.996973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.598141+0000","last_change":"2024-09-12T12:22:30.594561+0000","last_active":"2024-09-12T12:22:32.598141+0000","last_peered":"2024-09-12T12:22:32.598141+0000","last_clean":"2024-09-12T12:22:32.598141+0000","last_became_active":"2024-09-12T12:22:30.594226+0000","last_became_peered":"2024-09-12T12:22:30.594226+0000","last_unstale":"2024-09-12T12:22:32.598141+0000","last_undegraded":"2024-09-12T12:22:32.598141+0000","last_fullsized":"2024-09-12T12:22:32.598141+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:22:36.420998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597881+0000","last_change":"2024-09-12T12:22:31.595458+0000","last_active":"2024-09-12T12:22:32.597881+0000","last_peered":"2024-09-12T12:22:32.597881+0000","last_clean":"2024-09-12T12:22:32.597881+0000","last_became_active":"2024-09-12T12:22:31.593461+0000","last_became_peered":"2024-09-12T12:22:31.593461+0000","last_unstale":"2024-09-12T12:22:32.597881+0000","last_undegraded":"2024-09-12T12:22:32.597881+0000","last_fullsized":"2024-09-12T12:22:32.597881+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:38:31.747308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.619684+0000","last_change":"2024-09-12T12:22:30.600283+0000","last_active":"2024-09-12T12:22:31.619684+0000","last_peered":"2024-09-12T12:22:31.619684+0000","last_clean":"2024-09-12T12:22:31.619684+0000","last_became_active":"2024-09-12T12:22:30.599503+0000","last_became_peered":"2024-09-12T12:22:30.599503+0000","last_unstale":"2024-09-12T12:22:31.619684+0000","last_undegraded":"2024-09-12T12:22:31.619684+0000","last_fullsized":"2024-09-12T12:22:31.619684+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:49:53.700003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565364+0000","last_change":"2024-09-12T12:22:31.590291+0000","last_active":"2024-09-12T12:22:32.565364+0000","last_peered":"2024-09-12T12:22:32.565364+0000","last_clean":"2024-09-12T12:22:32.565364+0000","last_became_active":"2024-09-12T12:22:31.589945+0000","last_became_peered":"2024-09-12T12:22:31.589945+0000","last_unstale":"2024-09-12T12:22:32.565364+0000","last_undegraded":"2024-09-12T12:22:32.565364+0000","last_fullsized":"2024-09-12T12:22:32.565364+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:36.936983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.613406+0000","last_change":"2024-09-12T12:22:30.585642+0000","last_active":"2024-09-12T12:22:31.613406+0000","last_peered":"2024-09-12T12:22:31.613406+0000","last_clean":"2024-09-12T12:22:31.613406+0000","last_became_active":"2024-09-12T12:22:30.584940+0000","last_became_peered":"2024-09-12T12:22:30.584940+0000","last_unstale":"2024-09-12T12:22:31.613406+0000","last_undegraded":"2024-09-12T12:22:31.613406+0000","last_fullsized":"2024-09-12T12:22:31.613406+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:13:44.353244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565128+0000","last_change":"2024-09-12T12:22:31.602513+0000","last_active":"2024-09-12T12:22:32.565128+0000","last_peered":"2024-09-12T12:22:32.565128+0000","last_clean":"2024-09-12T12:22:32.565128+0000","last_became_active":"2024-09-12T12:22:31.602095+0000","last_became_peered":"2024-09-12T12:22:31.602095+0000","last_unstale":"2024-09-12T12:22:32.565128+0000","last_undegraded":"2024-09-12T12:22:32.565128+0000","last_fullsized":"2024-09-12T12:22:32.565128+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:35:37.411177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597822+0000","last_change":"2024-09-12T12:22:30.586415+0000","last_active":"2024-09-12T12:22:32.597822+0000","last_peered":"2024-09-12T12:22:32.597822+0000","last_clean":"2024-09-12T12:22:32.597822+0000","last_became_active":"2024-09-12T12:22:30.585980+0000","last_became_peered":"2024-09-12T12:22:30.585980+0000","last_unstale":"2024-09-12T12:22:32.597822+0000","last_undegraded":"2024-09-12T12:22:32.597822+0000","last_fullsized":"2024-09-12T12:22:32.597822+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:17:41.342306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567762+0000","last_change":"2024-09-12T12:22:31.580457+0000","last_active":"2024-09-12T12:22:32.567762+0000","last_peered":"2024-09-12T12:22:32.567762+0000","last_clean":"2024-09-12T12:22:32.567762+0000","last_became_active":"2024-09-12T12:22:31.579673+0000","last_became_peered":"2024-09-12T12:22:31.579673+0000","last_unstale":"2024-09-12T12:22:32.567762+0000","last_undegraded":"2024-09-12T12:22:32.567762+0000","last_fullsized":"2024-09-12T12:22:32.567762+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.986247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572190+0000","last_change":"2024-09-12T12:22:30.598094+0000","last_active":"2024-09-12T12:22:31.572190+0000","last_peered":"2024-09-12T12:22:31.572190+0000","last_clean":"2024-09-12T12:22:31.572190+0000","last_became_active":"2024-09-12T12:22:30.594331+0000","last_became_peered":"2024-09-12T12:22:30.594331+0000","last_unstale":"2024-09-12T12:22:31.572190+0000","last_undegraded":"2024-09-12T12:22:31.572190+0000","last_fullsized":"2024-09-12T12:22:31.572190+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:48:25.662443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.564902+0000","last_change":"2024-09-12T12:22:31.571232+0000","last_active":"2024-09-12T12:22:32.564902+0000","last_peered":"2024-09-12T12:22:32.564902+0000","last_clean":"2024-09-12T12:22:32.564902+0000","last_became_active":"2024-09-12T12:22:31.569217+0000","last_became_peered":"2024-09-12T12:22:31.569217+0000","last_unstale":"2024-09-12T12:22:32.564902+0000","last_undegraded":"2024-09-12T12:22:32.564902+0000","last_fullsized":"2024-09-12T12:22:32.564902+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:37:47.974909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.584879+0000","last_change":"2024-09-12T12:22:30.591999+0000","last_active":"2024-09-12T12:22:31.584879+0000","last_peered":"2024-09-12T12:22:31.584879+0000","last_clean":"2024-09-12T12:22:31.584879+0000","last_became_active":"2024-09-12T12:22:30.591371+0000","last_became_peered":"2024-09-12T12:22:30.591371+0000","last_unstale":"2024-09-12T12:22:31.584879+0000","last_undegraded":"2024-09-12T12:22:31.584879+0000","last_fullsized":"2024-09-12T12:22:31.584879+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:33:05.916427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.610279+0000","last_change":"2024-09-12T12:22:31.604147+0000","last_active":"2024-09-12T12:22:31.610279+0000","last_peered":"2024-09-12T12:22:31.610279+0000","last_clean":"2024-09-12T12:22:31.610279+0000","last_became_active":"2024-09-12T12:22:31.602898+0000","last_became_peered":"2024-09-12T12:22:31.602898+0000","last_unstale":"2024-09-12T12:22:31.610279+0000","last_undegraded":"2024-09-12T12:22:31.610279+0000","last_fullsized":"2024-09-12T12:22:31.610279+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:46.326145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.589560+0000","last_change":"2024-09-12T12:22:30.588745+0000","last_active":"2024-09-12T12:22:31.589560+0000","last_peered":"2024-09-12T12:22:31.589560+0000","last_clean":"2024-09-12T12:22:31.589560+0000","last_became_active":"2024-09-12T12:22:30.588031+0000","last_became_peered":"2024-09-12T12:22:30.588031+0000","last_unstale":"2024-09-12T12:22:31.589560+0000","last_undegraded":"2024-09-12T12:22:31.589560+0000","last_fullsized":"2024-09-12T12:22:31.589560+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:09.145112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.611310+0000","last_change":"2024-09-12T12:22:31.604015+0000","last_active":"2024-09-12T12:22:31.611310+0000","last_peered":"2024-09-12T12:22:31.611310+0000","last_clean":"2024-09-12T12:22:31.611310+0000","last_became_active":"2024-09-12T12:22:31.603530+0000","last_became_peered":"2024-09-12T12:22:31.603530+0000","last_unstale":"2024-09-12T12:22:31.611310+0000","last_undegraded":"2024-09-12T12:22:31.611310+0000","last_fullsized":"2024-09-12T12:22:31.611310+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:00.325171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.566860+0000","last_change":"2024-09-12T12:22:30.589525+0000","last_active":"2024-09-12T12:22:32.566860+0000","last_peered":"2024-09-12T12:22:32.566860+0000","last_clean":"2024-09-12T12:22:32.566860+0000","last_became_active":"2024-09-12T12:22:30.588737+0000","last_became_peered":"2024-09-12T12:22:30.588737+0000","last_unstale":"2024-09-12T12:22:32.566860+0000","last_undegraded":"2024-09-12T12:22:32.566860+0000","last_fullsized":"2024-09-12T12:22:32.566860+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:41:01.931950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.603509+0000","last_change":"2024-09-12T12:22:31.598907+0000","last_active":"2024-09-12T12:22:31.603509+0000","last_peered":"2024-09-12T12:22:31.603509+0000","last_clean":"2024-09-12T12:22:31.603509+0000","last_became_active":"2024-09-12T12:22:31.598575+0000","last_became_peered":"2024-09-12T12:22:31.598575+0000","last_unstale":"2024-09-12T12:22:31.603509+0000","last_undegraded":"2024-09-12T12:22:31.603509+0000","last_fullsized":"2024-09-12T12:22:31.603509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.986247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.619656+0000","last_change":"2024-09-12T12:22:30.596808+0000","last_active":"2024-09-12T12:22:31.619656+0000","last_peered":"2024-09-12T12:22:31.619656+0000","last_clean":"2024-09-12T12:22:31.619656+0000","last_became_active":"2024-09-12T12:22:30.595968+0000","last_became_peered":"2024-09-12T12:22:30.595968+0000","last_unstale":"2024-09-12T12:22:31.619656+0000","last_undegraded":"2024-09-12T12:22:31.619656+0000","last_fullsized":"2024-09-12T12:22:31.619656+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:49:53.700003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.611377+0000","last_change":"2024-09-12T12:22:31.604094+0000","last_active":"2024-09-12T12:22:31.611377+0000","last_peered":"2024-09-12T12:22:31.611377+0000","last_clean":"2024-09-12T12:22:31.611377+0000","last_became_active":"2024-09-12T12:22:31.603003+0000","last_became_peered":"2024-09-12T12:22:31.603003+0000","last_unstale":"2024-09-12T12:22:31.611377+0000","last_undegraded":"2024-09-12T12:22:31.611377+0000","last_fullsized":"2024-09-12T12:22:31.611377+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:44:57.296236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.620352+0000","last_change":"2024-09-12T12:22:30.583697+0000","last_active":"2024-09-12T12:22:31.620352+0000","last_peered":"2024-09-12T12:22:31.620352+0000","last_clean":"2024-09-12T12:22:31.620352+0000","last_became_active":"2024-09-12T12:22:30.582920+0000","last_became_peered":"2024-09-12T12:22:30.582920+0000","last_unstale":"2024-09-12T12:22:31.620352+0000","last_undegraded":"2024-09-12T12:22:31.620352+0000","last_fullsized":"2024-09-12T12:22:31.620352+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:48:20.728320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609619+0000","last_change":"2024-09-12T12:22:31.606102+0000","last_active":"2024-09-12T12:22:31.609619+0000","last_peered":"2024-09-12T12:22:31.609619+0000","last_clean":"2024-09-12T12:22:31.609619+0000","last_became_active":"2024-09-12T12:22:31.605453+0000","last_became_peered":"2024-09-12T12:22:31.605453+0000","last_unstale":"2024-09-12T12:22:31.609619+0000","last_undegraded":"2024-09-12T12:22:31.609619+0000","last_fullsized":"2024-09-12T12:22:31.609619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:00.325171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565175+0000","last_change":"2024-09-12T12:22:30.593679+0000","last_active":"2024-09-12T12:22:32.565175+0000","last_peered":"2024-09-12T12:22:32.565175+0000","last_clean":"2024-09-12T12:22:32.565175+0000","last_became_active":"2024-09-12T12:22:30.589351+0000","last_became_peered":"2024-09-12T12:22:30.589351+0000","last_unstale":"2024-09-12T12:22:32.565175+0000","last_undegraded":"2024-09-12T12:22:32.565175+0000","last_fullsized":"2024-09-12T12:22:32.565175+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:44.877398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594983+0000","last_change":"2024-09-12T12:22:31.589349+0000","last_active":"2024-09-12T12:22:31.594983+0000","last_peered":"2024-09-12T12:22:31.594983+0000","last_clean":"2024-09-12T12:22:31.594983+0000","last_became_active":"2024-09-12T12:22:31.588607+0000","last_became_peered":"2024-09-12T12:22:31.588607+0000","last_unstale":"2024-09-12T12:22:31.594983+0000","last_undegraded":"2024-09-12T12:22:31.594983+0000","last_fullsized":"2024-09-12T12:22:31.594983+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:11:25.011120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567873+0000","last_change":"2024-09-12T12:22:30.588538+0000","last_active":"2024-09-12T12:22:32.567873+0000","last_peered":"2024-09-12T12:22:32.567873+0000","last_clean":"2024-09-12T12:22:32.567873+0000","last_became_active":"2024-09-12T12:22:30.587825+0000","last_became_peered":"2024-09-12T12:22:30.587825+0000","last_unstale":"2024-09-12T12:22:32.567873+0000","last_undegraded":"2024-09-12T12:22:32.567873+0000","last_fullsized":"2024-09-12T12:22:32.567873+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:23:49.561010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.610118+0000","last_change":"2024-09-12T12:22:31.601216+0000","last_active":"2024-09-12T12:22:31.610118+0000","last_peered":"2024-09-12T12:22:31.610118+0000","last_clean":"2024-09-12T12:22:31.610118+0000","last_became_active":"2024-09-12T12:22:31.595458+0000","last_became_peered":"2024-09-12T12:22:31.595458+0000","last_unstale":"2024-09-12T12:22:31.610118+0000","last_undegraded":"2024-09-12T12:22:31.610118+0000","last_fullsized":"2024-09-12T12:22:31.610118+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:15.576446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572425+0000","last_change":"2024-09-12T12:22:30.596630+0000","last_active":"2024-09-12T12:22:31.572425+0000","last_peered":"2024-09-12T12:22:31.572425+0000","last_clean":"2024-09-12T12:22:31.572425+0000","last_became_active":"2024-09-12T12:22:30.595435+0000","last_became_peered":"2024-09-12T12:22:30.595435+0000","last_unstale":"2024-09-12T12:22:31.572425+0000","last_undegraded":"2024-09-12T12:22:31.572425+0000","last_fullsized":"2024-09-12T12:22:31.572425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:21:04.590421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594566+0000","last_change":"2024-09-12T12:22:31.591794+0000","last_active":"2024-09-12T12:22:31.594566+0000","last_peered":"2024-09-12T12:22:31.594566+0000","last_clean":"2024-09-12T12:22:31.594566+0000","last_became_active":"2024-09-12T12:22:31.589787+0000","last_became_peered":"2024-09-12T12:22:31.589787+0000","last_unstale":"2024-09-12T12:22:31.594566+0000","last_undegraded":"2024-09-12T12:22:31.594566+0000","last_fullsized":"2024-09-12T12:22:31.594566+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:09.362130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.622554+0000","last_change":"2024-09-12T12:22:30.589423+0000","last_active":"2024-09-12T12:22:31.622554+0000","last_peered":"2024-09-12T12:22:31.622554+0000","last_clean":"2024-09-12T12:22:31.622554+0000","last_became_active":"2024-09-12T12:22:30.588637+0000","last_became_peered":"2024-09-12T12:22:30.588637+0000","last_unstale":"2024-09-12T12:22:31.622554+0000","last_undegraded":"2024-09-12T12:22:31.622554+0000","last_fullsized":"2024-09-12T12:22:31.622554+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:03.758234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.605887+0000","last_change":"2024-09-12T12:22:31.600493+0000","last_active":"2024-09-12T12:22:31.605887+0000","last_peered":"2024-09-12T12:22:31.605887+0000","last_clean":"2024-09-12T12:22:31.605887+0000","last_became_active":"2024-09-12T12:22:31.599248+0000","last_became_peered":"2024-09-12T12:22:31.599248+0000","last_unstale":"2024-09-12T12:22:31.605887+0000","last_undegraded":"2024-09-12T12:22:31.605887+0000","last_fullsized":"2024-09-12T12:22:31.605887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:04.049615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567009+0000","last_change":"2024-09-12T12:22:30.590277+0000","last_active":"2024-09-12T12:22:32.567009+0000","last_peered":"2024-09-12T12:22:32.567009+0000","last_clean":"2024-09-12T12:22:32.567009+0000","last_became_active":"2024-09-12T12:22:30.588964+0000","last_became_peered":"2024-09-12T12:22:30.588964+0000","last_unstale":"2024-09-12T12:22:32.567009+0000","last_undegraded":"2024-09-12T12:22:32.567009+0000","last_fullsized":"2024-09-12T12:22:32.567009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:06.485582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609238+0000","last_change":"2024-09-12T12:22:31.604643+0000","last_active":"2024-09-12T12:22:31.609238+0000","last_peered":"2024-09-12T12:22:31.609238+0000","last_clean":"2024-09-12T12:22:31.609238+0000","last_became_active":"2024-09-12T12:22:31.603937+0000","last_became_peered":"2024-09-12T12:22:31.603937+0000","last_unstale":"2024-09-12T12:22:31.609238+0000","last_undegraded":"2024-09-12T12:22:31.609238+0000","last_fullsized":"2024-09-12T12:22:31.609238+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:18.099595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.598058+0000","last_change":"2024-09-12T12:22:30.582323+0000","last_active":"2024-09-12T12:22:32.598058+0000","last_peered":"2024-09-12T12:22:32.598058+0000","last_clean":"2024-09-12T12:22:32.598058+0000","last_became_active":"2024-09-12T12:22:30.582015+0000","last_became_peered":"2024-09-12T12:22:30.582015+0000","last_unstale":"2024-09-12T12:22:32.598058+0000","last_undegraded":"2024-09-12T12:22:32.598058+0000","last_fullsized":"2024-09-12T12:22:32.598058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:13:33.060254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.566367+0000","last_change":"2024-09-12T12:22:31.589048+0000","last_active":"2024-09-12T12:22:32.566367+0000","last_peered":"2024-09-12T12:22:32.566367+0000","last_clean":"2024-09-12T12:22:32.566367+0000","last_became_active":"2024-09-12T12:22:31.588253+0000","last_became_peered":"2024-09-12T12:22:31.588253+0000","last_unstale":"2024-09-12T12:22:32.566367+0000","last_undegraded":"2024-09-12T12:22:32.566367+0000","last_fullsized":"2024-09-12T12:22:32.566367+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:45:24.937289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.618084+0000","last_change":"2024-09-12T12:22:30.590249+0000","last_active":"2024-09-12T12:22:31.618084+0000","last_peered":"2024-09-12T12:22:31.618084+0000","last_clean":"2024-09-12T12:22:31.618084+0000","last_became_active":"2024-09-12T12:22:30.589576+0000","last_became_peered":"2024-09-12T12:22:30.589576+0000","last_unstale":"2024-09-12T12:22:31.618084+0000","last_undegraded":"2024-09-12T12:22:31.618084+0000","last_fullsized":"2024-09-12T12:22:31.618084+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:19:03.758234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.611339+0000","last_change":"2024-09-12T12:22:31.604114+0000","last_active":"2024-09-12T12:22:31.611339+0000","last_peered":"2024-09-12T12:22:31.611339+0000","last_clean":"2024-09-12T12:22:31.611339+0000","last_became_active":"2024-09-12T12:22:31.603548+0000","last_became_peered":"2024-09-12T12:22:31.603548+0000","last_unstale":"2024-09-12T12:22:31.611339+0000","last_undegraded":"2024-09-12T12:22:31.611339+0000","last_fullsized":"2024-09-12T12:22:31.611339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:29:38.122250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597131+0000","last_change":"2024-09-12T12:22:30.582935+0000","last_active":"2024-09-12T12:22:32.597131+0000","last_peered":"2024-09-12T12:22:32.597131+0000","last_clean":"2024-09-12T12:22:32.597131+0000","last_became_active":"2024-09-12T12:22:30.582550+0000","last_became_peered":"2024-09-12T12:22:30.582550+0000","last_unstale":"2024-09-12T12:22:32.597131+0000","last_undegraded":"2024-09-12T12:22:32.597131+0000","last_fullsized":"2024-09-12T12:22:32.597131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:25:11.432036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.609329+0000","last_change":"2024-09-12T12:22:31.605555+0000","last_active":"2024-09-12T12:22:31.609329+0000","last_peered":"2024-09-12T12:22:31.609329+0000","last_clean":"2024-09-12T12:22:31.609329+0000","last_became_active":"2024-09-12T12:22:31.604868+0000","last_became_peered":"2024-09-12T12:22:31.604868+0000","last_unstale":"2024-09-12T12:22:31.609329+0000","last_undegraded":"2024-09-12T12:22:31.609329+0000","last_fullsized":"2024-09-12T12:22:31.609329+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:04.049615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565639+0000","last_change":"2024-09-12T12:22:30.594333+0000","last_active":"2024-09-12T12:22:32.565639+0000","last_peered":"2024-09-12T12:22:32.565639+0000","last_clean":"2024-09-12T12:22:32.565639+0000","last_became_active":"2024-09-12T12:22:30.589837+0000","last_became_peered":"2024-09-12T12:22:30.589837+0000","last_unstale":"2024-09-12T12:22:32.565639+0000","last_undegraded":"2024-09-12T12:22:32.565639+0000","last_fullsized":"2024-09-12T12:22:32.565639+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:21.349454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597336+0000","last_change":"2024-09-12T12:22:31.595271+0000","last_active":"2024-09-12T12:22:32.597336+0000","last_peered":"2024-09-12T12:22:32.597336+0000","last_clean":"2024-09-12T12:22:32.597336+0000","last_became_active":"2024-09-12T12:22:31.594823+0000","last_became_peered":"2024-09-12T12:22:31.594823+0000","last_unstale":"2024-09-12T12:22:32.597336+0000","last_undegraded":"2024-09-12T12:22:32.597336+0000","last_fullsized":"2024-09-12T12:22:32.597336+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:45.881345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.587334+0000","last_change":"2024-09-12T12:22:30.597645+0000","last_active":"2024-09-12T12:22:31.587334+0000","last_peered":"2024-09-12T12:22:31.587334+0000","last_clean":"2024-09-12T12:22:31.587334+0000","last_became_active":"2024-09-12T12:22:30.597231+0000","last_became_peered":"2024-09-12T12:22:30.597231+0000","last_unstale":"2024-09-12T12:22:31.587334+0000","last_undegraded":"2024-09-12T12:22:31.587334+0000","last_fullsized":"2024-09-12T12:22:31.587334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:22.784957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597208+0000","last_change":"2024-09-12T12:22:31.592723+0000","last_active":"2024-09-12T12:22:32.597208+0000","last_peered":"2024-09-12T12:22:32.597208+0000","last_clean":"2024-09-12T12:22:32.597208+0000","last_became_active":"2024-09-12T12:22:31.592438+0000","last_became_peered":"2024-09-12T12:22:31.592438+0000","last_unstale":"2024-09-12T12:22:32.597208+0000","last_undegraded":"2024-09-12T12:22:32.597208+0000","last_fullsized":"2024-09-12T12:22:32.597208+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:33.115619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572009+0000","last_change":"2024-09-12T12:22:30.601472+0000","last_active":"2024-09-12T12:22:31.572009+0000","last_peered":"2024-09-12T12:22:31.572009+0000","last_clean":"2024-09-12T12:22:31.572009+0000","last_became_active":"2024-09-12T12:22:30.600618+0000","last_became_peered":"2024-09-12T12:22:30.600618+0000","last_unstale":"2024-09-12T12:22:31.572009+0000","last_undegraded":"2024-09-12T12:22:31.572009+0000","last_fullsized":"2024-09-12T12:22:31.572009+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:45.322199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.604619+0000","last_change":"2024-09-12T12:22:31.594578+0000","last_active":"2024-09-12T12:22:31.604619+0000","last_peered":"2024-09-12T12:22:31.604619+0000","last_clean":"2024-09-12T12:22:31.604619+0000","last_became_active":"2024-09-12T12:22:31.593911+0000","last_became_peered":"2024-09-12T12:22:31.593911+0000","last_unstale":"2024-09-12T12:22:31.604619+0000","last_undegraded":"2024-09-12T12:22:31.604619+0000","last_fullsized":"2024-09-12T12:22:31.604619+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:43.978839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.613350+0000","last_change":"2024-09-12T12:22:30.595429+0000","last_active":"2024-09-12T12:22:31.613350+0000","last_peered":"2024-09-12T12:22:31.613350+0000","last_clean":"2024-09-12T12:22:31.613350+0000","last_became_active":"2024-09-12T12:22:30.595014+0000","last_became_peered":"2024-09-12T12:22:30.595014+0000","last_unstale":"2024-09-12T12:22:31.613350+0000","last_undegraded":"2024-09-12T12:22:31.613350+0000","last_fullsized":"2024-09-12T12:22:31.613350+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:23:49.561010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.594304+0000","last_change":"2024-09-12T12:22:31.585082+0000","last_active":"2024-09-12T12:22:31.594304+0000","last_peered":"2024-09-12T12:22:31.594304+0000","last_clean":"2024-09-12T12:22:31.594304+0000","last_became_active":"2024-09-12T12:22:31.584621+0000","last_became_peered":"2024-09-12T12:22:31.584621+0000","last_unstale":"2024-09-12T12:22:31.594304+0000","last_undegraded":"2024-09-12T12:22:31.594304+0000","last_fullsized":"2024-09-12T12:22:31.594304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:10.149058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.601047+0000","last_change":"2024-09-12T12:22:30.586241+0000","last_active":"2024-09-12T12:22:32.601047+0000","last_peered":"2024-09-12T12:22:32.601047+0000","last_clean":"2024-09-12T12:22:32.601047+0000","last_became_active":"2024-09-12T12:22:30.585913+0000","last_became_peered":"2024-09-12T12:22:30.585913+0000","last_unstale":"2024-09-12T12:22:32.601047+0000","last_undegraded":"2024-09-12T12:22:32.601047+0000","last_fullsized":"2024-09-12T12:22:32.601047+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:30:15.972093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.602957+0000","last_change":"2024-09-12T12:22:31.598372+0000","last_active":"2024-09-12T12:22:31.602957+0000","last_peered":"2024-09-12T12:22:31.602957+0000","last_clean":"2024-09-12T12:22:31.602957+0000","last_became_active":"2024-09-12T12:22:31.598078+0000","last_became_peered":"2024-09-12T12:22:31.598078+0000","last_unstale":"2024-09-12T12:22:31.602957+0000","last_undegraded":"2024-09-12T12:22:31.602957+0000","last_fullsized":"2024-09-12T12:22:31.602957+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:39.300126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.568059+0000","last_change":"2024-09-12T12:22:30.583044+0000","last_active":"2024-09-12T12:22:32.568059+0000","last_peered":"2024-09-12T12:22:32.568059+0000","last_clean":"2024-09-12T12:22:32.568059+0000","last_became_active":"2024-09-12T12:22:30.582352+0000","last_became_peered":"2024-09-12T12:22:30.582352+0000","last_unstale":"2024-09-12T12:22:32.568059+0000","last_undegraded":"2024-09-12T12:22:32.568059+0000","last_fullsized":"2024-09-12T12:22:32.568059+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:31.490479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.597952+0000","last_change":"2024-09-12T12:22:31.594015+0000","last_active":"2024-09-12T12:22:32.597952+0000","last_peered":"2024-09-12T12:22:32.597952+0000","last_clean":"2024-09-12T12:22:32.597952+0000","last_became_active":"2024-09-12T12:22:31.593131+0000","last_became_peered":"2024-09-12T12:22:31.593131+0000","last_unstale":"2024-09-12T12:22:32.597952+0000","last_undegraded":"2024-09-12T12:22:32.597952+0000","last_fullsized":"2024-09-12T12:22:32.597952+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:17.091261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.574351+0000","last_change":"2024-09-12T12:22:30.602111+0000","last_active":"2024-09-12T12:22:31.574351+0000","last_peered":"2024-09-12T12:22:31.574351+0000","last_clean":"2024-09-12T12:22:31.574351+0000","last_became_active":"2024-09-12T12:22:30.601379+0000","last_became_peered":"2024-09-12T12:22:30.601379+0000","last_unstale":"2024-09-12T12:22:31.574351+0000","last_undegraded":"2024-09-12T12:22:31.574351+0000","last_fullsized":"2024-09-12T12:22:31.574351+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:11:24.007173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565840+0000","last_change":"2024-09-12T12:22:31.597577+0000","last_active":"2024-09-12T12:22:32.565840+0000","last_peered":"2024-09-12T12:22:32.565840+0000","last_clean":"2024-09-12T12:22:32.565840+0000","last_became_active":"2024-09-12T12:22:31.597228+0000","last_became_peered":"2024-09-12T12:22:31.597228+0000","last_unstale":"2024-09-12T12:22:32.565840+0000","last_undegraded":"2024-09-12T12:22:32.565840+0000","last_fullsized":"2024-09-12T12:22:32.565840+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:09.787892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.582047+0000","last_change":"2024-09-12T12:22:30.594273+0000","last_active":"2024-09-12T12:22:31.582047+0000","last_peered":"2024-09-12T12:22:31.582047+0000","last_clean":"2024-09-12T12:22:31.582047+0000","last_became_active":"2024-09-12T12:22:30.593662+0000","last_became_peered":"2024-09-12T12:22:30.593662+0000","last_unstale":"2024-09-12T12:22:31.582047+0000","last_undegraded":"2024-09-12T12:22:31.582047+0000","last_fullsized":"2024-09-12T12:22:31.582047+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:55:31.490479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.598043+0000","last_change":"2024-09-12T12:22:31.594190+0000","last_active":"2024-09-12T12:22:31.598043+0000","last_peered":"2024-09-12T12:22:31.598043+0000","last_clean":"2024-09-12T12:22:31.598043+0000","last_became_active":"2024-09-12T12:22:31.593504+0000","last_became_peered":"2024-09-12T12:22:31.593504+0000","last_unstale":"2024-09-12T12:22:31.598043+0000","last_undegraded":"2024-09-12T12:22:31.598043+0000","last_fullsized":"2024-09-12T12:22:31.598043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:43.978839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.580418+0000","last_change":"2024-09-12T12:22:30.583854+0000","last_active":"2024-09-12T12:22:31.580418+0000","last_peered":"2024-09-12T12:22:31.580418+0000","last_clean":"2024-09-12T12:22:31.580418+0000","last_became_active":"2024-09-12T12:22:30.583063+0000","last_became_peered":"2024-09-12T12:22:30.583063+0000","last_unstale":"2024-09-12T12:22:31.580418+0000","last_undegraded":"2024-09-12T12:22:31.580418+0000","last_fullsized":"2024-09-12T12:22:31.580418+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:43.721208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.602853+0000","last_change":"2024-09-12T12:22:31.598459+0000","last_active":"2024-09-12T12:22:31.602853+0000","last_peered":"2024-09-12T12:22:31.602853+0000","last_clean":"2024-09-12T12:22:31.602853+0000","last_became_active":"2024-09-12T12:22:31.598097+0000","last_became_peered":"2024-09-12T12:22:31.598097+0000","last_unstale":"2024-09-12T12:22:31.602853+0000","last_undegraded":"2024-09-12T12:22:31.602853+0000","last_fullsized":"2024-09-12T12:22:31.602853+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:10:28.631953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.574146+0000","last_change":"2024-09-12T12:22:30.593490+0000","last_active":"2024-09-12T12:22:31.574146+0000","last_peered":"2024-09-12T12:22:31.574146+0000","last_clean":"2024-09-12T12:22:31.574146+0000","last_became_active":"2024-09-12T12:22:30.592880+0000","last_became_peered":"2024-09-12T12:22:30.592880+0000","last_unstale":"2024-09-12T12:22:31.574146+0000","last_undegraded":"2024-09-12T12:22:31.574146+0000","last_fullsized":"2024-09-12T12:22:31.574146+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:49.609132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565092+0000","last_change":"2024-09-12T12:22:31.570839+0000","last_active":"2024-09-12T12:22:32.565092+0000","last_peered":"2024-09-12T12:22:32.565092+0000","last_clean":"2024-09-12T12:22:32.565092+0000","last_became_active":"2024-09-12T12:22:31.570080+0000","last_became_peered":"2024-09-12T12:22:31.570080+0000","last_unstale":"2024-09-12T12:22:32.565092+0000","last_undegraded":"2024-09-12T12:22:32.565092+0000","last_fullsized":"2024-09-12T12:22:32.565092+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:31.028861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565129+0000","last_change":"2024-09-12T12:22:30.590264+0000","last_active":"2024-09-12T12:22:32.565129+0000","last_peered":"2024-09-12T12:22:32.565129+0000","last_clean":"2024-09-12T12:22:32.565129+0000","last_became_active":"2024-09-12T12:22:30.589963+0000","last_became_peered":"2024-09-12T12:22:30.589963+0000","last_unstale":"2024-09-12T12:22:32.565129+0000","last_undegraded":"2024-09-12T12:22:32.565129+0000","last_fullsized":"2024-09-12T12:22:32.565129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:32.111672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.565004+0000","last_change":"2024-09-12T12:22:31.602488+0000","last_active":"2024-09-12T12:22:32.565004+0000","last_peered":"2024-09-12T12:22:32.565004+0000","last_clean":"2024-09-12T12:22:32.565004+0000","last_became_active":"2024-09-12T12:22:31.601281+0000","last_became_peered":"2024-09-12T12:22:31.601281+0000","last_unstale":"2024-09-12T12:22:32.565004+0000","last_undegraded":"2024-09-12T12:22:32.565004+0000","last_fullsized":"2024-09-12T12:22:32.565004+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:30.542282+0000","last_clean_scrub_stamp":"2024-09-12T12:22:30.542282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:12:21.619491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-12T12:22:32.567674+0000","last_change":"2024-09-12T12:22:30.594141+0000","last_active":"2024-09-12T12:22:32.567674+0000","last_peered":"2024-09-12T12:22:32.567674+0000","last_clean":"2024-09-12T12:22:32.567674+0000","last_became_active":"2024-09-12T12:22:30.593407+0000","last_became_peered":"2024-09-12T12:22:30.593407+0000","last_unstale":"2024-09-12T12:22:32.567674+0000","last_undegraded":"2024-09-12T12:22:32.567674+0000","last_fullsized":"2024-09-12T12:22:32.567674+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:22:29.538335+0000","last_clean_scrub_stamp":"2024-09-12T12:22:29.538335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:28:49.609132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-12T12:22:31.572877+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:22:31.572877+0000","last_peered":"2024-09-12T12:22:31.572877+0000","last_clean":"2024-09-12T12:22:31.572877+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:22:31.572877+0000","last_undegraded":"2024-09-12T12:22:31.572877+0000","last_fullsized":"2024-09-12T12:22:31.572877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":663552,"data_stored":270217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1245184,"data_stored":854389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":643072,"data_stored":263861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":264513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":265473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":651264,"data_stored":263917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":647168,"data_stored":263951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1253376,"data_stored":856423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:22:36.018 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:22:36.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T12:22:36.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 Processor -- start 2024-09-12T12:22:36.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 -- start start 2024-09-12T12:22:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82a415aae0 0x7f82a406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f82a406fa00 0x7f82a406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a40b4e60 0x7f82a406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f82a415aec0 con 0x7f82a415aae0 2024-09-12T12:22:36.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f82a40b3dc0 con 0x7f82a406fa00 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82aa4a6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82a40b40a0 con 0x7f82a40b4e60 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82a415aae0 0x7f82a406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f82a415aae0 0x7f82a406f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37704/0 (socket says 172.21.15.3:37704) 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 -- 172.21.15.3:0/4044280571 learned_addr learned my addr 172.21.15.3:0/4044280571 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a37fe640 1 --2- 172.21.15.3:0/4044280571 >> v2:172.21.15.142:3300/0 conn(0x7f82a406fa00 0x7f82a406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a8a43640 1 --2- 172.21.15.3:0/4044280571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a40b4e60 0x7f82a406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 -- 172.21.15.3:0/4044280571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a40b4e60 msgr2=0x7f82a406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 --2- 172.21.15.3:0/4044280571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a40b4e60 0x7f82a406eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 -- 172.21.15.3:0/4044280571 >> v2:172.21.15.142:3300/0 conn(0x7f82a406fa00 msgr2=0x7f82a406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 --2- 172.21.15.3:0/4044280571 >> v2:172.21.15.142:3300/0 conn(0x7f82a406fa00 0x7f82a406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a3fff640 1 -- 172.21.15.3:0/4044280571 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a40b4380 con 0x7f82a415aae0 2024-09-12T12:22:36.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.188+0000 7f82a37fe640 1 --2- 172.21.15.3:0/4044280571 >> v2:172.21.15.142:3300/0 conn(0x7f82a406fa00 0x7f82a406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82a3fff640 1 --2- 172.21.15.3:0/4044280571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a415aae0 0x7f82a406f4c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f82a406db00 tx=0x7f829401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0295f43b7b8699c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82a2ffd640 1 -- 172.21.15.3:0/4044280571 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940b1070 con 0x7f82a415aae0 2024-09-12T12:22:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82a2ffd640 1 -- 172.21.15.3:0/4044280571 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82940ac600 con 0x7f82a415aae0 2024-09-12T12:22:36.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82a2ffd640 1 -- 172.21.15.3:0/4044280571 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940ac9d0 con 0x7f82a415aae0 2024-09-12T12:22:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 -- 172.21.15.3:0/4044280571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a415aae0 msgr2=0x7f82a406f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/4044280571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a415aae0 0x7f82a406f4c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f82a406db00 tx=0x7f829401c460 comp rx=0 tx=0).stop 2024-09-12T12:22:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 -- 172.21.15.3:0/4044280571 shutdown_connections 2024-09-12T12:22:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/4044280571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a40b4e60 0x7f82a406eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/4044280571 >> v2:172.21.15.142:3300/0 conn(0x7f82a406fa00 0x7f82a406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/4044280571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a415aae0 0x7f82a406f4c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 -- 172.21.15.3:0/4044280571 >> 172.21.15.3:0/4044280571 conn(0x7f82a40b1d80 msgr2=0x7f82a40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:36.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 -- 172.21.15.3:0/4044280571 shutdown_connections 2024-09-12T12:22:36.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 -- 172.21.15.3:0/4044280571 wait complete. 2024-09-12T12:22:36.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 Processor -- start 2024-09-12T12:22:36.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.192+0000 7f82aa4a6640 1 -- start start 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a406fa00 0x7f82a40ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a40b4e60 0x7f82a4146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a415aae0 0x7f82a4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4071f20 con 0x7f82a40b4e60 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82a4071da0 con 0x7f82a406fa00 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f82a40720a0 con 0x7f82a415aae0 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a40b4e60 0x7f82a4146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a40b4e60 0x7f82a4146ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37706/0 (socket says 172.21.15.3:37706) 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 -- 172.21.15.3:0/3453863888 learned_addr learned my addr 172.21.15.3:0/3453863888 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a3fff640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a406fa00 0x7f82a40ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a8a43640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a415aae0 0x7f82a4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 -- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a415aae0 msgr2=0x7f82a4159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a415aae0 0x7f82a4159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 -- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a406fa00 msgr2=0x7f82a40ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a406fa00 0x7f82a40ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 -- 172.21.15.3:0/3453863888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a4147000 con 0x7f82a40b4e60 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a37fe640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a40b4e60 0x7f82a4146ac0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f829800c7f0 tx=0x7f829800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a0ff9640 1 -- 172.21.15.3:0/3453863888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298015070 con 0x7f82a40b4e60 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a0ff9640 1 -- 172.21.15.3:0/3453863888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8298007820 con 0x7f82a40b4e60 2024-09-12T12:22:36.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a0ff9640 1 -- 172.21.15.3:0/3453863888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298007bf0 con 0x7f82a40b4e60 2024-09-12T12:22:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a4158970 con 0x7f82a40b4e60 2024-09-12T12:22:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a4158f30 con 0x7f82a40b4e60 2024-09-12T12:22:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a406f4c0 con 0x7f82a40b4e60 2024-09-12T12:22:36.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a0ff9640 1 -- 172.21.15.3:0/3453863888 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8298018070 con 0x7f82a40b4e60 2024-09-12T12:22:36.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a0ff9640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82900678c0 0x7f8290069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a0ff9640 1 -- 172.21.15.3:0/3453863888 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f8298085e60 con 0x7f82a40b4e60 2024-09-12T12:22:36.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.196+0000 7f82a3fff640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82900678c0 0x7f8290069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.200+0000 7f82a3fff640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82900678c0 0x7f8290069d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f82940ae000 tx=0x7f8294008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:36.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.204+0000 7f82a0ff9640 1 -- 172.21.15.3:0/3453863888 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8298056910 con 0x7f82a40b4e60 2024-09-12T12:22:36.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.416+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f82a415b980 con 0x7f82a40b4e60 2024-09-12T12:22:36.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.420+0000 7f82a0ff9640 1 -- 172.21.15.3:0/3453863888 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f8298056210 con 0x7f82a40b4e60 2024-09-12T12:22:36.425 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T12:22:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:22:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-12T12:22:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-12T12:22:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T12:22:36.426 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-12T12:22:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82900678c0 msgr2=0x7f8290069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82900678c0 0x7f8290069d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f82940ae000 tx=0x7f8294008040 comp rx=0 tx=0).stop 2024-09-12T12:22:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a40b4e60 msgr2=0x7f82a4146ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a40b4e60 0x7f82a4146ac0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f829800c7f0 tx=0x7f829800ccb0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 shutdown_connections 2024-09-12T12:22:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f82900678c0 0x7f8290069d80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f82a415aae0 0x7f82a4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82a40b4e60 0x7f82a4146ac0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 --2- 172.21.15.3:0/3453863888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82a406fa00 0x7f82a40ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 >> 172.21.15.3:0/3453863888 conn(0x7f82a40b1d80 msgr2=0x7f82a415d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 shutdown_connections 2024-09-12T12:22:36.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.424+0000 7f82aa4a6640 1 -- 172.21.15.3:0/3453863888 wait complete. 2024-09-12T12:22:36.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:22:36.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b27d7f640 1 Processor -- start 2024-09-12T12:22:36.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b27d7f640 1 -- start start 2024-09-12T12:22:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b27d7f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b2015e5d0 0x7f7b200ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b27d7f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200d5050 0x7f7b200cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b27d7f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b200d31a0 con 0x7f7b200cfa10 2024-09-12T12:22:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b27d7f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7b200b1060 con 0x7f7b2015e5d0 2024-09-12T12:22:36.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b27d7f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7b200cfe40 con 0x7f7b200d5050 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b25b1b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b2015e5d0 0x7f7b200ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b25b1b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7b2015e5d0 0x7f7b200ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37718/0 (socket says 172.21.15.3:37718) 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b25b1b640 1 -- 172.21.15.3:0/3994004025 learned_addr learned my addr 172.21.15.3:0/3994004025 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b2531a640 1 --2- 172.21.15.3:0/3994004025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200d5050 0x7f7b200cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b25b1b640 1 -- 172.21.15.3:0/3994004025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200d5050 msgr2=0x7f7b200cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b25b1b640 1 --2- 172.21.15.3:0/3994004025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200d5050 0x7f7b200cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b25b1b640 1 -- 172.21.15.3:0/3994004025 >> v1:172.21.15.142:6789/0 conn(0x7f7b200cfa10 legacy=0x7f7b20162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:36.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.552+0000 7f7b25b1b640 1 -- 172.21.15.3:0/3994004025 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b200d3480 con 0x7f7b2015e5d0 2024-09-12T12:22:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b25b1b640 1 --2- 172.21.15.3:0/3994004025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b200ceb60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009870 tx=0x7f7b1001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391d0a0cde8a4ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b24b19640 1 -- 172.21.15.3:0/3994004025 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b100aa620 con 0x7f7b2015e5d0 2024-09-12T12:22:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b24b19640 1 -- 172.21.15.3:0/3994004025 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b100af070 con 0x7f7b2015e5d0 2024-09-12T12:22:36.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b24b19640 1 -- 172.21.15.3:0/3994004025 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b100aa9e0 con 0x7f7b2015e5d0 2024-09-12T12:22:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 -- 172.21.15.3:0/3994004025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 msgr2=0x7f7b200ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/3994004025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b200ceb60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009870 tx=0x7f7b1001c520 comp rx=0 tx=0).stop 2024-09-12T12:22:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 -- 172.21.15.3:0/3994004025 shutdown_connections 2024-09-12T12:22:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/3994004025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200d5050 0x7f7b200cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/3994004025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b200ceb60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 -- 172.21.15.3:0/3994004025 >> 172.21.15.3:0/3994004025 conn(0x7f7b2008c000 msgr2=0x7f7b200d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:36.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 -- 172.21.15.3:0/3994004025 shutdown_connections 2024-09-12T12:22:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 -- 172.21.15.3:0/3994004025 wait complete. 2024-09-12T12:22:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.556+0000 7f7b27d7f640 1 Processor -- start 2024-09-12T12:22:36.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 -- start start 2024-09-12T12:22:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200cfa10 0x7f7b20155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b200d5050 0x7f7b200b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b2014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2016b5c0 con 0x7f7b2015e5d0 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2015ebc0 con 0x7f7b200d5050 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2016b740 con 0x7f7b200cfa10 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b2014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b2014c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37726/0 (socket says 172.21.15.3:37726) 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 -- 172.21.15.3:0/2849838333 learned_addr learned my addr 172.21.15.3:0/2849838333 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2531a640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b200d5050 0x7f7b200b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b25b1b640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200cfa10 0x7f7b20155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 -- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200cfa10 msgr2=0x7f7b20155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200cfa10 0x7f7b20155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 -- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b200d5050 msgr2=0x7f7b200b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b200d5050 0x7f7b200b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 -- 172.21.15.3:0/2849838333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b200b2f40 con 0x7f7b2015e5d0 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b2631c640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b2014c6e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b440 tx=0x7f7b1800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b1e7fc640 1 -- 172.21.15.3:0/2849838333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18015020 con 0x7f7b2015e5d0 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b1e7fc640 1 -- 172.21.15.3:0/2849838333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b18011830 con 0x7f7b2015e5d0 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b1e7fc640 1 -- 172.21.15.3:0/2849838333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18011c20 con 0x7f7b2015e5d0 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2015a270 con 0x7f7b2015e5d0 2024-09-12T12:22:36.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2015a810 con 0x7f7b2015e5d0 2024-09-12T12:22:36.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b200cf4d0 con 0x7f7b2015e5d0 2024-09-12T12:22:36.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b1e7fc640 1 -- 172.21.15.3:0/2849838333 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7b18018020 con 0x7f7b2015e5d0 2024-09-12T12:22:36.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b1e7fc640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b0c0678c0 0x7f7b0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:36.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.560+0000 7f7b1e7fc640 1 -- 172.21.15.3:0/2849838333 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f7b18085750 con 0x7f7b2015e5d0 2024-09-12T12:22:36.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.564+0000 7f7b25b1b640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b0c0678c0 0x7f7b0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:36.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.564+0000 7f7b25b1b640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b0c0678c0 0x7f7b0c069d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7b100ae000 tx=0x7f7b100a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:36.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.568+0000 7f7b1e7fc640 1 -- 172.21.15.3:0/2849838333 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7b1801f030 con 0x7f7b2015e5d0 2024-09-12T12:22:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b200ba490 con 0x7f7b2015e5d0 2024-09-12T12:22:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b1e7fc640 1 -- 172.21.15.3:0/2849838333 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7f7b18056290 con 0x7f7b2015e5d0 2024-09-12T12:22:36.759 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:36.759 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"btime":"2024-09-12T12:22:34:602545+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/256387515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":256387515},{"type":"v1","addr":"172.21.15.142:6835","nonce":256387515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2534794519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2534794519},{"type":"v1","addr":"172.21.15.142:6837","nonce":2534794519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2676380928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2676380928},{"type":"v1","addr":"172.21.15.3:6833","nonce":2676380928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:22:30.937780+0000","modified":"2024-09-12T12:22:33.984511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4007447206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4007447206},{"type":"v1","addr":"172.21.15.3:6835","nonce":4007447206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5655,"qdb_cluster":[5655]},"id":4}]} 2024-09-12T12:22:36.760 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b0c0678c0 msgr2=0x7f7b0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b0c0678c0 0x7f7b0c069d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7b100ae000 tx=0x7f7b100a7000 comp rx=0 tx=0).stop 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 msgr2=0x7f7b2014c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b2014c6e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b440 tx=0x7f7b1800b900 comp rx=0 tx=0).stop 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 shutdown_connections 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b0c0678c0 0x7f7b0c069d80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b2015e5d0 0x7f7b2014c6e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b200d5050 0x7f7b200b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 --2- 172.21.15.3:0/2849838333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b200cfa10 0x7f7b20155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:36.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.756+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 >> 172.21.15.3:0/2849838333 conn(0x7f7b2008c000 msgr2=0x7f7b200b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:36.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.760+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 shutdown_connections 2024-09-12T12:22:36.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:36.760+0000 7f7b27d7f640 1 -- 172.21.15.3:0/2849838333 wait complete. 2024-09-12T12:22:37.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:22:30.937780+0000', 'modified': '2024-09-12T12:22:33.984511+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5655}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5655': {'gid': 5655, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/4007447206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 4007447206}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 4007447206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5655, 'qdb_cluster': [5655]} 2024-09-12T12:22:37.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:22:37.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T12:22:37.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.312+0000 7fb4be403640 1 Processor -- start 2024-09-12T12:22:37.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.312+0000 7fb4be403640 1 -- start start 2024-09-12T12:22:37.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4be403640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 0x7fb4b815f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4be403640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb4b80ced00 0x7fb4b80cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4be403640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b80d2d90 con 0x7fb4b80b5fc0 2024-09-12T12:22:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4be403640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b80d2b60 con 0x7fb4b80ced00 2024-09-12T12:22:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4be403640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b815fb60 con 0x7fb4b80d26f0 2024-09-12T12:22:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b7fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb4b80b5fc0 0x7fb4b80b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49012/0 (socket says 172.21.15.3:49012) 2024-09-12T12:22:37.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b7fff640 1 -- 172.21.15.3:0/728054547 learned_addr learned my addr 172.21.15.3:0/728054547 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 0x7fb4b815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/728054547 >> v2:172.21.15.142:3300/0 conn(0x7fb4b80ced00 0x7fb4b80cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1618529868 0 0) 0x7fb4b80d2d90 con 0x7fb4b80b5fc0 2024-09-12T12:22:37.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a4003530 con 0x7fb4b80b5fc0 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2124064533 0 0) 0x7fb4a4003530 con 0x7fb4b80b5fc0 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 msgr2=0x7fb4b815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 0x7fb4b815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 >> v2:172.21.15.142:3300/0 conn(0x7fb4b80ced00 msgr2=0x7fb4b80cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 --2- 172.21.15.3:0/728054547 >> v2:172.21.15.142:3300/0 conn(0x7fb4b80ced00 0x7fb4b80cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b80d3070 con 0x7fb4b80b5fc0 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fb4a8001680 con 0x7fb4b80b5fc0 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 >> v1:172.21.15.3:6789/0 conn(0x7fb4b80b5fc0 legacy=0x7fb4b80b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a40039c0 0x7fb4a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 0x7fb4a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b815fb60 con 0x7fb4a4004810 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a400b9d0 con 0x7fb4a40039c0 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a400bb50 con 0x7fb4b80d26f0 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 0x7fb4a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a40039c0 0x7fb4a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 -- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 msgr2=0x7fb4a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80d26f0 0x7fb4a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 -- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a40039c0 msgr2=0x7fb4a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a40039c0 0x7fb4a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 -- 172.21.15.3:0/728054547 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a4003530 con 0x7fb4a4004810 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4a40039c0 0x7fb4a4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4a4004c90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ae0 tx=0x7fb4a001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1ea639dbeef9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00b2070 con 0x7fb4a4004810 2024-09-12T12:22:37.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a00a5580 con 0x7fb4a4004810 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.316+0000 7fb4b77fe640 1 -- 172.21.15.3:0/728054547 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00096b0 con 0x7fb4a4004810 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- 172.21.15.3:0/728054547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 msgr2=0x7fb4a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 --2- 172.21.15.3:0/728054547 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4a4004c90 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ae0 tx=0x7fb4a001ca70 comp rx=0 tx=0).stop 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4bc9a0640 1 -- 172.21.15.3:0/728054547 reap_dead start 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- 172.21.15.3:0/728054547 shutdown_connections 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- 172.21.15.3:0/728054547 >> 172.21.15.3:0/728054547 conn(0x7fb4b808c000 msgr2=0x7fb4b80d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- 172.21.15.3:0/728054547 shutdown_connections 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- 172.21.15.3:0/728054547 wait complete. 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 Processor -- start 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- start start 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4b81f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80b8b30 0x7fb4b815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4b81f53b0 0x7fb4b806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b80d3910 con 0x7fb4a4004810 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b80d3630 con 0x7fb4b81f53b0 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b815e8b0 con 0x7fb4b80b8b30 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4b81f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4b81f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37740/0 (socket says 172.21.15.3:37740) 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 -- 172.21.15.3:0/3829966986 learned_addr learned my addr 172.21.15.3:0/3829966986 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 -- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80b8b30 msgr2=0x7fb4b815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80b8b30 0x7fb4b815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80b8b30 0x7fb4b815a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 -- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4b81f53b0 msgr2=0x7fb4b806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4b81f53b0 0x7fb4b806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4b81f53b0 0x7fb4b806e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 -- 172.21.15.3:0/3829966986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b80b7370 con 0x7fb4a4004810 2024-09-12T12:22:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4b81f5ad0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8010bc0 tx=0x7fb4a800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b57fa640 1 -- 172.21.15.3:0/3829966986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a801b070 con 0x7fb4a4004810 2024-09-12T12:22:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b57fa640 1 -- 172.21.15.3:0/3829966986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a8011040 con 0x7fb4a4004810 2024-09-12T12:22:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b81620c0 con 0x7fb4a4004810 2024-09-12T12:22:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4b57fa640 1 -- 172.21.15.3:0/3829966986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80143a0 con 0x7fb4a4004810 2024-09-12T12:22:37.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.320+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b80b92e0 con 0x7fb4a4004810 2024-09-12T12:22:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4b57fa640 1 -- 172.21.15.3:0/3829966986 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb4a8014540 con 0x7fb4a4004810 2024-09-12T12:22:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4b57fa640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb4980678f0 0x7fb498069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4b57fa640 1 -- 172.21.15.3:0/3829966986 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fb4a8088700 con 0x7fb4a4004810 2024-09-12T12:22:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb4980678f0 0x7fb498069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 Processor -- start 2024-09-12T12:22:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 -- start start 2024-09-12T12:22:37.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb4980678f0 0x7fb498069db0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00af000 tx=0x7fb4a00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703a7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a7035dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a7035e290 0x562a7035e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a70245b60 con 0x562a7035e290 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562a701acc70 con 0x562a703a8440 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562a70152ed0 con 0x562a703a7b20 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a7035e290 0x562a7035e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a7035e290 0x562a7035e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37742/0 (socket says 172.21.15.3:37742) 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 -- 172.21.15.3:0/737509863 learned_addr learned my addr 172.21.15.3:0/737509863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a7035dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703a7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 -- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 msgr2=0x562a703a7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703a7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 -- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 msgr2=0x562a7035dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a7035dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.324+0000 7fb4bc9a0640 1 -- 172.21.15.3:0/737509863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a70164e50 con 0x562a7035e290 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a7035e290 0x562a7035e670 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80b3f50 tx=0x7fb4b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ff43b55d71312b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4adffb640 1 -- 172.21.15.3:0/737509863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b001a020 con 0x562a7035e290 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4adffb640 1 -- 172.21.15.3:0/737509863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0016660 con 0x562a7035e290 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4adffb640 1 -- 172.21.15.3:0/737509863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0016a50 con 0x562a7035e290 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/737509863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a7035e290 msgr2=0x562a7035e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a7035e290 0x562a7035e670 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80b3f50 tx=0x7fb4b0009750 comp rx=0 tx=0).stop 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/737509863 shutdown_connections 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a7035e290 0x562a7035e670 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a7035dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/737509863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703a7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/737509863 >> 172.21.15.3:0/737509863 conn(0x562a70364600 msgr2=0x562a7035c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/737509863 shutdown_connections 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/737509863 wait complete. 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- start start 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 Processor -- start 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a7b20 0x562a70362df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a8440 0x562a70368cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a70363330 0x562a70369230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a7020eb00 con 0x562a703a8440 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562a70187360 con 0x562a703a7b20 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562a701842d0 con 0x562a70363330 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 10 client.0 _ll_register_callbacks cb 0x562a70349820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a8440 0x562a70368cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a8440 0x562a70368cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37752/0 (socket says 172.21.15.3:37752) 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2596743507 learned_addr learned my addr 172.21.15.3:0/2596743507 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a7b20 0x562a70362df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a70363330 0x562a70369230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a70363330 msgr2=0x562a70369230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a70363330 0x562a70369230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a7b20 msgr2=0x562a70362df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a7b20 0x562a70362df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2596743507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a70153120 con 0x562a703a8440 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a8440 0x562a70368cf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb4a001cf50 tx=0x7fb4a0019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f03d57bed5a8b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb496ffd640 1 -- 172.21.15.3:0/2596743507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00c7060 con 0x562a703a8440 2024-09-12T12:22:37.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb496ffd640 1 -- 172.21.15.3:0/2596743507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fb4a0003af0 con 0x562a703a8440 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b0016660 con 0x562a703a8440 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb496ffd640 1 -- 172.21.15.3:0/2596743507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb4a00cd050 con 0x562a703a8440 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 10 client.5781 fetch_fsmap learned FSMap version 36 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x562a7014c450 con 0x562a703a8440 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb496ffd640 1 -- 172.21.15.3:0/2596743507 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0016660 con 0x562a703a8440 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb496ffd640 1 -- 172.21.15.3:0/2596743507 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7fb4a00d0020 con 0x562a703a8440 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 10 client.5781 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 10 client.5781 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 20 client.5781 populate_metadata read hostname 'smithi003' 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703b6280 0x562a703b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 4 client.5781 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 --> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a70146c00 con 0x562a703b6280 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.328+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703b6280 0x562a703b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703b6280 0x562a703b8670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c827d4c63fe2a0 server_cookie=16842da1258eec49 in_seq=0 out_seq=0 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb496ffd640 10 client.5781 ms_handle_connect on v2:172.21.15.3:6834/4007447206 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb496ffd640 1 -- 172.21.15.3:0/2596743507 <== mds.0 v2:172.21.15.3:6834/4007447206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a70146c00 con 0x562a703b6280 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb496ffd640 10 client.5781 handle_command_reply: tid=0 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 client.5781 shutdown 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a8440 msgr2=0x562a70368cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a8440 0x562a70368cf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb4a001cf50 tx=0x7fb4a0019810 comp rx=0 tx=0).stop 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 shutdown_connections 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703b6280 0x562a703b8670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a70363330 0x562a70369230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a8440 0x562a70368cf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2596743507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a7b20 0x562a70362df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb496ffd640 0 client.5781 ms_handle_reset on v2:172.21.15.3:6834/4007447206 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 >> 172.21.15.3:0/2596743507 conn(0x562a7035e290 msgr2=0x562a7035cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 shutdown_connections 2024-09-12T12:22:37.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2596743507 wait complete. 2024-09-12T12:22:37.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.332+0000 7fb4c14bf000 20 client.5781 trim_cache size 0 max 16384 2024-09-12T12:22:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.348+0000 7fb4c14bf000 1 Processor -- start 2024-09-12T12:22:37.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.348+0000 7fb4c14bf000 1 -- start start 2024-09-12T12:22:37.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a70363330 0x562a703a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a7b20 0x562a703a7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a8440 0x562a703b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8160c80 con 0x562a703a7b20 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b815ffe0 con 0x562a70363330 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b8160630 con 0x562a703a8440 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a7b20 0x562a703a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a7b20 0x562a703a7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37766/0 (socket says 172.21.15.3:37766) 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2094565926 learned_addr learned my addr 172.21.15.3:0/2094565926 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a8440 0x562a703b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a8440 msgr2=0x562a703b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a70363330 0x562a703a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a8440 0x562a703b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a70363330 msgr2=0x562a703a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a70363330 0x562a703a6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 -- 172.21.15.3:0/2094565926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a70133fd0 con 0x562a703a7b20 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a8440 0x562a703b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a7b20 0x562a703a7230 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb4a001cfe0 tx=0x7fb4a00be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bbc4ba1bf46390 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4ad7fa640 1 -- 172.21.15.3:0/2094565926 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00b2040 con 0x562a703a7b20 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4ad7fa640 1 -- 172.21.15.3:0/2094565926 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a00a5580 con 0x562a703a7b20 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4ad7fa640 1 -- 172.21.15.3:0/2094565926 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00a5930 con 0x562a703a7b20 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2094565926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a7b20 msgr2=0x562a703a7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a7b20 0x562a703a7230 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb4a001cfe0 tx=0x7fb4a00be000 comp rx=0 tx=0).stop 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2094565926 shutdown_connections 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a8440 0x562a703b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a703a7b20 0x562a703a7230 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/2094565926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a70363330 0x562a703a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2094565926 >> 172.21.15.3:0/2094565926 conn(0x562a7035e290 msgr2=0x562a703625f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2094565926 shutdown_connections 2024-09-12T12:22:37.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- 172.21.15.3:0/2094565926 wait complete. 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- start start 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 Processor -- start 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a70363330 0x562a703bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703d6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a703d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a701abf80 con 0x562a70363330 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562a70164ca0 con 0x562a703a8440 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x562a701b12a0 con 0x562a703a7b20 2024-09-12T12:22:37.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4c14bf000 10 client.0 _ll_register_callbacks cb 0x7fb4a00b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a70363330 0x562a703bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a70363330 0x562a703bb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37772/0 (socket says 172.21.15.3:37772) 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 -- 172.21.15.3:0/1852757593 learned_addr learned my addr 172.21.15.3:0/1852757593 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703d6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a703d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 -- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 msgr2=0x562a703d6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703d6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 -- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 msgr2=0x562a703d6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a703d6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4b7fff640 1 -- 172.21.15.3:0/1852757593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a7022f750 con 0x562a70363330 2024-09-12T12:22:37.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.352+0000 7fb4bc9a0640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a703d6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4b7fff640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a70363330 0x562a703bb590 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80611e0 tx=0x7fb4a8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1407b3f3bba30c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 1 -- 172.21.15.3:0/1852757593 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8084070 con 0x562a70363330 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 1 -- 172.21.15.3:0/1852757593 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fb4a80551a0 con 0x562a70363330 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a00a5580 con 0x562a70363330 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 1 -- 172.21.15.3:0/1852757593 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb4a8077050 con 0x562a70363330 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 10 client.5787 fetch_fsmap learned FSMap version 36 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x562a7013aa30 con 0x562a70363330 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 1 -- 172.21.15.3:0/1852757593 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a00a5580 con 0x562a70363330 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 1 -- 172.21.15.3:0/1852757593 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7fb4a8080130 con 0x562a70363330 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 10 client.5787 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 10 client.5787 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:22:37.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 20 client.5787 populate_metadata read hostname 'smithi003' 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703d73e0 0x562a7041ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 4 client.5787 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 --> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] -- command(tid 0: {"prefix": "status"}) -- 0x562a701ba3a0 con 0x562a703d73e0 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703d73e0 0x562a7041ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4af7fe640 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703d73e0 0x562a7041ad50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa209f425eefcc6c server_cookie=a429217a8008127d in_seq=0 out_seq=0 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 10 client.5787 ms_handle_connect on v2:172.21.15.3:6834/4007447206 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 1 -- 172.21.15.3:0/1852757593 <== mds.0 v2:172.21.15.3:6834/4007447206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x562a701ba3a0 con 0x562a703d73e0 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 10 client.5787 handle_command_reply: tid=0 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 client.5787 shutdown 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a70363330 msgr2=0x562a703bb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a70363330 0x562a703bb590 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80611e0 tx=0x7fb4a8009040 comp rx=0 tx=0).stop 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 shutdown_connections 2024-09-12T12:22:37.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:6834/4007447206,v1:172.21.15.3:6835/4007447206] conn(0x562a703d73e0 0x562a7041ad50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562a703a8440 0x562a703d6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x562a703a7b20 0x562a703d6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb496ffd640 0 client.5787 ms_handle_reset on v2:172.21.15.3:6834/4007447206 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 --2- 172.21.15.3:0/1852757593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a70363330 0x562a703bb590 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 >> 172.21.15.3:0/1852757593 conn(0x562a7035e290 msgr2=0x562a703a5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 shutdown_connections 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 1 -- 172.21.15.3:0/1852757593 wait complete. 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4c14bf000 20 client.5787 trim_cache size 0 max 16384 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:22:37.366 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5655, 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 5.7692966659999998, 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 36, 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 55, 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 55, 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 10.807511129 2024-09-12T12:22:37.367 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:22:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb4980678f0 msgr2=0x7fb498069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb4980678f0 0x7fb498069db0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00af000 tx=0x7fb4a00046d0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 msgr2=0x7fb4b81f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4b81f5ad0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8010bc0 tx=0x7fb4a800e4d0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 shutdown_connections 2024-09-12T12:22:37.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb4980678f0 0x7fb498069db0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb4b81f53b0 0x7fb4b806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb4b80b8b30 0x7fb4b815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 --2- 172.21.15.3:0/3829966986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4a4004810 0x7fb4b81f5ad0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 >> 172.21.15.3:0/3829966986 conn(0x7fb4b808c000 msgr2=0x7fb4b815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 shutdown_connections 2024-09-12T12:22:37.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.356+0000 7fb4be403640 1 -- 172.21.15.3:0/3829966986 wait complete. 2024-09-12T12:22:37.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:22:37.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.484+0000 7f06aec2a640 1 Processor -- start 2024-09-12T12:22:37.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06aec2a640 1 -- start start 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06aec2a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a80c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06aec2a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06a808c410 0x7f06a8162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06aec2a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f06a80d2920 0x7f06a8162890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06aec2a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f06a8162dd0 con 0x7f06a808c410 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06aec2a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f06a80ce9b0 con 0x7f06a80d2920 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06aec2a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06a8163020 con 0x7f06a8161ed0 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06a808c410 0x7f06a8162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f06a808c410 0x7f06a8162350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37786/0 (socket says 172.21.15.3:37786) 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 -- 172.21.15.3:0/3871681626 learned_addr learned my addr 172.21.15.3:0/3871681626 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06ac9c6640 1 --2- 172.21.15.3:0/3871681626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a80c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06ad1c7640 1 --2- 172.21.15.3:0/3871681626 >> v2:172.21.15.142:3300/0 conn(0x7f06a80d2920 0x7f06a8162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 -- 172.21.15.3:0/3871681626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 msgr2=0x7f06a80c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 --2- 172.21.15.3:0/3871681626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a80c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 -- 172.21.15.3:0/3871681626 >> v2:172.21.15.142:3300/0 conn(0x7f06a80d2920 msgr2=0x7f06a8162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 --2- 172.21.15.3:0/3871681626 >> v2:172.21.15.142:3300/0 conn(0x7f06a80d2920 0x7f06a8162890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 -- 172.21.15.3:0/3871681626 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a80d3000 con 0x7f06a808c410 2024-09-12T12:22:37.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a7fff640 1 --2- 172.21.15.3:0/3871681626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a808c410 0x7f06a8162350 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06a808d100 tx=0x7f069001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1abae1c75bc514c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a77fe640 1 -- 172.21.15.3:0/3871681626 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900b5070 con 0x7f06a808c410 2024-09-12T12:22:37.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a77fe640 1 -- 172.21.15.3:0/3871681626 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06900a89d0 con 0x7f06a808c410 2024-09-12T12:22:37.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.488+0000 7f06a77fe640 1 -- 172.21.15.3:0/3871681626 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900a8da0 con 0x7f06a808c410 2024-09-12T12:22:37.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- 172.21.15.3:0/3871681626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a808c410 msgr2=0x7f06a8162350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 --2- 172.21.15.3:0/3871681626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a808c410 0x7f06a8162350 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06a808d100 tx=0x7f069001ed10 comp rx=0 tx=0).stop 2024-09-12T12:22:37.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- 172.21.15.3:0/3871681626 shutdown_connections 2024-09-12T12:22:37.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 --2- 172.21.15.3:0/3871681626 >> v2:172.21.15.142:3300/0 conn(0x7f06a80d2920 0x7f06a8162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 --2- 172.21.15.3:0/3871681626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a808c410 0x7f06a8162350 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 --2- 172.21.15.3:0/3871681626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a80c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- 172.21.15.3:0/3871681626 >> 172.21.15.3:0/3871681626 conn(0x7f06a808c000 msgr2=0x7f06a80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- 172.21.15.3:0/3871681626 shutdown_connections 2024-09-12T12:22:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- 172.21.15.3:0/3871681626 wait complete. 2024-09-12T12:22:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 Processor -- start 2024-09-12T12:22:37.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- start start 2024-09-12T12:22:37.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a80d2920 0x7f06a80a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a8160670 0x7f06a815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06a80cfea0 con 0x7f06a8160670 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06a80cfd20 con 0x7f06a80d2920 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06a80d0020 con 0x7f06a8161ed0 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a8160670 0x7f06a815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a8160670 0x7f06a815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37800/0 (socket says 172.21.15.3:37800) 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 -- 172.21.15.3:0/394072205 learned_addr learned my addr 172.21.15.3:0/394072205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ac9c6640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a80d2920 0x7f06a80a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06a7fff640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 -- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 msgr2=0x7f06a815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 -- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a80d2920 msgr2=0x7f06a80a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a80d2920 0x7f06a80a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 -- 172.21.15.3:0/394072205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a815ebd0 con 0x7f06a8160670 2024-09-12T12:22:37.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06a7fff640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:37.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06ad1c7640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a8160670 0x7f06a815d3c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0698007cc0 tx=0x7f069800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06a57fa640 1 -- 172.21.15.3:0/394072205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698014420 con 0x7f06a8160670 2024-09-12T12:22:37.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06a57fa640 1 -- 172.21.15.3:0/394072205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0698019070 con 0x7f06a8160670 2024-09-12T12:22:37.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81f4e80 con 0x7f06a8160670 2024-09-12T12:22:37.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06a57fa640 1 -- 172.21.15.3:0/394072205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698014800 con 0x7f06a8160670 2024-09-12T12:22:37.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.492+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a80cb900 con 0x7f06a8160670 2024-09-12T12:22:37.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.496+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a80c80a0 con 0x7f06a8160670 2024-09-12T12:22:37.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.496+0000 7f06a57fa640 1 -- 172.21.15.3:0/394072205 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06980149a0 con 0x7f06a8160670 2024-09-12T12:22:37.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.496+0000 7f06a57fa640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06880678c0 0x7f0688069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:37.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.496+0000 7f06a57fa640 1 -- 172.21.15.3:0/394072205 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f0698086ef0 con 0x7f06a8160670 2024-09-12T12:22:37.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.496+0000 7f06ac9c6640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06880678c0 0x7f0688069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:37.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.496+0000 7f06ac9c6640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06880678c0 0x7f0688069d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f06a80ce040 tx=0x7f06a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:37.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.500+0000 7f06a57fa640 1 -- 172.21.15.3:0/394072205 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0698057aa0 con 0x7f06a8160670 2024-09-12T12:22:37.529 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-12T12:22:37.528+0000 7f51d26b5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:22:37.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f06a80ba4d0 con 0x7f06a8160670 2024-09-12T12:22:37.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06a57fa640 1 -- 172.21.15.3:0/394072205 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f06980573a0 con 0x7f06a8160670 2024-09-12T12:22:37.702 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:22:37.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06880678c0 msgr2=0x7f0688069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06880678c0 0x7f0688069d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f06a80ce040 tx=0x7f06a0008040 comp rx=0 tx=0).stop 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a8160670 msgr2=0x7f06a815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a8160670 0x7f06a815d3c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0698007cc0 tx=0x7f069800e4b0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 shutdown_connections 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06880678c0 0x7f0688069d80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06a8160670 0x7f06a815d3c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a8161ed0 0x7f06a815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 --2- 172.21.15.3:0/394072205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a80d2920 0x7f06a80a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 >> 172.21.15.3:0/394072205 conn(0x7f06a808c000 msgr2=0x7f06a80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 shutdown_connections 2024-09-12T12:22:37.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37.700+0000 7f06aec2a640 1 -- 172.21.15.3:0/394072205 wait complete. 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:22:37.720 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:22:37.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:37.721 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:22:37.729 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:22:37.730 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42329sec preferred_lft 42329sec 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.731 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:22:37.732 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:22:37.733 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:37.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:37.734 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:22:37.785 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:22:37.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:37.786 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:22:37.835 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:22:37.836 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:22:37.836 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:22:37.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:22:37.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:22:37.836 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:22:37.836 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:22:37.836 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:22:37.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:22:37.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:22:37.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:22:37.837 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:22:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:22:37.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:37.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:37.962 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:22:37.967 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:22:37.967 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:22:37.967 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:22:37.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:37.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:22:38.026 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:22:38.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:38.026 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:38.074 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:22:38.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:38.075 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:38.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:22:38.180 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:38.228 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:22:38.228 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:22:38.228 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:22:38.229 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:22:38.230 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:22:38.231 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:22:38.231 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:38.231 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:38.231 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:38.231 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:38.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:38.232 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:22:38.282 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:22:38.282 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:22:38.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:22:38.341 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:38.390 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:22:38.390 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:22:38.390 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:22:38.390 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:22:38.390 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:22:38.390 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:22:38.391 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:38.392 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:38.393 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:38.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:38.394 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:22:38.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:22:38.408+0000 7f10b3d86580 -1 init, newargv = 0x560bb97ab010 newargc=15 2024-09-12T12:22:38.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[39971]: starting ceph client 2024-09-12T12:22:38.417 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[39971]: starting fuse 2024-09-12T12:22:39.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:22:39.533 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:39.583 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:22:39.584 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:22:39.585 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:22:39.586 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:39.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:39.587 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:22:39.635 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:22:39.635 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:22:39.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:22:39.635 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:22:39.635 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:22:39.636 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:22:39.637 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:22:39.762 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.39971.asok 2024-09-12T12:22:39.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:39.775 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39971.asok 2024-09-12T12:22:39.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:39.776 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39971.asok status 2024-09-12T12:22:39.932 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:22:39.932 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:22:39.932 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "39971", 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 5799, 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:22:39.933 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 5799 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3239766052 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3239766052 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.5799 192.168.0.1:0/3239766052", 2024-09-12T12:22:39.934 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3239766052", 2024-09-12T12:22:39.935 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:22:39.935 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 36, 2024-09-12T12:22:39.935 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 55, 2024-09-12T12:22:39.935 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:22:39.935 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:22:39.935 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:22:39.935 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:22:39.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:39.946 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:22:39.946 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:39.953 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:22:39.953 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:39.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:39.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:40.011 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:22:40.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681eb5d640 1 Processor -- start 2024-09-12T12:22:40.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681eb5d640 1 -- start start 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681eb5d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6818159bd0 0x7f681806dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681eb5d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6818159fb0 0x7f681815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681eb5d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f681806e400 con 0x7f681806f4a0 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681eb5d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f681806f010 con 0x7f6818159fb0 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681eb5d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f681806e6e0 con 0x7f6818159bd0 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f6817fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6818159fb0 0x7f681815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681d0fa640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f681806f4a0 0x7f68180ad060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40250/0 (socket says 172.21.15.3:40250) 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681d0fa640 1 -- 172.21.15.3:0/2071496193 learned_addr learned my addr 172.21.15.3:0/2071496193 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f681c8f9640 1 --2- 172.21.15.3:0/2071496193 >> v2:172.21.15.142:3300/0 conn(0x7f6818159bd0 0x7f681806dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f6817fff640 1 -- 172.21.15.3:0/2071496193 >> v2:172.21.15.142:3300/0 conn(0x7f6818159bd0 msgr2=0x7f681806dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:40.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f6817fff640 1 --2- 172.21.15.3:0/2071496193 >> v2:172.21.15.142:3300/0 conn(0x7f6818159bd0 0x7f681806dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f6817fff640 1 -- 172.21.15.3:0/2071496193 >> v1:172.21.15.3:6789/0 conn(0x7f681806f4a0 legacy=0x7f68180ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:40.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f68177fe640 1 -- 172.21.15.3:0/2071496193 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655829627 0 0) 0x7f681806e400 con 0x7f681806f4a0 2024-09-12T12:22:40.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.184+0000 7f6817fff640 1 -- 172.21.15.3:0/2071496193 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681806e9c0 con 0x7f6818159fb0 2024-09-12T12:22:40.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f6817fff640 1 --2- 172.21.15.3:0/2071496193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f681815d2d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6818071320 tx=0x7f680c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14264dd56ce81d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:40.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f68177fe640 1 -- 172.21.15.3:0/2071496193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c0b1070 con 0x7f6818159fb0 2024-09-12T12:22:40.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f68177fe640 1 -- 172.21.15.3:0/2071496193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680c0ac5f0 con 0x7f6818159fb0 2024-09-12T12:22:40.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f68177fe640 1 -- 172.21.15.3:0/2071496193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c0ac9c0 con 0x7f6818159fb0 2024-09-12T12:22:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- 172.21.15.3:0/2071496193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 msgr2=0x7f681815d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 --2- 172.21.15.3:0/2071496193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f681815d2d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6818071320 tx=0x7f680c01c480 comp rx=0 tx=0).stop 2024-09-12T12:22:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- 172.21.15.3:0/2071496193 shutdown_connections 2024-09-12T12:22:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 --2- 172.21.15.3:0/2071496193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f681815d2d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 --2- 172.21.15.3:0/2071496193 >> v2:172.21.15.142:3300/0 conn(0x7f6818159bd0 0x7f681806dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- 172.21.15.3:0/2071496193 >> 172.21.15.3:0/2071496193 conn(0x7f68180b1d80 msgr2=0x7f68180b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- 172.21.15.3:0/2071496193 shutdown_connections 2024-09-12T12:22:40.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- 172.21.15.3:0/2071496193 wait complete. 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 Processor -- start 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- start start 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681806f4a0 0x7f68181f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6818159bd0 0x7f68180ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f6818153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68181691e0 con 0x7f6818159fb0 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68180b3aa0 con 0x7f681806f4a0 2024-09-12T12:22:40.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681eb5d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6818169360 con 0x7f6818159bd0 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f6818153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f6818153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47046/0 (socket says 172.21.15.3:47046) 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 -- 172.21.15.3:0/55600912 learned_addr learned my addr 172.21.15.3:0/55600912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681c8f9640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681806f4a0 0x7f68181f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f6817fff640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6818159bd0 0x7f68180ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 -- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6818159bd0 msgr2=0x7f68180ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6818159bd0 0x7f68180ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 -- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681806f4a0 msgr2=0x7f68181f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681806f4a0 0x7f68181f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 -- 172.21.15.3:0/55600912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68180cf8f0 con 0x7f6818159fb0 2024-09-12T12:22:40.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681c8f9640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681806f4a0 0x7f68181f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.188+0000 7f681d0fa640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f6818153f00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f681000c7a0 tx=0x7f681000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f68157fa640 1 -- 172.21.15.3:0/55600912 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810018070 con 0x7f6818159fb0 2024-09-12T12:22:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f68157fa640 1 -- 172.21.15.3:0/55600912 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681000a640 con 0x7f6818159fb0 2024-09-12T12:22:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f68157fa640 1 -- 172.21.15.3:0/55600912 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681000aa70 con 0x7f6818159fb0 2024-09-12T12:22:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6818151460 con 0x7f6818159fb0 2024-09-12T12:22:40.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68180cf530 con 0x7f6818159fb0 2024-09-12T12:22:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f681815d2d0 con 0x7f6818159fb0 2024-09-12T12:22:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f68157fa640 1 -- 172.21.15.3:0/55600912 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f681001b070 con 0x7f6818159fb0 2024-09-12T12:22:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f68157fa640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68040678c0 0x7f6804069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:40.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f68157fa640 1 -- 172.21.15.3:0/55600912 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f6810085240 con 0x7f6818159fb0 2024-09-12T12:22:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f681c8f9640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68040678c0 0x7f6804069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:40.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.192+0000 7f681c8f9640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68040678c0 0x7f6804069d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6808009840 tx=0x7f6808008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:40.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.196+0000 7f68157fa640 1 -- 172.21.15.3:0/55600912 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6810055e10 con 0x7f6818159fb0 2024-09-12T12:22:40.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.400+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f68180b2b00 con 0x7f6818159fb0 2024-09-12T12:22:40.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.400+0000 7f68157fa640 1 -- 172.21.15.3:0/55600912 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f6810055710 con 0x7f6818159fb0 2024-09-12T12:22:40.405 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68040678c0 msgr2=0x7f6804069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68040678c0 0x7f6804069d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6808009840 tx=0x7f6808008040 comp rx=0 tx=0).stop 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 msgr2=0x7f6818153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f6818153f00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f681000c7a0 tx=0x7f681000cc60 comp rx=0 tx=0).stop 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 shutdown_connections 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f68040678c0 0x7f6804069d80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6818159fb0 0x7f6818153f00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6818159bd0 0x7f68180ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 --2- 172.21.15.3:0/55600912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f681806f4a0 0x7f68181f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 >> 172.21.15.3:0/55600912 conn(0x7f68180b1d80 msgr2=0x7f6818165360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 shutdown_connections 2024-09-12T12:22:40.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:40.404+0000 7f681eb5d640 1 -- 172.21.15.3:0/55600912 wait complete. 2024-09-12T12:22:40.423 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:22:40.424 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:22:40.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:40.424 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:22:40.431 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:22:40.431 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:22:40.431 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:40.432 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42324sec preferred_lft 42324sec 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.433 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:40.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:40.434 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:22:40.488 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:22:40.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:40.489 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:22:40.541 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:22:40.542 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:22:40.542 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:22:40.542 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:22:40.542 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:22:40.542 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:22:40.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:22:40.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:22:40.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:22:40.543 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:22:40.629 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:22:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:22:40.657 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:22:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:40.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:40.666 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:22:40.717 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:22:40.717 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:22:40.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:22:40.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:40.717 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:22:40.773 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:22:40.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:40.774 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:40.821 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:22:40.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:40.821 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:40.868 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:22:40.926 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:40.973 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:22:40.973 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:22:40.973 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:22:40.974 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:22:40.975 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:40.976 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:40.976 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:40.976 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:40.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:40.977 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:22:41.024 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:22:41.024 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:22:41.069 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:22:41.084 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:41.133 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:22:41.133 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:22:41.133 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:22:41.133 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:22:41.133 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:22:41.133 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:22:41.133 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:22:41.134 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:41.135 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:41.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:41.136 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:22:41.149 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:22:41.148+0000 7f941c783580 -1 init, newargv = 0x556dcfbe00c0 newargc=15 2024-09-12T12:22:41.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35897]: starting ceph client 2024-09-12T12:22:41.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35897]: starting fuse 2024-09-12T12:22:42.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:22:42.271 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:42.317 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:22:42.317 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:22:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:22:42.319 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:42.320 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:42.320 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:42.320 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:42.320 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:42.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:42.321 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:22:42.368 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:22:42.368 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:22:42.369 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:22:42.370 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:22:42.493 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.35897.asok 2024-09-12T12:22:42.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:42.505 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35897.asok 2024-09-12T12:22:42.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:42.505 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35897.asok status 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "35897", 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:42.662 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 4946, 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 4946 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 3792701178 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:22:42.663 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 3792701178 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.4946 192.168.0.1:0/3792701178", 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/3792701178", 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 36, 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 55, 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:22:42.664 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:22:42.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:42.675 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:22:42.675 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:42.681 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:22:42.681 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:42.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:42.682 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:42.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:42.734 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2024-09-12T12:22:42.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:42.751 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo/bar) 2024-09-12T12:22:42.804 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:22:42.804 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:42.851 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:43.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8c76c640 1 Processor -- start 2024-09-12T12:22:43.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8c76c640 1 -- start start 2024-09-12T12:22:43.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8c76c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec840b1980 0x7fec840ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8c76c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 0x7fec840d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8c76c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec840cd6e0 con 0x7fec8406bcc0 2024-09-12T12:22:43.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8c76c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fec840d0fb0 con 0x7fec840b1980 2024-09-12T12:22:43.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8c76c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fec840cd9c0 con 0x7fec840b38b0 2024-09-12T12:22:43.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8ad09640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fec8406bcc0 0x7fec840cb890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40262/0 (socket says 172.21.15.3:40262) 2024-09-12T12:22:43.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8ad09640 1 -- 172.21.15.3:0/2175502696 learned_addr learned my addr 172.21.15.3:0/2175502696 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec8a508640 1 --2- 172.21.15.3:0/2175502696 >> v2:172.21.15.142:3300/0 conn(0x7fec840b1980 0x7fec840ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89d07640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 0x7fec840d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584707166 0 0) 0x7fec840cd6e0 con 0x7fec8406bcc0 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec6c003530 con 0x7fec8406bcc0 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 281836441 0 0) 0x7fec6c003530 con 0x7fec8406bcc0 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 msgr2=0x7fec840d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 0x7fec840d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 >> v2:172.21.15.142:3300/0 conn(0x7fec840b1980 msgr2=0x7fec840ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 --2- 172.21.15.3:0/2175502696 >> v2:172.21.15.142:3300/0 conn(0x7fec840b1980 0x7fec840ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.024+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec840cdca0 con 0x7fec8406bcc0 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fec74001a00 con 0x7fec8406bcc0 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 >> v1:172.21.15.3:6789/0 conn(0x7fec8406bcc0 legacy=0x7fec840cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 0x7fec6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec6c004d50 0x7fec6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec840d0fb0 con 0x7fec6c004d50 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec840cd9c0 con 0x7fec6c0039c0 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fec6c00b9d0 con 0x7fec840b38b0 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec6c004d50 0x7fec6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8a508640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89d07640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 0x7fec6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 -- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 msgr2=0x7fec6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec840b38b0 0x7fec6c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 -- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c0039c0 msgr2=0x7fec6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 -- 172.21.15.3:0/2175502696 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c003530 con 0x7fec6c004d50 2024-09-12T12:22:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec6c004d50 0x7fec6c0051d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a8b0 tx=0x7fec7401d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6890821aac5adea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7400a4d0 con 0x7fec6c004d50 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec740072d0 con 0x7fec6c004d50 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec89506640 1 -- 172.21.15.3:0/2175502696 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec740076c0 con 0x7fec6c004d50 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8c76c640 1 -- 172.21.15.3:0/2175502696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec6c004d50 msgr2=0x7fec6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8c76c640 1 --2- 172.21.15.3:0/2175502696 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec6c004d50 0x7fec6c0051d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a8b0 tx=0x7fec7401d3e0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8ad09640 1 -- 172.21.15.3:0/2175502696 reap_dead start 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8c76c640 1 -- 172.21.15.3:0/2175502696 shutdown_connections 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8c76c640 1 -- 172.21.15.3:0/2175502696 >> 172.21.15.3:0/2175502696 conn(0x7fec840b2d90 msgr2=0x7fec840b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8c76c640 1 -- 172.21.15.3:0/2175502696 shutdown_connections 2024-09-12T12:22:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.028+0000 7fec8c76c640 1 -- 172.21.15.3:0/2175502696 wait complete. 2024-09-12T12:22:43.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 Processor -- start 2024-09-12T12:22:43.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 -- start start 2024-09-12T12:22:43.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c004d50 0x7fec840c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840af4e0 0x7fec84156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec84156460 0x7fec8415f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec840b1d60 con 0x7fec840af4e0 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec840ce9e0 con 0x7fec84156460 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fec840b1ee0 con 0x7fec6c004d50 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840af4e0 0x7fec84156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840af4e0 0x7fec84156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47054/0 (socket says 172.21.15.3:47054) 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 -- 172.21.15.3:0/6386261 learned_addr learned my addr 172.21.15.3:0/6386261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8a508640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c004d50 0x7fec840c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8ad09640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec84156460 0x7fec8415f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 -- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c004d50 msgr2=0x7fec840c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c004d50 0x7fec840c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 -- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec84156460 msgr2=0x7fec8415f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec84156460 0x7fec8415f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 -- 172.21.15.3:0/6386261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec841574e0 con 0x7fec840af4e0 2024-09-12T12:22:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec89d07640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840af4e0 0x7fec84156c90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec78010890 tx=0x7fec78010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec82ffd640 1 -- 172.21.15.3:0/6386261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7800f040 con 0x7fec840af4e0 2024-09-12T12:22:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec82ffd640 1 -- 172.21.15.3:0/6386261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7800eb60 con 0x7fec840af4e0 2024-09-12T12:22:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec840b55c0 con 0x7fec840af4e0 2024-09-12T12:22:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec82ffd640 1 -- 172.21.15.3:0/6386261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7801c400 con 0x7fec840af4e0 2024-09-12T12:22:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec840b5b80 con 0x7fec840af4e0 2024-09-12T12:22:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.032+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec840ce4a0 con 0x7fec840af4e0 2024-09-12T12:22:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.036+0000 7fec82ffd640 1 -- 172.21.15.3:0/6386261 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fec7801c5a0 con 0x7fec840af4e0 2024-09-12T12:22:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.036+0000 7fec82ffd640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fec700678f0 0x7fec70069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.036+0000 7fec82ffd640 1 -- 172.21.15.3:0/6386261 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fec780862b0 con 0x7fec840af4e0 2024-09-12T12:22:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.036+0000 7fec8a508640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fec700678f0 0x7fec70069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.036+0000 7fec8a508640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fec700678f0 0x7fec70069db0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009b60 tx=0x7fec7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:43.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.040+0000 7fec82ffd640 1 -- 172.21.15.3:0/6386261 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fec78018030 con 0x7fec840af4e0 2024-09-12T12:22:43.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.212+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fec841613a0 con 0x7fec840af4e0 2024-09-12T12:22:43.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.212+0000 7fec82ffd640 1 -- 172.21.15.3:0/6386261 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7fec78056d60 con 0x7fec840af4e0 2024-09-12T12:22:43.217 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fec700678f0 msgr2=0x7fec70069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fec700678f0 0x7fec70069db0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009b60 tx=0x7fec7c008040 comp rx=0 tx=0).stop 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840af4e0 msgr2=0x7fec84156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840af4e0 0x7fec84156c90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec78010890 tx=0x7fec78010d50 comp rx=0 tx=0).stop 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 shutdown_connections 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fec700678f0 0x7fec70069db0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec84156460 0x7fec8415f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec840af4e0 0x7fec84156c90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 --2- 172.21.15.3:0/6386261 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec6c004d50 0x7fec840c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 >> 172.21.15.3:0/6386261 conn(0x7fec840b2d90 msgr2=0x7fec840750d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:43.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 shutdown_connections 2024-09-12T12:22:43.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.216+0000 7fec8c76c640 1 -- 172.21.15.3:0/6386261 wait complete. 2024-09-12T12:22:43.234 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:22:43.235 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:43.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f4360254640 1 Processor -- start 2024-09-12T12:22:43.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f4360254640 1 -- start start 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f4360254640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43580b5fc0 0x7f43580d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f4360254640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f43580cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f4360254640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43580d2d90 con 0x7f435815f620 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f4360254640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f43580d3070 con 0x7f43580b5fc0 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f4360254640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f43580d3350 con 0x7f43580ced00 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435dff0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43580b5fc0 0x7f43580d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435d7ef640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f435815f620 0x7f43580b63a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40264/0 (socket says 172.21.15.3:40264) 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435d7ef640 1 -- 172.21.15.3:0/1841753953 learned_addr learned my addr 172.21.15.3:0/1841753953 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:43.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435dff0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f43580b5fc0 0x7f43580d26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47070/0 (socket says 172.21.15.3:47070) 2024-09-12T12:22:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435cfee640 1 -- 172.21.15.3:0/1841753953 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1327890472 0 0) 0x7f43580d2d90 con 0x7f435815f620 2024-09-12T12:22:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435cfee640 1 -- 172.21.15.3:0/1841753953 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4340003530 con 0x7f435815f620 2024-09-12T12:22:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435e7f1640 1 --2- 172.21.15.3:0/1841753953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f43580cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435dff0640 1 -- 172.21.15.3:0/1841753953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 msgr2=0x7f43580cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435dff0640 1 --2- 172.21.15.3:0/1841753953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f43580cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435dff0640 1 -- 172.21.15.3:0/1841753953 >> v1:172.21.15.3:6789/0 conn(0x7f435815f620 legacy=0x7f43580b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.400+0000 7f435dff0640 1 -- 172.21.15.3:0/1841753953 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43580d3630 con 0x7f43580b5fc0 2024-09-12T12:22:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f435dff0640 1 --2- 172.21.15.3:0/1841753953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43580b5fc0 0x7f43580d26f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43480097c0 tx=0x7f434801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c02d75b2d078857 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f435cfee640 1 -- 172.21.15.3:0/1841753953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480b1070 con 0x7f43580b5fc0 2024-09-12T12:22:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f435cfee640 1 -- 172.21.15.3:0/1841753953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43480ac630 con 0x7f43580b5fc0 2024-09-12T12:22:43.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f435cfee640 1 -- 172.21.15.3:0/1841753953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480a8050 con 0x7f43580b5fc0 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 -- 172.21.15.3:0/1841753953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43580b5fc0 msgr2=0x7f43580d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 --2- 172.21.15.3:0/1841753953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43580b5fc0 0x7f43580d26f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f43480097c0 tx=0x7f434801c470 comp rx=0 tx=0).stop 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 -- 172.21.15.3:0/1841753953 shutdown_connections 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 --2- 172.21.15.3:0/1841753953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f43580cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 --2- 172.21.15.3:0/1841753953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f43580b5fc0 0x7f43580d26f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 -- 172.21.15.3:0/1841753953 >> 172.21.15.3:0/1841753953 conn(0x7f435808c000 msgr2=0x7f43580d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 -- 172.21.15.3:0/1841753953 shutdown_connections 2024-09-12T12:22:43.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 -- 172.21.15.3:0/1841753953 wait complete. 2024-09-12T12:22:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 Processor -- start 2024-09-12T12:22:43.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.404+0000 7f4360254640 1 -- start start 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43580b5fc0 0x7f435815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f435815b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f435815f620 0x7f4358159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f435816b5d0 con 0x7f435815f620 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f435816b450 con 0x7f43580b5fc0 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f435816b750 con 0x7f43580ced00 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f435815f620 0x7f4358159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f435815f620 0x7f4358159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47076/0 (socket says 172.21.15.3:47076) 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 -- 172.21.15.3:0/1560063665 learned_addr learned my addr 172.21.15.3:0/1560063665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:43.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435d7ef640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f435815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435dff0640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43580b5fc0 0x7f435815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 -- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 msgr2=0x7f435815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f435815b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 -- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43580b5fc0 msgr2=0x7f435815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43580b5fc0 0x7f435815ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 -- 172.21.15.3:0/1560063665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43580d4d10 con 0x7f435815f620 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435dff0640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43580b5fc0 0x7f435815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435e7f1640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f435815f620 0x7f4358159830 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f435000d800 tx=0x7f435000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f43567fc640 1 -- 172.21.15.3:0/1560063665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435000b6c0 con 0x7f435815f620 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f43567fc640 1 -- 172.21.15.3:0/1560063665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4350019070 con 0x7f435815f620 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43580cccd0 con 0x7f435815f620 2024-09-12T12:22:43.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f43567fc640 1 -- 172.21.15.3:0/1560063665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435000baa0 con 0x7f435815f620 2024-09-12T12:22:43.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43580cd270 con 0x7f435815f620 2024-09-12T12:22:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43580d26f0 con 0x7f435815f620 2024-09-12T12:22:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f43567fc640 1 -- 172.21.15.3:0/1560063665 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f435000bc40 con 0x7f435815f620 2024-09-12T12:22:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f43567fc640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f43440678c0 0x7f4344069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f43567fc640 1 -- 172.21.15.3:0/1560063665 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f4350086100 con 0x7f435815f620 2024-09-12T12:22:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.408+0000 7f435dff0640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f43440678c0 0x7f4344069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:43.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.412+0000 7f435dff0640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f43440678c0 0x7f4344069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f43480ae000 tx=0x7f4348008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:43.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.412+0000 7f43567fc640 1 -- 172.21.15.3:0/1560063665 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4350056c40 con 0x7f435815f620 2024-09-12T12:22:43.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.588+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43580d42b0 con 0x7f435815f620 2024-09-12T12:22:43.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.588+0000 7f43567fc640 1 -- 172.21.15.3:0/1560063665 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7f4350056540 con 0x7f435815f620 2024-09-12T12:22:43.592 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:22:43.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f43440678c0 msgr2=0x7f4344069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f43440678c0 0x7f4344069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f43480ae000 tx=0x7f4348008040 comp rx=0 tx=0).stop 2024-09-12T12:22:43.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f435815f620 msgr2=0x7f4358159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:43.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f435815f620 0x7f4358159830 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f435000d800 tx=0x7f435000dcc0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 shutdown_connections 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f43440678c0 0x7f4344069d80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f435815f620 0x7f4358159830 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43580ced00 0x7f435815b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 --2- 172.21.15.3:0/1560063665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43580b5fc0 0x7f435815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 >> 172.21.15.3:0/1560063665 conn(0x7f435808c000 msgr2=0x7f435806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 shutdown_connections 2024-09-12T12:22:43.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:43.592+0000 7f4360254640 1 -- 172.21.15.3:0/1560063665 wait complete. 2024-09-12T12:22:43.610 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:22:43.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:43.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:22:43.620 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[39971]: fuse finished with error 0 and tester_r 0 2024-09-12T12:22:43.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:22:46.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:22:46.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:22:49.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:22:49.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:49.723 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:22:49.731 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:49.731 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:49.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:49.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf10ecc640 1 Processor -- start 2024-09-12T12:22:49.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf10ecc640 1 -- start start 2024-09-12T12:22:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf10ecc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 0x7fdf0c0ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf10ecc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf0c15a9d0 0x7fdf0c071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf10ecc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf0c0b4980 con 0x7fdf0c0b4190 2024-09-12T12:22:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf10ecc640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdf0c0b46a0 con 0x7fdf0c15a9d0 2024-09-12T12:22:49.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf10ecc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c06de50 con 0x7fdf0c15a5f0 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf0ad9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdf0c0b4190 0x7fdf0c072720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56820/0 (socket says 172.21.15.3:56820) 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf0ad9d640 1 -- 172.21.15.3:0/264116685 learned_addr learned my addr 172.21.15.3:0/264116685 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf0a59c640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 0x7fdf0c0ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3828725048 0 0) 0x7fdf0c0b4980 con 0x7fdf0c0b4190 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf09d9b640 1 --2- 172.21.15.3:0/264116685 >> v2:172.21.15.142:3300/0 conn(0x7fdf0c15a9d0 0x7fdf0c071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.904+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdef4003530 con 0x7fdf0c0b4190 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1979613663 0 0) 0x7fdef4003530 con 0x7fdf0c0b4190 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 msgr2=0x7fdf0c0ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 0x7fdf0c0ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 >> v2:172.21.15.142:3300/0 conn(0x7fdf0c15a9d0 msgr2=0x7fdf0c071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 --2- 172.21.15.3:0/264116685 >> v2:172.21.15.142:3300/0 conn(0x7fdf0c15a9d0 0x7fdf0c071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c06e130 con 0x7fdf0c0b4190 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fdf04001a00 con 0x7fdf0c0b4190 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 >> v1:172.21.15.3:6789/0 conn(0x7fdf0c0b4190 legacy=0x7fdf0c072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef40039c0 0x7fdef4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef4004810 0x7fdef4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 0x7fdef40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c0b46a0 con 0x7fdef4004810 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c06de50 con 0x7fdef40039c0 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdef400b9d0 con 0x7fdf0c15a5f0 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf09d9b640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef4004810 0x7fdef4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0ad9d640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 0x7fdef40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0a59c640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef40039c0 0x7fdef4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf09d9b640 1 -- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 msgr2=0x7fdef40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf09d9b640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c15a5f0 0x7fdef40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf09d9b640 1 -- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef40039c0 msgr2=0x7fdef4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf09d9b640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef40039c0 0x7fdef4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf09d9b640 1 -- 172.21.15.3:0/264116685 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef4003530 con 0x7fdef4004810 2024-09-12T12:22:49.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf09d9b640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef4004810 0x7fdef4004c90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000aa60 tx=0x7fdf0001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7a18a873ebb8ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf000b2070 con 0x7fdef4004810 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf000a5ac0 con 0x7fdef4004810 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0959a640 1 -- 172.21.15.3:0/264116685 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000b040 con 0x7fdef4004810 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf10ecc640 1 -- 172.21.15.3:0/264116685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef4004810 msgr2=0x7fdef4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf10ecc640 1 --2- 172.21.15.3:0/264116685 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdef4004810 0x7fdef4004c90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000aa60 tx=0x7fdf0001cc70 comp rx=0 tx=0).stop 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf0ad9d640 1 -- 172.21.15.3:0/264116685 reap_dead start 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf10ecc640 1 -- 172.21.15.3:0/264116685 shutdown_connections 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf10ecc640 1 -- 172.21.15.3:0/264116685 >> 172.21.15.3:0/264116685 conn(0x7fdf0c06be90 msgr2=0x7fdf0c06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf10ecc640 1 -- 172.21.15.3:0/264116685 shutdown_connections 2024-09-12T12:22:49.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.908+0000 7fdf10ecc640 1 -- 172.21.15.3:0/264116685 wait complete. 2024-09-12T12:22:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 Processor -- start 2024-09-12T12:22:49.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 -- start start 2024-09-12T12:22:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef4004810 0x7fdf0c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0af210 0x7fdf0c0af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf0c1e3ed0 0x7fdf0c1e4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c072e30 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c0722e0 con 0x7fdef4004810 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c072fb0 con 0x7fdf0c0af210 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf0c1e3ed0 0x7fdf0c1e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf0c1e3ed0 0x7fdf0c1e4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54886/0 (socket says 172.21.15.3:54886) 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 -- 172.21.15.3:0/238164050 learned_addr learned my addr 172.21.15.3:0/238164050 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0a59c640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef4004810 0x7fdf0c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf09d9b640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0af210 0x7fdf0c0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 -- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0af210 msgr2=0x7fdf0c0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:49.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0af210 0x7fdf0c0af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 -- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef4004810 msgr2=0x7fdf0c152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef4004810 0x7fdf0c152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 -- 172.21.15.3:0/238164050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c1e4ab0 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf0ad9d640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf0c1e3ed0 0x7fdf0c1e4360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c06fd30 tx=0x7fdf04011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdefaffd640 1 -- 172.21.15.3:0/238164050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04010040 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdefaffd640 1 -- 172.21.15.3:0/238164050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf0401a070 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c1e4d90 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdefaffd640 1 -- 172.21.15.3:0/238164050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400fba0 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c1e5350 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.912+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c071da0 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.916+0000 7fdefaffd640 1 -- 172.21.15.3:0/238164050 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf0400fd40 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.916+0000 7fdefaffd640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdeec0678c0 0x7fdeec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.916+0000 7fdefaffd640 1 -- 172.21.15.3:0/238164050 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fdf04088ac0 con 0x7fdf0c1e3ed0 2024-09-12T12:22:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.916+0000 7fdf0a59c640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdeec0678c0 0x7fdeec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.916+0000 7fdf0a59c640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdeec0678c0 0x7fdeec069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c09a550 tx=0x7fdefc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:49.920+0000 7fdefaffd640 1 -- 172.21.15.3:0/238164050 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdf04058f10 con 0x7fdf0c1e3ed0 2024-09-12T12:22:50.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.092+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf0c0b3360 con 0x7fdf0c1e3ed0 2024-09-12T12:22:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.092+0000 7fdefaffd640 1 -- 172.21.15.3:0/238164050 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7fdf0405d210 con 0x7fdf0c1e3ed0 2024-09-12T12:22:50.096 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:22:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdeec0678c0 msgr2=0x7fdeec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdeec0678c0 0x7fdeec069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c09a550 tx=0x7fdefc008040 comp rx=0 tx=0).stop 2024-09-12T12:22:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf0c1e3ed0 msgr2=0x7fdf0c1e4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf0c1e3ed0 0x7fdf0c1e4360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c06fd30 tx=0x7fdf04011c20 comp rx=0 tx=0).stop 2024-09-12T12:22:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 shutdown_connections 2024-09-12T12:22:50.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdeec0678c0 0x7fdeec069d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf0c1e3ed0 0x7fdf0c1e4360 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf0c0af210 0x7fdf0c0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 --2- 172.21.15.3:0/238164050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdef4004810 0x7fdf0c152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 >> 172.21.15.3:0/238164050 conn(0x7fdf0c06be90 msgr2=0x7fdf0c1575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 shutdown_connections 2024-09-12T12:22:50.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.096+0000 7fdf10ecc640 1 -- 172.21.15.3:0/238164050 wait complete. 2024-09-12T12:22:50.114 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:50.114 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:50.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:50.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f53009fb640 1 Processor -- start 2024-09-12T12:22:50.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f53009fb640 1 -- start start 2024-09-12T12:22:50.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f53009fb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52fc159ea0 0x7f52fc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f53009fb640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52fc15a280 con 0x7f52fc0b4ca0 2024-09-12T12:22:50.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f53009fb640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52fc0b5280 con 0x7f52fc06d210 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f53009fb640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f52fc06d680 con 0x7f52fc159ea0 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52fad9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f52fc0b4ca0 0x7f52fc0cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56826/0 (socket says 172.21.15.3:56826) 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52fad9d640 1 -- 172.21.15.3:0/4138012968 learned_addr learned my addr 172.21.15.3:0/4138012968 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52fa59c640 1 --2- 172.21.15.3:0/4138012968 >> v2:172.21.15.3:3300/0 conn(0x7f52fc159ea0 0x7f52fc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52f959a640 1 -- 172.21.15.3:0/4138012968 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123998049 0 0) 0x7f52fc15a280 con 0x7f52fc0b4ca0 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52f959a640 1 -- 172.21.15.3:0/4138012968 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52e4003530 con 0x7f52fc0b4ca0 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52fa59c640 1 -- 172.21.15.3:0/4138012968 >> v1:172.21.15.142:6789/0 conn(0x7f52fc06d210 legacy=0x7f52fc0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52fa59c640 1 -- 172.21.15.3:0/4138012968 >> v1:172.21.15.3:6789/0 conn(0x7f52fc0b4ca0 legacy=0x7f52fc0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.240+0000 7f52fa59c640 1 -- 172.21.15.3:0/4138012968 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc0b5560 con 0x7f52fc159ea0 2024-09-12T12:22:50.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f52fa59c640 1 --2- 172.21.15.3:0/4138012968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc06ccd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f52f40097c0 tx=0x7f52f401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241e108dd329d9bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:50.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f52f959a640 1 -- 172.21.15.3:0/4138012968 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b0070 con 0x7f52fc159ea0 2024-09-12T12:22:50.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f52f959a640 1 -- 172.21.15.3:0/4138012968 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f40ab600 con 0x7f52fc159ea0 2024-09-12T12:22:50.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f52f959a640 1 -- 172.21.15.3:0/4138012968 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40ab9d0 con 0x7f52fc159ea0 2024-09-12T12:22:50.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- 172.21.15.3:0/4138012968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 msgr2=0x7f52fc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 --2- 172.21.15.3:0/4138012968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc06ccd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f52f40097c0 tx=0x7f52f401c4a0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- 172.21.15.3:0/4138012968 shutdown_connections 2024-09-12T12:22:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 --2- 172.21.15.3:0/4138012968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc06ccd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- 172.21.15.3:0/4138012968 >> 172.21.15.3:0/4138012968 conn(0x7f52fc072720 msgr2=0x7f52fc072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- 172.21.15.3:0/4138012968 shutdown_connections 2024-09-12T12:22:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- 172.21.15.3:0/4138012968 wait complete. 2024-09-12T12:22:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 Processor -- start 2024-09-12T12:22:50.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- start start 2024-09-12T12:22:50.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52fc06d210 0x7f52fc145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52fc0b4ca0 0x7f52fc158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc16a060 con 0x7f52fc159ea0 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc0d04f0 con 0x7f52fc0b4ca0 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f53009fb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f52fc16a1e0 con 0x7f52fc06d210 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f52fad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f52fad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54902/0 (socket says 172.21.15.3:54902) 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.244+0000 7f52fad9d640 1 -- 172.21.15.3:0/1858865827 learned_addr learned my addr 172.21.15.3:0/1858865827 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fa59c640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52fc06d210 0x7f52fc145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52f9d9b640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52fc0b4ca0 0x7f52fc158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fad9d640 1 -- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52fc06d210 msgr2=0x7f52fc145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fad9d640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52fc06d210 0x7f52fc145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fad9d640 1 -- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52fc0b4ca0 msgr2=0x7f52fc158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fad9d640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52fc0b4ca0 0x7f52fc158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fad9d640 1 -- 172.21.15.3:0/1858865827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc15fb90 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fa59c640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52fc06d210 0x7f52fc145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fad9d640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc157400 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00cb30 tx=0x7f52ec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52eaffd640 1 -- 172.21.15.3:0/1858865827 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec017070 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52eaffd640 1 -- 172.21.15.3:0/1858865827 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ec00aca0 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc156640 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52eaffd640 1 -- 172.21.15.3:0/1858865827 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec012600 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc156c00 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52fc06ccd0 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52eaffd640 1 -- 172.21.15.3:0/1858865827 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52ec01a070 con 0x7f52fc159ea0 2024-09-12T12:22:50.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52eaffd640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52dc0678c0 0x7f52dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52eaffd640 1 -- 172.21.15.3:0/1858865827 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f52ec0859d0 con 0x7f52fc159ea0 2024-09-12T12:22:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fa59c640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52dc0678c0 0x7f52dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.248+0000 7f52fa59c640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52dc0678c0 0x7f52dc069d80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52f40ad000 tx=0x7f52f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:50.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.252+0000 7f52eaffd640 1 -- 172.21.15.3:0/1858865827 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f52ec055ec0 con 0x7f52fc159ea0 2024-09-12T12:22:50.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.428+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52fc15abe0 con 0x7f52fc159ea0 2024-09-12T12:22:50.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.428+0000 7f52eaffd640 1 -- 172.21.15.3:0/1858865827 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7f52ec05a1c0 con 0x7f52fc159ea0 2024-09-12T12:22:50.432 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:22:50.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52dc0678c0 msgr2=0x7f52dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52dc0678c0 0x7f52dc069d80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52f40ad000 tx=0x7f52f4008040 comp rx=0 tx=0).stop 2024-09-12T12:22:50.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 msgr2=0x7f52fc157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc157400 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00cb30 tx=0x7f52ec00a060 comp rx=0 tx=0).stop 2024-09-12T12:22:50.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 shutdown_connections 2024-09-12T12:22:50.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52dc0678c0 0x7f52dc069d80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52fc159ea0 0x7f52fc157400 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52fc0b4ca0 0x7f52fc158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 --2- 172.21.15.3:0/1858865827 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52fc06d210 0x7f52fc145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 >> 172.21.15.3:0/1858865827 conn(0x7f52fc072720 msgr2=0x7f52fc155320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:50.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 shutdown_connections 2024-09-12T12:22:50.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.432+0000 7f53009fb640 1 -- 172.21.15.3:0/1858865827 wait complete. 2024-09-12T12:22:50.451 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:22:50.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:50.452 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:50.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[35897]: fuse finished with error 0 and tester_r 0 2024-09-12T12:22:50.465 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:22:50.554 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:22:50.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:50.554 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:22:50.560 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.badguy mds 'allow rw path=/foo/bar' osd 'allow rw' mon 'allow r' 2024-09-12T12:22:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.696+0000 7fd506392640 1 Processor -- start 2024-09-12T12:22:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.696+0000 7fd506392640 1 -- start start 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd50015b130 0x7fd5000b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd50006f6e0 0x7fd500072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd50015f0b0 con 0x7fd5000cc270 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd5000ae920 con 0x7fd50006f6e0 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd50015edd0 con 0x7fd50015b130 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4ff7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd50006f6e0 0x7fd500072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd50492f640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd5000cc270 0x7fd5000aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56832/0 (socket says 172.21.15.3:56832) 2024-09-12T12:22:50.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd50492f640 1 -- 172.21.15.3:0/2922541663 learned_addr learned my addr 172.21.15.3:0/2922541663 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4fffff640 1 --2- 172.21.15.3:0/2922541663 >> v2:172.21.15.142:3300/0 conn(0x7fd50015b130 0x7fd5000b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4ff7fe640 1 -- 172.21.15.3:0/2922541663 >> v2:172.21.15.142:3300/0 conn(0x7fd50015b130 msgr2=0x7fd5000b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4ff7fe640 1 --2- 172.21.15.3:0/2922541663 >> v2:172.21.15.142:3300/0 conn(0x7fd50015b130 0x7fd5000b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4ff7fe640 1 -- 172.21.15.3:0/2922541663 >> v1:172.21.15.3:6789/0 conn(0x7fd5000cc270 legacy=0x7fd5000aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4ff7fe640 1 -- 172.21.15.3:0/2922541663 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5000adff0 con 0x7fd50006f6e0 2024-09-12T12:22:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4feffd640 1 -- 172.21.15.3:0/2922541663 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2115201927 0 0) 0x7fd50015f0b0 con 0x7fd5000cc270 2024-09-12T12:22:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4ff7fe640 1 --2- 172.21.15.3:0/2922541663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500072f30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd50006fe00 tx=0x7fd4f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800805c99d4a179e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4feffd640 1 -- 172.21.15.3:0/2922541663 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40a7070 con 0x7fd50006f6e0 2024-09-12T12:22:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4feffd640 1 -- 172.21.15.3:0/2922541663 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f40b0070 con 0x7fd50006f6e0 2024-09-12T12:22:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd4feffd640 1 -- 172.21.15.3:0/2922541663 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40ab890 con 0x7fd50006f6e0 2024-09-12T12:22:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- 172.21.15.3:0/2922541663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 msgr2=0x7fd500072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 --2- 172.21.15.3:0/2922541663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500072f30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd50006fe00 tx=0x7fd4f401c450 comp rx=0 tx=0).stop 2024-09-12T12:22:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- 172.21.15.3:0/2922541663 shutdown_connections 2024-09-12T12:22:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 --2- 172.21.15.3:0/2922541663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500072f30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 --2- 172.21.15.3:0/2922541663 >> v2:172.21.15.142:3300/0 conn(0x7fd50015b130 0x7fd5000b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- 172.21.15.3:0/2922541663 >> 172.21.15.3:0/2922541663 conn(0x7fd5000b1d80 msgr2=0x7fd5000b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- 172.21.15.3:0/2922541663 shutdown_connections 2024-09-12T12:22:50.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.700+0000 7fd506392640 1 -- 172.21.15.3:0/2922541663 wait complete. 2024-09-12T12:22:50.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 Processor -- start 2024-09-12T12:22:50.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 -- start start 2024-09-12T12:22:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5000cc270 0x7fd50014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50015b130 0x7fd50009dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd50015b510 con 0x7fd50006f6e0 2024-09-12T12:22:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd500073880 con 0x7fd5000cc270 2024-09-12T12:22:50.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd50015b690 con 0x7fd50015b130 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54920/0 (socket says 172.21.15.3:54920) 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 -- 172.21.15.3:0/57913962 learned_addr learned my addr 172.21.15.3:0/57913962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4ff7fe640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5000cc270 0x7fd50014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd50492f640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50015b130 0x7fd50009dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 -- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50015b130 msgr2=0x7fd50009dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50015b130 0x7fd50009dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 -- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5000cc270 msgr2=0x7fd50014fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5000cc270 0x7fd50014fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 -- 172.21.15.3:0/57913962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50009a430 con 0x7fd50006f6e0 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4ff7fe640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5000cc270 0x7fd50014fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd50492f640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50015b130 0x7fd50009dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fffff640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500152ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000d5f0 tx=0x7fd4f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fcff9640 1 -- 172.21.15.3:0/57913962 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00125f0 con 0x7fd50006f6e0 2024-09-12T12:22:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fcff9640 1 -- 172.21.15.3:0/57913962 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f0019070 con 0x7fd50006f6e0 2024-09-12T12:22:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50009a710 con 0x7fd50006f6e0 2024-09-12T12:22:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd4fcff9640 1 -- 172.21.15.3:0/57913962 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00129b0 con 0x7fd50006f6e0 2024-09-12T12:22:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.704+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50009acd0 con 0x7fd50006f6e0 2024-09-12T12:22:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.708+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50015d860 con 0x7fd50006f6e0 2024-09-12T12:22:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.708+0000 7fd4fcff9640 1 -- 172.21.15.3:0/57913962 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd4f0012b50 con 0x7fd50006f6e0 2024-09-12T12:22:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.708+0000 7fd4fcff9640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4ec0678f0 0x7fd4ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.708+0000 7fd4fcff9640 1 -- 172.21.15.3:0/57913962 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fd4f0085fd0 con 0x7fd50006f6e0 2024-09-12T12:22:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.708+0000 7fd4ff7fe640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4ec0678f0 0x7fd4ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.708+0000 7fd4ff7fe640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4ec0678f0 0x7fd4ec069db0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40af000 tx=0x7fd4f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.712+0000 7fd4fcff9640 1 -- 172.21.15.3:0/57913962 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd4f0056ba0 con 0x7fd50006f6e0 2024-09-12T12:22:50.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.932+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.badguy", "caps": ["mds", "allow rw path=/foo/bar", "osd", "allow rw", "mon", "allow r"]} v 0) -- 0x7fd50006f360 con 0x7fd50006f6e0 2024-09-12T12:22:50.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.936+0000 7fd4fcff9640 1 -- 172.21.15.3:0/57913962 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.badguy", "caps": ["mds", "allow rw path=/foo/bar", "osd", "allow rw", "mon", "allow r"]}]=0 v16) ==== 173+0+64 (secure 0 0 0) 0x7fd4f00564a0 con 0x7fd50006f6e0 2024-09-12T12:22:50.940 INFO:teuthology.orchestra.run.smithi003.stdout:[client.badguy] 2024-09-12T12:22:50.940 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAa3eJmiaLANxAA6CEjP8cJQrqnH02M0tfzTg== 2024-09-12T12:22:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4ec0678f0 msgr2=0x7fd4ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4ec0678f0 0x7fd4ec069db0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40af000 tx=0x7fd4f4008040 comp rx=0 tx=0).stop 2024-09-12T12:22:50.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 msgr2=0x7fd500152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500152ea0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000d5f0 tx=0x7fd4f000dab0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 shutdown_connections 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4ec0678f0 0x7fd4ec069db0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd50015b130 0x7fd50009dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5000cc270 0x7fd50014fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 --2- 172.21.15.3:0/57913962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd50006f6e0 0x7fd500152ea0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 >> 172.21.15.3:0/57913962 conn(0x7fd5000b1d80 msgr2=0x7fd5000affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 shutdown_connections 2024-09-12T12:22:50.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:50.940+0000 7fd506392640 1 -- 172.21.15.3:0/57913962 wait complete. 2024-09-12T12:22:50.958 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:50.958 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.badguy.keyring 2024-09-12T12:22:50.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:22:50.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:22:51.018 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T12:22:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-12T12:22:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:22:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-12T12:22:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.018 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:22:51.019 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.020 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-12T12:22:51.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-12T12:22:51.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.023 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-12T12:22:51.024 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-12T12:22:51.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-12T12:22:51.026 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:22:51.027 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-12T12:22:51.028 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:22:51.029 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout:[client.badguy] 2024-09-12T12:22:51.030 INFO:teuthology.orchestra.run.smithi003.stdout: keyring = /etc/ceph/ceph.client.badguy.keyring 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-09-12T12:22:51.045 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.046 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-09-12T12:22:51.047 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:22:51.048 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-09-12T12:22:51.049 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.050 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-09-12T12:22:51.051 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:22:51.052 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-09-12T12:22:51.053 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:22:51.054 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on pool no app = False 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:22:51.055 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-09-12T12:22:51.056 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout:[client.badguy] 2024-09-12T12:22:51.057 INFO:teuthology.orchestra.run.smithi142.stdout: keyring = /etc/ceph/ceph.client.badguy.keyring 2024-09-12T12:22:51.058 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:22:51.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.196+0000 7ff72ed28640 1 Processor -- start 2024-09-12T12:22:51.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.196+0000 7ff72ed28640 1 -- start start 2024-09-12T12:22:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7280cd5c0 0x7ff7280b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7280ad1c0 0x7ff7280b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7280b4400 con 0x7ff72815c2e0 2024-09-12T12:22:51.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff7280b2d70 con 0x7ff7280cd5c0 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7280cb550 con 0x7ff7280ad1c0 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff727fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7280cd5c0 0x7ff7280b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72cac4640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff72815c2e0 0x7ff7280cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56848/0 (socket says 172.21.15.3:56848) 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72cac4640 1 -- 172.21.15.3:0/327259155 learned_addr learned my addr 172.21.15.3:0/327259155 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72d2c5640 1 --2- 172.21.15.3:0/327259155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7280ad1c0 0x7ff7280b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff727fff640 1 -- 172.21.15.3:0/327259155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7280ad1c0 msgr2=0x7ff7280b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff727fff640 1 --2- 172.21.15.3:0/327259155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7280ad1c0 0x7ff7280b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff727fff640 1 -- 172.21.15.3:0/327259155 >> v1:172.21.15.3:6789/0 conn(0x7ff72815c2e0 legacy=0x7ff7280cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff727fff640 1 -- 172.21.15.3:0/327259155 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7280b46e0 con 0x7ff7280cd5c0 2024-09-12T12:22:51.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72d2c5640 1 --2- 172.21.15.3:0/327259155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7280ad1c0 0x7ff7280b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:51.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff727fff640 1 --2- 172.21.15.3:0/327259155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280cd5c0 0x7ff7280b3d60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0097c0 tx=0x7ff71c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ac6096ad419aff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:51.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff7277fe640 1 -- 172.21.15.3:0/327259155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0b0070 con 0x7ff7280cd5c0 2024-09-12T12:22:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff7277fe640 1 -- 172.21.15.3:0/327259155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c0ab600 con 0x7ff7280cd5c0 2024-09-12T12:22:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff7277fe640 1 -- 172.21.15.3:0/327259155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0ab9d0 con 0x7ff7280cd5c0 2024-09-12T12:22:51.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 -- 172.21.15.3:0/327259155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280cd5c0 msgr2=0x7ff7280b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 --2- 172.21.15.3:0/327259155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280cd5c0 0x7ff7280b3d60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0097c0 tx=0x7ff71c01c4a0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 -- 172.21.15.3:0/327259155 shutdown_connections 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 --2- 172.21.15.3:0/327259155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7280ad1c0 0x7ff7280b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 --2- 172.21.15.3:0/327259155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280cd5c0 0x7ff7280b3d60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.200+0000 7ff72ed28640 1 -- 172.21.15.3:0/327259155 >> 172.21.15.3:0/327259155 conn(0x7ff728074e90 msgr2=0x7ff7280752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- 172.21.15.3:0/327259155 shutdown_connections 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- 172.21.15.3:0/327259155 wait complete. 2024-09-12T12:22:51.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 Processor -- start 2024-09-12T12:22:51.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- start start 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280ad1c0 0x7ff728070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7280cd5c0 0x7ff728147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff72815c2e0 0x7ff72814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff72806e760 con 0x7ff7280ad1c0 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff72806e5e0 con 0x7ff7280cd5c0 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff72806e8e0 con 0x7ff72815c2e0 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280ad1c0 0x7ff728070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280ad1c0 0x7ff728070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54940/0 (socket says 172.21.15.3:54940) 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 -- 172.21.15.3:0/3648566338 learned_addr learned my addr 172.21.15.3:0/3648566338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72d2c5640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff72815c2e0 0x7ff72814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff727fff640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7280cd5c0 0x7ff728147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 -- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff72815c2e0 msgr2=0x7ff72814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff72815c2e0 0x7ff72814df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 -- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7280cd5c0 msgr2=0x7ff728147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7280cd5c0 0x7ff728147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 -- 172.21.15.3:0/3648566338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7280713e0 con 0x7ff7280ad1c0 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72d2c5640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff72815c2e0 0x7ff72814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff727fff640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7280cd5c0 0x7ff728147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72cac4640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280ad1c0 0x7ff728070ea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff71800c370 tx=0x7ff71800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff7257fa640 1 -- 172.21.15.3:0/3648566338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718017070 con 0x7ff7280ad1c0 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff7257fa640 1 -- 172.21.15.3:0/3648566338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff718005cf0 con 0x7ff7280ad1c0 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff728147bd0 con 0x7ff7280ad1c0 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff7257fa640 1 -- 172.21.15.3:0/3648566338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7180072f0 con 0x7ff7280ad1c0 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7280ce1f0 con 0x7ff7280ad1c0 2024-09-12T12:22:51.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.204+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7280b2830 con 0x7ff7280ad1c0 2024-09-12T12:22:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.208+0000 7ff7257fa640 1 -- 172.21.15.3:0/3648566338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff71801a070 con 0x7ff7280ad1c0 2024-09-12T12:22:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.208+0000 7ff7257fa640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7140678f0 0x7ff714069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.208+0000 7ff7257fa640 1 -- 172.21.15.3:0/3648566338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7ff7180853a0 con 0x7ff7280ad1c0 2024-09-12T12:22:51.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.208+0000 7ff727fff640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7140678f0 0x7ff714069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.208+0000 7ff727fff640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7140678f0 0x7ff714069db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0ad000 tx=0x7ff71c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:51.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.212+0000 7ff7257fa640 1 -- 172.21.15.3:0/3648566338 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff718021020 con 0x7ff7280ad1c0 2024-09-12T12:22:51.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.412+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff72815f190 con 0x7ff7280ad1c0 2024-09-12T12:22:51.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.412+0000 7ff7257fa640 1 -- 172.21.15.3:0/3648566338 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7ff718055fd0 con 0x7ff7280ad1c0 2024-09-12T12:22:51.416 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:22:51.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7140678f0 msgr2=0x7ff714069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7140678f0 0x7ff714069db0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0ad000 tx=0x7ff71c008040 comp rx=0 tx=0).stop 2024-09-12T12:22:51.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280ad1c0 msgr2=0x7ff728070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280ad1c0 0x7ff728070ea0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff71800c370 tx=0x7ff71800c830 comp rx=0 tx=0).stop 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 shutdown_connections 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7140678f0 0x7ff714069db0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff72815c2e0 0x7ff72814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7280cd5c0 0x7ff728147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 --2- 172.21.15.3:0/3648566338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7280ad1c0 0x7ff728070ea0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 >> 172.21.15.3:0/3648566338 conn(0x7ff728074e90 msgr2=0x7ff7280b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 shutdown_connections 2024-09-12T12:22:51.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:51.416+0000 7ff72ed28640 1 -- 172.21.15.3:0/3648566338 wait complete. 2024-09-12T12:22:51.435 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:22:51.435 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:22:51.435 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:22:51.436 INFO:tasks.cephfs.mount:self.client.name = client.badguy 2024-09-12T12:22:51.436 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:51.436 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:22:51.436 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /foo/bar 2024-09-12T12:22:51.436 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:22:51.436 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:22:51.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:51.436 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:22:51.443 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:22:51.443 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:22:51.443 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:22:51.443 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.443 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:22:51.444 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42313sec preferred_lft 42313sec 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:51.445 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:51.446 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:22:51.446 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.446 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:22:51.446 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:51.446 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:22:51.446 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:22:51.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:51.446 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:22:51.500 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:22:51.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:51.501 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:22:51.553 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:22:51.553 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:22:51.553 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:22:51.553 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:22:51.553 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:22:51.554 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:22:51.554 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:22:51.554 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:22:51.554 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:22:51.554 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:22:51.554 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:22:51.554 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:22:51.554 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:22:51.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:22:51.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:22:51.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:22:51.555 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:22:51.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:22:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:22:51.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:22:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:51.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:51.680 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:22:51.686 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:22:51.686 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:22:51.687 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:22:51.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:51.687 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:22:51.741 INFO:tasks.cephfs.fuse_mount:Client client.badguy config is {} 2024-09-12T12:22:51.741 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:51.741 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:51.788 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:22:51.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:22:51.789 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:51.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:22:51.894 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:51.942 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:22:51.943 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:22:51.944 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:22:51.945 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:22:51.945 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:22:51.945 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:22:51.945 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:51.945 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:51.945 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:51.945 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:51.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:51.946 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:22:51.993 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:22:51.993 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id badguy --client_mountpoint=/foo/bar) 2024-09-12T12:22:52.037 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:22:52.052 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:22:52.101 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:52.102 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:22:52.103 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:52.104 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:52.104 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:52.104 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:52.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:52.105 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:22:52.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:22:52.116+0000 7f9f1e0a3580 -1 init, newargv = 0x55d1e4bec330 newargc=15 2024-09-12T12:22:52.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36002]: starting ceph client 2024-09-12T12:22:52.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36002]: starting fuse 2024-09-12T12:22:53.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:22:53.241 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:22:53.291 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:22:53.292 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:53.294 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:22:53.340 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:22:53.340 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:22:53.341 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.badguy.*.asok" 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:22:53.342 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:22:53.343 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:22:53.343 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:22:53.343 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:22:53.343 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:22:53.343 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:22:53.343 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.badguy")) 2024-09-12T12:22:53.343 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:22:53.465 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.badguy.36002.asok 2024-09-12T12:22:53.469 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:22:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:22:53.477 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.badguy.36002.asok 2024-09-12T12:22:53.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:53.477 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.badguy.36002.asok status 2024-09-12T12:22:53.631 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:22:53.631 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:22:53.631 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:22:53.631 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:22:53.631 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "badguy", 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "36002", 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/foo/bar" 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 4970, 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 4970 2024-09-12T12:22:53.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1543956586 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:22:53.633 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1543956586 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.4970 192.168.0.1:0/1543956586", 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/1543956586", 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 36, 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 55, 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:22:53.634 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:22:53.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:53.645 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:22:53.645 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:53.651 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:22:53.651 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:53.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:53.651 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:53.709 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:53.709 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:53.757 DEBUG:tasks.cephfs.mount:Creating test file 2024-09-12 12:22:53.756972 badguy 2024-09-12T12:22:53.757 DEBUG:teuthology.orchestra.run.smithi142:> touch '/home/ubuntu/cephtest/mnt.1/2024-09-12 12:22:53.756972 badguy' 2024-09-12T12:22:53.813 DEBUG:tasks.cephfs.mount:Deleting test file 2024-09-12 12:22:53.756972 badguy 2024-09-12T12:22:53.813 DEBUG:teuthology.orchestra.run.smithi142:> rm -f '/home/ubuntu/cephtest/mnt.1/2024-09-12 12:22:53.756972 badguy' 2024-09-12T12:22:53.864 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:53.864 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:53.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:54.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbffd54640 1 Processor -- start 2024-09-12T12:22:54.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbffd54640 1 -- start start 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbffd54640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf815b2d0 0x7fcbf80b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbffd54640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcbf806e780 0x7fcbf80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbffd54640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf806ecf0 con 0x7fcbf80cd4e0 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbffd54640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf80b1e70 con 0x7fcbf806e780 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbffd54640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf80b2710 con 0x7fcbf815b2d0 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.048+0000 7fcbfd2ef640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcbf806e780 0x7fcbf80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfd2ef640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcbf806e780 0x7fcbf80ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54944/0 (socket says 172.21.15.3:54944) 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfd2ef640 1 -- 172.21.15.3:0/2990499072 learned_addr learned my addr 172.21.15.3:0/2990499072 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfdaf0640 1 --2- 172.21.15.3:0/2990499072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf815b2d0 0x7fcbf80b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfd2ef640 1 -- 172.21.15.3:0/2990499072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf815b2d0 msgr2=0x7fcbf80b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:54.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfd2ef640 1 --2- 172.21.15.3:0/2990499072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf815b2d0 0x7fcbf80b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfd2ef640 1 -- 172.21.15.3:0/2990499072 >> v1:172.21.15.142:6789/0 conn(0x7fcbf80cd4e0 legacy=0x7fcbf80b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfd2ef640 1 -- 172.21.15.3:0/2990499072 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf80b20c0 con 0x7fcbf806e780 2024-09-12T12:22:54.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfd2ef640 1 --2- 172.21.15.3:0/2990499072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf806e780 0x7fcbf80ccfa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800bb10 tx=0x7fcbe801eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a67a14949922cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:54.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfcaee640 1 -- 172.21.15.3:0/2990499072 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80b4070 con 0x7fcbf806e780 2024-09-12T12:22:54.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfcaee640 1 -- 172.21.15.3:0/2990499072 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe80a7900 con 0x7fcbf806e780 2024-09-12T12:22:54.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbfcaee640 1 -- 172.21.15.3:0/2990499072 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80a7cd0 con 0x7fcbf806e780 2024-09-12T12:22:54.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 -- 172.21.15.3:0/2990499072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf806e780 msgr2=0x7fcbf80ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 --2- 172.21.15.3:0/2990499072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf806e780 0x7fcbf80ccfa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800bb10 tx=0x7fcbe801eb10 comp rx=0 tx=0).stop 2024-09-12T12:22:54.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 -- 172.21.15.3:0/2990499072 shutdown_connections 2024-09-12T12:22:54.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 --2- 172.21.15.3:0/2990499072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf806e780 0x7fcbf80ccfa0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 --2- 172.21.15.3:0/2990499072 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf815b2d0 0x7fcbf80b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 -- 172.21.15.3:0/2990499072 >> 172.21.15.3:0/2990499072 conn(0x7fcbf8074e90 msgr2=0x7fcbf80752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:54.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 -- 172.21.15.3:0/2990499072 shutdown_connections 2024-09-12T12:22:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.052+0000 7fcbffd54640 1 -- 172.21.15.3:0/2990499072 wait complete. 2024-09-12T12:22:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 Processor -- start 2024-09-12T12:22:54.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 -- start start 2024-09-12T12:22:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf806e780 0x7fcbf8154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf80cd4e0 0x7fcbf8153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf815b2d0 0x7fcbf8145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf806d140 con 0x7fcbf815b2d0 2024-09-12T12:22:54.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf806cfc0 con 0x7fcbf80cd4e0 2024-09-12T12:22:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf806d2c0 con 0x7fcbf806e780 2024-09-12T12:22:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf815b2d0 0x7fcbf8145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf815b2d0 0x7fcbf8145600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54954/0 (socket says 172.21.15.3:54954) 2024-09-12T12:22:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 -- 172.21.15.3:0/524057656 learned_addr learned my addr 172.21.15.3:0/524057656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfdaf0640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf806e780 0x7fcbf8154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfd2ef640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf80cd4e0 0x7fcbf8153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 -- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf806e780 msgr2=0x7fcbf8154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf806e780 0x7fcbf8154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 -- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf80cd4e0 msgr2=0x7fcbf8153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf80cd4e0 0x7fcbf8153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 -- 172.21.15.3:0/524057656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf8153720 con 0x7fcbf815b2d0 2024-09-12T12:22:54.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbfe2f1640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf815b2d0 0x7fcbf8145600 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000c710 tx=0x7fcbf000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbee7fc640 1 -- 172.21.15.3:0/524057656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0016020 con 0x7fcbf815b2d0 2024-09-12T12:22:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf8153a00 con 0x7fcbf815b2d0 2024-09-12T12:22:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbee7fc640 1 -- 172.21.15.3:0/524057656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf0006d60 con 0x7fcbf815b2d0 2024-09-12T12:22:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbee7fc640 1 -- 172.21.15.3:0/524057656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0003bc0 con 0x7fcbf815b2d0 2024-09-12T12:22:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf8153fc0 con 0x7fcbf815b2d0 2024-09-12T12:22:54.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.056+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf80ccfa0 con 0x7fcbf815b2d0 2024-09-12T12:22:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.060+0000 7fcbee7fc640 1 -- 172.21.15.3:0/524057656 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcbf000a850 con 0x7fcbf815b2d0 2024-09-12T12:22:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.060+0000 7fcbee7fc640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcbdc0678c0 0x7fcbdc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.060+0000 7fcbee7fc640 1 -- 172.21.15.3:0/524057656 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fcbf0085800 con 0x7fcbf815b2d0 2024-09-12T12:22:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.060+0000 7fcbfdaf0640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcbdc0678c0 0x7fcbdc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.060+0000 7fcbfdaf0640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcbdc0678c0 0x7fcbdc069d80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40079c0 tx=0x7fcbf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:54.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.064+0000 7fcbee7fc640 1 -- 172.21.15.3:0/524057656 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcbf0020020 con 0x7fcbf815b2d0 2024-09-12T12:22:54.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.236+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcbf80d0410 con 0x7fcbf815b2d0 2024-09-12T12:22:54.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.236+0000 7fcbee7fc640 1 -- 172.21.15.3:0/524057656 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7fcbf000ce20 con 0x7fcbf815b2d0 2024-09-12T12:22:54.241 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:22:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcbdc0678c0 msgr2=0x7fcbdc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcbdc0678c0 0x7fcbdc069d80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40079c0 tx=0x7fcbf4008040 comp rx=0 tx=0).stop 2024-09-12T12:22:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf815b2d0 msgr2=0x7fcbf8145600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf815b2d0 0x7fcbf8145600 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000c710 tx=0x7fcbf000cbd0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 shutdown_connections 2024-09-12T12:22:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcbdc0678c0 0x7fcbdc069d80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcbf815b2d0 0x7fcbf8145600 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbf80cd4e0 0x7fcbf8153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 --2- 172.21.15.3:0/524057656 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcbf806e780 0x7fcbf8154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 >> 172.21.15.3:0/524057656 conn(0x7fcbf8074e90 msgr2=0x7fcbf80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 shutdown_connections 2024-09-12T12:22:54.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.240+0000 7fcbffd54640 1 -- 172.21.15.3:0/524057656 wait complete. 2024-09-12T12:22:54.259 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:22:54.259 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:22:54.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:22:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5e0055640 1 Processor -- start 2024-09-12T12:22:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5e0055640 1 -- start start 2024-09-12T12:22:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5e0055640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5d8074cc0 0x7fb5d80b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5e0055640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d80b1230 con 0x7fb5d815c750 2024-09-12T12:22:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5e0055640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d80b0f90 con 0x7fb5d80af5a0 2024-09-12T12:22:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5e0055640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d80b3de0 con 0x7fb5d8074cc0 2024-09-12T12:22:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5dd5f0640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb5d815c750 0x7fb5d80750a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56854/0 (socket says 172.21.15.3:56854) 2024-09-12T12:22:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5dddf1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5d8074cc0 0x7fb5d80b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5dd5f0640 1 -- 172.21.15.3:0/2606772860 learned_addr learned my addr 172.21.15.3:0/2606772860 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:22:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5dddf1640 1 -- 172.21.15.3:0/2606772860 >> v1:172.21.15.142:6789/0 conn(0x7fb5d80af5a0 legacy=0x7fb5d80af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5dddf1640 1 -- 172.21.15.3:0/2606772860 >> v1:172.21.15.3:6789/0 conn(0x7fb5d815c750 legacy=0x7fb5d80750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5dddf1640 1 -- 172.21.15.3:0/2606772860 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d80b3b40 con 0x7fb5d8074cc0 2024-09-12T12:22:54.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.376+0000 7fb5dcdef640 1 -- 172.21.15.3:0/2606772860 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 79483676 0 0) 0x7fb5d80b1230 con 0x7fb5d815c750 2024-09-12T12:22:54.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5dddf1640 1 --2- 172.21.15.3:0/2606772860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d8074cc0 0x7fb5d80b2340 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00097c0 tx=0x7fb5d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7640d30a5abd2214 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5dcdef640 1 -- 172.21.15.3:0/2606772860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00b0070 con 0x7fb5d8074cc0 2024-09-12T12:22:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5dcdef640 1 -- 172.21.15.3:0/2606772860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d00ab600 con 0x7fb5d8074cc0 2024-09-12T12:22:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5dcdef640 1 -- 172.21.15.3:0/2606772860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00ab9d0 con 0x7fb5d8074cc0 2024-09-12T12:22:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- 172.21.15.3:0/2606772860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d8074cc0 msgr2=0x7fb5d80b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 --2- 172.21.15.3:0/2606772860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d8074cc0 0x7fb5d80b2340 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00097c0 tx=0x7fb5d001c460 comp rx=0 tx=0).stop 2024-09-12T12:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- 172.21.15.3:0/2606772860 shutdown_connections 2024-09-12T12:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 --2- 172.21.15.3:0/2606772860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d8074cc0 0x7fb5d80b2340 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- 172.21.15.3:0/2606772860 >> 172.21.15.3:0/2606772860 conn(0x7fb5d80b2ef0 msgr2=0x7fb5d80b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- 172.21.15.3:0/2606772860 shutdown_connections 2024-09-12T12:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- 172.21.15.3:0/2606772860 wait complete. 2024-09-12T12:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 Processor -- start 2024-09-12T12:22:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- start start 2024-09-12T12:22:54.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d8074cc0 0x7fb5d809a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d80af5a0 0x7fb5d809bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d815c750 0x7fb5d80b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d815cb80 con 0x7fb5d815c750 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d80af100 con 0x7fb5d80af5a0 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5e0055640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d8151170 con 0x7fb5d8074cc0 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d815c750 0x7fb5d80b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d815c750 0x7fb5d80b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54978/0 (socket says 172.21.15.3:54978) 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 -- 172.21.15.3:0/506669382 learned_addr learned my addr 172.21.15.3:0/506669382 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5dddf1640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d8074cc0 0x7fb5d809a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5dd5f0640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d80af5a0 0x7fb5d809bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 -- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d8074cc0 msgr2=0x7fb5d809a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d8074cc0 0x7fb5d809a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 -- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d80af5a0 msgr2=0x7fb5d809bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d80af5a0 0x7fb5d809bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.384+0000 7fb5de5f2640 1 -- 172.21.15.3:0/506669382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d8151450 con 0x7fb5d815c750 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5de5f2640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d815c750 0x7fb5d80b5a20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c780 tx=0x7fb5c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5d67fc640 1 -- 172.21.15.3:0/506669382 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8004b70 con 0x7fb5d815c750 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5d67fc640 1 -- 172.21.15.3:0/506669382 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c8004d10 con 0x7fb5d815c750 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d8151730 con 0x7fb5d815c750 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5d67fc640 1 -- 172.21.15.3:0/506669382 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c800aa90 con 0x7fb5d815c750 2024-09-12T12:22:54.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d8151cf0 con 0x7fb5d815c750 2024-09-12T12:22:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d8151ef0 con 0x7fb5d815c750 2024-09-12T12:22:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5d67fc640 1 -- 172.21.15.3:0/506669382 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5c800ac30 con 0x7fb5d815c750 2024-09-12T12:22:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5d67fc640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678c0 0x7fb5c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5d67fc640 1 -- 172.21.15.3:0/506669382 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fb5c8085ea0 con 0x7fb5d815c750 2024-09-12T12:22:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5dddf1640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678c0 0x7fb5c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:54.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.388+0000 7fb5dddf1640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678c0 0x7fb5c4069d80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00ad000 tx=0x7fb5d00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:54.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.392+0000 7fb5d67fc640 1 -- 172.21.15.3:0/506669382 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb5c8020070 con 0x7fb5d815c750 2024-09-12T12:22:54.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.564+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5d806ca00 con 0x7fb5d815c750 2024-09-12T12:22:54.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5d67fc640 1 -- 172.21.15.3:0/506669382 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7fb5c80562f0 con 0x7fb5d815c750 2024-09-12T12:22:54.569 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:22:54.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678c0 msgr2=0x7fb5c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678c0 0x7fb5c4069d80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00ad000 tx=0x7fb5d00a6000 comp rx=0 tx=0).stop 2024-09-12T12:22:54.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d815c750 msgr2=0x7fb5d80b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d815c750 0x7fb5d80b5a20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c780 tx=0x7fb5c800cc40 comp rx=0 tx=0).stop 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 shutdown_connections 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678c0 0x7fb5c4069d80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d815c750 0x7fb5d80b5a20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d80af5a0 0x7fb5d809bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 --2- 172.21.15.3:0/506669382 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d8074cc0 0x7fb5d809a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 >> 172.21.15.3:0/506669382 conn(0x7fb5d80b2ef0 msgr2=0x7fb5d815a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 shutdown_connections 2024-09-12T12:22:54.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:22:54.568+0000 7fb5e0055640 1 -- 172.21.15.3:0/506669382 wait complete. 2024-09-12T12:22:54.588 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:22:54.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:22:54.588 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:22:54.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36002]: fuse finished with error 0 and tester_r 0 2024-09-12T12:22:54.603 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:00.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:23:00.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:00.705 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:23:00.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:23:00.716 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:23:00.762 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T12:23:00.762 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-12T12:23:00.762 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:23:00.762 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-12T12:23:00.762 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.762 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:23:00.763 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:23:00.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.765 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-12T12:23:00.766 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-12T12:23:00.767 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:23:00.768 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.769 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:23:00.770 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-12T12:23:00.771 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-12T12:23:00.772 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-12T12:23:00.773 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-12T12:23:00.774 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:23:00.774 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:23:00.774 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:23:00.774 INFO:teuthology.orchestra.run.smithi003.stdout:[client.badguy] 2024-09-12T12:23:00.774 INFO:teuthology.orchestra.run.smithi003.stdout: keyring = /etc/ceph/ceph.client.badguy.keyring 2024-09-12T12:23:00.774 INFO:teuthology.orchestra.run.smithi003.stdout: client_metadata = root=/baz 2024-09-12T12:23:00.790 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-09-12T12:23:00.790 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-09-12T12:23:00.790 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:23:00.790 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-09-12T12:23:00.790 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.791 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:23:00.792 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:23:00.793 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.794 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.795 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-09-12T12:23:00.796 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:00.797 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-09-12T12:23:00.798 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:23:00.799 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on pool no app = False 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-09-12T12:23:00.800 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-09-12T12:23:00.801 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout:[client.badguy] 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: keyring = /etc/ceph/ceph.client.badguy.keyring 2024-09-12T12:23:00.802 INFO:teuthology.orchestra.run.smithi142.stdout: client_metadata = root=/baz 2024-09-12T12:23:00.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-12T12:23:00.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:23:00.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.948+0000 7f8712466640 1 Processor -- start 2024-09-12T12:23:00.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.948+0000 7f8712466640 1 -- start start 2024-09-12T12:23:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8712466640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f870c15e7f0 0x7f870c0c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8712466640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870c15ebd0 con 0x7f870c0c84e0 2024-09-12T12:23:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8712466640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f870c070e40 con 0x7f870c0705f0 2024-09-12T12:23:00.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8712466640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f870c06c690 con 0x7f870c15e7f0 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8710a03640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f870c0c84e0 0x7f870c1682f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55064/0 (socket says 172.21.15.3:55064) 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8710a03640 1 -- 172.21.15.3:0/451927440 learned_addr learned my addr 172.21.15.3:0/451927440 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870bfff640 1 --2- 172.21.15.3:0/451927440 >> v2:172.21.15.142:3300/0 conn(0x7f870c15e7f0 0x7f870c0c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242670438 0 0) 0x7f870c15ebd0 con 0x7f870c0c84e0 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86ec003530 con 0x7f870c0c84e0 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216146183 0 0) 0x7f86ec003530 con 0x7f870c0c84e0 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 >> v2:172.21.15.142:3300/0 conn(0x7f870c15e7f0 msgr2=0x7f870c0c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 --2- 172.21.15.3:0/451927440 >> v2:172.21.15.142:3300/0 conn(0x7f870c15e7f0 0x7f870c0c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 >> v1:172.21.15.142:6789/0 conn(0x7f870c0705f0 legacy=0x7f870c0c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c06bd60 con 0x7f870c0c84e0 2024-09-12T12:23:00.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f86fc001680 con 0x7f870c0c84e0 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 >> v1:172.21.15.3:6789/0 conn(0x7f870c0c84e0 legacy=0x7f870c1682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039c0 0x7f86ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f86ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86ec0051d0 0x7f86ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f870c06c690 con 0x7f86ec004810 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86ec00c170 con 0x7f86ec0039c0 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f86ec00c370 con 0x7f86ec0051d0 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870b7fe640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f86ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870bfff640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039c0 0x7f86ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870b7fe640 1 -- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86ec0051d0 msgr2=0x7f86ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8710a03640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86ec0051d0 0x7f86ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870b7fe640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f86ec0051d0 0x7f86ec00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870b7fe640 1 -- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039c0 msgr2=0x7f86ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:00.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870b7fe640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86ec0039c0 0x7f86ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f870b7fe640 1 -- 172.21.15.3:0/451927440 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ec003530 con 0x7f86ec004810 2024-09-12T12:23:00.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.952+0000 7f8710a03640 1 -- 172.21.15.3:0/451927440 reap_dead start 2024-09-12T12:23:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f870b7fe640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f86ec004c90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001780 tx=0x7f86f401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb0a8a4c75e4149 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40013a0 con 0x7f86ec004810 2024-09-12T12:23:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86f40052f0 con 0x7f86ec004810 2024-09-12T12:23:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f870affd640 1 -- 172.21.15.3:0/451927440 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4003840 con 0x7f86ec004810 2024-09-12T12:23:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- 172.21.15.3:0/451927440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 msgr2=0x7f86ec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f86ec004c90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001780 tx=0x7f86f401c3d0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- 172.21.15.3:0/451927440 shutdown_connections 2024-09-12T12:23:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 --2- 172.21.15.3:0/451927440 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f86ec004c90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- 172.21.15.3:0/451927440 >> 172.21.15.3:0/451927440 conn(0x7f870c0b41e0 msgr2=0x7f870c0b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- 172.21.15.3:0/451927440 shutdown_connections 2024-09-12T12:23:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- 172.21.15.3:0/451927440 wait complete. 2024-09-12T12:23:00.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 Processor -- start 2024-09-12T12:23:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- start start 2024-09-12T12:23:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f870c1f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870c155960 0x7f870c1f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f870c1f8580 0x7f870c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f870c070b50 con 0x7f86ec004810 2024-09-12T12:23:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f870c0709d0 con 0x7f870c1f8580 2024-09-12T12:23:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f8712466640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f870c06d310 con 0x7f870c155960 2024-09-12T12:23:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f870bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f870c1f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f870bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f870c1f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51620/0 (socket says 172.21.15.3:51620) 2024-09-12T12:23:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.956+0000 7f870bfff640 1 -- 172.21.15.3:0/1333027484 learned_addr learned my addr 172.21.15.3:0/1333027484 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f870b7fe640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870c155960 0x7f870c1f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8710a03640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f870c1f8580 0x7f870c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f870bfff640 1 -- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870c155960 msgr2=0x7f870c1f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f870bfff640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870c155960 0x7f870c1f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f870bfff640 1 -- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f870c1f8580 msgr2=0x7f870c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f870bfff640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f870c1f8580 0x7f870c0d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f870bfff640 1 -- 172.21.15.3:0/1333027484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c1f8a90 con 0x7f86ec004810 2024-09-12T12:23:00.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8710a03640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f870c1f8580 0x7f870c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f870bfff640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f870c1f9200 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8704009f50 tx=0x7f87040115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8708ff9640 1 -- 172.21.15.3:0/1333027484 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870401a020 con 0x7f86ec004810 2024-09-12T12:23:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8708ff9640 1 -- 172.21.15.3:0/1333027484 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8704016470 con 0x7f86ec004810 2024-09-12T12:23:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c1f8d70 con 0x7f86ec004810 2024-09-12T12:23:00.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8708ff9640 1 -- 172.21.15.3:0/1333027484 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704016860 con 0x7f86ec004810 2024-09-12T12:23:00.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1534f0 con 0x7f86ec004810 2024-09-12T12:23:00.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.960+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870c06d6a0 con 0x7f86ec004810 2024-09-12T12:23:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.964+0000 7f8708ff9640 1 -- 172.21.15.3:0/1333027484 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f870401d020 con 0x7f86ec004810 2024-09-12T12:23:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.964+0000 7f8708ff9640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86f00678c0 0x7f86f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.964+0000 7f8708ff9640 1 -- 172.21.15.3:0/1333027484 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f8704087040 con 0x7f86ec004810 2024-09-12T12:23:00.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.964+0000 7f870b7fe640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86f00678c0 0x7f86f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.964+0000 7f870b7fe640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86f00678c0 0x7f86f0069d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001750 tx=0x7f86f40011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:00.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f8708ff9640 1 -- 172.21.15.3:0/1333027484 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8704024030 con 0x7f86ec004810 2024-09-12T12:23:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79febc1640 1 Processor -- start 2024-09-12T12:23:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79febc1640 1 -- start start 2024-09-12T12:23:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79febc1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 0x7f79f80b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79febc1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f806eef0 con 0x7f79f815cf90 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79febc1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f806f1d0 con 0x7f79f815b340 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79febc1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79f806f4b0 con 0x7f79f8072850 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79fd15e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f79f815cf90 0x7f79f80b49e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55076/0 (socket says 172.21.15.3:55076) 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79fd15e640 1 -- 172.21.15.3:0/2044594911 learned_addr learned my addr 172.21.15.3:0/2044594911 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f7fff640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 0x7f79f80b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1677051428 0 0) 0x7f79f806eef0 con 0x7f79f815cf90 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79e8003530 con 0x7f79f815cf90 2024-09-12T12:23:00.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1466196698 0 0) 0x7f79e8003530 con 0x7f79f815cf90 2024-09-12T12:23:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 msgr2=0x7f79f80b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 0x7f79f80b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 >> v1:172.21.15.142:6789/0 conn(0x7f79f815b340 legacy=0x7f79f8072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f815ba60 con 0x7f79f815cf90 2024-09-12T12:23:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f79f0001a00 con 0x7f79f815cf90 2024-09-12T12:23:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 >> v1:172.21.15.3:6789/0 conn(0x7f79f815cf90 legacy=0x7f79f80b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e80039f0 0x7f79e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e8004810 0x7f79e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 0x7f79e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79f806f4b0 con 0x7f79e8004810 2024-09-12T12:23:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79e800bab0 con 0x7f79e80039f0 2024-09-12T12:23:00.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79e800bcb0 con 0x7f79f8072850 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f7fff640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e8004810 0x7f79e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79fd15e640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 0x7f79e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79fc95d640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e80039f0 0x7f79e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f7fff640 1 -- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 msgr2=0x7f79e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f7fff640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79f8072850 0x7f79e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f7fff640 1 -- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e80039f0 msgr2=0x7f79e8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f7fff640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79e80039f0 0x7f79e8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.972+0000 7f79f7fff640 1 -- 172.21.15.3:0/2044594911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e8003530 con 0x7f79e8004810 2024-09-12T12:23:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79f7fff640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e8004810 0x7f79e8004c90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009ab0 tx=0x7f79e401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace239043c9445ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40b2070 con 0x7f79e8004810 2024-09-12T12:23:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e40a5510 con 0x7f79e8004810 2024-09-12T12:23:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79f77fe640 1 -- 172.21.15.3:0/2044594911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40096b0 con 0x7f79e8004810 2024-09-12T12:23:00.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 -- 172.21.15.3:0/2044594911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e8004810 msgr2=0x7f79e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 --2- 172.21.15.3:0/2044594911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79e8004810 0x7f79e8004c90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f79e4009ab0 tx=0x7f79e401ca30 comp rx=0 tx=0).stop 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79fd15e640 1 -- 172.21.15.3:0/2044594911 reap_dead start 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 -- 172.21.15.3:0/2044594911 shutdown_connections 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 -- 172.21.15.3:0/2044594911 >> 172.21.15.3:0/2044594911 conn(0x7f79f80750b0 msgr2=0x7f79f806bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 -- 172.21.15.3:0/2044594911 shutdown_connections 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 -- 172.21.15.3:0/2044594911 wait complete. 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 Processor -- start 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.976+0000 7f79febc1640 1 -- start start 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e8004810 0x7f79f8154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f8145710 0x7f79f8144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8157da0 0x7f79f815be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f79f815b720 con 0x7f79f8157da0 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79f806f790 con 0x7f79f8145710 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f79f806fe10 con 0x7f79e8004810 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8157da0 0x7f79f815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8157da0 0x7f79f815be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51638/0 (socket says 172.21.15.3:51638) 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 -- 172.21.15.3:0/4237564289 learned_addr learned my addr 172.21.15.3:0/4237564289 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fc95d640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e8004810 0x7f79f8154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f7fff640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f8145710 0x7f79f8144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 -- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e8004810 msgr2=0x7f79f8154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f79e8004810 0x7f79f8154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 -- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f8145710 msgr2=0x7f79f8144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79f8145710 0x7f79f8144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 -- 172.21.15.3:0/4237564289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f8145420 con 0x7f79f8157da0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fd15e640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f79f8157da0 0x7f79f815be80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f79f00012f0 tx=0x7f79f0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f57fa640 1 -- 172.21.15.3:0/4237564289 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f0010040 con 0x7f79f8157da0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f57fa640 1 -- 172.21.15.3:0/4237564289 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f001a070 con 0x7f79f8157da0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 -- 172.21.15.3:0/4237564289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f8155630 con 0x7f79f8157da0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f57fa640 1 -- 172.21.15.3:0/4237564289 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f000fb80 con 0x7f79f8157da0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 -- 172.21.15.3:0/4237564289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f8153710 con 0x7f79f8157da0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 -- 172.21.15.3:0/4237564289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f79f806eb50 con 0x7f79f8157da0 2024-09-12T12:23:00.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f57fa640 1 -- 172.21.15.3:0/4237564289 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f79f000fd20 con 0x7f79f8157da0 2024-09-12T12:23:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f57fa640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79dc0678f0 0x7f79dc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f57fa640 1 -- 172.21.15.3:0/4237564289 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f79f0089510 con 0x7f79f8157da0 2024-09-12T12:23:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79f57fa640 1 -- 172.21.15.3:0/4237564289 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7f79f00899a0 con 0x7f79f8157da0 2024-09-12T12:23:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79febc1640 1 -- 172.21.15.3:0/4237564289 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f79f8145b90 con 0x7f79f8157da0 2024-09-12T12:23:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.980+0000 7f79fc95d640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79dc0678f0 0x7f79dc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:00.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:00.984+0000 7f79fc95d640 1 --2- 172.21.15.3:0/4237564289 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f79dc0678f0 0x7f79dc069db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f79ec007fd0 tx=0x7f79ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:01.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.176+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f870c1f9740 con 0x7f86ec004810 2024-09-12T12:23:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.176+0000 7f8708ff9640 1 -- 172.21.15.3:0/1333027484 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f87040576a0 con 0x7f86ec004810 2024-09-12T12:23:01.181 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86f00678c0 msgr2=0x7f86f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86f00678c0 0x7f86f0069d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f86f4001750 tx=0x7f86f40011a0 comp rx=0 tx=0).stop 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 msgr2=0x7f870c1f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f870c1f9200 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8704009f50 tx=0x7f87040115a0 comp rx=0 tx=0).stop 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 shutdown_connections 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f86f00678c0 0x7f86f0069d80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f870c1f8580 0x7f870c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f870c155960 0x7f870c1f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 --2- 172.21.15.3:0/1333027484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86ec004810 0x7f870c1f9200 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 >> 172.21.15.3:0/1333027484 conn(0x7f870c0b41e0 msgr2=0x7f870c0b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 shutdown_connections 2024-09-12T12:23:01.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:01.180+0000 7f8712466640 1 -- 172.21.15.3:0/1333027484 wait complete. 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:self.client.name = client.badguy 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /foo/bar 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:23:01.201 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:23:01.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:01.202 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:23:01.208 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:23:01.208 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:23:01.208 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:23:01.208 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.208 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42303sec preferred_lft 42303sec 2024-09-12T12:23:01.209 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:01.210 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:23:01.211 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.211 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:23:01.211 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:01.211 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:23:01.211 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:01.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:01.211 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:23:01.268 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:23:01.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:01.269 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:23:01.320 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:23:01.320 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:23:01.320 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:23:01.321 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:23:01.321 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:23:01.322 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:23:01.322 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:23:01.322 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:23:01.322 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:23:01.322 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:23:01.414 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:23:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:23:01.444 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:23:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:23:01.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:01.452 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:23:01.458 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:23:01.458 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:23:01.458 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:23:01.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:01.458 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:23:01.516 INFO:tasks.cephfs.fuse_mount:Client client.badguy config is {} 2024-09-12T12:23:01.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:23:01.517 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:01.564 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:23:01.564 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:23:01.564 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:01.612 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:23:01.673 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:01.722 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:23:01.722 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:23:01.722 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:23:01.722 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:23:01.723 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:01.724 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:01.725 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:01.725 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:01.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:01.726 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:23:01.772 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:23:01.772 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id badguy --client_mountpoint=/foo/bar) 2024-09-12T12:23:01.816 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:23:01.831 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:01.876 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:23:01.876 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:23:01.877 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:23:01.878 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:23:01.879 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:01.879 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:01.879 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:01.879 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:01.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:01.880 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:23:01.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:23:01.892+0000 7f4402880580 -1 init, newargv = 0x560256a37630 newargc=15 2024-09-12T12:23:01.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36103]: starting ceph client 2024-09-12T12:23:01.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:23:01.896+0000 7f43e7fff640 -1 client.5380 mds.0 rejected us (non-allowable root '/baz') 2024-09-12T12:23:01.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36103]: ceph mount failed with (1) Operation not permitted 2024-09-12T12:23:02.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:daemon-helper: command failed with exit status 1 2024-09-12T12:23:02.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:02.968+0000 7f79f57fa640 1 -- 172.21.15.3:0/4237564289 <== mon.0 v2:172.21.15.3:3300/0 7 ==== log(1 entries from seq 1 at 2024-09-12T12:23:01.896198+0000) ==== 260+0+0 (secure 0 0 0) 0x7f79f0087b50 con 0x7f79f8157da0 2024-09-12T12:23:02.972 INFO:teuthology.orchestra.run.smithi003.stdout: cluster: 2024-09-12T12:23:02.972 INFO:teuthology.orchestra.run.smithi003.stdout: id: 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:23:02.972 INFO:teuthology.orchestra.run.smithi003.stdout: health: HEALTH_OK 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: services: 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: mon: 3 daemons, quorum a,b,c (age 3m) 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: mgr: x(active, since 3m), standbys: y 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: mds: 1/1 daemons up, 3 standby 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: data: 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: volumes: 1/1 healthy 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: pools: 3 pools, 97 pgs 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: objects: 25 objects, 597 KiB 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: usage: 226 MiB used, 720 GiB / 720 GiB avail 2024-09-12T12:23:02.973 INFO:teuthology.orchestra.run.smithi003.stdout: pgs: 97 active+clean 2024-09-12T12:23:02.974 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:02.974 INFO:teuthology.orchestra.run.smithi003.stdout: io: 2024-09-12T12:23:02.974 INFO:teuthology.orchestra.run.smithi003.stdout: client: 85 B/s rd, 2.0 KiB/s wr, 0 op/s rd, 1 op/s wr 2024-09-12T12:23:02.974 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:02.974 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:02.974 INFO:teuthology.orchestra.run.smithi003.stdout:2024-09-12T12:23:01.896198+0000 mds.c [WRN] client session with non-allowable root '/baz' denied (client.5380 192.168.0.1:0/523779649) 2024-09-12T12:23:02.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:23:03.014 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:23:03.063 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:23:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:03.065 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:03.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:03.066 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:23:03.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:03.112 INFO:tasks.cephfs.fuse_mount:mount command failed. 2024-09-12T12:23:03.213 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:23:03.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:03.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:03.301 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:23:03.302 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:03.302 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:03.302 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:03.307 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:03.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:23:03.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:03.307 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:23:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-12T12:23:03.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:03.359 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:03.359 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:03.359 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:03.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:03.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:23:03.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:03.366 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:23:03.413 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:03.413 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:03.413 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:03.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:03.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:23:03.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:03.419 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T12:23:03.467 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-12T12:23:03.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:03.468 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:03.468 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:03.468 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:03.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:03.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:23:03.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:03.474 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T12:23:03.520 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-12T12:23:03.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:03.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:03.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef88a0640 1 Processor -- start 2024-09-12T12:23:03.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef88a0640 1 -- start start 2024-09-12T12:23:03.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef88a0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6ef415e900 0x7f6ef40d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef88a0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 0x7f6ef40d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef88a0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef40b7950 con 0x7f6ef40d1cb0 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef88a0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef40d2120 con 0x7f6ef415e900 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef88a0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef40b7c80 con 0x7f6ef40cfc40 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef2d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6ef40d1cb0 0x7f6ef40d2b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55078/0 (socket says 172.21.15.3:55078) 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef2d9d640 1 -- 172.21.15.3:0/2560694516 learned_addr learned my addr 172.21.15.3:0/2560694516 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 0x7f6ef40d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef259c640 1 --2- 172.21.15.3:0/2560694516 >> v2:172.21.15.142:3300/0 conn(0x7f6ef415e900 0x7f6ef40d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134455184 0 0) 0x7f6ef40b7950 con 0x7f6ef40d1cb0 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6edc003530 con 0x7f6ef40d1cb0 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3701393907 0 0) 0x7f6edc003530 con 0x7f6ef40d1cb0 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 msgr2=0x7f6ef40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 0x7f6ef40d1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 >> v2:172.21.15.142:3300/0 conn(0x7f6ef415e900 msgr2=0x7f6ef40d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 --2- 172.21.15.3:0/2560694516 >> v2:172.21.15.142:3300/0 conn(0x7f6ef415e900 0x7f6ef40d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef415ece0 con 0x7f6ef40d1cb0 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef259c640 1 --2- 172.21.15.3:0/2560694516 >> v2:172.21.15.142:3300/0 conn(0x7f6ef415e900 0x7f6ef40d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f6eec001a00 con 0x7f6ef40d1cb0 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 >> v1:172.21.15.3:6789/0 conn(0x7f6ef40d1cb0 legacy=0x7f6ef40d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:03.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 0x7f6edc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6edc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004d50 0x7f6edc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40d2120 con 0x7f6edc003f40 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40b7c80 con 0x7f6edc004d50 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6edc00ba20 con 0x7f6ef40cfc40 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6edc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef259c640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 0x7f6edc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef2d9d640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004d50 0x7f6edc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef1d9b640 1 -- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 msgr2=0x7f6edc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cfc40 0x7f6edc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef1d9b640 1 -- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004d50 msgr2=0x7f6edc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:03.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004d50 0x7f6edc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef1d9b640 1 -- 172.21.15.3:0/2560694516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc003530 con 0x7f6edc003f40 2024-09-12T12:23:03.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.656+0000 7f6ef2d9d640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6edc004d50 0x7f6edc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6edc004810 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009af0 tx=0x7f6ee801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb01fe0ab3d2f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80b2070 con 0x7f6edc003f40 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee80a5510 con 0x7f6edc003f40 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef159a640 1 -- 172.21.15.3:0/2560694516 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8009690 con 0x7f6edc003f40 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 -- 172.21.15.3:0/2560694516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 msgr2=0x7f6edc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 --2- 172.21.15.3:0/2560694516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6edc004810 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009af0 tx=0x7f6ee801ca30 comp rx=0 tx=0).stop 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef2d9d640 1 -- 172.21.15.3:0/2560694516 reap_dead start 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 -- 172.21.15.3:0/2560694516 shutdown_connections 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 -- 172.21.15.3:0/2560694516 >> 172.21.15.3:0/2560694516 conn(0x7f6ef406c0c0 msgr2=0x7f6ef406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 -- 172.21.15.3:0/2560694516 shutdown_connections 2024-09-12T12:23:03.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 -- 172.21.15.3:0/2560694516 wait complete. 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 Processor -- start 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.660+0000 7f6ef88a0640 1 -- start start 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6ef415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4160ba0 0x7f6ef40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cd5e0 0x7f6ef40cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40d4bf0 con 0x7f6edc003f40 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40d4a70 con 0x7f6ef4160ba0 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef4169430 con 0x7f6ef40cd5e0 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6ef415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6ef415c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51656/0 (socket says 172.21.15.3:51656) 2024-09-12T12:23:03.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 -- 172.21.15.3:0/583262439 learned_addr learned my addr 172.21.15.3:0/583262439 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef2d9d640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cd5e0 0x7f6ef40cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4160ba0 0x7f6ef40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 -- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cd5e0 msgr2=0x7f6ef40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cd5e0 0x7f6ef40cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 -- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4160ba0 msgr2=0x7f6ef40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4160ba0 0x7f6ef40cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 -- 172.21.15.3:0/583262439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef415fb00 con 0x7f6edc003f40 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4160ba0 0x7f6ef40cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef259c640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6ef415c8e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011900 tx=0x7f6ee4011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ee2ffd640 1 -- 172.21.15.3:0/583262439 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400fbd0 con 0x7f6edc003f40 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ee2ffd640 1 -- 172.21.15.3:0/583262439 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee400fd70 con 0x7f6edc003f40 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef415fde0 con 0x7f6edc003f40 2024-09-12T12:23:03.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ee2ffd640 1 -- 172.21.15.3:0/583262439 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4015660 con 0x7f6edc003f40 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef41603a0 con 0x7f6edc003f40 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef40d4530 con 0x7f6edc003f40 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ee2ffd640 1 -- 172.21.15.3:0/583262439 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6ee4015800 con 0x7f6edc003f40 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ee2ffd640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6ed4067910 0x7f6ed4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ee2ffd640 1 -- 172.21.15.3:0/583262439 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f6ee4086760 con 0x7f6edc003f40 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.664+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6ed4067910 0x7f6ed4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.668+0000 7f6ef1d9b640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6ed4067910 0x7f6ed4069dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80af000 tx=0x7f6ee80046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:03.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.668+0000 7f6ee2ffd640 1 -- 172.21.15.3:0/583262439 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6ee4019030 con 0x7f6edc003f40 2024-09-12T12:23:03.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.868+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef41691f0 con 0x7f6edc003f40 2024-09-12T12:23:03.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.868+0000 7f6ee2ffd640 1 -- 172.21.15.3:0/583262439 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7f6ee4057380 con 0x7f6edc003f40 2024-09-12T12:23:03.872 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:03.872 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"btime":"2024-09-12T12:22:34:602545+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/256387515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":256387515},{"type":"v1","addr":"172.21.15.142:6835","nonce":256387515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2534794519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2534794519},{"type":"v1","addr":"172.21.15.142:6837","nonce":2534794519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2676380928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2676380928},{"type":"v1","addr":"172.21.15.3:6833","nonce":2676380928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:22:30.937780+0000","modified":"2024-09-12T12:22:33.984511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4007447206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4007447206},{"type":"v1","addr":"172.21.15.3:6835","nonce":4007447206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5655,"qdb_cluster":[5655]},"id":4}]} 2024-09-12T12:23:03.873 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-12T12:23:03.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6ed4067910 msgr2=0x7f6ed4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6ed4067910 0x7f6ed4069dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80af000 tx=0x7f6ee80046d0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 msgr2=0x7f6ef415c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6ef415c8e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011900 tx=0x7f6ee4011dc0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 shutdown_connections 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6ed4067910 0x7f6ed4069dd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6ef40cd5e0 0x7f6ef40cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6ef4160ba0 0x7f6ef40cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 --2- 172.21.15.3:0/583262439 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6edc003f40 0x7f6ef415c8e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 >> 172.21.15.3:0/583262439 conn(0x7f6ef406c0c0 msgr2=0x7f6ef4167fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 shutdown_connections 2024-09-12T12:23:03.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:03.872+0000 7f6ef88a0640 1 -- 172.21.15.3:0/583262439 wait complete. 2024-09-12T12:23:03.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:04.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee16c3a640 1 Processor -- start 2024-09-12T12:23:04.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee16c3a640 1 -- start start 2024-09-12T12:23:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee16c3a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee1015fb70 0x7fee100b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee16c3a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee100b25d0 0x7fee100b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee16c3a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee100b3bc0 con 0x7fee1015ff50 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee16c3a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fee100afe80 con 0x7fee1015fb70 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee16c3a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee100b0440 con 0x7fee100b25d0 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee149d6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee1015fb70 0x7fee100b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee0ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee1015ff50 0x7fee100b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55090/0 (socket says 172.21.15.3:55090) 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee0ffff640 1 -- 172.21.15.3:0/209323733 learned_addr learned my addr 172.21.15.3:0/209323733 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee151d7640 1 --2- 172.21.15.3:0/209323733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee100b25d0 0x7fee100b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee149d6640 1 -- 172.21.15.3:0/209323733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee100b25d0 msgr2=0x7fee100b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee149d6640 1 --2- 172.21.15.3:0/209323733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee100b25d0 0x7fee100b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee149d6640 1 -- 172.21.15.3:0/209323733 >> v1:172.21.15.3:6789/0 conn(0x7fee1015ff50 legacy=0x7fee100b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee149d6640 1 -- 172.21.15.3:0/209323733 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee100b3ea0 con 0x7fee1015fb70 2024-09-12T12:23:04.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee0f7fe640 1 -- 172.21.15.3:0/209323733 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2508595703 0 0) 0x7fee100b3bc0 con 0x7fee1015ff50 2024-09-12T12:23:04.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.016+0000 7fee149d6640 1 --2- 172.21.15.3:0/209323733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1015fb70 0x7fee100b2ff0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fee000097c0 tx=0x7fee0001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c01ce3d050f3303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee0f7fe640 1 -- 172.21.15.3:0/209323733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000b1070 con 0x7fee1015fb70 2024-09-12T12:23:04.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee0f7fe640 1 -- 172.21.15.3:0/209323733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee000ac600 con 0x7fee1015fb70 2024-09-12T12:23:04.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee0f7fe640 1 -- 172.21.15.3:0/209323733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000ac9d0 con 0x7fee1015fb70 2024-09-12T12:23:04.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- 172.21.15.3:0/209323733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1015fb70 msgr2=0x7fee100b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 --2- 172.21.15.3:0/209323733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1015fb70 0x7fee100b2ff0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fee000097c0 tx=0x7fee0001c470 comp rx=0 tx=0).stop 2024-09-12T12:23:04.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- 172.21.15.3:0/209323733 shutdown_connections 2024-09-12T12:23:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 --2- 172.21.15.3:0/209323733 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee100b25d0 0x7fee100b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 --2- 172.21.15.3:0/209323733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee1015fb70 0x7fee100b2ff0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- 172.21.15.3:0/209323733 >> 172.21.15.3:0/209323733 conn(0x7fee100b4200 msgr2=0x7fee100b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- 172.21.15.3:0/209323733 shutdown_connections 2024-09-12T12:23:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- 172.21.15.3:0/209323733 wait complete. 2024-09-12T12:23:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 Processor -- start 2024-09-12T12:23:04.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- start start 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee100b25d0 0x7fee1014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee1015fb70 0x7fee100cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1015ff50 0x7fee10157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee100b19b0 con 0x7fee100b25d0 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee100b1830 con 0x7fee1015ff50 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee16c3a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee100b1b30 con 0x7fee1015fb70 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee100b25d0 0x7fee1014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee100b25d0 0x7fee1014ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51680/0 (socket says 172.21.15.3:51680) 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 -- 172.21.15.3:0/2540258791 learned_addr learned my addr 172.21.15.3:0/2540258791 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:04.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee0ffff640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee1015fb70 0x7fee100cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 -- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee1015fb70 msgr2=0x7fee100cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee1015fb70 0x7fee100cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee151d7640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1015ff50 0x7fee10157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 -- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1015ff50 msgr2=0x7fee10157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1015ff50 0x7fee10157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 -- 172.21.15.3:0/2540258791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee100c6390 con 0x7fee100b25d0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee149d6640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee100b25d0 0x7fee1014ee10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee0001ab20 tx=0x7fee0001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.020+0000 7fee0d7fa640 1 -- 172.21.15.3:0/2540258791 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000b1040 con 0x7fee100b25d0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee0d7fa640 1 -- 172.21.15.3:0/2540258791 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee000bd070 con 0x7fee100b25d0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee100c6670 con 0x7fee100b25d0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee0d7fa640 1 -- 172.21.15.3:0/2540258791 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000b7660 con 0x7fee100b25d0 2024-09-12T12:23:04.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee100c6c30 con 0x7fee100b25d0 2024-09-12T12:23:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee100b3bc0 con 0x7fee100b25d0 2024-09-12T12:23:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee0d7fa640 1 -- 172.21.15.3:0/2540258791 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fee000b7800 con 0x7fee100b25d0 2024-09-12T12:23:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee0d7fa640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedfc0678c0 0x7fedfc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee0d7fa640 1 -- 172.21.15.3:0/2540258791 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fee001200f0 con 0x7fee100b25d0 2024-09-12T12:23:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee0ffff640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedfc0678c0 0x7fedfc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.024+0000 7fee0ffff640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedfc0678c0 0x7fedfc069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee100d0dd0 tx=0x7fee04007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.028+0000 7fee0d7fa640 1 -- 172.21.15.3:0/2540258791 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fee00016090 con 0x7fee100b25d0 2024-09-12T12:23:04.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee100d0b20 con 0x7fee100b25d0 2024-09-12T12:23:04.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee0d7fa640 1 -- 172.21.15.3:0/2540258791 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7fee000f0d50 con 0x7fee100b25d0 2024-09-12T12:23:04.231 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:04.232 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"btime":"2024-09-12T12:22:34:602545+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/256387515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":256387515},{"type":"v1","addr":"172.21.15.142:6835","nonce":256387515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2534794519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2534794519},{"type":"v1","addr":"172.21.15.142:6837","nonce":2534794519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2676380928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2676380928},{"type":"v1","addr":"172.21.15.3:6833","nonce":2676380928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:22:30.937780+0000","modified":"2024-09-12T12:22:33.984511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4007447206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4007447206},{"type":"v1","addr":"172.21.15.3:6835","nonce":4007447206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5655,"qdb_cluster":[5655]},"id":4}]} 2024-09-12T12:23:04.232 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-12T12:23:04.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedfc0678c0 msgr2=0x7fedfc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedfc0678c0 0x7fedfc069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fee100d0dd0 tx=0x7fee04007c10 comp rx=0 tx=0).stop 2024-09-12T12:23:04.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee100b25d0 msgr2=0x7fee1014ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee100b25d0 0x7fee1014ee10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fee0001ab20 tx=0x7fee0001acb0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 shutdown_connections 2024-09-12T12:23:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fedfc0678c0 0x7fedfc069d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee1015ff50 0x7fee10157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee1015fb70 0x7fee100cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 --2- 172.21.15.3:0/2540258791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee100b25d0 0x7fee1014ee10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 >> 172.21.15.3:0/2540258791 conn(0x7fee100b4200 msgr2=0x7fee100b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:04.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 shutdown_connections 2024-09-12T12:23:04.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.228+0000 7fee16c3a640 1 -- 172.21.15.3:0/2540258791 wait complete. 2024-09-12T12:23:04.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e440d0640 1 Processor -- start 2024-09-12T12:23:04.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e440d0640 1 -- start start 2024-09-12T12:23:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e440d0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e3c15fb70 0x7f4e3c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e440d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b25d0 0x7f4e3c0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e440d0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3c0b3bc0 con 0x7f4e3c15ff50 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e440d0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3c0afe80 con 0x7f4e3c15fb70 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e440d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c0b0440 con 0x7f4e3c0b25d0 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4e3c15ff50 0x7f4e3c0b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55094/0 (socket says 172.21.15.3:55094) 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 -- 172.21.15.3:0/4245144374 learned_addr learned my addr 172.21.15.3:0/4245144374 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4266d640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b25d0 0x7f4e3c0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916073807 0 0) 0x7f4e3c0b3bc0 con 0x7f4e3c15ff50 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e41e6c640 1 --2- 172.21.15.3:0/4245144374 >> v2:172.21.15.142:3300/0 conn(0x7f4e3c15fb70 0x7f4e3c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e28003530 con 0x7f4e3c15ff50 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 30049216 0 0) 0x7f4e28003530 con 0x7f4e3c15ff50 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b25d0 msgr2=0x7f4e3c0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b25d0 0x7f4e3c0b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 >> v2:172.21.15.142:3300/0 conn(0x7f4e3c15fb70 msgr2=0x7f4e3c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 --2- 172.21.15.3:0/4245144374 >> v2:172.21.15.142:3300/0 conn(0x7f4e3c15fb70 0x7f4e3c0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c0b3ea0 con 0x7f4e3c15ff50 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f4e2c001680 con 0x7f4e3c15ff50 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 >> v1:172.21.15.3:6789/0 conn(0x7f4e3c15ff50 legacy=0x7f4e3c0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e280039c0 0x7f4e28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e280051d0 0x7f4e2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0afe80 con 0x7f4e28004810 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0b0440 con 0x7f4e280039c0 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2800c010 con 0x7f4e280051d0 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e41e6c640 1 --2- 172.21.15.3:0/4245144374 >> v2:172.21.15.142:3300/0 conn(0x7f4e3c15fb70 0x7f4e3c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4266d640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b25d0 0x7f4e3c0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:04.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 -- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e280051d0 msgr2=0x7f4e2800b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T12:23:04.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e280051d0 0x7f4e2800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 -- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e280039c0 msgr2=0x7f4e28003de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T12:23:04.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e280039c0 0x7f4e28003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.360+0000 7f4e4166b640 1 -- 172.21.15.3:0/4245144374 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e28003530 con 0x7f4e28004810 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e4166b640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e28004c90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00c240 tx=0x7f4e2c01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feb34b875fa0600 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c007d60 con 0x7f4e28004810 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2c005b00 con 0x7f4e28004810 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e40e6a640 1 -- 172.21.15.3:0/4245144374 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c007070 con 0x7f4e28004810 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- 172.21.15.3:0/4245144374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 msgr2=0x7f4e28004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 --2- 172.21.15.3:0/4245144374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e28004c90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00c240 tx=0x7f4e2c01f3f0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e4266d640 1 -- 172.21.15.3:0/4245144374 reap_dead start 2024-09-12T12:23:04.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- 172.21.15.3:0/4245144374 shutdown_connections 2024-09-12T12:23:04.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- 172.21.15.3:0/4245144374 >> 172.21.15.3:0/4245144374 conn(0x7f4e3c0b4200 msgr2=0x7f4e3c0b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:04.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- 172.21.15.3:0/4245144374 shutdown_connections 2024-09-12T12:23:04.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- 172.21.15.3:0/4245144374 wait complete. 2024-09-12T12:23:04.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 Processor -- start 2024-09-12T12:23:04.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- start start 2024-09-12T12:23:04.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e3c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e3c1534b0 0x7f4e3c0b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b7bf0 0x7f4e3c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0d1580 con 0x7f4e28004810 2024-09-12T12:23:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0d1400 con 0x7f4e3c1534b0 2024-09-12T12:23:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e440d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c0b2a60 con 0x7f4e3c0b7bf0 2024-09-12T12:23:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e41e6c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e3c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.364+0000 7f4e41e6c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e3c14d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51690/0 (socket says 172.21.15.3:51690) 2024-09-12T12:23:04.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e41e6c640 1 -- 172.21.15.3:0/887853001 learned_addr learned my addr 172.21.15.3:0/887853001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:04.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e4266d640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b7bf0 0x7f4e3c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e41e6c640 1 -- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b7bf0 msgr2=0x7f4e3c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e41e6c640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b7bf0 0x7f4e3c14f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e41e6c640 1 -- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e3c1534b0 msgr2=0x7f4e3c0b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e41e6c640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e3c1534b0 0x7f4e3c0b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e41e6c640 1 -- 172.21.15.3:0/887853001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c153910 con 0x7f4e28004810 2024-09-12T12:23:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e41e6c640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e3c14d6a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e840 tx=0x7f4e3800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e327fc640 1 -- 172.21.15.3:0/887853001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38017070 con 0x7f4e28004810 2024-09-12T12:23:04.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e327fc640 1 -- 172.21.15.3:0/887853001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3800c8d0 con 0x7f4e28004810 2024-09-12T12:23:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c0b8150 con 0x7f4e28004810 2024-09-12T12:23:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e327fc640 1 -- 172.21.15.3:0/887853001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800f040 con 0x7f4e28004810 2024-09-12T12:23:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c0c75a0 con 0x7f4e28004810 2024-09-12T12:23:04.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c0b3bc0 con 0x7f4e28004810 2024-09-12T12:23:04.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e327fc640 1 -- 172.21.15.3:0/887853001 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4e3801a070 con 0x7f4e28004810 2024-09-12T12:23:04.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e327fc640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e200678c0 0x7f4e20069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e327fc640 1 -- 172.21.15.3:0/887853001 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f4e38085940 con 0x7f4e28004810 2024-09-12T12:23:04.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e4166b640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e200678c0 0x7f4e20069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.368+0000 7f4e4166b640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e200678c0 0x7f4e20069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0ae000 tx=0x7f4e2c001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.372+0000 7f4e327fc640 1 -- 172.21.15.3:0/887853001 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4e38056510 con 0x7f4e28004810 2024-09-12T12:23:04.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.536+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e3c0b1680 con 0x7f4e28004810 2024-09-12T12:23:04.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e327fc640 1 -- 172.21.15.3:0/887853001 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16665 (secure 0 0 0) 0x7f4e38055e10 con 0x7f4e28004810 2024-09-12T12:23:04.543 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:04.543 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":55,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:32.556186+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:22:28.609722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T12:22:29.925008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:04.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e200678c0 msgr2=0x7f4e20069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e200678c0 0x7f4e20069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0ae000 tx=0x7f4e2c001a50 comp rx=0 tx=0).stop 2024-09-12T12:23:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 msgr2=0x7f4e3c14d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e3c14d6a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e840 tx=0x7f4e3800ed00 comp rx=0 tx=0).stop 2024-09-12T12:23:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 shutdown_connections 2024-09-12T12:23:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4e200678c0 0x7f4e20069d80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e3c0b7bf0 0x7f4e3c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e3c1534b0 0x7f4e3c0b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 --2- 172.21.15.3:0/887853001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e28004810 0x7f4e3c14d6a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 >> 172.21.15.3:0/887853001 conn(0x7f4e3c0b4200 msgr2=0x7f4e3c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:04.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 shutdown_connections 2024-09-12T12:23:04.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.540+0000 7f4e440d0640 1 -- 172.21.15.3:0/887853001 wait complete. 2024-09-12T12:23:04.562 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:23:04.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8724468640 1 Processor -- start 2024-09-12T12:23:04.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8724468640 1 -- start start 2024-09-12T12:23:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8724468640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c0cfc40 0x7f871c0d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8724468640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f871c0d1cb0 0x7f871c0b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8724468640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871c0b7950 con 0x7f871c15e900 2024-09-12T12:23:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8724468640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f871c0d2120 con 0x7f871c0d1cb0 2024-09-12T12:23:04.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8724468640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f871c0d4c40 con 0x7f871c0cfc40 2024-09-12T12:23:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8721a03640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c0cfc40 0x7f871c0d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8722a05640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f871c0d1cb0 0x7f871c0b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8721a03640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c0cfc40 0x7f871c0d1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:58348/0 (socket says 172.21.15.3:58348) 2024-09-12T12:23:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8721a03640 1 -- 172.21.15.3:0/704433138 learned_addr learned my addr 172.21.15.3:0/704433138 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:04.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8722204640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f871c15e900 0x7f871c0d4530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:44926/0 (socket says 172.21.15.3:44926) 2024-09-12T12:23:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8721a03640 1 -- 172.21.15.3:0/704433138 >> v2:172.21.15.142:3300/0 conn(0x7f871c0d1cb0 msgr2=0x7f871c0b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8721a03640 1 --2- 172.21.15.3:0/704433138 >> v2:172.21.15.142:3300/0 conn(0x7f871c0d1cb0 0x7f871c0b8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8721a03640 1 -- 172.21.15.3:0/704433138 >> v1:172.21.15.142:6789/0 conn(0x7f871c15e900 legacy=0x7f871c0d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8721a03640 1 -- 172.21.15.3:0/704433138 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c15ece0 con 0x7f871c0cfc40 2024-09-12T12:23:04.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.680+0000 7f8722a05640 1 --2- 172.21.15.3:0/704433138 >> v2:172.21.15.142:3300/0 conn(0x7f871c0d1cb0 0x7f871c0b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:04.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8721a03640 1 --2- 172.21.15.3:0/704433138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c0cfc40 0x7f871c0d1770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8718009870 tx=0x7f871801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc19ac30c29dbb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8721202640 1 -- 172.21.15.3:0/704433138 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87180b0070 con 0x7f871c0cfc40 2024-09-12T12:23:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8721202640 1 -- 172.21.15.3:0/704433138 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87180ab600 con 0x7f871c0cfc40 2024-09-12T12:23:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8721202640 1 -- 172.21.15.3:0/704433138 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87180ab9d0 con 0x7f871c0cfc40 2024-09-12T12:23:04.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 -- 172.21.15.3:0/704433138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c0cfc40 msgr2=0x7f871c0d1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 --2- 172.21.15.3:0/704433138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c0cfc40 0x7f871c0d1770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8718009870 tx=0x7f871801c4d0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 -- 172.21.15.3:0/704433138 shutdown_connections 2024-09-12T12:23:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 --2- 172.21.15.3:0/704433138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c0cfc40 0x7f871c0d1770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 --2- 172.21.15.3:0/704433138 >> v2:172.21.15.142:3300/0 conn(0x7f871c0d1cb0 0x7f871c0b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 -- 172.21.15.3:0/704433138 >> 172.21.15.3:0/704433138 conn(0x7f871c06c0c0 msgr2=0x7f871c06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 -- 172.21.15.3:0/704433138 shutdown_connections 2024-09-12T12:23:04.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 -- 172.21.15.3:0/704433138 wait complete. 2024-09-12T12:23:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 Processor -- start 2024-09-12T12:23:04.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.684+0000 7f8724468640 1 -- start start 2024-09-12T12:23:04.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f871c0cfc40 0x7f871c0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0d1cb0 0x7f871c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c15e900 0x7f871c156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0b8fe0 con 0x7f871c0d1cb0 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f871c0b8e60 con 0x7f871c0cfc40 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f871c0b9160 con 0x7f871c15e900 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0d1cb0 0x7f871c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0d1cb0 0x7f871c156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51698/0 (socket says 172.21.15.3:51698) 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 -- 172.21.15.3:0/1390055610 learned_addr learned my addr 172.21.15.3:0/1390055610 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 -- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c15e900 msgr2=0x7f871c156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c15e900 0x7f871c156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 -- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f871c0cfc40 msgr2=0x7f871c0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f871c0cfc40 0x7f871c0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 -- 172.21.15.3:0/1390055610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c157290 con 0x7f871c0d1cb0 2024-09-12T12:23:04.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8721a03640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0d1cb0 0x7f871c156720 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f87180ae040 tx=0x7f8718019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8712ffd640 1 -- 172.21.15.3:0/1390055610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87180ba070 con 0x7f871c0d1cb0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8712ffd640 1 -- 172.21.15.3:0/1390055610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87180b4430 con 0x7f871c0d1cb0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c157570 con 0x7f871c0d1cb0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8712ffd640 1 -- 172.21.15.3:0/1390055610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87180b0040 con 0x7f871c0d1cb0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c1f7ad0 con 0x7f871c0d1cb0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c157a90 con 0x7f871c0d1cb0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8712ffd640 1 -- 172.21.15.3:0/1390055610 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f87180bd070 con 0x7f871c0d1cb0 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8712ffd640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87000678c0 0x7f8700069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:04.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8712ffd640 1 -- 172.21.15.3:0/1390055610 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f871811ed40 con 0x7f871c0d1cb0 2024-09-12T12:23:04.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.688+0000 7f8722204640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87000678c0 0x7f8700069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:04.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.692+0000 7f8722204640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87000678c0 0x7f8700069d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f870c000f70 tx=0x7f870c007a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:04.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.696+0000 7f8712ffd640 1 -- 172.21.15.3:0/1390055610 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8718016150 con 0x7f871c0d1cb0 2024-09-12T12:23:04.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.892+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f871c0d1770 con 0x7f871c0d1cb0 2024-09-12T12:23:04.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8712ffd640 1 -- 172.21.15.3:0/1390055610 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7f87180ef350 con 0x7f871c0d1cb0 2024-09-12T12:23:04.898 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:04.899 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"btime":"2024-09-12T12:22:34:602545+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/256387515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":256387515},{"type":"v1","addr":"172.21.15.142:6835","nonce":256387515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2534794519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2534794519},{"type":"v1","addr":"172.21.15.142:6837","nonce":2534794519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2676380928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2676380928},{"type":"v1","addr":"172.21.15.3:6833","nonce":2676380928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:22:30.937780+0000","modified":"2024-09-12T12:22:33.984511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4007447206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4007447206},{"type":"v1","addr":"172.21.15.3:6835","nonce":4007447206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5655,"qdb_cluster":[5655]},"id":4}]} 2024-09-12T12:23:04.899 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-12T12:23:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87000678c0 msgr2=0x7f8700069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87000678c0 0x7f8700069d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f870c000f70 tx=0x7f870c007a10 comp rx=0 tx=0).stop 2024-09-12T12:23:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0d1cb0 msgr2=0x7f871c156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0d1cb0 0x7f871c156720 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f87180ae040 tx=0x7f8718019270 comp rx=0 tx=0).stop 2024-09-12T12:23:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 shutdown_connections 2024-09-12T12:23:04.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87000678c0 0x7f8700069d80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f871c15e900 0x7f871c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f871c0d1cb0 0x7f871c156720 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 --2- 172.21.15.3:0/1390055610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f871c0cfc40 0x7f871c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 >> 172.21.15.3:0/1390055610 conn(0x7f871c06c0c0 msgr2=0x7f871c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 shutdown_connections 2024-09-12T12:23:04.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:04.896+0000 7f8724468640 1 -- 172.21.15.3:0/1390055610 wait complete. 2024-09-12T12:23:04.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee91a79640 1 Processor -- start 2024-09-12T12:23:05.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee91a79640 1 -- start start 2024-09-12T12:23:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee91a79640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c15e910 0x7fee8c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee91a79640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee8c0d1dd0 0x7fee8c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee91a79640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee8c15ecf0 con 0x7fee8c0d5310 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee91a79640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fee8c0d4550 con 0x7fee8c0d1dd0 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee91a79640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c0d4830 con 0x7fee8c15e910 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee8c0d1dd0 0x7fee8c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fee8c0d1dd0 0x7fee8c0b78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51714/0 (socket says 172.21.15.3:51714) 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8bfff640 1 -- 172.21.15.3:0/872187230 learned_addr learned my addr 172.21.15.3:0/872187230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8b7fe640 1 --2- 172.21.15.3:0/872187230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c15e910 0x7fee8c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8bfff640 1 -- 172.21.15.3:0/872187230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c15e910 msgr2=0x7fee8c0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8bfff640 1 --2- 172.21.15.3:0/872187230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c15e910 0x7fee8c0cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8bfff640 1 -- 172.21.15.3:0/872187230 >> v1:172.21.15.142:6789/0 conn(0x7fee8c0d5310 legacy=0x7fee8c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.028+0000 7fee8bfff640 1 -- 172.21.15.3:0/872187230 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c0d4b10 con 0x7fee8c0d1dd0 2024-09-12T12:23:05.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8bfff640 1 --2- 172.21.15.3:0/872187230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c0d1dd0 0x7fee8c0b78d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee84009e50 tx=0x7fee8401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa87f858df8e955 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8a7fc640 1 -- 172.21.15.3:0/872187230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840b4070 con 0x7fee8c0d1dd0 2024-09-12T12:23:05.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8a7fc640 1 -- 172.21.15.3:0/872187230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee840af400 con 0x7fee8c0d1dd0 2024-09-12T12:23:05.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8a7fc640 1 -- 172.21.15.3:0/872187230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840af7d0 con 0x7fee8c0d1dd0 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- 172.21.15.3:0/872187230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c0d1dd0 msgr2=0x7fee8c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 --2- 172.21.15.3:0/872187230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c0d1dd0 0x7fee8c0b78d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee84009e50 tx=0x7fee8401f210 comp rx=0 tx=0).stop 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- 172.21.15.3:0/872187230 shutdown_connections 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 --2- 172.21.15.3:0/872187230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c0d1dd0 0x7fee8c0b78d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 --2- 172.21.15.3:0/872187230 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c15e910 0x7fee8c0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- 172.21.15.3:0/872187230 >> 172.21.15.3:0/872187230 conn(0x7fee8c06c0c0 msgr2=0x7fee8c06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- 172.21.15.3:0/872187230 shutdown_connections 2024-09-12T12:23:05.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- 172.21.15.3:0/872187230 wait complete. 2024-09-12T12:23:05.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 Processor -- start 2024-09-12T12:23:05.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- start start 2024-09-12T12:23:05.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8c0d1dd0 0x7fee8c15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c0d5310 0x7fee8c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c15e910 0x7fee8c1f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0cfe00 con 0x7fee8c15e910 2024-09-12T12:23:05.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c0d2610 con 0x7fee8c0d1dd0 2024-09-12T12:23:05.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee91a79640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c0b3340 con 0x7fee8c0d5310 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c15e910 0x7fee8c1f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c15e910 0x7fee8c1f4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51728/0 (socket says 172.21.15.3:51728) 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8bfff640 1 -- 172.21.15.3:0/821391735 learned_addr learned my addr 172.21.15.3:0/821391735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8b7fe640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8c0d1dd0 0x7fee8c15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.032+0000 7fee8affd640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c0d5310 0x7fee8c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee8bfff640 1 -- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c0d5310 msgr2=0x7fee8c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee8bfff640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c0d5310 0x7fee8c157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee8bfff640 1 -- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8c0d1dd0 msgr2=0x7fee8c15b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee8bfff640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8c0d1dd0 0x7fee8c15b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee8bfff640 1 -- 172.21.15.3:0/821391735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c0cce00 con 0x7fee8c15e910 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee8bfff640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c15e910 0x7fee8c1f4240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee8401ce40 tx=0x7fee8401bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee73fff640 1 -- 172.21.15.3:0/821391735 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840c0070 con 0x7fee8c15e910 2024-09-12T12:23:05.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee73fff640 1 -- 172.21.15.3:0/821391735 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee840090d0 con 0x7fee8c15e910 2024-09-12T12:23:05.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c1f5400 con 0x7fee8c15e910 2024-09-12T12:23:05.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee73fff640 1 -- 172.21.15.3:0/821391735 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840b4040 con 0x7fee8c15e910 2024-09-12T12:23:05.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c1f4de0 con 0x7fee8c15e910 2024-09-12T12:23:05.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee8c0cf770 con 0x7fee8c15e910 2024-09-12T12:23:05.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee73fff640 1 -- 172.21.15.3:0/821391735 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fee840c3070 con 0x7fee8c15e910 2024-09-12T12:23:05.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee73fff640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fee780678c0 0x7fee78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee73fff640 1 -- 172.21.15.3:0/821391735 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fee84123270 con 0x7fee8c15e910 2024-09-12T12:23:05.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.036+0000 7fee8b7fe640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fee780678c0 0x7fee78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.040+0000 7fee8b7fe640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fee780678c0 0x7fee78069d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fee800079c0 tx=0x7fee80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.040+0000 7fee73fff640 1 -- 172.21.15.3:0/821391735 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fee84018d70 con 0x7fee8c15e910 2024-09-12T12:23:05.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.228+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee8c072670 con 0x7fee8c15e910 2024-09-12T12:23:05.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee73fff640 1 -- 172.21.15.3:0/821391735 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5220 (secure 0 0 0) 0x7fee840f3890 con 0x7fee8c15e910 2024-09-12T12:23:05.234 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":36,"btime":"2024-09-12T12:22:34:602545+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/256387515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":256387515},{"type":"v1","addr":"172.21.15.142:6835","nonce":256387515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2534794519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2534794519},{"type":"v1","addr":"172.21.15.142:6837","nonce":2534794519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2676380928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2676380928},{"type":"v1","addr":"172.21.15.3:6833","nonce":2676380928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:22:30.937780+0000","modified":"2024-09-12T12:22:33.984511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/4007447206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":4007447206},{"type":"v1","addr":"172.21.15.3:6835","nonce":4007447206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5655,"qdb_cluster":[5655]},"id":4}]} 2024-09-12T12:23:05.234 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 36 2024-09-12T12:23:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fee780678c0 msgr2=0x7fee78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fee780678c0 0x7fee78069d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fee800079c0 tx=0x7fee80008040 comp rx=0 tx=0).stop 2024-09-12T12:23:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c15e910 msgr2=0x7fee8c1f4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c15e910 0x7fee8c1f4240 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee8401ce40 tx=0x7fee8401bfb0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 shutdown_connections 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fee780678c0 0x7fee78069d80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee8c15e910 0x7fee8c1f4240 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fee8c0d5310 0x7fee8c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 --2- 172.21.15.3:0/821391735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee8c0d1dd0 0x7fee8c15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 >> 172.21.15.3:0/821391735 conn(0x7fee8c06c0c0 msgr2=0x7fee8c06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 shutdown_connections 2024-09-12T12:23:05.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.232+0000 7fee91a79640 1 -- 172.21.15.3:0/821391735 wait complete. 2024-09-12T12:23:05.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b42967640 1 Processor -- start 2024-09-12T12:23:05.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b42967640 1 -- start start 2024-09-12T12:23:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b42967640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c15bf90 0x7f1b3c06d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b42967640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b3c0d0a70 0x7f1b3c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b42967640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c0b56a0 con 0x7f1b3c0b4560 2024-09-12T12:23:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b42967640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3c1557f0 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b42967640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c0ceeb0 con 0x7f1b3c15bf90 2024-09-12T12:23:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b40f04640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b3c0d0a70 0x7f1b3c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b3bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c15bf90 0x7f1b3c06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b3bfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c15bf90 0x7f1b3c06d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:58396/0 (socket says 172.21.15.3:58396) 2024-09-12T12:23:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b3bfff640 1 -- 172.21.15.3:0/4281772208 learned_addr learned my addr 172.21.15.3:0/4281772208 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b3bfff640 1 -- 172.21.15.3:0/4281772208 >> v2:172.21.15.142:3300/0 conn(0x7f1b3c0d0a70 msgr2=0x7f1b3c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b3bfff640 1 --2- 172.21.15.3:0/4281772208 >> v2:172.21.15.142:3300/0 conn(0x7f1b3c0d0a70 0x7f1b3c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b3bfff640 1 -- 172.21.15.3:0/4281772208 >> v1:172.21.15.142:6789/0 conn(0x7f1b3c0b4560 legacy=0x7f1b3c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.368+0000 7f1b3bfff640 1 -- 172.21.15.3:0/4281772208 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c0b50e0 con 0x7f1b3c15bf90 2024-09-12T12:23:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b3bfff640 1 --2- 172.21.15.3:0/4281772208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c15bf90 0x7f1b3c06d120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009870 tx=0x7f1b2c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dbe77e47069ebe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b3affd640 1 -- 172.21.15.3:0/4281772208 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0b1070 con 0x7f1b3c15bf90 2024-09-12T12:23:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b3affd640 1 -- 172.21.15.3:0/4281772208 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b2c0ac630 con 0x7f1b3c15bf90 2024-09-12T12:23:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b3affd640 1 -- 172.21.15.3:0/4281772208 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0a8050 con 0x7f1b3c15bf90 2024-09-12T12:23:05.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 -- 172.21.15.3:0/4281772208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c15bf90 msgr2=0x7f1b3c06d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 --2- 172.21.15.3:0/4281772208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c15bf90 0x7f1b3c06d120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009870 tx=0x7f1b2c01c560 comp rx=0 tx=0).stop 2024-09-12T12:23:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 -- 172.21.15.3:0/4281772208 shutdown_connections 2024-09-12T12:23:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 --2- 172.21.15.3:0/4281772208 >> v2:172.21.15.142:3300/0 conn(0x7f1b3c0d0a70 0x7f1b3c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 --2- 172.21.15.3:0/4281772208 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c15bf90 0x7f1b3c06d120 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 -- 172.21.15.3:0/4281772208 >> 172.21.15.3:0/4281772208 conn(0x7f1b3c067a00 msgr2=0x7f1b3c067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 -- 172.21.15.3:0/4281772208 shutdown_connections 2024-09-12T12:23:05.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 -- 172.21.15.3:0/4281772208 wait complete. 2024-09-12T12:23:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 Processor -- start 2024-09-12T12:23:05.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.372+0000 7f1b42967640 1 -- start start 2024-09-12T12:23:05.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c0b4560 0x7f1b3c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b3c0d0a70 0x7f1b3c155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b3c15bf90 0x7f1b3c155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c06d7e0 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c06d660 con 0x7f1b3c15bf90 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c0cf340 con 0x7f1b3c0b4560 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b3c0d0a70 0x7f1b3c155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b3c0d0a70 0x7f1b3c155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51732/0 (socket says 172.21.15.3:51732) 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 -- 172.21.15.3:0/2061773744 learned_addr learned my addr 172.21.15.3:0/2061773744 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b40f04640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b3c15bf90 0x7f1b3c155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3bfff640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c0b4560 0x7f1b3c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 -- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c0b4560 msgr2=0x7f1b3c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c0b4560 0x7f1b3c158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 -- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b3c15bf90 msgr2=0x7f1b3c155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b3c15bf90 0x7f1b3c155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 -- 172.21.15.3:0/2061773744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c0b0b90 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3b7fe640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b3c0d0a70 0x7f1b3c155980 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c450 tx=0x7f1b3000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b38ff9640 1 -- 172.21.15.3:0/2061773744 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30016020 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b38ff9640 1 -- 172.21.15.3:0/2061773744 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b300052d0 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b38ff9640 1 -- 172.21.15.3:0/2061773744 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30007260 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c0b0e70 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c0b1360 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0cbcd0 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b38ff9640 1 -- 172.21.15.3:0/2061773744 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1b30007400 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b38ff9640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b280678c0 0x7f1b28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b38ff9640 1 -- 172.21.15.3:0/2061773744 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f1b300862a0 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.376+0000 7f1b3bfff640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b280678c0 0x7f1b28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.380+0000 7f1b3bfff640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b280678c0 0x7f1b28069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ae000 tx=0x7f1b2c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.380+0000 7f1b38ff9640 1 -- 172.21.15.3:0/2061773744 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1b3c0cbcd0 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.540+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b3c06ec00 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b38ff9640 1 -- 172.21.15.3:0/2061773744 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16665 (secure 0 0 0) 0x7f1b30020020 con 0x7f1b3c0d0a70 2024-09-12T12:23:05.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:05.546 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":55,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:22:32.556186+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:22:28.609722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-12T12:22:29.925008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b280678c0 msgr2=0x7f1b28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b280678c0 0x7f1b28069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ae000 tx=0x7f1b2c0a7000 comp rx=0 tx=0).stop 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b3c0d0a70 msgr2=0x7f1b3c155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b3c0d0a70 0x7f1b3c155980 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c450 tx=0x7f1b3000c910 comp rx=0 tx=0).stop 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 shutdown_connections 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b280678c0 0x7f1b28069d80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b3c15bf90 0x7f1b3c155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b3c0d0a70 0x7f1b3c155980 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 --2- 172.21.15.3:0/2061773744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b3c0b4560 0x7f1b3c158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 >> 172.21.15.3:0/2061773744 conn(0x7f1b3c067a00 msgr2=0x7f1b3c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:05.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 shutdown_connections 2024-09-12T12:23:05.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.544+0000 7f1b42967640 1 -- 172.21.15.3:0/2061773744 wait complete. 2024-09-12T12:23:05.563 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T12:23:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9227288640 1 Processor -- start 2024-09-12T12:23:05.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9227288640 1 -- start start 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9227288640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f92200d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9227288640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92200d1e80 0x7f92200b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9227288640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92200b7950 con 0x7f92200cfc40 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9227288640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f92200b7c80 con 0x7f92200d1e80 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9227288640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f922015ece0 con 0x7f922015e900 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9225825640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f92200d1e80 0x7f92200b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9224823640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f92200cfc40 0x7f92200d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55098/0 (socket says 172.21.15.3:55098) 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9224823640 1 -- 172.21.15.3:0/4229712203 learned_addr learned my addr 172.21.15.3:0/4229712203 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:05.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9225024640 1 --2- 172.21.15.3:0/4229712203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f92200d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9225825640 1 -- 172.21.15.3:0/4229712203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 msgr2=0x7f92200d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9225825640 1 --2- 172.21.15.3:0/4229712203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f92200d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9225825640 1 -- 172.21.15.3:0/4229712203 >> v1:172.21.15.3:6789/0 conn(0x7f92200cfc40 legacy=0x7f92200d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.672+0000 7f9225825640 1 -- 172.21.15.3:0/4229712203 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92200d4e80 con 0x7f92200d1e80 2024-09-12T12:23:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9225825640 1 --2- 172.21.15.3:0/4229712203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b8110 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9218009d90 tx=0x7f921801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a9b8cb8ab7a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9217fff640 1 -- 172.21.15.3:0/4229712203 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92180b2070 con 0x7f92200d1e80 2024-09-12T12:23:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9217fff640 1 -- 172.21.15.3:0/4229712203 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92180a58b0 con 0x7f92200d1e80 2024-09-12T12:23:05.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9217fff640 1 -- 172.21.15.3:0/4229712203 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92180a5c80 con 0x7f92200d1e80 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 -- 172.21.15.3:0/4229712203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 msgr2=0x7f92200b8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 --2- 172.21.15.3:0/4229712203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b8110 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9218009d90 tx=0x7f921801c8e0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 -- 172.21.15.3:0/4229712203 shutdown_connections 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 --2- 172.21.15.3:0/4229712203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b8110 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 --2- 172.21.15.3:0/4229712203 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f92200d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 -- 172.21.15.3:0/4229712203 >> 172.21.15.3:0/4229712203 conn(0x7f922006c0c0 msgr2=0x7f922006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 -- 172.21.15.3:0/4229712203 shutdown_connections 2024-09-12T12:23:05.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 -- 172.21.15.3:0/4229712203 wait complete. 2024-09-12T12:23:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 Processor -- start 2024-09-12T12:23:05.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.676+0000 7f9227288640 1 -- start start 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92200cfc40 0x7f9220168c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f9220156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f92200b8fe0 con 0x7f92200d1e80 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92200b8e60 con 0x7f92200cfc40 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f92200b9160 con 0x7f922015e900 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51752/0 (socket says 172.21.15.3:51752) 2024-09-12T12:23:05.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 -- 172.21.15.3:0/1922505123 learned_addr learned my addr 172.21.15.3:0/1922505123 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9225825640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f9220156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9225024640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92200cfc40 0x7f9220168c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 -- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 msgr2=0x7f9220156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f9220156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 -- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92200cfc40 msgr2=0x7f9220168c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92200cfc40 0x7f9220168c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 -- 172.21.15.3:0/1922505123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9220159480 con 0x7f92200d1e80 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9224823640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b97f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f921c00c420 tx=0x7f921c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9215ffb640 1 -- 172.21.15.3:0/1922505123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c017070 con 0x7f92200d1e80 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9215ffb640 1 -- 172.21.15.3:0/1922505123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c005cf0 con 0x7f92200d1e80 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92200ce420 con 0x7f92200d1e80 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9215ffb640 1 -- 172.21.15.3:0/1922505123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c0072f0 con 0x7f92200d1e80 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9220159720 con 0x7f92200d1e80 2024-09-12T12:23:05.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92200b8110 con 0x7f92200d1e80 2024-09-12T12:23:05.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9215ffb640 1 -- 172.21.15.3:0/1922505123 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f921c01a070 con 0x7f92200d1e80 2024-09-12T12:23:05.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9215ffb640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92040678f0 0x7f9204069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:05.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9215ffb640 1 -- 172.21.15.3:0/1922505123 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f921c085510 con 0x7f92200d1e80 2024-09-12T12:23:05.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.680+0000 7f9225024640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92040678f0 0x7f9204069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:05.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.684+0000 7f9225024640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92040678f0 0x7f9204069db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f92100096f0 tx=0x7f9210008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:05.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.684+0000 7f9215ffb640 1 -- 172.21.15.3:0/1922505123 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f921c021020 con 0x7f92200d1e80 2024-09-12T12:23:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.880+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f92200d4540 con 0x7f92200d1e80 2024-09-12T12:23:05.884 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:23:05.880+0000 7f0a5d435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:23:05.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.884+0000 7f9215ffb640 1 -- 172.21.15.3:0/1922505123 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f921c055fa0 con 0x7f92200d1e80 2024-09-12T12:23:05.887 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:23:05.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92040678f0 msgr2=0x7f9204069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92040678f0 0x7f9204069db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f92100096f0 tx=0x7f9210008040 comp rx=0 tx=0).stop 2024-09-12T12:23:05.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 msgr2=0x7f92200b97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:05.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b97f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f921c00c420 tx=0x7f921c00c8e0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 shutdown_connections 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f92040678f0 0x7f9204069db0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f922015e900 0x7f9220156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f92200d1e80 0x7f92200b97f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 --2- 172.21.15.3:0/1922505123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92200cfc40 0x7f9220168c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 >> 172.21.15.3:0/1922505123 conn(0x7f922006c0c0 msgr2=0x7f922006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 shutdown_connections 2024-09-12T12:23:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:05.888+0000 7f9227288640 1 -- 172.21.15.3:0/1922505123 wait complete. 2024-09-12T12:23:05.895 INFO:tasks.ceph.mds.c.smithi003.stderr: -3973> 2024-09-12T12:22:37.528+0000 7f51d26b5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:23:05.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:23:05.926 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:23:06.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.032+0000 7fd4d3d66640 1 Processor -- start 2024-09-12T12:23:06.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.032+0000 7fd4d3d66640 1 -- start start 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.032+0000 7fd4d3d66640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd4cc0725d0 0x7fd4cc06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.032+0000 7fd4d3d66640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4cc072a30 0x7fd4cc06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.032+0000 7fd4d3d66640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4cc15b8c0 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.032+0000 7fd4d3d66640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc074ce0 con 0x7fd4cc072a30 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.032+0000 7fd4d3d66640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc0734c0 con 0x7fd4cc0725d0 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4cc072a30 0x7fd4cc06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd4cc072a30 0x7fd4cc06ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51782/0 (socket says 172.21.15.3:51782) 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 -- 172.21.15.3:0/1964424531 learned_addr learned my addr 172.21.15.3:0/1964424531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d1301640 1 --2- 172.21.15.3:0/1964424531 >> v2:172.21.15.142:3300/0 conn(0x7fd4cc0725d0 0x7fd4cc06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 -- 172.21.15.3:0/1964424531 >> v2:172.21.15.142:3300/0 conn(0x7fd4cc0725d0 msgr2=0x7fd4cc06e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 --2- 172.21.15.3:0/1964424531 >> v2:172.21.15.142:3300/0 conn(0x7fd4cc0725d0 0x7fd4cc06e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 -- 172.21.15.3:0/1964424531 >> v1:172.21.15.142:6789/0 conn(0x7fd4cc15b4e0 legacy=0x7fd4cc0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:06.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 -- 172.21.15.3:0/1964424531 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc0737a0 con 0x7fd4cc072a30 2024-09-12T12:23:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d2303640 1 --2- 172.21.15.3:0/1964424531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc072a30 0x7fd4cc06ef20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc050880 tx=0x7fd4c401ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc25b64cf10d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d0b00640 1 -- 172.21.15.3:0/1964424531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400b2d0 con 0x7fd4cc072a30 2024-09-12T12:23:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d0b00640 1 -- 172.21.15.3:0/1964424531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c400b470 con 0x7fd4cc072a30 2024-09-12T12:23:06.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d0b00640 1 -- 172.21.15.3:0/1964424531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40a7cb0 con 0x7fd4cc072a30 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 -- 172.21.15.3:0/1964424531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc072a30 msgr2=0x7fd4cc06ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/1964424531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc072a30 0x7fd4cc06ef20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc050880 tx=0x7fd4c401ebc0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 -- 172.21.15.3:0/1964424531 shutdown_connections 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/1964424531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc072a30 0x7fd4cc06ef20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/1964424531 >> v2:172.21.15.142:3300/0 conn(0x7fd4cc0725d0 0x7fd4cc06e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 -- 172.21.15.3:0/1964424531 >> 172.21.15.3:0/1964424531 conn(0x7fd4cc0d0b70 msgr2=0x7fd4cc0d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 -- 172.21.15.3:0/1964424531 shutdown_connections 2024-09-12T12:23:06.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.036+0000 7fd4d3d66640 1 -- 172.21.15.3:0/1964424531 wait complete. 2024-09-12T12:23:06.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 Processor -- start 2024-09-12T12:23:06.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 -- start start 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4cc0725d0 0x7fd4cc148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc072a30 0x7fd4cc157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc15b4e0 0x7fd4cc1440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d0970 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d07f0 con 0x7fd4cc072a30 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd4cc06d8d0 con 0x7fd4cc0725d0 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc15b4e0 0x7fd4cc1440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:06.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc15b4e0 0x7fd4cc1440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51794/0 (socket says 172.21.15.3:51794) 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 -- 172.21.15.3:0/2205089582 learned_addr learned my addr 172.21.15.3:0/2205089582 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d1b02640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4cc0725d0 0x7fd4cc148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d1301640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc072a30 0x7fd4cc157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 -- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4cc0725d0 msgr2=0x7fd4cc148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4cc0725d0 0x7fd4cc148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 -- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc072a30 msgr2=0x7fd4cc157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc072a30 0x7fd4cc157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 -- 172.21.15.3:0/2205089582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc1493f0 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d2303640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc15b4e0 0x7fd4cc1440e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd4c401c7f0 tx=0x7fd4c401b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4c27fc640 1 -- 172.21.15.3:0/2205089582 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009800 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4c27fc640 1 -- 172.21.15.3:0/2205089582 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c40c0070 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4c27fc640 1 -- 172.21.15.3:0/2205089582 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009c00 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc155600 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc155bc0 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.040+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc0cbbd0 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.044+0000 7fd4c27fc640 1 -- 172.21.15.3:0/2205089582 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd4c40ba470 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.044+0000 7fd4c27fc640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4b00678c0 0x7fd4b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:06.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.044+0000 7fd4c27fc640 1 -- 172.21.15.3:0/2205089582 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5795+0+0 (secure 0 0 0) 0x7fd4c4123d40 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.044+0000 7fd4d1b02640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4b00678c0 0x7fd4b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:06.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.044+0000 7fd4d1b02640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4b00678c0 0x7fd4b0069d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc144df0 tx=0x7fd4c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:06.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.048+0000 7fd4c27fc640 1 -- 172.21.15.3:0/2205089582 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd4c40187e0 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.244+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd4cc073090 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4c27fc640 1 -- 172.21.15.3:0/2205089582 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7fd4c40f4ae0 con 0x7fd4cc15b4e0 2024-09-12T12:23:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4b00678c0 msgr2=0x7fd4b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4b00678c0 0x7fd4b0069d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc144df0 tx=0x7fd4c8009040 comp rx=0 tx=0).stop 2024-09-12T12:23:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc15b4e0 msgr2=0x7fd4cc1440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc15b4e0 0x7fd4cc1440e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd4c401c7f0 tx=0x7fd4c401b960 comp rx=0 tx=0).stop 2024-09-12T12:23:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 shutdown_connections 2024-09-12T12:23:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd4b00678c0 0x7fd4b0069d80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4cc15b4e0 0x7fd4cc1440e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd4cc072a30 0x7fd4cc157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 --2- 172.21.15.3:0/2205089582 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd4cc0725d0 0x7fd4cc148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:06.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 >> 172.21.15.3:0/2205089582 conn(0x7fd4cc0d0b70 msgr2=0x7fd4cc0d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:06.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 shutdown_connections 2024-09-12T12:23:06.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:06.888+0000 7fd4d3d66640 1 -- 172.21.15.3:0/2205089582 wait complete. 2024-09-12T12:23:06.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:23:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.092+0000 7fc9507e5640 1 Processor -- start 2024-09-12T12:23:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.092+0000 7fc9507e5640 1 -- start start 2024-09-12T12:23:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc9507e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc9480716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc9507e5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9480d04a0 0x7fc9480d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc9507e5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9480ad3b0 con 0x7fc94815b4f0 2024-09-12T12:23:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc9507e5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc9480ad160 con 0x7fc9480d04a0 2024-09-12T12:23:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc9507e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc9480ad690 con 0x7fc94815b110 2024-09-12T12:23:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94e581640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc9480716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94ed82640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc9480d04a0 0x7fc9480d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94dd80640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fc94815b4f0 0x7fc9480b3b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:44958/0 (socket says 172.21.15.3:44958) 2024-09-12T12:23:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94dd80640 1 -- 172.21.15.3:0/4195112300 learned_addr learned my addr 172.21.15.3:0/4195112300 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94e581640 1 -- 172.21.15.3:0/4195112300 >> v2:172.21.15.142:3300/0 conn(0x7fc9480d04a0 msgr2=0x7fc9480d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94e581640 1 --2- 172.21.15.3:0/4195112300 >> v2:172.21.15.142:3300/0 conn(0x7fc9480d04a0 0x7fc9480d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94e581640 1 -- 172.21.15.3:0/4195112300 >> v1:172.21.15.142:6789/0 conn(0x7fc94815b4f0 legacy=0x7fc9480b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94e581640 1 -- 172.21.15.3:0/4195112300 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9480ad970 con 0x7fc94815b110 2024-09-12T12:23:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94e581640 1 --2- 172.21.15.3:0/4195112300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc9480716a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9380097c0 tx=0x7fc93801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00c01688834aa12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94d57f640 1 -- 172.21.15.3:0/4195112300 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b0070 con 0x7fc94815b110 2024-09-12T12:23:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94d57f640 1 -- 172.21.15.3:0/4195112300 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9380ab600 con 0x7fc94815b110 2024-09-12T12:23:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.096+0000 7fc94d57f640 1 -- 172.21.15.3:0/4195112300 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380ab9d0 con 0x7fc94815b110 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- 172.21.15.3:0/4195112300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 msgr2=0x7fc9480716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 --2- 172.21.15.3:0/4195112300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc9480716a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9380097c0 tx=0x7fc93801c410 comp rx=0 tx=0).stop 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- 172.21.15.3:0/4195112300 shutdown_connections 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 --2- 172.21.15.3:0/4195112300 >> v2:172.21.15.142:3300/0 conn(0x7fc9480d04a0 0x7fc9480d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 --2- 172.21.15.3:0/4195112300 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc9480716a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- 172.21.15.3:0/4195112300 >> 172.21.15.3:0/4195112300 conn(0x7fc948067bd0 msgr2=0x7fc948067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- 172.21.15.3:0/4195112300 shutdown_connections 2024-09-12T12:23:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- 172.21.15.3:0/4195112300 wait complete. 2024-09-12T12:23:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 Processor -- start 2024-09-12T12:23:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- start start 2024-09-12T12:23:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480d04a0 0x7fc948154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc948154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc94815b4f0 0x7fc94815c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9480aed50 con 0x7fc9480d04a0 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc9480aebd0 con 0x7fc94815b4f0 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc9507e5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc9480aeed0 con 0x7fc94815b110 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480d04a0 0x7fc948154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480d04a0 0x7fc948154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51810/0 (socket says 172.21.15.3:51810) 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 -- 172.21.15.3:0/3810040474 learned_addr learned my addr 172.21.15.3:0/3810040474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94ed82640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc94815b4f0 0x7fc94815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94dd80640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc948154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 -- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 msgr2=0x7fc948154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc948154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 -- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc94815b4f0 msgr2=0x7fc94815c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc94815b4f0 0x7fc94815c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.100+0000 7fc94e581640 1 -- 172.21.15.3:0/3810040474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc948148dd0 con 0x7fc9480d04a0 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc94e581640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480d04a0 0x7fc948154450 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ae040 tx=0x7fc93801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc93effd640 1 -- 172.21.15.3:0/3810040474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b0050 con 0x7fc9480d04a0 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc93effd640 1 -- 172.21.15.3:0/3810040474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9380bc070 con 0x7fc9480d04a0 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc948155100 con 0x7fc9480d04a0 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc93effd640 1 -- 172.21.15.3:0/3810040474 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380b5870 con 0x7fc9480d04a0 2024-09-12T12:23:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9481553c0 con 0x7fc9480d04a0 2024-09-12T12:23:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9480716a0 con 0x7fc9480d04a0 2024-09-12T12:23:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc93effd640 1 -- 172.21.15.3:0/3810040474 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9380b5a10 con 0x7fc9480d04a0 2024-09-12T12:23:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc93effd640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc92c0678c0 0x7fc92c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc93effd640 1 -- 172.21.15.3:0/3810040474 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5795+0+0 (secure 0 0 0) 0x7fc938120500 con 0x7fc9480d04a0 2024-09-12T12:23:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc94dd80640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc92c0678c0 0x7fc92c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:07.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.104+0000 7fc94dd80640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc92c0678c0 0x7fc92c069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc94814c6c0 tx=0x7fc944009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:07.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.108+0000 7fc93effd640 1 -- 172.21.15.3:0/3810040474 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc938016090 con 0x7fc9480d04a0 2024-09-12T12:23:07.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.276+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc948154ed0 con 0x7fc9480d04a0 2024-09-12T12:23:07.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:07.996+0000 7fc93effd640 1 -- 172.21.15.3:0/3810040474 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) ==== 181+0+0 (secure 0 0 0) 0x7fc9380f1080 con 0x7fc9480d04a0 2024-09-12T12:23:07.999 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:23:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc92c0678c0 msgr2=0x7fc92c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc92c0678c0 0x7fc92c069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc94814c6c0 tx=0x7fc944009040 comp rx=0 tx=0).stop 2024-09-12T12:23:08.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480d04a0 msgr2=0x7fc948154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480d04a0 0x7fc948154450 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc9380ae040 tx=0x7fc93801ac50 comp rx=0 tx=0).stop 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 shutdown_connections 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc92c0678c0 0x7fc92c069d80 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc94814c6c0 tx=0x7fc944009040 comp rx=0 tx=0).stop 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc94815b4f0 0x7fc94815c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc94815b110 0x7fc948154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 --2- 172.21.15.3:0/3810040474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9480d04a0 0x7fc948154450 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 >> 172.21.15.3:0/3810040474 conn(0x7fc948067bd0 msgr2=0x7fc948071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 shutdown_connections 2024-09-12T12:23:08.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.000+0000 7fc9507e5640 1 -- 172.21.15.3:0/3810040474 wait complete. 2024-09-12T12:23:08.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:23:08.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa927dd0640 1 Processor -- start 2024-09-12T12:23:08.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa927dd0640 1 -- start start 2024-09-12T12:23:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa927dd0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 0x7fa9200b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa927dd0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa92006f270 con 0x7fa9200cda80 2024-09-12T12:23:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa927dd0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9200cdf40 con 0x7fa9200d0bf0 2024-09-12T12:23:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa927dd0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9200ce220 con 0x7fa92015c660 2024-09-12T12:23:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa92536b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa9200cda80 0x7fa9200d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54574/0 (socket says 172.21.15.3:54574) 2024-09-12T12:23:08.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.148+0000 7fa92536b640 1 -- 172.21.15.3:0/430225195 learned_addr learned my addr 172.21.15.3:0/430225195 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa925b6c640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 0x7fa9200b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505073200 0 0) 0x7fa92006f270 con 0x7fa9200cda80 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa908003530 con 0x7fa9200cda80 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2169718410 0 0) 0x7fa908003530 con 0x7fa9200cda80 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 msgr2=0x7fa9200b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 0x7fa9200b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 >> v1:172.21.15.142:6789/0 conn(0x7fa9200d0bf0 legacy=0x7fa92006fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92006f510 con 0x7fa9200cda80 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fa91c001a00 con 0x7fa9200cda80 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 >> v1:172.21.15.3:6789/0 conn(0x7fa9200cda80 legacy=0x7fa9200d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9080039f0 0x7fa908003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa908004810 0x7fa908004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 0x7fa9080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9200ce220 con 0x7fa908004810 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa90800bb10 con 0x7fa9080039f0 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa90800bd10 con 0x7fa92015c660 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92536b640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa908004810 0x7fa908004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa925b6c640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9080039f0 0x7fa908003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92636d640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 0x7fa9080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92536b640 1 -- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 msgr2=0x7fa9080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92536b640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 0x7fa9080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92536b640 1 -- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9080039f0 msgr2=0x7fa908003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92536b640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9080039f0 0x7fa908003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92536b640 1 -- 172.21.15.3:0/430225195 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa908003530 con 0x7fa908004810 2024-09-12T12:23:08.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92636d640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa92015c660 0x7fa9080051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92536b640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa908004810 0x7fa908004c90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a810 tx=0x7fa91c01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51a978cf540c34f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00a460 con 0x7fa908004810 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c0ad650 con 0x7fa908004810 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa924b6a640 1 -- 172.21.15.3:0/430225195 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0ada40 con 0x7fa908004810 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa927dd0640 1 -- 172.21.15.3:0/430225195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa908004810 msgr2=0x7fa908004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa927dd0640 1 --2- 172.21.15.3:0/430225195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa908004810 0x7fa908004c90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a810 tx=0x7fa91c01d320 comp rx=0 tx=0).stop 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa92636d640 1 -- 172.21.15.3:0/430225195 reap_dead start 2024-09-12T12:23:08.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.152+0000 7fa927dd0640 1 -- 172.21.15.3:0/430225195 shutdown_connections 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- 172.21.15.3:0/430225195 >> 172.21.15.3:0/430225195 conn(0x7fa9200b3ff0 msgr2=0x7fa9200b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- 172.21.15.3:0/430225195 shutdown_connections 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- 172.21.15.3:0/430225195 wait complete. 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 Processor -- start 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- start start 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa908004810 0x7fa9200c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92014c440 0x7fa9200c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9200c7ee0 0x7fa9200c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa92015ca40 con 0x7fa92014c440 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9200d1280 con 0x7fa9200c7ee0 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa92015cbc0 con 0x7fa908004810 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92014c440 0x7fa9200c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92014c440 0x7fa9200c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39442/0 (socket says 172.21.15.3:39442) 2024-09-12T12:23:08.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 -- 172.21.15.3:0/1182756824 learned_addr learned my addr 172.21.15.3:0/1182756824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa925b6c640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa908004810 0x7fa9200c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92636d640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9200c7ee0 0x7fa9200c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 -- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa908004810 msgr2=0x7fa9200c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa908004810 0x7fa9200c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 -- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9200c7ee0 msgr2=0x7fa9200c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9200c7ee0 0x7fa9200c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 -- 172.21.15.3:0/1182756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9200c88a0 con 0x7fa92014c440 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa92536b640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92014c440 0x7fa9200c79a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a810 tx=0x7fa908005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa9167fc640 1 -- 172.21.15.3:0/1182756824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0bb070 con 0x7fa92014c440 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa9167fc640 1 -- 172.21.15.3:0/1182756824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c0ad430 con 0x7fa92014c440 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9200b6df0 con 0x7fa92014c440 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa9167fc640 1 -- 172.21.15.3:0/1182756824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0019c0 con 0x7fa92014c440 2024-09-12T12:23:08.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.156+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9200b73b0 con 0x7fa92014c440 2024-09-12T12:23:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.160+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9200b4ae0 con 0x7fa92014c440 2024-09-12T12:23:08.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.160+0000 7fa9167fc640 1 -- 172.21.15.3:0/1182756824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa91c0be070 con 0x7fa92014c440 2024-09-12T12:23:08.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.160+0000 7fa9167fc640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa90c0678c0 0x7fa90c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.160+0000 7fa9167fc640 1 -- 172.21.15.3:0/1182756824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5357+0+0 (secure 0 0 0) 0x7fa91c11fb70 con 0x7fa92014c440 2024-09-12T12:23:08.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.160+0000 7fa925b6c640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa90c0678c0 0x7fa90c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.160+0000 7fa925b6c640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa90c0678c0 0x7fa90c069d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa918009a30 tx=0x7fa918008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.164+0000 7fa9167fc640 1 -- 172.21.15.3:0/1182756824 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa91c017030 con 0x7fa92014c440 2024-09-12T12:23:08.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:08.336+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9200d0110 con 0x7fa92014c440 2024-09-12T12:23:09.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.004+0000 7fa9167fc640 1 -- 172.21.15.3:0/1182756824 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) ==== 169+0+0 (secure 0 0 0) 0x7fa91c0f0770 con 0x7fa92014c440 2024-09-12T12:23:09.008 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T12:23:09.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa90c0678c0 msgr2=0x7fa90c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa90c0678c0 0x7fa90c069d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa918009a30 tx=0x7fa918008040 comp rx=0 tx=0).stop 2024-09-12T12:23:09.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92014c440 msgr2=0x7fa9200c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92014c440 0x7fa9200c79a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a810 tx=0x7fa908005840 comp rx=0 tx=0).stop 2024-09-12T12:23:09.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 shutdown_connections 2024-09-12T12:23:09.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa90c0678c0 0x7fa90c069d80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9200c7ee0 0x7fa9200c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa92014c440 0x7fa9200c79a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 --2- 172.21.15.3:0/1182756824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa908004810 0x7fa9200c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 >> 172.21.15.3:0/1182756824 conn(0x7fa9200b3ff0 msgr2=0x7fa9200b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:09.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 shutdown_connections 2024-09-12T12:23:09.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.012+0000 7fa927dd0640 1 -- 172.21.15.3:0/1182756824 wait complete. 2024-09-12T12:23:09.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:23:09.040 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:23:09.070 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T12:23:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-12T12:23:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:23:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-12T12:23:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.070 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:23:09.071 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-12T12:23:09.072 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-12T12:23:09.073 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-12T12:23:09.074 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.075 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:23:09.076 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.077 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-12T12:23:09.078 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.079 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:23:09.080 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-12T12:23:09.081 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout:[client.badguy] 2024-09-12T12:23:09.082 INFO:teuthology.orchestra.run.smithi003.stdout: client_metadata = root=/baz 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-09-12T12:23:09.084 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-09-12T12:23:09.085 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-09-12T12:23:09.086 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:23:09.087 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.088 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-09-12T12:23:09.089 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-09-12T12:23:09.090 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.091 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.092 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on pool no app = False 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.093 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-09-12T12:23:09.094 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-09-12T12:23:09.095 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-09-12T12:23:09.095 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-09-12T12:23:09.095 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:23:09.095 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.095 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.095 INFO:teuthology.orchestra.run.smithi142.stdout:[client.badguy] 2024-09-12T12:23:09.095 INFO:teuthology.orchestra.run.smithi142.stdout: client_metadata = root=/baz 2024-09-12T12:23:09.096 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:23:09.127 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:23:09.152 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T12:23:09.152 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-12T12:23:09.152 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-12T12:23:09.153 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-12T12:23:09.154 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-12T12:23:09.155 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:23:09.156 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-12T12:23:09.157 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-12T12:23:09.158 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-12T12:23:09.159 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.160 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-12T12:23:09.161 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:23:09.162 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-12T12:23:09.163 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-12T12:23:09.164 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:23:09.165 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.165 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-12T12:23:09.165 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.165 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.165 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-12T12:23:09.165 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-12T12:23:09.165 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-12T12:23:09.166 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.167 INFO:teuthology.orchestra.run.smithi003.stdout:[client.badguy] 2024-09-12T12:23:09.169 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-09-12T12:23:09.169 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-09-12T12:23:09.169 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.170 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-09-12T12:23:09.171 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.172 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:23:09.173 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.174 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.175 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-09-12T12:23:09.176 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-09-12T12:23:09.177 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-09-12T12:23:09.178 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.179 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:23:09.180 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.181 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on pool no app = False 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-09-12T12:23:09.182 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-09-12T12:23:09.184 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-09-12T12:23:09.185 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:23:09.186 INFO:teuthology.orchestra.run.smithi142.stdout:[client.badguy] 2024-09-12T12:23:09.187 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T12:23:09.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.344+0000 7f1de1e45640 1 Processor -- start 2024-09-12T12:23:09.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1de1e45640 1 -- start start 2024-09-12T12:23:09.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1de1e45640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 0x7f1ddc0d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1de1e45640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0cf7f0 0x7f1ddc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1de1e45640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ddc0d27c0 con 0x7f1ddc0d3580 2024-09-12T12:23:09.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1de1e45640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1ddc0d2570 con 0x7f1ddc0cf7f0 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1de1e45640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ddc0d2aa0 con 0x7f1ddc15e8e0 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1dd3fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1ddc0d3580 0x7f1ddc0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54580/0 (socket says 172.21.15.3:54580) 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1dd3fff640 1 -- 172.21.15.3:0/79691390 learned_addr learned my addr 172.21.15.3:0/79691390 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddb7fe640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 0x7f1ddc0d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/79691390 >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0cf7f0 0x7f1ddc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712149807 0 0) 0x7f1ddc0d27c0 con 0x7f1ddc0d3580 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dc8003530 con 0x7f1ddc0d3580 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2986483227 0 0) 0x7f1dc8003530 con 0x7f1ddc0d3580 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 msgr2=0x7f1ddc0d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 0x7f1ddc0d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0cf7f0 msgr2=0x7f1ddc0b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 --2- 172.21.15.3:0/79691390 >> v2:172.21.15.142:3300/0 conn(0x7f1ddc0cf7f0 0x7f1ddc0b7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ddc0d2d80 con 0x7f1ddc0d3580 2024-09-12T12:23:09.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1dc4001680 con 0x7f1ddc0d3580 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 >> v1:172.21.15.3:6789/0 conn(0x7f1ddc0d3580 legacy=0x7f1ddc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc80039c0 0x7f1dc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 0x7f1dc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc8004d50 0x7f1dc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc0d2570 con 0x7f1dc8004d50 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc0d2aa0 con 0x7f1dc80039c0 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc800ba00 con 0x7f1ddc15e8e0 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc8004d50 0x7f1dc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1dd3fff640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 0x7f1dc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddb7fe640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc80039c0 0x7f1dc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddbfff640 1 -- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 msgr2=0x7f1dc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 0x7f1dc8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddbfff640 1 -- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc80039c0 msgr2=0x7f1dc8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc80039c0 0x7f1dc8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddbfff640 1 -- 172.21.15.3:0/79691390 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc8003530 con 0x7f1dc8004d50 2024-09-12T12:23:09.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1ddb7fe640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc80039c0 0x7f1dc8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:09.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.348+0000 7f1dd3fff640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc15e8e0 0x7f1dc8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc8004d50 0x7f1dc80051d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009fc0 tx=0x7f1dd401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea6211e7efaeea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40b2070 con 0x7f1dc8004d50 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd40a5bc0 con 0x7f1dc8004d50 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1ddaffd640 1 -- 172.21.15.3:0/79691390 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4009c10 con 0x7f1dc8004d50 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1de1e45640 1 -- 172.21.15.3:0/79691390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc8004d50 msgr2=0x7f1dc80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1de1e45640 1 --2- 172.21.15.3:0/79691390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc8004d50 0x7f1dc80051d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009fc0 tx=0x7f1dd401ccc0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1ddbfff640 1 -- 172.21.15.3:0/79691390 reap_dead start 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1de1e45640 1 -- 172.21.15.3:0/79691390 shutdown_connections 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1de1e45640 1 -- 172.21.15.3:0/79691390 >> 172.21.15.3:0/79691390 conn(0x7f1ddc067a00 msgr2=0x7f1ddc067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:09.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1de1e45640 1 -- 172.21.15.3:0/79691390 shutdown_connections 2024-09-12T12:23:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1de1e45640 1 -- 172.21.15.3:0/79691390 wait complete. 2024-09-12T12:23:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.352+0000 7f1de1e45640 1 Processor -- start 2024-09-12T12:23:09.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 -- start start 2024-09-12T12:23:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc154a50 0x7f1ddc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b8e30 0x7f1ddc0b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ddc0cb620 0x7f1ddc0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc0d1db0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ddc0d1c30 con 0x7f1ddc154a50 2024-09-12T12:23:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1ddc0d1f30 con 0x7f1ddc0b8e30 2024-09-12T12:23:09.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ddc0cb620 0x7f1ddc0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ddc0cb620 0x7f1ddc0cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39464/0 (socket says 172.21.15.3:39464) 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 -- 172.21.15.3:0/1800977589 learned_addr learned my addr 172.21.15.3:0/1800977589 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1dd3fff640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b8e30 0x7f1ddc0b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddb7fe640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc154a50 0x7f1ddc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 -- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b8e30 msgr2=0x7f1ddc0b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b8e30 0x7f1ddc0b92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 -- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc154a50 msgr2=0x7f1ddc0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc154a50 0x7f1ddc0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 -- 172.21.15.3:0/1800977589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ddc0cbfe0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1dd3fff640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b8e30 0x7f1ddc0b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1ddbfff640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ddc0cb620 0x7f1ddc0cbaa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40986e0 tx=0x7f1dd4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1dd8ff9640 1 -- 172.21.15.3:0/1800977589 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40bd070 con 0x7f1ddc0cb620 2024-09-12T12:23:09.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1dd8ff9640 1 -- 172.21.15.3:0/1800977589 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd40a59c0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ddc0cc270 con 0x7f1ddc0cb620 2024-09-12T12:23:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1dd8ff9640 1 -- 172.21.15.3:0/1800977589 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40b2040 con 0x7f1ddc0cb620 2024-09-12T12:23:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ddc0cc830 con 0x7f1ddc0cb620 2024-09-12T12:23:09.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ddc0b97f0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1dd8ff9640 1 -- 172.21.15.3:0/1800977589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1dd40039a0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.356+0000 7f1dd8ff9640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1dbc0678c0 0x7f1dbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.360+0000 7f1dd8ff9640 1 -- 172.21.15.3:0/1800977589 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7f1dd411f8f0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.360+0000 7f1ddb7fe640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1dbc0678c0 0x7f1dbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.360+0000 7f1ddb7fe640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1dbc0678c0 0x7f1dbc069d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0099b0 tx=0x7f1dcc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:09.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.360+0000 7f1dd8ff9640 1 -- 172.21.15.3:0/1800977589 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1dd40169d0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.532+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1ddc0cf0e0 con 0x7f1ddc0cb620 2024-09-12T12:23:09.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.536+0000 7f1dd8ff9640 1 -- 172.21.15.3:0/1800977589 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f1dd40f0130 con 0x7f1ddc0cb620 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1dbc0678c0 msgr2=0x7f1dbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1dbc0678c0 0x7f1dbc069d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0099b0 tx=0x7f1dcc008040 comp rx=0 tx=0).stop 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ddc0cb620 msgr2=0x7f1ddc0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ddc0cb620 0x7f1ddc0cbaa0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40986e0 tx=0x7f1dd4009750 comp rx=0 tx=0).stop 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 shutdown_connections 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1dbc0678c0 0x7f1dbc069d80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ddc0cb620 0x7f1ddc0cbaa0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ddc0b8e30 0x7f1ddc0b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 --2- 172.21.15.3:0/1800977589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ddc154a50 0x7f1ddc0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 >> 172.21.15.3:0/1800977589 conn(0x7f1ddc067a00 msgr2=0x7f1ddc0d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:09.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 shutdown_connections 2024-09-12T12:23:09.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.540+0000 7f1de1e45640 1 -- 172.21.15.3:0/1800977589 wait complete. 2024-09-12T12:23:09.559 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject' 2024-09-12T12:23:09.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62c134640 1 Processor -- start 2024-09-12T12:23:09.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62c134640 1 -- start start 2024-09-12T12:23:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62c134640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd62415c000 0x7fd6240b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62c134640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6240d05b0 0x7fd6240d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62c134640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62415f160 con 0x7fd6240af410 2024-09-12T12:23:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62c134640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd62415f9f0 con 0x7fd6240d05b0 2024-09-12T12:23:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62c134640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6240d0f20 con 0x7fd62415c000 2024-09-12T12:23:09.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd629ed0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd62415c000 0x7fd6240b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62a6d1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd6240d05b0 0x7fd6240d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd6296cf640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fd6240af410 0x7fd62415ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:52276/0 (socket says 172.21.15.3:52276) 2024-09-12T12:23:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd6296cf640 1 -- 172.21.15.3:0/623639594 learned_addr learned my addr 172.21.15.3:0/623639594 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62a6d1640 1 -- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd62415c000 msgr2=0x7fd6240b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62a6d1640 1 --2- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd62415c000 0x7fd6240b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62a6d1640 1 -- 172.21.15.3:0/623639594 >> v1:172.21.15.142:6789/0 conn(0x7fd6240af410 legacy=0x7fd62415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd62a6d1640 1 -- 172.21.15.3:0/623639594 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62415f430 con 0x7fd6240d05b0 2024-09-12T12:23:09.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.684+0000 7fd629ed0640 1 --2- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd62415c000 0x7fd6240b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:09.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62a6d1640 1 --2- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240d05b0 0x7fd6240d09e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009d90 tx=0x7fd61c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f074a37133c1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:09.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd628ece640 1 -- 172.21.15.3:0/623639594 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b2070 con 0x7fd6240d05b0 2024-09-12T12:23:09.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd628ece640 1 -- 172.21.15.3:0/623639594 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c0a5970 con 0x7fd6240d05b0 2024-09-12T12:23:09.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd628ece640 1 -- 172.21.15.3:0/623639594 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0a5d40 con 0x7fd6240d05b0 2024-09-12T12:23:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 -- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240d05b0 msgr2=0x7fd6240d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 --2- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240d05b0 0x7fd6240d09e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009d90 tx=0x7fd61c01c8b0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 -- 172.21.15.3:0/623639594 shutdown_connections 2024-09-12T12:23:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 --2- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240d05b0 0x7fd6240d09e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 --2- 172.21.15.3:0/623639594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd62415c000 0x7fd6240b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 -- 172.21.15.3:0/623639594 >> 172.21.15.3:0/623639594 conn(0x7fd624067e00 msgr2=0x7fd624068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:09.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 -- 172.21.15.3:0/623639594 shutdown_connections 2024-09-12T12:23:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 -- 172.21.15.3:0/623639594 wait complete. 2024-09-12T12:23:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 Processor -- start 2024-09-12T12:23:09.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.688+0000 7fd62c134640 1 -- start start 2024-09-12T12:23:09.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240af410 0x7fd6240af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6240d05b0 0x7fd62415fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd62415c000 0x7fd62409d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd6240b3f50 con 0x7fd62415c000 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd62415c3e0 con 0x7fd6240af410 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd62406d9f0 con 0x7fd6240d05b0 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd629ed0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240af410 0x7fd6240af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd629ed0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240af410 0x7fd6240af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:44694/0 (socket says 172.21.15.3:44694) 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd629ed0640 1 -- 172.21.15.3:0/381147211 learned_addr learned my addr 172.21.15.3:0/381147211 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62a6d1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd62415c000 0x7fd62409d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd6296cf640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6240d05b0 0x7fd62415fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62a6d1640 1 -- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6240d05b0 msgr2=0x7fd62415fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62a6d1640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6240d05b0 0x7fd62415fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62a6d1640 1 -- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240af410 msgr2=0x7fd6240af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62a6d1640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240af410 0x7fd6240af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62a6d1640 1 -- 172.21.15.3:0/381147211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62409b160 con 0x7fd62415c000 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62a6d1640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd62415c000 0x7fd62409d630 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0ac000 tx=0x7fd61c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd61a7fc640 1 -- 172.21.15.3:0/381147211 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0bd020 con 0x7fd62415c000 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd61a7fc640 1 -- 172.21.15.3:0/381147211 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c0b3040 con 0x7fd62415c000 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62409c870 con 0x7fd62415c000 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd61a7fc640 1 -- 172.21.15.3:0/381147211 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0b2050 con 0x7fd62415c000 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62409ce30 con 0x7fd62415c000 2024-09-12T12:23:09.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd624159240 con 0x7fd62415c000 2024-09-12T12:23:09.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd61a7fc640 1 -- 172.21.15.3:0/381147211 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd61c0c0070 con 0x7fd62415c000 2024-09-12T12:23:09.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6080678c0 0x7fd608069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:09.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd61a7fc640 1 -- 172.21.15.3:0/381147211 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7fd61c11fd70 con 0x7fd62415c000 2024-09-12T12:23:09.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.692+0000 7fd629ed0640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6080678c0 0x7fd608069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:09.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.696+0000 7fd629ed0640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6080678c0 0x7fd608069d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd624160ca0 tx=0x7fd620009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:09.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.696+0000 7fd61a7fc640 1 -- 172.21.15.3:0/381147211 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd61c016560 con 0x7fd62415c000 2024-09-12T12:23:09.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:09.920+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]} v 0) -- 0x7fd6240afd50 con 0x7fd62415c000 2024-09-12T12:23:10.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd61a7fc640 1 -- 172.21.15.3:0/381147211 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_session_reject"]}]=0 v182) ==== 142+0+0 (secure 0 0 0) 0x7fd61c0f04d0 con 0x7fd62415c000 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6080678c0 msgr2=0x7fd608069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6080678c0 0x7fd608069d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd624160ca0 tx=0x7fd620009190 comp rx=0 tx=0).stop 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd62415c000 msgr2=0x7fd62409d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd62415c000 0x7fd62409d630 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0ac000 tx=0x7fd61c019650 comp rx=0 tx=0).stop 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 shutdown_connections 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd6080678c0 0x7fd608069d80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd62415c000 0x7fd62409d630 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6240d05b0 0x7fd62415fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 --2- 172.21.15.3:0/381147211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd6240af410 0x7fd6240af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 >> 172.21.15.3:0/381147211 conn(0x7fd624067e00 msgr2=0x7fd624161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 shutdown_connections 2024-09-12T12:23:10.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.008+0000 7fd62c134640 1 -- 172.21.15.3:0/381147211 wait complete. 2024-09-12T12:23:10.031 INFO:tasks.cephfs_test_runner:test_session_reject (tasks.cephfs.test_sessionmap.TestSessionMap) ... ok 2024-09-12T12:23:10.032 INFO:tasks.cephfs_test_runner:Starting test: test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-09-12T12:23:10.032 INFO:tasks.cephfs_test_runner:test_tell_conn_close (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-12T12:23:10.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:10.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.148+0000 7f18c8b31640 1 Processor -- start 2024-09-12T12:23:10.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.148+0000 7f18c8b31640 1 -- start start 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c8b31640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c8b31640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18c40cd200 0x7f18c40cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c8b31640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c406f200 con 0x7f18c40b49c0 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c8b31640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f18c40d18e0 con 0x7f18c40cd200 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c8b31640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18c406f4e0 con 0x7f18c415e6c0 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18c40cd200 0x7f18c40cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f18c40cd200 0x7f18c40cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39496/0 (socket says 172.21.15.3:39496) 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 -- 172.21.15.3:0/437947954 learned_addr learned my addr 172.21.15.3:0/437947954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c1d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f18c40b49c0 0x7f18c40cf0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54590/0 (socket says 172.21.15.3:54590) 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c259c640 1 --2- 172.21.15.3:0/437947954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 -- 172.21.15.3:0/437947954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 msgr2=0x7f18c40ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:10.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 --2- 172.21.15.3:0/437947954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c40ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 -- 172.21.15.3:0/437947954 >> v1:172.21.15.3:6789/0 conn(0x7f18c40b49c0 legacy=0x7f18c40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 -- 172.21.15.3:0/437947954 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c406f7c0 con 0x7f18c40cd200 2024-09-12T12:23:10.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c159a640 1 -- 172.21.15.3:0/437947954 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724506648 0 0) 0x7f18c406f200 con 0x7f18c40b49c0 2024-09-12T12:23:10.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c2d9d640 1 --2- 172.21.15.3:0/437947954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c40cd6b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009e50 tx=0x7f18bc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85336f9dd2fe3033 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c159a640 1 -- 172.21.15.3:0/437947954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0a5830 con 0x7f18c40cd200 2024-09-12T12:23:10.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c159a640 1 -- 172.21.15.3:0/437947954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc0a59d0 con 0x7f18c40cd200 2024-09-12T12:23:10.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.152+0000 7f18c159a640 1 -- 172.21.15.3:0/437947954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0a5da0 con 0x7f18c40cd200 2024-09-12T12:23:10.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/437947954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 msgr2=0x7f18c40cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 --2- 172.21.15.3:0/437947954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c40cd6b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009e50 tx=0x7f18bc01c970 comp rx=0 tx=0).stop 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/437947954 shutdown_connections 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 --2- 172.21.15.3:0/437947954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c40cd6b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 --2- 172.21.15.3:0/437947954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c40ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/437947954 >> 172.21.15.3:0/437947954 conn(0x7f18c40b3ea0 msgr2=0x7f18c40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/437947954 shutdown_connections 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/437947954 wait complete. 2024-09-12T12:23:10.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 Processor -- start 2024-09-12T12:23:10.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- start start 2024-09-12T12:23:10.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18c40b49c0 0x7f18c414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c41521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c41526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18c40cf960 con 0x7f18c40cd200 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18c40cf7e0 con 0x7f18c40b49c0 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f18c416b340 con 0x7f18c415e6c0 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c41521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c41521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39512/0 (socket says 172.21.15.3:39512) 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 -- 172.21.15.3:0/2954110616 learned_addr learned my addr 172.21.15.3:0/2954110616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c2d9d640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c41526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c259c640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18c40b49c0 0x7f18c414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 -- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 msgr2=0x7f18c41526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c41526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 -- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18c40b49c0 msgr2=0x7f18c414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18c40b49c0 0x7f18c414aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 -- 172.21.15.3:0/2954110616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c4154cf0 con 0x7f18c40cd200 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c1d9b640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c41521b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c4b0 tx=0x7f18b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2954110616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8018070 con 0x7f18c40cd200 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2954110616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b8007500 con 0x7f18c40cd200 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c4153880 con 0x7f18c40cd200 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2954110616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8004a30 con 0x7f18c40cd200 2024-09-12T12:23:10.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c4153e40 con 0x7f18c40cd200 2024-09-12T12:23:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.156+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c40cd6b0 con 0x7f18c40cd200 2024-09-12T12:23:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.160+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2954110616 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f18b801b070 con 0x7f18c40cd200 2024-09-12T12:23:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.160+0000 7f18b2ffd640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18a40678f0 0x7f18a4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.160+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2954110616 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7f18b8084cf0 con 0x7f18c40cd200 2024-09-12T12:23:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.160+0000 7f18c259c640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18a40678f0 0x7f18a4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.160+0000 7f18c259c640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18a40678f0 0x7f18a4069db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18b40096f0 tx=0x7f18b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.164+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2954110616 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f18b8055be0 con 0x7f18c40cd200 2024-09-12T12:23:10.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.336+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18c41690f0 con 0x7f18c40cd200 2024-09-12T12:23:10.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.336+0000 7f18b2ffd640 1 -- 172.21.15.3:0/2954110616 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+12970 (secure 0 0 0) 0x7f18b80554e0 con 0x7f18c40cd200 2024-09-12T12:23:10.341 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:10.341 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":58,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:08.999222+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/4007447206":"2024-09-13T12:23:05.884069+0000","172.21.15.3:6834/4007447206":"2024-09-13T12:23:05.884069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:10.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18a40678f0 msgr2=0x7f18a4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18a40678f0 0x7f18a4069db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18b40096f0 tx=0x7f18b4008040 comp rx=0 tx=0).stop 2024-09-12T12:23:10.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 msgr2=0x7f18c41521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c41521b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c4b0 tx=0x7f18b800c970 comp rx=0 tx=0).stop 2024-09-12T12:23:10.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 shutdown_connections 2024-09-12T12:23:10.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f18a40678f0 0x7f18a4069db0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f18c415e6c0 0x7f18c41526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18c40cd200 0x7f18c41521b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 --2- 172.21.15.3:0/2954110616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18c40b49c0 0x7f18c414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 >> 172.21.15.3:0/2954110616 conn(0x7f18c40b3ea0 msgr2=0x7f18c406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:10.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 shutdown_connections 2024-09-12T12:23:10.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.340+0000 7f18c8b31640 1 -- 172.21.15.3:0/2954110616 wait complete. 2024-09-12T12:23:10.361 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:19:37.735209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:23:10.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T12:23:10.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349f6e2640 1 Processor -- start 2024-09-12T12:23:10.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349f6e2640 1 -- start start 2024-09-12T12:23:10.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349f6e2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f34980adf60 0x7f349806e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349f6e2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3498160600 con 0x7f34980d0cd0 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349f6e2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34981608e0 con 0x7f349815c210 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349f6e2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3498160bc0 con 0x7f34980adf60 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349cc7d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f34980d0cd0 0x7f34980cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54606/0 (socket says 172.21.15.3:54606) 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349cc7d640 1 -- 172.21.15.3:0/3333222916 learned_addr learned my addr 172.21.15.3:0/3333222916 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349dc7f640 1 --2- 172.21.15.3:0/3333222916 >> v2:172.21.15.3:3300/0 conn(0x7f34980adf60 0x7f349806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f348ffff640 1 -- 172.21.15.3:0/3333222916 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1765074527 0 0) 0x7f3498160600 con 0x7f34980d0cd0 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f348ffff640 1 -- 172.21.15.3:0/3333222916 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3484003530 con 0x7f34980d0cd0 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349dc7f640 1 -- 172.21.15.3:0/3333222916 >> v1:172.21.15.142:6789/0 conn(0x7f349815c210 legacy=0x7f34980d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349dc7f640 1 -- 172.21.15.3:0/3333222916 >> v1:172.21.15.3:6789/0 conn(0x7f34980d0cd0 legacy=0x7f34980cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.488+0000 7f349dc7f640 1 -- 172.21.15.3:0/3333222916 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34980cec00 con 0x7f34980adf60 2024-09-12T12:23:10.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349dc7f640 1 --2- 172.21.15.3:0/3333222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349806e450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3490009d90 tx=0x7f349001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a42150607dddaf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f348ffff640 1 -- 172.21.15.3:0/3333222916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900b2070 con 0x7f34980adf60 2024-09-12T12:23:10.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f348ffff640 1 -- 172.21.15.3:0/3333222916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34900a58b0 con 0x7f34980adf60 2024-09-12T12:23:10.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f348ffff640 1 -- 172.21.15.3:0/3333222916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900a5c80 con 0x7f34980adf60 2024-09-12T12:23:10.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 -- 172.21.15.3:0/3333222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 msgr2=0x7f349806e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 --2- 172.21.15.3:0/3333222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349806e450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3490009d90 tx=0x7f349001c8e0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 -- 172.21.15.3:0/3333222916 shutdown_connections 2024-09-12T12:23:10.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 --2- 172.21.15.3:0/3333222916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349806e450 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 -- 172.21.15.3:0/3333222916 >> 172.21.15.3:0/3333222916 conn(0x7f3498067e00 msgr2=0x7f3498068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:10.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 -- 172.21.15.3:0/3333222916 shutdown_connections 2024-09-12T12:23:10.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 -- 172.21.15.3:0/3333222916 wait complete. 2024-09-12T12:23:10.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 Processor -- start 2024-09-12T12:23:10.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.492+0000 7f349f6e2640 1 -- start start 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349809d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34980d0cd0 0x7f349815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349815c210 0x7f349815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f349816a5f0 con 0x7f34980adf60 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f349816a470 con 0x7f34980d0cd0 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f349816a770 con 0x7f349815c210 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349809d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349809d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39526/0 (socket says 172.21.15.3:39526) 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 -- 172.21.15.3:0/649319652 learned_addr learned my addr 172.21.15.3:0/649319652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349cc7d640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34980d0cd0 0x7f349815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349dc7f640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349815c210 0x7f349815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 -- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349815c210 msgr2=0x7f349815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349815c210 0x7f349815ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 -- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34980d0cd0 msgr2=0x7f349815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34980d0cd0 0x7f349815c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 -- 172.21.15.3:0/649319652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34980720e0 con 0x7f34980adf60 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349cc7d640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34980d0cd0 0x7f349815c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349d47e640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349809d360 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f349400c3c0 tx=0x7f349400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f348dffb640 1 -- 172.21.15.3:0/649319652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494017070 con 0x7f34980adf60 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f348dffb640 1 -- 172.21.15.3:0/649319652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34940052d0 con 0x7f34980adf60 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f348dffb640 1 -- 172.21.15.3:0/649319652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3494003840 con 0x7f34980adf60 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3498150a10 con 0x7f34980adf60 2024-09-12T12:23:10.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3498150fd0 con 0x7f34980adf60 2024-09-12T12:23:10.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349806e450 con 0x7f34980adf60 2024-09-12T12:23:10.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f348dffb640 1 -- 172.21.15.3:0/649319652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f349401a070 con 0x7f34980adf60 2024-09-12T12:23:10.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f348dffb640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f347c0678c0 0x7f347c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f348dffb640 1 -- 172.21.15.3:0/649319652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7f3494084d20 con 0x7f34980adf60 2024-09-12T12:23:10.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.496+0000 7f349cc7d640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f347c0678c0 0x7f347c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.500+0000 7f349cc7d640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f347c0678c0 0x7f347c069d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34981507b0 tx=0x7f3488009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.500+0000 7f348dffb640 1 -- 172.21.15.3:0/649319652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3494005480 con 0x7f34980adf60 2024-09-12T12:23:10.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.668+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f34980ae610 con 0x7f34980adf60 2024-09-12T12:23:10.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.668+0000 7f348dffb640 1 -- 172.21.15.3:0/649319652 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f3494055b60 con 0x7f34980adf60 2024-09-12T12:23:10.671 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f347c0678c0 msgr2=0x7f347c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f347c0678c0 0x7f347c069d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34981507b0 tx=0x7f3488009040 comp rx=0 tx=0).stop 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 msgr2=0x7f349809d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349809d360 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f349400c3c0 tx=0x7f349400c880 comp rx=0 tx=0).stop 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 shutdown_connections 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f347c0678c0 0x7f347c069d80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f349815c210 0x7f349815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34980d0cd0 0x7f349815c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 --2- 172.21.15.3:0/649319652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34980adf60 0x7f349809d360 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 >> 172.21.15.3:0/649319652 conn(0x7f3498067e00 msgr2=0x7f3498069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 shutdown_connections 2024-09-12T12:23:10.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.672+0000 7f349f6e2640 1 -- 172.21.15.3:0/649319652 wait complete. 2024-09-12T12:23:10.691 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-09-12T12:23:10.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 Processor -- start 2024-09-12T12:23:10.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 -- start start 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1c4073cb0 0x7fe1c415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe1c40b30e0 0x7fe1c40adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40ae150 0x7fe1c40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c4074090 con 0x7fe1c4073cb0 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c40b5000 con 0x7fe1c40b30e0 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1cb318640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c40b3530 con 0x7fe1c40ae150 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1c4073cb0 0x7fe1c415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe1c4073cb0 0x7fe1c415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39532/0 (socket says 172.21.15.3:39532) 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 -- 172.21.15.3:0/279314701 learned_addr learned my addr 172.21.15.3:0/279314701 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c88b3640 1 --2- 172.21.15.3:0/279314701 >> v2:172.21.15.142:3300/0 conn(0x7fe1c40b30e0 0x7fe1c40adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c98b5640 1 --2- 172.21.15.3:0/279314701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40ae150 0x7fe1c40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 -- 172.21.15.3:0/279314701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40ae150 msgr2=0x7fe1c40ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:10.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 --2- 172.21.15.3:0/279314701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40ae150 0x7fe1c40ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 -- 172.21.15.3:0/279314701 >> v2:172.21.15.142:3300/0 conn(0x7fe1c40b30e0 msgr2=0x7fe1c40adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:10.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 --2- 172.21.15.3:0/279314701 >> v2:172.21.15.142:3300/0 conn(0x7fe1c40b30e0 0x7fe1c40adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c90b4640 1 -- 172.21.15.3:0/279314701 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c40b37c0 con 0x7fe1c4073cb0 2024-09-12T12:23:10.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.808+0000 7fe1c98b5640 1 --2- 172.21.15.3:0/279314701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40ae150 0x7fe1c40ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:10.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1c90b4640 1 --2- 172.21.15.3:0/279314701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c415c710 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe1c406efc0 tx=0x7fe1c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab8318e5a9b91ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1bbfff640 1 -- 172.21.15.3:0/279314701 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00b0070 con 0x7fe1c4073cb0 2024-09-12T12:23:10.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1bbfff640 1 -- 172.21.15.3:0/279314701 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c00ab600 con 0x7fe1c4073cb0 2024-09-12T12:23:10.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1bbfff640 1 -- 172.21.15.3:0/279314701 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00ab9d0 con 0x7fe1c4073cb0 2024-09-12T12:23:10.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- 172.21.15.3:0/279314701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 msgr2=0x7fe1c415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 --2- 172.21.15.3:0/279314701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c415c710 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe1c406efc0 tx=0x7fe1c001c520 comp rx=0 tx=0).stop 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- 172.21.15.3:0/279314701 shutdown_connections 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 --2- 172.21.15.3:0/279314701 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40ae150 0x7fe1c40ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 --2- 172.21.15.3:0/279314701 >> v2:172.21.15.142:3300/0 conn(0x7fe1c40b30e0 0x7fe1c40adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 --2- 172.21.15.3:0/279314701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c415c710 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- 172.21.15.3:0/279314701 >> 172.21.15.3:0/279314701 conn(0x7fe1c4067e00 msgr2=0x7fe1c4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- 172.21.15.3:0/279314701 shutdown_connections 2024-09-12T12:23:10.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- 172.21.15.3:0/279314701 wait complete. 2024-09-12T12:23:10.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 Processor -- start 2024-09-12T12:23:10.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- start start 2024-09-12T12:23:10.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c41f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c40ae150 0x7fe1c4156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40b30e0 0x7fe1c409a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c415fe80 con 0x7fe1c4073cb0 2024-09-12T12:23:10.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c415fd00 con 0x7fe1c40ae150 2024-09-12T12:23:10.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1cb318640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c4160000 con 0x7fe1c40b30e0 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.812+0000 7fe1c90b4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c41f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c41f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39546/0 (socket says 172.21.15.3:39546) 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 -- 172.21.15.3:0/4052965328 learned_addr learned my addr 172.21.15.3:0/4052965328 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c88b3640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c40ae150 0x7fe1c4156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c98b5640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40b30e0 0x7fe1c409a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 -- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40b30e0 msgr2=0x7fe1c409a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40b30e0 0x7fe1c409a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 -- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c40ae150 msgr2=0x7fe1c4156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c40ae150 0x7fe1c4156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 -- 172.21.15.3:0/4052965328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c40ccc20 con 0x7fe1c4073cb0 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c88b3640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c40ae150 0x7fe1c4156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c90b4640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c41f51a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe1c001abd0 tx=0x7fe1c00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1b9ffb640 1 -- 172.21.15.3:0/4052965328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00ba070 con 0x7fe1c4073cb0 2024-09-12T12:23:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1b9ffb640 1 -- 172.21.15.3:0/4052965328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c00b4620 con 0x7fe1c4073cb0 2024-09-12T12:23:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c40ccf00 con 0x7fe1c4073cb0 2024-09-12T12:23:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1b9ffb640 1 -- 172.21.15.3:0/4052965328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00b0040 con 0x7fe1c4073cb0 2024-09-12T12:23:10.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c40cd4c0 con 0x7fe1c4073cb0 2024-09-12T12:23:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c40ae880 con 0x7fe1c4073cb0 2024-09-12T12:23:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1b9ffb640 1 -- 172.21.15.3:0/4052965328 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe1c00bd070 con 0x7fe1c4073cb0 2024-09-12T12:23:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1b9ffb640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe1a80678c0 0x7fe1a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1b9ffb640 1 -- 172.21.15.3:0/4052965328 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7fe1c011eba0 con 0x7fe1c4073cb0 2024-09-12T12:23:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c88b3640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe1a80678c0 0x7fe1a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:10.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.816+0000 7fe1c88b3640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe1a80678c0 0x7fe1a8069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40ad620 tx=0x7fe1b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:10.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:10.820+0000 7fe1b9ffb640 1 -- 172.21.15.3:0/4052965328 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe1c0016110 con 0x7fe1c4073cb0 2024-09-12T12:23:11.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:11.044+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7fe1c415f290 con 0x7fe1c4073cb0 2024-09-12T12:23:12.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.016+0000 7fe1b9ffb640 1 -- 172.21.15.3:0/4052965328 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v184) ==== 146+0+0 (secure 0 0 0) 0x7fe1c00ef9e0 con 0x7fe1c4073cb0 2024-09-12T12:23:12.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe1a80678c0 msgr2=0x7fe1a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe1a80678c0 0x7fe1a8069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40ad620 tx=0x7fe1b4008040 comp rx=0 tx=0).stop 2024-09-12T12:23:12.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 msgr2=0x7fe1c41f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c41f51a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe1c001abd0 tx=0x7fe1c00192c0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 shutdown_connections 2024-09-12T12:23:12.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe1a80678c0 0x7fe1a8069d80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe1c40b30e0 0x7fe1c409a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe1c40ae150 0x7fe1c4156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 --2- 172.21.15.3:0/4052965328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe1c4073cb0 0x7fe1c41f51a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 >> 172.21.15.3:0/4052965328 conn(0x7fe1c4067e00 msgr2=0x7fe1c40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:12.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 shutdown_connections 2024-09-12T12:23:12.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.020+0000 7fe1cb318640 1 -- 172.21.15.3:0/4052965328 wait complete. 2024-09-12T12:23:12.044 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T12:23:12.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd1d623640 1 Processor -- start 2024-09-12T12:23:12.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd1d623640 1 -- start start 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd1d623640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd180b3710 0x7fcd180ced90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd1d623640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd180cfae0 con 0x7fcd180cf2d0 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd1d623640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd18072240 con 0x7fcd1815e610 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd1d623640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcd180cf720 con 0x7fcd180b3710 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd177fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcd180cf2d0 0x7fcd180b75c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54610/0 (socket says 172.21.15.3:54610) 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd177fe640 1 -- 172.21.15.3:0/2594285984 learned_addr learned my addr 172.21.15.3:0/2594285984 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd167fc640 1 --2- 172.21.15.3:0/2594285984 >> v2:172.21.15.142:3300/0 conn(0x7fcd180b3710 0x7fcd180ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286617713 0 0) 0x7fcd180cfae0 con 0x7fcd180cf2d0 2024-09-12T12:23:12.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.172+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd00003530 con 0x7fcd180cf2d0 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1816558686 0 0) 0x7fcd00003530 con 0x7fcd180cf2d0 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 >> v2:172.21.15.142:3300/0 conn(0x7fcd180b3710 msgr2=0x7fcd180ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 --2- 172.21.15.3:0/2594285984 >> v2:172.21.15.142:3300/0 conn(0x7fcd180b3710 0x7fcd180ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 >> v1:172.21.15.142:6789/0 conn(0x7fcd1815e610 legacy=0x7fcd180735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd18072520 con 0x7fcd180cf2d0 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fcd10001680 con 0x7fcd180cf2d0 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd167fc640 1 --2- 172.21.15.3:0/2594285984 >> v2:172.21.15.142:3300/0 conn(0x7fcd180b3710 0x7fcd180ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 >> v1:172.21.15.3:6789/0 conn(0x7fcd180cf2d0 legacy=0x7fcd180b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd000039c0 0x7fcd00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd00004810 0x7fcd00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd000051d0 0x7fcd0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd180cf720 con 0x7fcd000039c0 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0000c100 con 0x7fcd000051d0 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0000c300 con 0x7fcd00004810 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd000039c0 0x7fcd00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd167fc640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd00004810 0x7fcd00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd16ffd640 1 -- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd00004810 msgr2=0x7fcd00004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd00004810 0x7fcd00004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd177fe640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd000051d0 0x7fcd0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd16ffd640 1 -- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd000051d0 msgr2=0x7fcd0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd000051d0 0x7fcd0000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd16ffd640 1 -- 172.21.15.3:0/2594285984 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd00003530 con 0x7fcd000039c0 2024-09-12T12:23:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd177fe640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd000051d0 0x7fcd0000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:12.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd177fe640 1 -- 172.21.15.3:0/2594285984 reap_dead start 2024-09-12T12:23:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd000039c0 0x7fcd00003de0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcd08001e70 tx=0x7fcd0801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b5dfdf5623871c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08001a90 con 0x7fcd000039c0 2024-09-12T12:23:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd08003b80 con 0x7fcd000039c0 2024-09-12T12:23:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd15ffb640 1 -- 172.21.15.3:0/2594285984 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08005080 con 0x7fcd000039c0 2024-09-12T12:23:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd1d623640 1 -- 172.21.15.3:0/2594285984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd000039c0 msgr2=0x7fcd00003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd1d623640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd000039c0 0x7fcd00003de0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcd08001e70 tx=0x7fcd0801c760 comp rx=0 tx=0).stop 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd1d623640 1 -- 172.21.15.3:0/2594285984 shutdown_connections 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd1d623640 1 --2- 172.21.15.3:0/2594285984 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd000039c0 0x7fcd00003de0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd1d623640 1 -- 172.21.15.3:0/2594285984 >> 172.21.15.3:0/2594285984 conn(0x7fcd180d4530 msgr2=0x7fcd180d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd1d623640 1 -- 172.21.15.3:0/2594285984 shutdown_connections 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.176+0000 7fcd1d623640 1 -- 172.21.15.3:0/2594285984 wait complete. 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 Processor -- start 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 -- start start 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd000039c0 0x7fcd180cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1815bb50 0x7fcd1815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1815aea0 0x7fcd1815b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18072ae0 con 0x7fcd1815bb50 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18073130 con 0x7fcd1815aea0 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcd18072dc0 con 0x7fcd000039c0 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1815bb50 0x7fcd1815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1815bb50 0x7fcd1815a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39554/0 (socket says 172.21.15.3:39554) 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 -- 172.21.15.3:0/1110553046 learned_addr learned my addr 172.21.15.3:0/1110553046 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd000039c0 0x7fcd180cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd177fe640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1815aea0 0x7fcd1815b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 -- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd000039c0 msgr2=0x7fcd180cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd000039c0 0x7fcd180cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 -- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1815aea0 msgr2=0x7fcd1815b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1815aea0 0x7fcd1815b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 -- 172.21.15.3:0/1110553046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd180c8190 con 0x7fcd1815bb50 2024-09-12T12:23:12.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd000039c0 0x7fcd180cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd177fe640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1815aea0 0x7fcd1815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd167fc640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1815bb50 0x7fcd1815a960 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00f580 tx=0x7fcd0c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fccff7fe640 1 -- 172.21.15.3:0/1110553046 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c018070 con 0x7fcd1815bb50 2024-09-12T12:23:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fccff7fe640 1 -- 172.21.15.3:0/1110553046 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c014890 con 0x7fcd1815bb50 2024-09-12T12:23:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1815b890 con 0x7fcd1815bb50 2024-09-12T12:23:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fccff7fe640 1 -- 172.21.15.3:0/1110553046 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c014c60 con 0x7fcd1815bb50 2024-09-12T12:23:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd180cd200 con 0x7fcd1815bb50 2024-09-12T12:23:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.180+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1815e9f0 con 0x7fcd1815bb50 2024-09-12T12:23:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.184+0000 7fccff7fe640 1 -- 172.21.15.3:0/1110553046 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd0c01b070 con 0x7fcd1815bb50 2024-09-12T12:23:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.184+0000 7fccff7fe640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd040678c0 0x7fcd04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.184+0000 7fccff7fe640 1 -- 172.21.15.3:0/1110553046 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7fcd0c085c60 con 0x7fcd1815bb50 2024-09-12T12:23:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.184+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd040678c0 0x7fcd04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.184+0000 7fcd16ffd640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd040678c0 0x7fcd04069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005000 tx=0x7fcd080a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:12.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.188+0000 7fccff7fe640 1 -- 172.21.15.3:0/1110553046 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcd1815b890 con 0x7fcd1815bb50 2024-09-12T12:23:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.344+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcd180b9810 con 0x7fcd1815bb50 2024-09-12T12:23:12.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.348+0000 7fccff7fe640 1 -- 172.21.15.3:0/1110553046 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fcd0c056ab0 con 0x7fcd1815bb50 2024-09-12T12:23:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd040678c0 msgr2=0x7fcd04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd040678c0 0x7fcd04069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005000 tx=0x7fcd080a9040 comp rx=0 tx=0).stop 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1815bb50 msgr2=0x7fcd1815a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1815bb50 0x7fcd1815a960 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00f580 tx=0x7fcd0c00fa40 comp rx=0 tx=0).stop 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 shutdown_connections 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd040678c0 0x7fcd04069d80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd1815aea0 0x7fcd1815b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd1815bb50 0x7fcd1815a960 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 --2- 172.21.15.3:0/1110553046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd000039c0 0x7fcd180cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 >> 172.21.15.3:0/1110553046 conn(0x7fcd180d4530 msgr2=0x7fcd1815ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 shutdown_connections 2024-09-12T12:23:12.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.352+0000 7fcd1d623640 1 -- 172.21.15.3:0/1110553046 wait complete. 2024-09-12T12:23:12.371 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:23:12.371 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:12.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:12.469 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:23:12.469 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:12.469 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:12.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:12.474 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:12.474 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:12.516 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:12.517 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:12.517 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:12.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:12.521 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:12.521 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:12.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:12.569 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T12:23:12.569 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:12.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e7fcf640 1 Processor -- start 2024-09-12T12:23:12.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e7fcf640 1 -- start start 2024-09-12T12:23:12.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e7fcf640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25e0073cb0 0x7f25e015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e7fcf640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e00b5000 con 0x7f25e00ae880 2024-09-12T12:23:12.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e7fcf640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e00b3530 con 0x7f25e00b30e0 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e7fcf640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f25e0074090 con 0x7f25e0073cb0 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e5d6b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f25e0073cb0 0x7f25e015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e656c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f25e00ae880 0x7f25e00d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54620/0 (socket says 172.21.15.3:54620) 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e656c640 1 -- 172.21.15.3:0/2919473883 learned_addr learned my addr 172.21.15.3:0/2919473883 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e5d6b640 1 -- 172.21.15.3:0/2919473883 >> v1:172.21.15.142:6789/0 conn(0x7f25e00b30e0 legacy=0x7f25e00adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e5d6b640 1 -- 172.21.15.3:0/2919473883 >> v1:172.21.15.3:6789/0 conn(0x7f25e00ae880 legacy=0x7f25e00d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e5d6b640 1 -- 172.21.15.3:0/2919473883 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e00b37c0 con 0x7f25e0073cb0 2024-09-12T12:23:12.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.696+0000 7f25e4d69640 1 -- 172.21.15.3:0/2919473883 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756471262 0 0) 0x7f25e00b5000 con 0x7f25e00ae880 2024-09-12T12:23:12.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e5d6b640 1 --2- 172.21.15.3:0/2919473883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e0073cb0 0x7f25e015c710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f25d00097c0 tx=0x7f25d001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88449b0a7f5fcb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:12.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e4d69640 1 -- 172.21.15.3:0/2919473883 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00b1070 con 0x7f25e0073cb0 2024-09-12T12:23:12.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e4d69640 1 -- 172.21.15.3:0/2919473883 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d00ac5f0 con 0x7f25e0073cb0 2024-09-12T12:23:12.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e4d69640 1 -- 172.21.15.3:0/2919473883 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00ac9c0 con 0x7f25e0073cb0 2024-09-12T12:23:12.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- 172.21.15.3:0/2919473883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e0073cb0 msgr2=0x7f25e015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/2919473883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e0073cb0 0x7f25e015c710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f25d00097c0 tx=0x7f25d001c4a0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- 172.21.15.3:0/2919473883 shutdown_connections 2024-09-12T12:23:12.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/2919473883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e0073cb0 0x7f25e015c710 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- 172.21.15.3:0/2919473883 >> 172.21.15.3:0/2919473883 conn(0x7f25e0067e00 msgr2=0x7f25e0068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:12.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- 172.21.15.3:0/2919473883 shutdown_connections 2024-09-12T12:23:12.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- 172.21.15.3:0/2919473883 wait complete. 2024-09-12T12:23:12.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 Processor -- start 2024-09-12T12:23:12.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- start start 2024-09-12T12:23:12.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0073cb0 0x7f25e00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00ae880 0x7f25e0148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e00b30e0 0x7f25e00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25e00ae4a0 con 0x7f25e00b30e0 2024-09-12T12:23:12.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25e00ae320 con 0x7f25e0073cb0 2024-09-12T12:23:12.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e7fcf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f25e00d0cc0 con 0x7f25e00ae880 2024-09-12T12:23:12.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e00b30e0 0x7f25e00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e00b30e0 0x7f25e00b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39562/0 (socket says 172.21.15.3:39562) 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 -- 172.21.15.3:0/3109870975 learned_addr learned my addr 172.21.15.3:0/3109870975 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e5d6b640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0073cb0 0x7f25e00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e556a640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00ae880 0x7f25e0148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 -- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00ae880 msgr2=0x7f25e0148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00ae880 0x7f25e0148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 -- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0073cb0 msgr2=0x7f25e00afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0073cb0 0x7f25e00afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.700+0000 7f25e656c640 1 -- 172.21.15.3:0/3109870975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e01486b0 con 0x7f25e00b30e0 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25e656c640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e00b30e0 0x7f25e00b0300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c7a0 tx=0x7f25d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:12.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25d67fc640 1 -- 172.21.15.3:0/3109870975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8018070 con 0x7f25e00b30e0 2024-09-12T12:23:12.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25d67fc640 1 -- 172.21.15.3:0/3109870975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d800a640 con 0x7f25e00b30e0 2024-09-12T12:23:12.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25d67fc640 1 -- 172.21.15.3:0/3109870975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800aab0 con 0x7f25e00b30e0 2024-09-12T12:23:12.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e00cee00 con 0x7f25e00b30e0 2024-09-12T12:23:12.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e00cf3c0 con 0x7f25e00b30e0 2024-09-12T12:23:12.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e015c710 con 0x7f25e00b30e0 2024-09-12T12:23:12.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25d67fc640 1 -- 172.21.15.3:0/3109870975 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f25d800ac50 con 0x7f25e00b30e0 2024-09-12T12:23:12.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25d67fc640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25c40678c0 0x7f25c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:12.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25d67fc640 1 -- 172.21.15.3:0/3109870975 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7f25d8084ff0 con 0x7f25e00b30e0 2024-09-12T12:23:12.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25e5d6b640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25c40678c0 0x7f25c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:12.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.704+0000 7f25e5d6b640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25c40678c0 0x7f25c4069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f25d00ae000 tx=0x7f25d00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:12.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.708+0000 7f25d67fc640 1 -- 172.21.15.3:0/3109870975 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f25d8055e30 con 0x7f25e00b30e0 2024-09-12T12:23:12.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.908+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25e014f580 con 0x7f25e00b30e0 2024-09-12T12:23:12.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.908+0000 7f25d67fc640 1 -- 172.21.15.3:0/3109870975 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3908 (secure 0 0 0) 0x7f25d8055730 con 0x7f25e00b30e0 2024-09-12T12:23:12.913 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:12.913 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":38,"btime":"2024-09-12T12:23:06:887138+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/256387515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":256387515},{"type":"v1","addr":"172.21.15.142:6835","nonce":256387515}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2534794519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2534794519},{"type":"v1","addr":"172.21.15.142:6837","nonce":2534794519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2676380928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2676380928},{"type":"v1","addr":"172.21.15.3:6833","nonce":2676380928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3483379033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3483379033},{"type":"v1","addr":"172.21.15.3:6835","nonce":3483379033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-09-12T12:23:12.913 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 38 2024-09-12T12:23:12.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25c40678c0 msgr2=0x7f25c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25c40678c0 0x7f25c4069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f25d00ae000 tx=0x7f25d00a7000 comp rx=0 tx=0).stop 2024-09-12T12:23:12.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e00b30e0 msgr2=0x7f25e00b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e00b30e0 0x7f25e00b0300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f25d800c7a0 tx=0x7f25d800cc60 comp rx=0 tx=0).stop 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 shutdown_connections 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f25c40678c0 0x7f25c4069d80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25e00b30e0 0x7f25e00b0300 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25e00ae880 0x7f25e0148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 --2- 172.21.15.3:0/3109870975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25e0073cb0 0x7f25e00afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 >> 172.21.15.3:0/3109870975 conn(0x7f25e0067e00 msgr2=0x7f25e00b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 shutdown_connections 2024-09-12T12:23:12.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:12.912+0000 7f25e7fcf640 1 -- 172.21.15.3:0/3109870975 wait complete. 2024-09-12T12:23:12.932 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:23:12.932 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T12:23:12.932 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:23:12.932 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:12.933 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:23:12.933 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T12:23:12.933 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:23:12.933 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:12.933 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:23:12.933 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T12:23:12.933 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:23:12.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:12.934 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:23:12.934 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T12:23:12.934 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:23:12.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:12.944 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:23:12.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:23:12.945 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:23:12.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:23:12.948 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:23:12.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:23:12.949 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:23:12.949 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:23:12.990 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:23:12.991 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:12.992 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:23:12.992 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:12.993 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:23:12.993 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:12.993 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:23:12.994 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:12.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T12:23:13.050 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:23:13.050 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:23:13.062 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:23:13.065 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:23:13.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.108+0000 7f47e12c1640 1 Processor -- start 2024-09-12T12:23:13.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.108+0000 7f47e12c1640 1 -- start start 2024-09-12T12:23:13.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47dc0cb870 0x7f47dc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc0b0200 0x7f47dc06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47dc0cc190 con 0x7f47dc06fa80 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f47dc06de90 con 0x7f47dc0cb870 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47dc06e450 con 0x7f47dc0b0200 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47dc0cb870 0x7f47dc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47dc0cb870 0x7f47dc0cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39600/0 (socket says 172.21.15.3:39600) 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 -- 172.21.15.3:0/1091349897 learned_addr learned my addr 172.21.15.3:0/1091349897 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47db7fe640 1 --2- 172.21.15.3:0/1091349897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc0b0200 0x7f47dc06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 -- 172.21.15.3:0/1091349897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc0b0200 msgr2=0x7f47dc06eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 --2- 172.21.15.3:0/1091349897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc0b0200 0x7f47dc06eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 -- 172.21.15.3:0/1091349897 >> v1:172.21.15.142:6789/0 conn(0x7f47dc06fa80 legacy=0x7f47dc06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 -- 172.21.15.3:0/1091349897 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc15f970 con 0x7f47dc0cb870 2024-09-12T12:23:13.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47db7fe640 1 --2- 172.21.15.3:0/1091349897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc0b0200 0x7f47dc06eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:13.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47daffd640 1 --2- 172.21.15.3:0/1091349897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc0cbc50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009e50 tx=0x7f47cc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2afd5ef8159d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47d9ffb640 1 -- 172.21.15.3:0/1091349897 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0a57b0 con 0x7f47dc0cb870 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47d9ffb640 1 -- 172.21.15.3:0/1091349897 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc0a5950 con 0x7f47dc0cb870 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47d9ffb640 1 -- 172.21.15.3:0/1091349897 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0a5d20 con 0x7f47dc0cb870 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 -- 172.21.15.3:0/1091349897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 msgr2=0x7f47dc0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 --2- 172.21.15.3:0/1091349897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc0cbc50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009e50 tx=0x7f47cc01c930 comp rx=0 tx=0).stop 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 -- 172.21.15.3:0/1091349897 shutdown_connections 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 --2- 172.21.15.3:0/1091349897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc0b0200 0x7f47dc06eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 --2- 172.21.15.3:0/1091349897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc0cbc50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.112+0000 7f47e12c1640 1 -- 172.21.15.3:0/1091349897 >> 172.21.15.3:0/1091349897 conn(0x7f47dc0d1220 msgr2=0x7f47dc0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- 172.21.15.3:0/1091349897 shutdown_connections 2024-09-12T12:23:13.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- 172.21.15.3:0/1091349897 wait complete. 2024-09-12T12:23:13.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 Processor -- start 2024-09-12T12:23:13.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- start start 2024-09-12T12:23:13.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc06fa80 0x7f47dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47dc0b0200 0x7f47dc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc067bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc0b08c0 con 0x7f47dc0cb870 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc0b0a40 con 0x7f47dc0b0200 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47dc15c5b0 con 0x7f47dc06fa80 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc067bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc067bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39608/0 (socket says 172.21.15.3:39608) 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 -- 172.21.15.3:0/2134916800 learned_addr learned my addr 172.21.15.3:0/2134916800 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47daffd640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc06fa80 0x7f47dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47da7fc640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47dc0b0200 0x7f47dc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 -- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc06fa80 msgr2=0x7f47dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc06fa80 0x7f47dc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 -- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47dc0b0200 msgr2=0x7f47dc0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47dc0b0200 0x7f47dc0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 -- 172.21.15.3:0/2134916800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc15e740 con 0x7f47dc0cb870 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47db7fe640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc067bd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f47d400c820 tx=0x7f47d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47d37fe640 1 -- 172.21.15.3:0/2134916800 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4015070 con 0x7f47dc0cb870 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47d37fe640 1 -- 172.21.15.3:0/2134916800 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d4007a70 con 0x7f47dc0cb870 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc15e460 con 0x7f47dc0cb870 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47d37fe640 1 -- 172.21.15.3:0/2134916800 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d4007e60 con 0x7f47dc0cb870 2024-09-12T12:23:13.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc15cdd0 con 0x7f47dc0cb870 2024-09-12T12:23:13.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.116+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47dc15d2b0 con 0x7f47dc0cb870 2024-09-12T12:23:13.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.120+0000 7f47d37fe640 1 -- 172.21.15.3:0/2134916800 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47d4018070 con 0x7f47dc0cb870 2024-09-12T12:23:13.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.120+0000 7f47d37fe640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f47bc0678c0 0x7f47bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.120+0000 7f47d37fe640 1 -- 172.21.15.3:0/2134916800 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4943+0+0 (secure 0 0 0) 0x7f47d4085830 con 0x7f47dc0cb870 2024-09-12T12:23:13.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.120+0000 7f47daffd640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f47bc0678c0 0x7f47bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.120+0000 7f47daffd640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f47bc0678c0 0x7f47bc069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0ae000 tx=0x7f47cc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:13.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.124+0000 7f47d37fe640 1 -- 172.21.15.3:0/2134916800 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f47dc15d2b0 con 0x7f47dc0cb870 2024-09-12T12:23:13.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:13.292+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f47dc0cbc50 con 0x7f47dc0cb870 2024-09-12T12:23:14.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.032+0000 7f47d37fe640 1 -- 172.21.15.3:0/2134916800 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) ==== 97+0+0 (secure 0 0 0) 0x7f47d4056020 con 0x7f47dc0cb870 2024-09-12T12:23:14.037 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-12T12:23:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f47bc0678c0 msgr2=0x7f47bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f47bc0678c0 0x7f47bc069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0ae000 tx=0x7f47cc0a7070 comp rx=0 tx=0).stop 2024-09-12T12:23:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 msgr2=0x7f47dc067bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc067bd0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f47d400c820 tx=0x7f47d400cce0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 shutdown_connections 2024-09-12T12:23:14.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f47bc0678c0 0x7f47bc069d80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47dc0cb870 0x7f47dc067bd0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47dc0b0200 0x7f47dc0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 --2- 172.21.15.3:0/2134916800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47dc06fa80 0x7f47dc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 >> 172.21.15.3:0/2134916800 conn(0x7f47dc0d1220 msgr2=0x7f47dc0d1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.036+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 shutdown_connections 2024-09-12T12:23:14.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.040+0000 7f47e12c1640 1 -- 172.21.15.3:0/2134916800 wait complete. 2024-09-12T12:23:14.064 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T12:23:14.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c2cf8640 1 Processor -- start 2024-09-12T12:23:14.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c2cf8640 1 -- start start 2024-09-12T12:23:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c2cf8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f03bc0d4150 0x7f03bc0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c2cf8640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03bc0d30b0 con 0x7f03bc0d1190 2024-09-12T12:23:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c2cf8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03bc0d3390 con 0x7f03bc0cf950 2024-09-12T12:23:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c2cf8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f03bc0d45e0 con 0x7f03bc0d4150 2024-09-12T12:23:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c0a94640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f03bc0d1190 0x7f03bc0d1570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54636/0 (socket says 172.21.15.3:54636) 2024-09-12T12:23:14.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c0a94640 1 -- 172.21.15.3:0/3172150415 learned_addr learned my addr 172.21.15.3:0/3172150415 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582389036 0 0) 0x7f03bc0d30b0 con 0x7f03bc0d1190 2024-09-12T12:23:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03a4003530 con 0x7f03bc0d1190 2024-09-12T12:23:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2501244172 0 0) 0x7f03a4003530 con 0x7f03bc0d1190 2024-09-12T12:23:14.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bbfff640 1 --2- 172.21.15.3:0/3172150415 >> v2:172.21.15.142:3300/0 conn(0x7f03bc0d4150 0x7f03bc0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 >> v2:172.21.15.142:3300/0 conn(0x7f03bc0d4150 msgr2=0x7f03bc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 --2- 172.21.15.3:0/3172150415 >> v2:172.21.15.142:3300/0 conn(0x7f03bc0d4150 0x7f03bc0cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 >> v1:172.21.15.142:6789/0 conn(0x7f03bc0cf950 legacy=0x7f03bc0b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc0d3670 con 0x7f03bc0d1190 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f03ac001680 con 0x7f03bc0d1190 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 >> v1:172.21.15.3:6789/0 conn(0x7f03bc0d1190 legacy=0x7f03bc0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a40039c0 0x7f03a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03a40052f0 0x7f03a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc0d45e0 con 0x7f03a4004810 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03a400c1e0 con 0x7f03a40039c0 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03a400c3e0 con 0x7f03a40052f0 2024-09-12T12:23:14.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bbfff640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c1295640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03a40052f0 0x7f03a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c0a94640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a40039c0 0x7f03a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bbfff640 1 -- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03a40052f0 msgr2=0x7f03a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bbfff640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03a40052f0 0x7f03a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bbfff640 1 -- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a40039c0 msgr2=0x7f03a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bbfff640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a40039c0 0x7f03a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03bbfff640 1 -- 172.21.15.3:0/3172150415 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a4003530 con 0x7f03a4004810 2024-09-12T12:23:14.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.184+0000 7f03c1295640 1 -- 172.21.15.3:0/3172150415 reap_dead start 2024-09-12T12:23:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03bbfff640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f03b0006fd0 tx=0x7f03b001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda17588afd4886e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b000a590 con 0x7f03a4004810 2024-09-12T12:23:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b00b0070 con 0x7f03a4004810 2024-09-12T12:23:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03bb7fe640 1 -- 172.21.15.3:0/3172150415 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b00ab8c0 con 0x7f03a4004810 2024-09-12T12:23:14.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 -- 172.21.15.3:0/3172150415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4004810 msgr2=0x7f03a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f03b0006fd0 tx=0x7f03b001c530 comp rx=0 tx=0).stop 2024-09-12T12:23:14.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 -- 172.21.15.3:0/3172150415 shutdown_connections 2024-09-12T12:23:14.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/3172150415 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4004810 0x7f03a4004c90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 -- 172.21.15.3:0/3172150415 >> 172.21.15.3:0/3172150415 conn(0x7f03bc067a00 msgr2=0x7f03bc067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:14.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 -- 172.21.15.3:0/3172150415 shutdown_connections 2024-09-12T12:23:14.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 -- 172.21.15.3:0/3172150415 wait complete. 2024-09-12T12:23:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 Processor -- start 2024-09-12T12:23:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.188+0000 7f03c2cf8640 1 -- start start 2024-09-12T12:23:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a4004810 0x7f03bc150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc0b91b0 0x7f03bc0b9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03bc1f5d90 0x7f03bc1f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc0d2cd0 con 0x7f03bc1f5d90 2024-09-12T12:23:14.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc0d3cc0 con 0x7f03a4004810 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f03bc0d2e50 con 0x7f03bc0b91b0 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03bc1f5d90 0x7f03bc1f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03bc1f5d90 0x7f03bc1f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39620/0 (socket says 172.21.15.3:39620) 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 -- 172.21.15.3:0/188862829 learned_addr learned my addr 172.21.15.3:0/188862829 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03bbfff640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc0b91b0 0x7f03bc0b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c0a94640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a4004810 0x7f03bc150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 -- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc0b91b0 msgr2=0x7f03bc0b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc0b91b0 0x7f03bc0b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 -- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a4004810 msgr2=0x7f03bc150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a4004810 0x7f03bc150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 -- 172.21.15.3:0/188862829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc1f6750 con 0x7f03bc1f5d90 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c1295640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03bc1f5d90 0x7f03bc1f6210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0ba100 tx=0x7f03b400e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03b97fa640 1 -- 172.21.15.3:0/188862829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b401b070 con 0x7f03bc1f5d90 2024-09-12T12:23:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03b97fa640 1 -- 172.21.15.3:0/188862829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b4013630 con 0x7f03bc1f5d90 2024-09-12T12:23:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc1511f0 con 0x7f03bc1f5d90 2024-09-12T12:23:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03b97fa640 1 -- 172.21.15.3:0/188862829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4013a00 con 0x7f03bc1f5d90 2024-09-12T12:23:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc1f6cc0 con 0x7f03bc1f5d90 2024-09-12T12:23:14.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03bc0b6d00 con 0x7f03bc1f5d90 2024-09-12T12:23:14.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03b97fa640 1 -- 172.21.15.3:0/188862829 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f03b401e070 con 0x7f03bc1f5d90 2024-09-12T12:23:14.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03b97fa640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f03a80678f0 0x7f03a8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03b97fa640 1 -- 172.21.15.3:0/188862829 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4857+0+0 (secure 0 0 0) 0x7f03b40854c0 con 0x7f03bc1f5d90 2024-09-12T12:23:14.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.192+0000 7f03c0a94640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f03a80678f0 0x7f03a8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.196+0000 7f03c0a94640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f03a80678f0 0x7f03a8069db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f03ac001d00 tx=0x7f03ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.196+0000 7f03b97fa640 1 -- 172.21.15.3:0/188862829 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f03b40563e0 con 0x7f03bc1f5d90 2024-09-12T12:23:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.424+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f03bc06bf40 con 0x7f03bc1f5d90 2024-09-12T12:23:14.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.424+0000 7f03b97fa640 1 -- 172.21.15.3:0/188862829 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5580 (secure 0 0 0) 0x7f03b4055ce0 con 0x7f03bc1f5d90 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg==", 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:14.428 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA==", 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg==", 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-12T12:23:14.430 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw==", 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w==", 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.431 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA==", 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.432 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw==", 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg==", 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.433 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA==", 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.434 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ==", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ==", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.435 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA==", 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.436 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA==", 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw==", 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw==", 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA==", 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-12T12:23:14.439 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKNVOAxAAMmejHiBys/ZYsGdB0oiroQ==", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.badguy", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAa3eJmiaLANxAA6CEjP8cJQrqnH02M0tfzTg==", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow rw path=/foo/bar", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-12T12:23:14.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw" 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA==", 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-12T12:23:14.441 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg==", 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:23:14.442 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:23:14.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f03a80678f0 msgr2=0x7f03a8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f03a80678f0 0x7f03a8069db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f03ac001d00 tx=0x7f03ac009040 comp rx=0 tx=0).stop 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03bc1f5d90 msgr2=0x7f03bc1f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03bc1f5d90 0x7f03bc1f6210 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0ba100 tx=0x7f03b400e980 comp rx=0 tx=0).stop 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 shutdown_connections 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f03a80678f0 0x7f03a8069db0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03bc1f5d90 0x7f03bc1f6210 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f03bc0b91b0 0x7f03bc0b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 --2- 172.21.15.3:0/188862829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03a4004810 0x7f03bc150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 >> 172.21.15.3:0/188862829 conn(0x7f03bc067a00 msgr2=0x7f03bc0b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 shutdown_connections 2024-09-12T12:23:14.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.428+0000 7f03c2cf8640 1 -- 172.21.15.3:0/188862829 wait complete. 2024-09-12T12:23:14.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.badguy 2024-09-12T12:23:14.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59d484b640 1 Processor -- start 2024-09-12T12:23:14.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59d484b640 1 -- start start 2024-09-12T12:23:14.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59d484b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f59d00ae250 0x7f59d00ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59d484b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59d015b620 0x7f59d015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59d484b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59d00ad0c0 con 0x7f59d00d05b0 2024-09-12T12:23:14.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59d484b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f59d00ad3a0 con 0x7f59d015b620 2024-09-12T12:23:14.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59d484b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f59d00ad680 con 0x7f59d00ae250 2024-09-12T12:23:14.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59ced9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59d015b620 0x7f59d015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59ced9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59d015b620 0x7f59d015ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39622/0 (socket says 172.21.15.3:39622) 2024-09-12T12:23:14.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.568+0000 7f59ced9d640 1 -- 172.21.15.3:0/1987500674 learned_addr learned my addr 172.21.15.3:0/1987500674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:14.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59c7fff640 1 --2- 172.21.15.3:0/1987500674 >> v2:172.21.15.142:3300/0 conn(0x7f59d00ae250 0x7f59d00ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59ced9d640 1 -- 172.21.15.3:0/1987500674 >> v2:172.21.15.142:3300/0 conn(0x7f59d00ae250 msgr2=0x7f59d00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59ced9d640 1 --2- 172.21.15.3:0/1987500674 >> v2:172.21.15.142:3300/0 conn(0x7f59d00ae250 0x7f59d00ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59ced9d640 1 -- 172.21.15.3:0/1987500674 >> v1:172.21.15.142:6789/0 conn(0x7f59d00d05b0 legacy=0x7f59d00d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59ced9d640 1 -- 172.21.15.3:0/1987500674 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d00ad960 con 0x7f59d015b620 2024-09-12T12:23:14.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59ced9d640 1 --2- 172.21.15.3:0/1987500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d015b620 0x7f59d015ba70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009e50 tx=0x7f59c801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3e52582e2a99e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59cdd9b640 1 -- 172.21.15.3:0/1987500674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a5730 con 0x7f59d015b620 2024-09-12T12:23:14.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59cdd9b640 1 -- 172.21.15.3:0/1987500674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c80a58d0 con 0x7f59d015b620 2024-09-12T12:23:14.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59cdd9b640 1 -- 172.21.15.3:0/1987500674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c80a5ca0 con 0x7f59d015b620 2024-09-12T12:23:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 -- 172.21.15.3:0/1987500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d015b620 msgr2=0x7f59d015ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 --2- 172.21.15.3:0/1987500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d015b620 0x7f59d015ba70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009e50 tx=0x7f59c801c980 comp rx=0 tx=0).stop 2024-09-12T12:23:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 -- 172.21.15.3:0/1987500674 shutdown_connections 2024-09-12T12:23:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 --2- 172.21.15.3:0/1987500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d015b620 0x7f59d015ba70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 --2- 172.21.15.3:0/1987500674 >> v2:172.21.15.142:3300/0 conn(0x7f59d00ae250 0x7f59d00ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 -- 172.21.15.3:0/1987500674 >> 172.21.15.3:0/1987500674 conn(0x7f59d0067e00 msgr2=0x7f59d0068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:14.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 -- 172.21.15.3:0/1987500674 shutdown_connections 2024-09-12T12:23:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.572+0000 7f59d484b640 1 -- 172.21.15.3:0/1987500674 wait complete. 2024-09-12T12:23:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 Processor -- start 2024-09-12T12:23:14.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 -- start start 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d00ae250 0x7f59d01586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59d00d05b0 0x7f59d009da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59d015b620 0x7f59d00cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59d015c9b0 con 0x7f59d00ae250 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59d015c830 con 0x7f59d00d05b0 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f59d0167cb0 con 0x7f59d015b620 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d00ae250 0x7f59d01586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d00ae250 0x7f59d01586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39626/0 (socket says 172.21.15.3:39626) 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 -- 172.21.15.3:0/3677423327 learned_addr learned my addr 172.21.15.3:0/3677423327 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:14.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59c7fff640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59d00d05b0 0x7f59d009da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ced9d640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59d015b620 0x7f59d00cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 -- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59d015b620 msgr2=0x7f59d00cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59d015b620 0x7f59d00cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 -- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59d00d05b0 msgr2=0x7f59d009da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59d00d05b0 0x7f59d009da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 -- 172.21.15.3:0/3677423327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d009dfb0 con 0x7f59d00ae250 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ced9d640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59d015b620 0x7f59d00cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59ce59c640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d00ae250 0x7f59d01586f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c450 tx=0x7f59c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59c6ffd640 1 -- 172.21.15.3:0/3677423327 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0012830 con 0x7f59d00ae250 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59c6ffd640 1 -- 172.21.15.3:0/3677423327 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c0017070 con 0x7f59d00ae250 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d0157650 con 0x7f59d00ae250 2024-09-12T12:23:14.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59c6ffd640 1 -- 172.21.15.3:0/3677423327 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0012c30 con 0x7f59d00ae250 2024-09-12T12:23:14.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.576+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d0157c10 con 0x7f59d00ae250 2024-09-12T12:23:14.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.580+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59d00af050 con 0x7f59d00ae250 2024-09-12T12:23:14.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.580+0000 7f59c6ffd640 1 -- 172.21.15.3:0/3677423327 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f59c001c460 con 0x7f59d00ae250 2024-09-12T12:23:14.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.580+0000 7f59c6ffd640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59b00678f0 0x7f59b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.580+0000 7f59c6ffd640 1 -- 172.21.15.3:0/3677423327 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4857+0+0 (secure 0 0 0) 0x7f59c0085790 con 0x7f59d00ae250 2024-09-12T12:23:14.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.580+0000 7f59c7fff640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59b00678f0 0x7f59b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.580+0000 7f59c7fff640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59b00678f0 0x7f59b0069db0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f59d0150c70 tx=0x7f59b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.584+0000 7f59c6ffd640 1 -- 172.21.15.3:0/3677423327 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f59c00564f0 con 0x7f59d00ae250 2024-09-12T12:23:14.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.800+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.badguy"} v 0) -- 0x7f59d009e180 con 0x7f59d00ae250 2024-09-12T12:23:14.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59c6ffd640 1 -- 172.21.15.3:0/3677423327 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.badguy"}]=0 v17) ==== 83+0+0 (secure 0 0 0) 0x7f59c0055df0 con 0x7f59d00ae250 2024-09-12T12:23:14.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59b00678f0 msgr2=0x7f59b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59b00678f0 0x7f59b0069db0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f59d0150c70 tx=0x7f59b8008040 comp rx=0 tx=0).stop 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d00ae250 msgr2=0x7f59d01586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d00ae250 0x7f59d01586f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f59c000c450 tx=0x7f59c000c910 comp rx=0 tx=0).stop 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 shutdown_connections 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59b00678f0 0x7f59b0069db0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59d015b620 0x7f59d00cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59d00d05b0 0x7f59d009da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 --2- 172.21.15.3:0/3677423327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59d00ae250 0x7f59d01586f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 >> 172.21.15.3:0/3677423327 conn(0x7f59d0067e00 msgr2=0x7f59d00681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 shutdown_connections 2024-09-12T12:23:14.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.804+0000 7f59d484b640 1 -- 172.21.15.3:0/3677423327 wait complete. 2024-09-12T12:23:14.824 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T12:23:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689fa8c640 1 Processor -- start 2024-09-12T12:23:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689fa8c640 1 -- start start 2024-09-12T12:23:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689fa8c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f689815c000 0x7f68980b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689fa8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980d05b0 0x7f68980d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689fa8c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f689815f160 con 0x7f68980af410 2024-09-12T12:23:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689fa8c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f689815f9f0 con 0x7f689815c000 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689fa8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f689815f430 con 0x7f68980d05b0 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689d828640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f689815c000 0x7f68980b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689d828640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f689815c000 0x7f68980b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39634/0 (socket says 172.21.15.3:39634) 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689d828640 1 -- 172.21.15.3:0/343479166 learned_addr learned my addr 172.21.15.3:0/343479166 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689e029640 1 --2- 172.21.15.3:0/343479166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980d05b0 0x7f68980d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689d828640 1 -- 172.21.15.3:0/343479166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980d05b0 msgr2=0x7f68980d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689d828640 1 --2- 172.21.15.3:0/343479166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980d05b0 0x7f68980d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689d828640 1 -- 172.21.15.3:0/343479166 >> v1:172.21.15.142:6789/0 conn(0x7f68980af410 legacy=0x7f689815ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.948+0000 7f689d828640 1 -- 172.21.15.3:0/343479166 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689815f710 con 0x7f689815c000 2024-09-12T12:23:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689d828640 1 --2- 172.21.15.3:0/343479166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f689815c000 0x7f68980b3a10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f68940098a0 tx=0x7f689401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e64617e86ed5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689c826640 1 -- 172.21.15.3:0/343479166 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940b0070 con 0x7f689815c000 2024-09-12T12:23:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689c826640 1 -- 172.21.15.3:0/343479166 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68940ab600 con 0x7f689815c000 2024-09-12T12:23:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689c826640 1 -- 172.21.15.3:0/343479166 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940ab9d0 con 0x7f689815c000 2024-09-12T12:23:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 -- 172.21.15.3:0/343479166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f689815c000 msgr2=0x7f68980b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 --2- 172.21.15.3:0/343479166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f689815c000 0x7f68980b3a10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f68940098a0 tx=0x7f689401c470 comp rx=0 tx=0).stop 2024-09-12T12:23:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 -- 172.21.15.3:0/343479166 shutdown_connections 2024-09-12T12:23:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 --2- 172.21.15.3:0/343479166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f68980d05b0 0x7f68980d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 --2- 172.21.15.3:0/343479166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f689815c000 0x7f68980b3a10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 -- 172.21.15.3:0/343479166 >> 172.21.15.3:0/343479166 conn(0x7f6898067e00 msgr2=0x7f6898068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 -- 172.21.15.3:0/343479166 shutdown_connections 2024-09-12T12:23:14.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 -- 172.21.15.3:0/343479166 wait complete. 2024-09-12T12:23:14.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.952+0000 7f689fa8c640 1 Processor -- start 2024-09-12T12:23:14.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 -- start start 2024-09-12T12:23:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68980af410 0x7f68981471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68980d05b0 0x7f68981476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689815c000 0x7f6898072ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f689806d9f0 con 0x7f68980af410 2024-09-12T12:23:14.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f68980b3f50 con 0x7f68980d05b0 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f689806db70 con 0x7f689815c000 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68980af410 0x7f68981471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68980af410 0x7f68981471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39650/0 (socket says 172.21.15.3:39650) 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 -- 172.21.15.3:0/1975071976 learned_addr learned my addr 172.21.15.3:0/1975071976 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689e029640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689815c000 0x7f6898072ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d027640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68980d05b0 0x7f68981476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 -- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689815c000 msgr2=0x7f6898072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689815c000 0x7f6898072ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 -- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68980d05b0 msgr2=0x7f68981476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68980d05b0 0x7f68981476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 -- 172.21.15.3:0/1975071976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6898150ee0 con 0x7f68980af410 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689e029640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689815c000 0x7f6898072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689d828640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68980af410 0x7f68981471b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f68940ae040 tx=0x7f689401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f688e7fc640 1 -- 172.21.15.3:0/1975071976 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940ba020 con 0x7f68980af410 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f688e7fc640 1 -- 172.21.15.3:0/1975071976 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68940b6640 con 0x7f68980af410 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68981511c0 con 0x7f68980af410 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f688e7fc640 1 -- 172.21.15.3:0/1975071976 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940b0040 con 0x7f68980af410 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6898151780 con 0x7f68980af410 2024-09-12T12:23:14.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68980d0a40 con 0x7f68980af410 2024-09-12T12:23:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f688e7fc640 1 -- 172.21.15.3:0/1975071976 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f68940bd020 con 0x7f68980af410 2024-09-12T12:23:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.956+0000 7f688e7fc640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f687c0678f0 0x7f687c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:14.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.960+0000 7f688e7fc640 1 -- 172.21.15.3:0/1975071976 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4857+0+0 (secure 0 0 0) 0x7f689411fa60 con 0x7f68980af410 2024-09-12T12:23:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.960+0000 7f689d027640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f687c0678f0 0x7f687c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:14.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.960+0000 7f689d027640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f687c0678f0 0x7f687c069db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6898070ce0 tx=0x7f6888009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:14.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:14.964+0000 7f688e7fc640 1 -- 172.21.15.3:0/1975071976 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f68940c4030 con 0x7f68980af410 2024-09-12T12:23:15.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.132+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6898159430 con 0x7f68980af410 2024-09-12T12:23:15.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.132+0000 7f688e7fc640 1 -- 172.21.15.3:0/1975071976 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) ==== 84+0+65 (secure 0 0 0) 0x7f68940160f0 con 0x7f68980af410 2024-09-12T12:23:15.136 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:15.136 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T12:23:15.136 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:15.137 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T12:23:15.137 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T12:23:15.137 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:15.137 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T12:23:15.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f687c0678f0 msgr2=0x7f687c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f687c0678f0 0x7f687c069db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6898070ce0 tx=0x7f6888009060 comp rx=0 tx=0).stop 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68980af410 msgr2=0x7f68981471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68980af410 0x7f68981471b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f68940ae040 tx=0x7f689401acb0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 shutdown_connections 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f687c0678f0 0x7f687c069db0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f689815c000 0x7f6898072ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68980d05b0 0x7f68981476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 --2- 172.21.15.3:0/1975071976 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68980af410 0x7f68981471b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 >> 172.21.15.3:0/1975071976 conn(0x7f6898067e00 msgr2=0x7f68980681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 shutdown_connections 2024-09-12T12:23:15.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.136+0000 7f689fa8c640 1 -- 172.21.15.3:0/1975071976 wait complete. 2024-09-12T12:23:15.156 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:23:15.156 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:23:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f82e8640 1 Processor -- start 2024-09-12T12:23:15.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f82e8640 1 -- start start 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f82e8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f015e610 0x7f99f0162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f82e8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f99f0162770 0x7f99f0162bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f82e8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f00d1050 con 0x7f99f0067a00 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f82e8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f99f00b1350 con 0x7f99f0162770 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f82e8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f99f00cfc10 con 0x7f99f015e610 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f5883640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f99f0162770 0x7f99f0162bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f6084640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f015e610 0x7f99f0162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.268+0000 7f99f6885640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f99f0067a00 0x7f99f0067eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:52354/0 (socket says 172.21.15.3:52354) 2024-09-12T12:23:15.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f6885640 1 -- 172.21.15.3:0/310986385 learned_addr learned my addr 172.21.15.3:0/310986385 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:15.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5883640 1 -- 172.21.15.3:0/310986385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f015e610 msgr2=0x7f99f0162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:15.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5883640 1 --2- 172.21.15.3:0/310986385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f015e610 0x7f99f0162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5883640 1 -- 172.21.15.3:0/310986385 >> v1:172.21.15.142:6789/0 conn(0x7f99f0067a00 legacy=0x7f99f0067eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:15.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5883640 1 -- 172.21.15.3:0/310986385 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f00b22a0 con 0x7f99f0162770 2024-09-12T12:23:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5883640 1 --2- 172.21.15.3:0/310986385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f0162770 0x7f99f0162bb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f99e00097c0 tx=0x7f99e001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8036e5ffa792896a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5082640 1 -- 172.21.15.3:0/310986385 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e00b0070 con 0x7f99f0162770 2024-09-12T12:23:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5082640 1 -- 172.21.15.3:0/310986385 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e00ab600 con 0x7f99f0162770 2024-09-12T12:23:15.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f5082640 1 -- 172.21.15.3:0/310986385 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e00ab9d0 con 0x7f99f0162770 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 -- 172.21.15.3:0/310986385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f0162770 msgr2=0x7f99f0162bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 --2- 172.21.15.3:0/310986385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f0162770 0x7f99f0162bb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f99e00097c0 tx=0x7f99e001c410 comp rx=0 tx=0).stop 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 -- 172.21.15.3:0/310986385 shutdown_connections 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 --2- 172.21.15.3:0/310986385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f0162770 0x7f99f0162bb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 --2- 172.21.15.3:0/310986385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f015e610 0x7f99f0162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 -- 172.21.15.3:0/310986385 >> 172.21.15.3:0/310986385 conn(0x7f99f00b2d80 msgr2=0x7f99f00b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 -- 172.21.15.3:0/310986385 shutdown_connections 2024-09-12T12:23:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.272+0000 7f99f82e8640 1 -- 172.21.15.3:0/310986385 wait complete. 2024-09-12T12:23:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 Processor -- start 2024-09-12T12:23:15.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 -- start start 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99f0067a00 0x7f99f00b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f015e610 0x7f99f009a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f0162770 0x7f99f009a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00b1ee0 con 0x7f99f0067a00 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00b2580 con 0x7f99f015e610 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f99f00b2060 con 0x7f99f0162770 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99f0067a00 0x7f99f00b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99f0067a00 0x7f99f00b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39656/0 (socket says 172.21.15.3:39656) 2024-09-12T12:23:15.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 -- 172.21.15.3:0/3990383333 learned_addr learned my addr 172.21.15.3:0/3990383333 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f5883640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f015e610 0x7f99f009a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6885640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f0162770 0x7f99f009a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 -- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f0162770 msgr2=0x7f99f009a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f0162770 0x7f99f009a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 -- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f015e610 msgr2=0x7f99f009a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f015e610 0x7f99f009a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 -- 172.21.15.3:0/3990383333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f00b6440 con 0x7f99f0067a00 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f5883640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f015e610 0x7f99f009a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f6084640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99f0067a00 0x7f99f00b5db0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00e8a0 tx=0x7f99ec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99e6ffd640 1 -- 172.21.15.3:0/3990383333 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec017070 con 0x7f99f0067a00 2024-09-12T12:23:15.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99e6ffd640 1 -- 172.21.15.3:0/3990383333 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ec00c940 con 0x7f99f0067a00 2024-09-12T12:23:15.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f00b6720 con 0x7f99f0067a00 2024-09-12T12:23:15.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99e6ffd640 1 -- 172.21.15.3:0/3990383333 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00ccf0 con 0x7f99f0067a00 2024-09-12T12:23:15.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f0155760 con 0x7f99f0067a00 2024-09-12T12:23:15.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.276+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f00b3bd0 con 0x7f99f0067a00 2024-09-12T12:23:15.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.280+0000 7f99e6ffd640 1 -- 172.21.15.3:0/3990383333 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99ec01a070 con 0x7f99f0067a00 2024-09-12T12:23:15.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.280+0000 7f99e6ffd640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99d40678f0 0x7f99d4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:15.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.280+0000 7f99e6ffd640 1 -- 172.21.15.3:0/3990383333 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4857+0+0 (secure 0 0 0) 0x7f99ec085f30 con 0x7f99f0067a00 2024-09-12T12:23:15.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.280+0000 7f99f5883640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99d40678f0 0x7f99d4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:15.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.280+0000 7f99f5883640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99d40678f0 0x7f99d4069db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ad000 tx=0x7f99e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:15.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.280+0000 7f99e6ffd640 1 -- 172.21.15.3:0/3990383333 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f99ec056ea0 con 0x7f99f0067a00 2024-09-12T12:23:15.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:15.444+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f99f00b71d0 con 0x7f99f0067a00 2024-09-12T12:23:16.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.040+0000 7f99e6ffd640 1 -- 172.21.15.3:0/3990383333 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) ==== 138+0+0 (secure 0 0 0) 0x7f99ec0567a0 con 0x7f99f0067a00 2024-09-12T12:23:16.045 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T12:23:16.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99d40678f0 msgr2=0x7f99d4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99d40678f0 0x7f99d4069db0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ad000 tx=0x7f99e0008040 comp rx=0 tx=0).stop 2024-09-12T12:23:16.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99f0067a00 msgr2=0x7f99f00b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99f0067a00 0x7f99f00b5db0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00e8a0 tx=0x7f99ec00eca0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 shutdown_connections 2024-09-12T12:23:16.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f99d40678f0 0x7f99d4069db0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f99f0162770 0x7f99f009a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f99f015e610 0x7f99f009a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 --2- 172.21.15.3:0/3990383333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99f0067a00 0x7f99f00b5db0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 >> 172.21.15.3:0/3990383333 conn(0x7f99f00b2d80 msgr2=0x7f99f00af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 shutdown_connections 2024-09-12T12:23:16.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.044+0000 7f99f82e8640 1 -- 172.21.15.3:0/3990383333 wait complete. 2024-09-12T12:23:16.073 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:23:16.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdceac5640 1 Processor -- start 2024-09-12T12:23:16.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdceac5640 1 -- start start 2024-09-12T12:23:16.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdceac5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 0x7efdc815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdceac5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efdc806e670 0x7efdc806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdceac5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc80b4b40 con 0x7efdc80ccbd0 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdceac5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efdc80b53e0 con 0x7efdc806e670 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdceac5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efdc80b4e20 con 0x7efdc815ad40 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdcc861640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efdc80ccbd0 0x7efdc80ccfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54648/0 (socket says 172.21.15.3:54648) 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdcc861640 1 -- 172.21.15.3:0/3662074069 learned_addr learned my addr 172.21.15.3:0/3662074069 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc7fff640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 0x7efdc815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdcd062640 1 --2- 172.21.15.3:0/3662074069 >> v2:172.21.15.142:3300/0 conn(0x7efdc806e670 0x7efdc806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1768190558 0 0) 0x7efdc80b4b40 con 0x7efdc80ccbd0 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdb4003530 con 0x7efdc80ccbd0 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2608073814 0 0) 0x7efdb4003530 con 0x7efdc80ccbd0 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 msgr2=0x7efdc815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 0x7efdc815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 >> v2:172.21.15.142:3300/0 conn(0x7efdc806e670 msgr2=0x7efdc806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 --2- 172.21.15.3:0/3662074069 >> v2:172.21.15.142:3300/0 conn(0x7efdc806e670 0x7efdc806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc80b5100 con 0x7efdc80ccbd0 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7efdb8001680 con 0x7efdc80ccbd0 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 >> v1:172.21.15.3:6789/0 conn(0x7efdc80ccbd0 legacy=0x7efdc80ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.208+0000 7efdc77fe640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdb40039c0 0x7efdb4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4004810 0x7efdb4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 0x7efdb40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efdc80b4e20 con 0x7efdb40039c0 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efdb400b9d0 con 0x7efdb4004810 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efdb400bb50 con 0x7efdc815ad40 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcc861640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdb40039c0 0x7efdb4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcd062640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 0x7efdb40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc7fff640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4004810 0x7efdb4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcc861640 1 -- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 msgr2=0x7efdb40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcc861640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 0x7efdb40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcc861640 1 -- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4004810 msgr2=0x7efdb4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcc861640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdb4004810 0x7efdb4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcc861640 1 -- 172.21.15.3:0/3662074069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb4003530 con 0x7efdb40039c0 2024-09-12T12:23:16.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcd062640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc815ad40 0x7efdb40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:16.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdcc861640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdb40039c0 0x7efdb4003de0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c210 tx=0x7efdb801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8107684d1685dd27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8007d80 con 0x7efdb40039c0 2024-09-12T12:23:16.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb80a9020 con 0x7efdb40039c0 2024-09-12T12:23:16.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.212+0000 7efdc77fe640 1 -- 172.21.15.3:0/3662074069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8005b00 con 0x7efdb40039c0 2024-09-12T12:23:16.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- 172.21.15.3:0/3662074069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdb40039c0 msgr2=0x7efdb4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 --2- 172.21.15.3:0/3662074069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdb40039c0 0x7efdb4003de0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c210 tx=0x7efdb801cca0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdcd062640 1 -- 172.21.15.3:0/3662074069 reap_dead start 2024-09-12T12:23:16.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- 172.21.15.3:0/3662074069 shutdown_connections 2024-09-12T12:23:16.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- 172.21.15.3:0/3662074069 >> 172.21.15.3:0/3662074069 conn(0x7efdc8067e00 msgr2=0x7efdc8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- 172.21.15.3:0/3662074069 shutdown_connections 2024-09-12T12:23:16.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- 172.21.15.3:0/3662074069 wait complete. 2024-09-12T12:23:16.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 Processor -- start 2024-09-12T12:23:16.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- start start 2024-09-12T12:23:16.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc81536e0 0x7efdc80b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdc8153ac0 0x7efdc81474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdc814a620 0x7efdc8147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efdc80cff50 con 0x7efdc8153ac0 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efdc80cfdd0 con 0x7efdc814a620 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efdc806ea80 con 0x7efdc81536e0 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdc8153ac0 0x7efdc81474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdc8153ac0 0x7efdc81474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39684/0 (socket says 172.21.15.3:39684) 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 -- 172.21.15.3:0/1173953554 learned_addr learned my addr 172.21.15.3:0/1173953554 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdcc861640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc81536e0 0x7efdc80b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 -- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc81536e0 msgr2=0x7efdc80b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc81536e0 0x7efdc80b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 -- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdc814a620 msgr2=0x7efdc8147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdc814a620 0x7efdc8147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 -- 172.21.15.3:0/1173953554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc80b0840 con 0x7efdc8153ac0 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc7fff640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdc8153ac0 0x7efdc81474f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efdb0011560 tx=0x7efdb0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc57fa640 1 -- 172.21.15.3:0/1173953554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0015930 con 0x7efdc8153ac0 2024-09-12T12:23:16.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc57fa640 1 -- 172.21.15.3:0/1173953554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb0015ad0 con 0x7efdc8153ac0 2024-09-12T12:23:16.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc80b0b20 con 0x7efdc8153ac0 2024-09-12T12:23:16.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdc57fa640 1 -- 172.21.15.3:0/1173953554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb001d400 con 0x7efdc8153ac0 2024-09-12T12:23:16.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.216+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc80b10e0 con 0x7efdc8153ac0 2024-09-12T12:23:16.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.220+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc806cd50 con 0x7efdc8153ac0 2024-09-12T12:23:16.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.220+0000 7efdc57fa640 1 -- 172.21.15.3:0/1173953554 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efdb001d5a0 con 0x7efdc8153ac0 2024-09-12T12:23:16.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.220+0000 7efdc57fa640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efda80678f0 0x7efda8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.220+0000 7efdc57fa640 1 -- 172.21.15.3:0/1173953554 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5227+0+0 (secure 0 0 0) 0x7efdb0086390 con 0x7efdc8153ac0 2024-09-12T12:23:16.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.220+0000 7efdcc861640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efda80678f0 0x7efda8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.220+0000 7efdcc861640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efda80678f0 0x7efda8069db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdb80aa000 tx=0x7efdb8001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.224+0000 7efdc57fa640 1 -- 172.21.15.3:0/1173953554 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efdb0019030 con 0x7efdc8153ac0 2024-09-12T12:23:16.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:16.388+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efdc814b510 con 0x7efdc8153ac0 2024-09-12T12:23:16.940 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:23:16.943 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:23:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.044+0000 7efdc57fa640 1 -- 172.21.15.3:0/1173953554 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) ==== 170+0+0 (secure 0 0 0) 0x7efdb0084e20 con 0x7efdc8153ac0 2024-09-12T12:23:17.049 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T12:23:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efda80678f0 msgr2=0x7efda8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efda80678f0 0x7efda8069db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efdb80aa000 tx=0x7efdb8001a50 comp rx=0 tx=0).stop 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdc8153ac0 msgr2=0x7efdc81474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdc8153ac0 0x7efdc81474f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efdb0011560 tx=0x7efdb0011a20 comp rx=0 tx=0).stop 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 shutdown_connections 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efda80678f0 0x7efda8069db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efdc814a620 0x7efdc8147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efdc8153ac0 0x7efdc81474f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 --2- 172.21.15.3:0/1173953554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efdc81536e0 0x7efdc80b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 >> 172.21.15.3:0/1173953554 conn(0x7efdc8067e00 msgr2=0x7efdc80aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 shutdown_connections 2024-09-12T12:23:17.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.048+0000 7efdceac5640 1 -- 172.21.15.3:0/1173953554 wait complete. 2024-09-12T12:23:17.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:23:17.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.204+0000 7f42ed22a640 1 Processor -- start 2024-09-12T12:23:17.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.204+0000 7f42ed22a640 1 -- start start 2024-09-12T12:23:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42ed22a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 0x7f42e8165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42ed22a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e80ad1f0 con 0x7f42e815b420 2024-09-12T12:23:17.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42ed22a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e806da70 con 0x7f42e80ce0c0 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42ed22a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f42e806e310 con 0x7f42e80d0cd0 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e6d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f42e815b420 0x7f42e806d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54660/0 (socket says 172.21.15.3:54660) 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e6d9d640 1 -- 172.21.15.3:0/3711645172 learned_addr learned my addr 172.21.15.3:0/3711645172 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e759e640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 0x7f42e8165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420856619 0 0) 0x7f42e80ad1f0 con 0x7f42e815b420 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d4003530 con 0x7f42e815b420 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3798384727 0 0) 0x7f42d4003530 con 0x7f42e815b420 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 msgr2=0x7f42e8165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 0x7f42e8165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 >> v1:172.21.15.142:6789/0 conn(0x7f42e80ce0c0 legacy=0x7f42e80d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e806dd50 con 0x7f42e815b420 2024-09-12T12:23:17.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f42d80016a0 con 0x7f42e815b420 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 >> v1:172.21.15.3:6789/0 conn(0x7f42e815b420 legacy=0x7f42e806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42d40039f0 0x7f42d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 0x7f42d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f42e806e310 con 0x7f42d4004d50 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40099f0 con 0x7f42d40039f0 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f42d4009bf0 con 0x7f42e80d0cd0 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e759e640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e6d9d640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42d40039f0 0x7f42d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42dffff640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 0x7f42d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e759e640 1 -- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 msgr2=0x7f42d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e759e640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e80d0cd0 0x7f42d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e759e640 1 -- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42d40039f0 msgr2=0x7f42d4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:17.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e759e640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42d40039f0 0x7f42d4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.208+0000 7f42e759e640 1 -- 172.21.15.3:0/3711645172 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d4003530 con 0x7f42d4004d50 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e759e640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42d40051d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f42e806eb30 tx=0x7f42e001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5042224d0ad4281 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e00b2070 con 0x7f42d4004d50 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e00a5c30 con 0x7f42d4004d50 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e659c640 1 -- 172.21.15.3:0/3711645172 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0009c10 con 0x7f42d4004d50 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- 172.21.15.3:0/3711645172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 msgr2=0x7f42d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 --2- 172.21.15.3:0/3711645172 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42d40051d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f42e806eb30 tx=0x7f42e001cc20 comp rx=0 tx=0).stop 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e759e640 1 -- 172.21.15.3:0/3711645172 reap_dead start 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- 172.21.15.3:0/3711645172 shutdown_connections 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- 172.21.15.3:0/3711645172 >> 172.21.15.3:0/3711645172 conn(0x7f42e8067e00 msgr2=0x7f42e8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- 172.21.15.3:0/3711645172 shutdown_connections 2024-09-12T12:23:17.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- 172.21.15.3:0/3711645172 wait complete. 2024-09-12T12:23:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 Processor -- start 2024-09-12T12:23:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- start start 2024-09-12T12:23:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42e81f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e81f31b0 0x7f42e81f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e81f1c50 0x7f42e81f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f42e80ae6e0 con 0x7f42d4004d50 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f42e815b800 con 0x7f42e81f1c50 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42ed22a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f42e80ae860 con 0x7f42e81f31b0 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42e81f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42e81f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39696/0 (socket says 172.21.15.3:39696) 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 -- 172.21.15.3:0/2008801527 learned_addr learned my addr 172.21.15.3:0/2008801527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42dffff640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e81f31b0 0x7f42e81f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e759e640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e81f1c50 0x7f42e81f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 -- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e81f31b0 msgr2=0x7f42e81f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e81f31b0 0x7f42e81f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 -- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e81f1c50 msgr2=0x7f42e81f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e81f1c50 0x7f42e81f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 -- 172.21.15.3:0/2008801527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81f0e90 con 0x7f42d4004d50 2024-09-12T12:23:17.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.212+0000 7f42e6d9d640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42e81f04f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f42d800e780 tx=0x7f42d800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42df7fe640 1 -- 172.21.15.3:0/2008801527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d800c6c0 con 0x7f42d4004d50 2024-09-12T12:23:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42df7fe640 1 -- 172.21.15.3:0/2008801527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d8018070 con 0x7f42d4004d50 2024-09-12T12:23:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81f1170 con 0x7f42d4004d50 2024-09-12T12:23:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42df7fe640 1 -- 172.21.15.3:0/2008801527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d800cac0 con 0x7f42d4004d50 2024-09-12T12:23:17.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e81f1730 con 0x7f42d4004d50 2024-09-12T12:23:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e81f1930 con 0x7f42d4004d50 2024-09-12T12:23:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42df7fe640 1 -- 172.21.15.3:0/2008801527 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f42d800cc60 con 0x7f42d4004d50 2024-09-12T12:23:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42df7fe640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f42c80678f0 0x7f42c8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42df7fe640 1 -- 172.21.15.3:0/2008801527 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5609+0+0 (secure 0 0 0) 0x7f42d8086200 con 0x7f42d4004d50 2024-09-12T12:23:17.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42dffff640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f42c80678f0 0x7f42c8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.216+0000 7f42dffff640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f42c80678f0 0x7f42c8069db0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f42e80b2d20 tx=0x7f42d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:17.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.220+0000 7f42df7fe640 1 -- 172.21.15.3:0/2008801527 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f42e81f1930 con 0x7f42d4004d50 2024-09-12T12:23:17.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:17.420+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f42e80d05c0 con 0x7f42d4004d50 2024-09-12T12:23:18.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.104+0000 7f42df7fe640 1 -- 172.21.15.3:0/2008801527 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v41) ==== 174+0+0 (secure 0 0 0) 0x7f42d8056e80 con 0x7f42d4004d50 2024-09-12T12:23:18.109 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 10 and data pool 11 2024-09-12T12:23:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f42c80678f0 msgr2=0x7f42c8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f42c80678f0 0x7f42c8069db0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f42e80b2d20 tx=0x7f42d0009040 comp rx=0 tx=0).stop 2024-09-12T12:23:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 msgr2=0x7f42e81f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42e81f04f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f42d800e780 tx=0x7f42d800ec40 comp rx=0 tx=0).stop 2024-09-12T12:23:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 shutdown_connections 2024-09-12T12:23:18.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f42c80678f0 0x7f42c8069db0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42e81f1c50 0x7f42e81f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42e81f31b0 0x7f42e81f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 --2- 172.21.15.3:0/2008801527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42d4004d50 0x7f42e81f04f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 >> 172.21.15.3:0/2008801527 conn(0x7f42e8067e00 msgr2=0x7f42e8159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 shutdown_connections 2024-09-12T12:23:18.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.116+0000 7f42ed22a640 1 -- 172.21.15.3:0/2008801527 wait complete. 2024-09-12T12:23:18.148 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:18.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.260+0000 7f32e9564640 1 Processor -- start 2024-09-12T12:23:18.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.260+0000 7f32e9564640 1 -- start start 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e9564640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f32e415c700 0x7f32e40b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e9564640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e4160c10 0x7f32e40cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e9564640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f32e40cdec0 0x7f32e406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e9564640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f32e40d01e0 con 0x7f32e415c700 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e9564640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f32e40d04c0 con 0x7f32e40cdec0 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e9564640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32e40cd180 con 0x7f32e4160c10 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f32e415c700 0x7f32e40b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f32e415c700 0x7f32e40b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38334/0 (socket says 172.21.15.3:38334) 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 -- 172.21.15.3:0/3191050615 learned_addr learned my addr 172.21.15.3:0/3191050615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:18.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e27fc640 1 --2- 172.21.15.3:0/3191050615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e4160c10 0x7f32e40cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 -- 172.21.15.3:0/3191050615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e4160c10 msgr2=0x7f32e40cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 --2- 172.21.15.3:0/3191050615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e4160c10 0x7f32e40cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e37fe640 1 --2- 172.21.15.3:0/3191050615 >> v2:172.21.15.142:3300/0 conn(0x7f32e40cdec0 0x7f32e406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 -- 172.21.15.3:0/3191050615 >> v2:172.21.15.142:3300/0 conn(0x7f32e40cdec0 msgr2=0x7f32e406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 --2- 172.21.15.3:0/3191050615 >> v2:172.21.15.142:3300/0 conn(0x7f32e40cdec0 0x7f32e406efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 -- 172.21.15.3:0/3191050615 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e40cd460 con 0x7f32e415c700 2024-09-12T12:23:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.264+0000 7f32e2ffd640 1 --2- 172.21.15.3:0/3191050615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e415c700 0x7f32e40b4950 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f32e4073f20 tx=0x7f32d401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a20afef877b28a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e1ffb640 1 -- 172.21.15.3:0/3191050615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40b0070 con 0x7f32e415c700 2024-09-12T12:23:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e1ffb640 1 -- 172.21.15.3:0/3191050615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d40ab600 con 0x7f32e415c700 2024-09-12T12:23:18.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e1ffb640 1 -- 172.21.15.3:0/3191050615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40ab9d0 con 0x7f32e415c700 2024-09-12T12:23:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- 172.21.15.3:0/3191050615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e415c700 msgr2=0x7f32e40b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 --2- 172.21.15.3:0/3191050615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e415c700 0x7f32e40b4950 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f32e4073f20 tx=0x7f32d401c3a0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- 172.21.15.3:0/3191050615 shutdown_connections 2024-09-12T12:23:18.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 --2- 172.21.15.3:0/3191050615 >> v2:172.21.15.142:3300/0 conn(0x7f32e40cdec0 0x7f32e406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 --2- 172.21.15.3:0/3191050615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e4160c10 0x7f32e40cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 --2- 172.21.15.3:0/3191050615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e415c700 0x7f32e40b4950 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- 172.21.15.3:0/3191050615 >> 172.21.15.3:0/3191050615 conn(0x7f32e40b3e30 msgr2=0x7f32e40b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- 172.21.15.3:0/3191050615 shutdown_connections 2024-09-12T12:23:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- 172.21.15.3:0/3191050615 wait complete. 2024-09-12T12:23:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 Processor -- start 2024-09-12T12:23:18.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- start start 2024-09-12T12:23:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e40cdec0 0x7f32e406cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32e415c700 0x7f32e4156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e4160c10 0x7f32e41fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f32e40cf0a0 con 0x7f32e4160c10 2024-09-12T12:23:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f32e40cef20 con 0x7f32e415c700 2024-09-12T12:23:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e9564640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f32e40cf220 con 0x7f32e40cdec0 2024-09-12T12:23:18.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e4160c10 0x7f32e41fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e4160c10 0x7f32e41fb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38344/0 (socket says 172.21.15.3:38344) 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 -- 172.21.15.3:0/2864347411 learned_addr learned my addr 172.21.15.3:0/2864347411 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e27fc640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32e415c700 0x7f32e4156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e2ffd640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e40cdec0 0x7f32e406cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 -- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e40cdec0 msgr2=0x7f32e406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e40cdec0 0x7f32e406cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 -- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32e415c700 msgr2=0x7f32e4156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32e415c700 0x7f32e4156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 -- 172.21.15.3:0/2864347411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e40af990 con 0x7f32e4160c10 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.268+0000 7f32e37fe640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e4160c10 0x7f32e41fb4e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00cd20 tx=0x7f32dc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32db7fe640 1 -- 172.21.15.3:0/2864347411 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc003ba0 con 0x7f32e4160c10 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32db7fe640 1 -- 172.21.15.3:0/2864347411 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc003d40 con 0x7f32e4160c10 2024-09-12T12:23:18.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32db7fe640 1 -- 172.21.15.3:0/2864347411 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc004240 con 0x7f32e4160c10 2024-09-12T12:23:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e406dc40 con 0x7f32e4160c10 2024-09-12T12:23:18.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e4156da0 con 0x7f32e4160c10 2024-09-12T12:23:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e40b4950 con 0x7f32e4160c10 2024-09-12T12:23:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32db7fe640 1 -- 172.21.15.3:0/2864347411 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f32dc0043e0 con 0x7f32e4160c10 2024-09-12T12:23:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32db7fe640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f32c40678c0 0x7f32c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32db7fe640 1 -- 172.21.15.3:0/2864347411 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5709+0+0 (secure 0 0 0) 0x7f32dc0866d0 con 0x7f32e4160c10 2024-09-12T12:23:18.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32e2ffd640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f32c40678c0 0x7f32c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.272+0000 7f32e2ffd640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f32c40678c0 0x7f32c4069d80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f32d40ad000 tx=0x7f32d40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.276+0000 7f32db7fe640 1 -- 172.21.15.3:0/2864347411 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f32dc01d340 con 0x7f32e4160c10 2024-09-12T12:23:18.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.440+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32e40b8280 con 0x7f32e4160c10 2024-09-12T12:23:18.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32db7fe640 1 -- 172.21.15.3:0/2864347411 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16681 (secure 0 0 0) 0x7f32dc057200 con 0x7f32e4160c10 2024-09-12T12:23:18.446 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:18.446 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":65,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:17.422821+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:15.490182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:16.430656+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f32c40678c0 msgr2=0x7f32c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f32c40678c0 0x7f32c4069d80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f32d40ad000 tx=0x7f32d40a6000 comp rx=0 tx=0).stop 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e4160c10 msgr2=0x7f32e41fb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e4160c10 0x7f32e41fb4e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00cd20 tx=0x7f32dc00a660 comp rx=0 tx=0).stop 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 shutdown_connections 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f32c40678c0 0x7f32c4069d80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32e4160c10 0x7f32e41fb4e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32e415c700 0x7f32e4156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 --2- 172.21.15.3:0/2864347411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f32e40cdec0 0x7f32e406cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 >> 172.21.15.3:0/2864347411 conn(0x7f32e40b3e30 msgr2=0x7f32e40af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:18.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 shutdown_connections 2024-09-12T12:23:18.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.444+0000 7f32e9564640 1 -- 172.21.15.3:0/2864347411 wait complete. 2024-09-12T12:23:18.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.580+0000 7f52bd7b4640 1 Processor -- start 2024-09-12T12:23:18.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.580+0000 7f52bd7b4640 1 -- start start 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52bd7b4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f52b8073cb0 0x7f52b815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52bd7b4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f52b80ae880 0x7f52b80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52bd7b4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b80b5000 con 0x7f52b80b30e0 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52bd7b4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f52b8074090 con 0x7f52b80ae880 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52bd7b4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f52b80b3530 con 0x7f52b8073cb0 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b67fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f52b80b30e0 0x7f52b80adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47102/0 (socket says 172.21.15.3:47102) 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b67fc640 1 -- 172.21.15.3:0/3841866470 learned_addr learned my addr 172.21.15.3:0/3841866470 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b77fe640 1 --2- 172.21.15.3:0/3841866470 >> v2:172.21.15.3:3300/0 conn(0x7f52b80ae880 0x7f52b80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b6ffd640 1 --2- 172.21.15.3:0/3841866470 >> v2:172.21.15.142:3300/0 conn(0x7f52b8073cb0 0x7f52b815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b5ffb640 1 -- 172.21.15.3:0/3841866470 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420239285 0 0) 0x7f52b80b5000 con 0x7f52b80b30e0 2024-09-12T12:23:18.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b77fe640 1 -- 172.21.15.3:0/3841866470 >> v2:172.21.15.142:3300/0 conn(0x7f52b8073cb0 msgr2=0x7f52b815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:18.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b77fe640 1 --2- 172.21.15.3:0/3841866470 >> v2:172.21.15.142:3300/0 conn(0x7f52b8073cb0 0x7f52b815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b77fe640 1 -- 172.21.15.3:0/3841866470 >> v1:172.21.15.3:6789/0 conn(0x7f52b80b30e0 legacy=0x7f52b80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.584+0000 7f52b77fe640 1 -- 172.21.15.3:0/3841866470 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b80b37c0 con 0x7f52b80ae880 2024-09-12T12:23:18.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52b77fe640 1 --2- 172.21.15.3:0/3841866470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80aecb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f52b80d1170 tx=0x7f52b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef015e84c9c728be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52b5ffb640 1 -- 172.21.15.3:0/3841866470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00b2070 con 0x7f52b80ae880 2024-09-12T12:23:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52b5ffb640 1 -- 172.21.15.3:0/3841866470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b00a58e0 con 0x7f52b80ae880 2024-09-12T12:23:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52b5ffb640 1 -- 172.21.15.3:0/3841866470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00a5cb0 con 0x7f52b80ae880 2024-09-12T12:23:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3841866470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 msgr2=0x7f52b80aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3841866470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80aecb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f52b80d1170 tx=0x7f52b001c930 comp rx=0 tx=0).stop 2024-09-12T12:23:18.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3841866470 shutdown_connections 2024-09-12T12:23:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3841866470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80aecb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3841866470 >> v2:172.21.15.142:3300/0 conn(0x7f52b8073cb0 0x7f52b815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3841866470 >> 172.21.15.3:0/3841866470 conn(0x7f52b8067e00 msgr2=0x7f52b8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3841866470 shutdown_connections 2024-09-12T12:23:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.588+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3841866470 wait complete. 2024-09-12T12:23:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 Processor -- start 2024-09-12T12:23:18.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 -- start start 2024-09-12T12:23:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52b8073cb0 0x7f52b814cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52b80b30e0 0x7f52b8150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80ae4a0 con 0x7f52b80ae880 2024-09-12T12:23:18.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80ae320 con 0x7f52b8073cb0 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f52b8168dc0 con 0x7f52b80b30e0 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38368/0 (socket says 172.21.15.3:38368) 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 -- 172.21.15.3:0/3351288093 learned_addr learned my addr 172.21.15.3:0/3351288093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b6ffd640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52b8073cb0 0x7f52b814cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b77fe640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52b80b30e0 0x7f52b8150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 -- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52b80b30e0 msgr2=0x7f52b8150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52b80b30e0 0x7f52b8150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 -- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52b8073cb0 msgr2=0x7f52b814cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52b8073cb0 0x7f52b814cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 -- 172.21.15.3:0/3351288093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b8138db0 con 0x7f52b80ae880 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b6ffd640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52b8073cb0 0x7f52b814cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b77fe640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52b80b30e0 0x7f52b8150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52b67fc640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80ccbc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c4b0 tx=0x7f52a800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52af7fe640 1 -- 172.21.15.3:0/3351288093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8018070 con 0x7f52b80ae880 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52af7fe640 1 -- 172.21.15.3:0/3351288093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8007500 con 0x7f52b80ae880 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b8137d10 con 0x7f52b80ae880 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52af7fe640 1 -- 172.21.15.3:0/3351288093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8004a30 con 0x7f52b80ae880 2024-09-12T12:23:18.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81382d0 con 0x7f52b80ae880 2024-09-12T12:23:18.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.592+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b80aecb0 con 0x7f52b80ae880 2024-09-12T12:23:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.596+0000 7f52af7fe640 1 -- 172.21.15.3:0/3351288093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52a801b070 con 0x7f52b80ae880 2024-09-12T12:23:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.596+0000 7f52af7fe640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52980678c0 0x7f5298069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.596+0000 7f52af7fe640 1 -- 172.21.15.3:0/3351288093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5709+0+0 (secure 0 0 0) 0x7f52a8084f10 con 0x7f52b80ae880 2024-09-12T12:23:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.596+0000 7f52b6ffd640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52980678c0 0x7f5298069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.596+0000 7f52b6ffd640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52980678c0 0x7f5298069d80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009840 tx=0x7f52a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.600+0000 7f52af7fe640 1 -- 172.21.15.3:0/3351288093 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f52a8055ae0 con 0x7f52b80ae880 2024-09-12T12:23:18.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.760+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52b80685f0 con 0x7f52b80ae880 2024-09-12T12:23:18.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.760+0000 7f52af7fe640 1 -- 172.21.15.3:0/3351288093 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16681 (secure 0 0 0) 0x7f52a80553e0 con 0x7f52b80ae880 2024-09-12T12:23:18.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:18.765 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":65,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:17.422821+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:15.490182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:16.430656+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:18.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52980678c0 msgr2=0x7f5298069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52980678c0 0x7f5298069d80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009840 tx=0x7f52a0008040 comp rx=0 tx=0).stop 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 msgr2=0x7f52b80ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80ccbc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c4b0 tx=0x7f52a800c970 comp rx=0 tx=0).stop 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 shutdown_connections 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f52980678c0 0x7f5298069d80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f52b80b30e0 0x7f52b8150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52b80ae880 0x7f52b80ccbc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 --2- 172.21.15.3:0/3351288093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52b8073cb0 0x7f52b814cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 >> 172.21.15.3:0/3351288093 conn(0x7f52b8067e00 msgr2=0x7f52b80adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:18.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 shutdown_connections 2024-09-12T12:23:18.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.764+0000 7f52bd7b4640 1 -- 172.21.15.3:0/3351288093 wait complete. 2024-09-12T12:23:18.784 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:23:18.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1ea23640 1 Processor -- start 2024-09-12T12:23:18.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1ea23640 1 -- start start 2024-09-12T12:23:18.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1ea23640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 0x7fdc1806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1ea23640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc180b4b40 con 0x7fdc1815ad40 2024-09-12T12:23:18.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1ea23640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc180b4e20 con 0x7fdc180ccbd0 2024-09-12T12:23:18.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1ea23640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc180b53e0 con 0x7fdc1806e670 2024-09-12T12:23:18.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdc1815ad40 0x7fdc1815b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47116/0 (socket says 172.21.15.3:47116) 2024-09-12T12:23:18.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 -- 172.21.15.3:0/1516494106 learned_addr learned my addr 172.21.15.3:0/1516494106 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:18.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1cfc0640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 0x7fdc1806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463610695 0 0) 0x7fdc180b4b40 con 0x7fdc1815ad40 2024-09-12T12:23:18.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc00003530 con 0x7fdc1815ad40 2024-09-12T12:23:18.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 508037644 0 0) 0x7fdc00003530 con 0x7fdc1815ad40 2024-09-12T12:23:18.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 msgr2=0x7fdc1806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:18.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 0x7fdc1806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 >> v1:172.21.15.142:6789/0 conn(0x7fdc180ccbd0 legacy=0x7fdc180ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc180b5100 con 0x7fdc1815ad40 2024-09-12T12:23:18.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fdc0c0016a0 con 0x7fdc1815ad40 2024-09-12T12:23:18.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 >> v1:172.21.15.3:6789/0 conn(0x7fdc1815ad40 legacy=0x7fdc1815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 0x7fdc000039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc00004d50 0x7fdc000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0000ba80 con 0x7fdc00003f70 2024-09-12T12:23:18.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0000bc30 con 0x7fdc00004d50 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0000be30 con 0x7fdc1806e670 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc17fff640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 0x7fdc000039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1cfc0640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc00004d50 0x7fdc000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 -- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 msgr2=0x7fdc000039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc1806e670 0x7fdc000039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 -- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc00004d50 msgr2=0x7fdc000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc00004d50 0x7fdc000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc177fe640 1 -- 172.21.15.3:0/1516494106 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc00003530 con 0x7fdc00003f70 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.912+0000 7fdc1cfc0640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc00004d50 0x7fdc000051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc177fe640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc00004810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c210 tx=0x7fdc0c01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384a4f2acbfea078 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c007d20 con 0x7fdc00003f70 2024-09-12T12:23:18.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc0c005b00 con 0x7fdc00003f70 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc16ffd640 1 -- 172.21.15.3:0/1516494106 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c007070 con 0x7fdc00003f70 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1516494106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 msgr2=0x7fdc00004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 --2- 172.21.15.3:0/1516494106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc00004810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c210 tx=0x7fdc0c01f390 comp rx=0 tx=0).stop 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1cfc0640 1 -- 172.21.15.3:0/1516494106 reap_dead start 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1516494106 shutdown_connections 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1516494106 >> 172.21.15.3:0/1516494106 conn(0x7fdc18067e00 msgr2=0x7fdc18068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1516494106 shutdown_connections 2024-09-12T12:23:18.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1516494106 wait complete. 2024-09-12T12:23:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 Processor -- start 2024-09-12T12:23:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- start start 2024-09-12T12:23:18.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc1814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc18157980 0x7fdc18149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc180b1e70 0x7fdc180b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdc180cfdd0 con 0x7fdc00003f70 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1815b4a0 con 0x7fdc18157980 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1ea23640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdc180cff50 con 0x7fdc180b1e70 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc1814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc1814e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38384/0 (socket says 172.21.15.3:38384) 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc17fff640 1 -- 172.21.15.3:0/1451885048 learned_addr learned my addr 172.21.15.3:0/1451885048 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc177fe640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc18157980 0x7fdc18149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.916+0000 7fdc1cfc0640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc180b1e70 0x7fdc180b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc17fff640 1 -- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc180b1e70 msgr2=0x7fdc180b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc17fff640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc180b1e70 0x7fdc180b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc17fff640 1 -- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc18157980 msgr2=0x7fdc18149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc17fff640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc18157980 0x7fdc18149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc17fff640 1 -- 172.21.15.3:0/1451885048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1814de10 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc17fff640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc1814e5d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e4e0 tx=0x7fdc0800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc14ff9640 1 -- 172.21.15.3:0/1451885048 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc080052e0 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc14ff9640 1 -- 172.21.15.3:0/1451885048 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc08018070 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc18157e10 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc14ff9640 1 -- 172.21.15.3:0/1451885048 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08003840 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc18073980 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1806ea80 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc14ff9640 1 -- 172.21.15.3:0/1451885048 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc080039e0 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc14ff9640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc040678c0 0x7fdc04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc14ff9640 1 -- 172.21.15.3:0/1451885048 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5709+0+0 (secure 0 0 0) 0x7fdc080857b0 con 0x7fdc00003f70 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc177fe640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc040678c0 0x7fdc04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.920+0000 7fdc177fe640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc040678c0 0x7fdc04069d80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0ae000 tx=0x7fdc0c0b4040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:18.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:18.924+0000 7fdc14ff9640 1 -- 172.21.15.3:0/1451885048 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdc08055ca0 con 0x7fdc00003f70 2024-09-12T12:23:19.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:19.128+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdc18159110 con 0x7fdc00003f70 2024-09-12T12:23:20.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.108+0000 7fdc14ff9640 1 -- 172.21.15.3:0/1451885048 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7fdc08059fa0 con 0x7fdc00003f70 2024-09-12T12:23:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.108+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc040678c0 msgr2=0x7fdc04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.108+0000 7fdc1ea23640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc040678c0 0x7fdc04069d80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0ae000 tx=0x7fdc0c0b4040 comp rx=0 tx=0).stop 2024-09-12T12:23:20.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.108+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 msgr2=0x7fdc1814e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.108+0000 7fdc1ea23640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc1814e5d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdc0800e4e0 tx=0x7fdc0800e9a0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 shutdown_connections 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdc040678c0 0x7fdc04069d80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdc180b1e70 0x7fdc180b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc18157980 0x7fdc18149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 --2- 172.21.15.3:0/1451885048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdc00003f70 0x7fdc1814e5d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 >> 172.21.15.3:0/1451885048 conn(0x7fdc18067e00 msgr2=0x7fdc1806dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 shutdown_connections 2024-09-12T12:23:20.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.112+0000 7fdc1ea23640 1 -- 172.21.15.3:0/1451885048 wait complete. 2024-09-12T12:23:20.134 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T12:23:20.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T12:23:20.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a59b7f640 1 Processor -- start 2024-09-12T12:23:20.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a59b7f640 1 -- start start 2024-09-12T12:23:20.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a59b7f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5415c320 0x7f3a5415c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:20.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a59b7f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a540d1f90 0x7f3a54067a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:20.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a59b7f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a540b2640 con 0x7f3a540b1de0 2024-09-12T12:23:20.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a59b7f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3a540b2360 con 0x7f3a540d1f90 2024-09-12T12:23:20.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a59b7f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3a540b0970 con 0x7f3a5415c320 2024-09-12T12:23:20.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a537fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5415c320 0x7f3a5415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:20.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a52ffd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a540d1f90 0x7f3a54067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a537fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5415c320 0x7f3a5415c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:50930/0 (socket says 172.21.15.3:50930) 2024-09-12T12:23:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a537fe640 1 -- 172.21.15.3:0/3979791110 learned_addr learned my addr 172.21.15.3:0/3979791110 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a537fe640 1 -- 172.21.15.3:0/3979791110 >> v2:172.21.15.142:3300/0 conn(0x7f3a540d1f90 msgr2=0x7f3a54067a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a537fe640 1 --2- 172.21.15.3:0/3979791110 >> v2:172.21.15.142:3300/0 conn(0x7f3a540d1f90 0x7f3a54067a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a537fe640 1 -- 172.21.15.3:0/3979791110 >> v1:172.21.15.142:6789/0 conn(0x7f3a540b1de0 legacy=0x7f3a54073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a537fe640 1 -- 172.21.15.3:0/3979791110 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a540b0c50 con 0x7f3a5415c320 2024-09-12T12:23:20.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.264+0000 7f3a52ffd640 1 --2- 172.21.15.3:0/3979791110 >> v2:172.21.15.142:3300/0 conn(0x7f3a540d1f90 0x7f3a54067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:20.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a537fe640 1 --2- 172.21.15.3:0/3979791110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5415c320 0x7f3a5415c700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009870 tx=0x7f3a3c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa7dffe1e2607068 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a527fc640 1 -- 172.21.15.3:0/3979791110 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0a9640 con 0x7f3a5415c320 2024-09-12T12:23:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a527fc640 1 -- 172.21.15.3:0/3979791110 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a3c0a97e0 con 0x7f3a5415c320 2024-09-12T12:23:20.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a527fc640 1 -- 172.21.15.3:0/3979791110 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0a9bb0 con 0x7f3a5415c320 2024-09-12T12:23:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 -- 172.21.15.3:0/3979791110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5415c320 msgr2=0x7f3a5415c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 --2- 172.21.15.3:0/3979791110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5415c320 0x7f3a5415c700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009870 tx=0x7f3a3c01c560 comp rx=0 tx=0).stop 2024-09-12T12:23:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 -- 172.21.15.3:0/3979791110 shutdown_connections 2024-09-12T12:23:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 --2- 172.21.15.3:0/3979791110 >> v2:172.21.15.142:3300/0 conn(0x7f3a540d1f90 0x7f3a54067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 --2- 172.21.15.3:0/3979791110 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5415c320 0x7f3a5415c700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 -- 172.21.15.3:0/3979791110 >> 172.21.15.3:0/3979791110 conn(0x7f3a540d1260 msgr2=0x7f3a540d1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:20.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 -- 172.21.15.3:0/3979791110 shutdown_connections 2024-09-12T12:23:20.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 -- 172.21.15.3:0/3979791110 wait complete. 2024-09-12T12:23:20.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 Processor -- start 2024-09-12T12:23:20.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.268+0000 7f3a59b7f640 1 -- start start 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a540b1de0 0x7f3a5415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a540d1f90 0x7f3a540c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5415c320 0x7f3a5409b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3a541693a0 con 0x7f3a5415c320 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54068350 con 0x7f3a540b1de0 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3a54169520 con 0x7f3a540d1f90 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5415c320 0x7f3a5409b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5415c320 0x7f3a5409b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38390/0 (socket says 172.21.15.3:38390) 2024-09-12T12:23:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 -- 172.21.15.3:0/1875215058 learned_addr learned my addr 172.21.15.3:0/1875215058 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a537fe640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a540b1de0 0x7f3a5415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a52ffd640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a540d1f90 0x7f3a540c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 -- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a540d1f90 msgr2=0x7f3a540c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a540d1f90 0x7f3a540c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 -- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a540b1de0 msgr2=0x7f3a5415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a540b1de0 0x7f3a5415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 -- 172.21.15.3:0/1875215058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5415bba0 con 0x7f3a5415c320 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a53fff640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5415c320 0x7f3a5409b8d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00c710 tx=0x7f3a4c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a4bfff640 1 -- 172.21.15.3:0/1875215058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c003ba0 con 0x7f3a5415c320 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a4bfff640 1 -- 172.21.15.3:0/1875215058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4c003d40 con 0x7f3a5415c320 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 -- 172.21.15.3:0/1875215058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5409e5d0 con 0x7f3a5415c320 2024-09-12T12:23:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a4bfff640 1 -- 172.21.15.3:0/1875215058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00aa90 con 0x7f3a5415c320 2024-09-12T12:23:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a59b7f640 1 -- 172.21.15.3:0/1875215058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5409eb90 con 0x7f3a5415c320 2024-09-12T12:23:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2c0052a0 con 0x7f3a5415c320 2024-09-12T12:23:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a4bfff640 1 -- 172.21.15.3:0/1875215058 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3a4c00ac30 con 0x7f3a5415c320 2024-09-12T12:23:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a4bfff640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a340678c0 0x7f3a34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a4bfff640 1 -- 172.21.15.3:0/1875215058 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f3a4c085a00 con 0x7f3a5415c320 2024-09-12T12:23:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.272+0000 7f3a537fe640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a340678c0 0x7f3a34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.276+0000 7f3a537fe640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a340678c0 0x7f3a34069d80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0ae000 tx=0x7f3a3c0b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:20.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.280+0000 7f3a4bfff640 1 -- 172.21.15.3:0/1875215058 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3a540b2640 con 0x7f3a5415c320 2024-09-12T12:23:20.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:20.480+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3a2c005590 con 0x7f3a5415c320 2024-09-12T12:23:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a4bfff640 1 -- 172.21.15.3:0/1875215058 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f3a4c00ce20 con 0x7f3a5415c320 2024-09-12T12:23:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a340678c0 msgr2=0x7f3a34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a340678c0 0x7f3a34069d80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0ae000 tx=0x7f3a3c0b3040 comp rx=0 tx=0).stop 2024-09-12T12:23:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5415c320 msgr2=0x7f3a5409b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5415c320 0x7f3a5409b8d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00c710 tx=0x7f3a4c00cbd0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 shutdown_connections 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3a340678c0 0x7f3a34069d80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a5415c320 0x7f3a5409b8d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a540d1f90 0x7f3a540c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 --2- 172.21.15.3:0/1875215058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a540b1de0 0x7f3a5415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 >> 172.21.15.3:0/1875215058 conn(0x7f3a540d1260 msgr2=0x7f3a540ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 shutdown_connections 2024-09-12T12:23:21.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.116+0000 7f3a49ffb640 1 -- 172.21.15.3:0/1875215058 wait complete. 2024-09-12T12:23:21.141 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T12:23:21.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:21.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f8b144640 1 Processor -- start 2024-09-12T12:23:21.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f8b144640 1 -- start start 2024-09-12T12:23:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f8b144640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f84073cb0 0x7f5f8415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f8b144640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5f840ae880 0x7f5f840aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f8b144640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f840b5000 con 0x7f5f840b30e0 2024-09-12T12:23:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f8b144640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5f84074090 con 0x7f5f840ae880 2024-09-12T12:23:21.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f8b144640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5f840b3500 con 0x7f5f84073cb0 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f896e1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5f840ae880 0x7f5f840aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f88ee0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f84073cb0 0x7f5f8415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f83fff640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f5f840b30e0 0x7f5f840adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:34756/0 (socket says 172.21.15.3:34756) 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f83fff640 1 -- 172.21.15.3:0/1580885868 learned_addr learned my addr 172.21.15.3:0/1580885868 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f896e1640 1 -- 172.21.15.3:0/1580885868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f84073cb0 msgr2=0x7f5f8415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f896e1640 1 --2- 172.21.15.3:0/1580885868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f84073cb0 0x7f5f8415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f896e1640 1 -- 172.21.15.3:0/1580885868 >> v1:172.21.15.142:6789/0 conn(0x7f5f840b30e0 legacy=0x7f5f840adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.264+0000 7f5f896e1640 1 -- 172.21.15.3:0/1580885868 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f840b37c0 con 0x7f5f840ae880 2024-09-12T12:23:21.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f896e1640 1 --2- 172.21.15.3:0/1580885868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f840ae880 0x7f5f840aecb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009d90 tx=0x7f5f7c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41417af1ded867d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f837fe640 1 -- 172.21.15.3:0/1580885868 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0b2070 con 0x7f5f840ae880 2024-09-12T12:23:21.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f837fe640 1 -- 172.21.15.3:0/1580885868 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7c0a5920 con 0x7f5f840ae880 2024-09-12T12:23:21.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f837fe640 1 -- 172.21.15.3:0/1580885868 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0a5cf0 con 0x7f5f840ae880 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 -- 172.21.15.3:0/1580885868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f840ae880 msgr2=0x7f5f840aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 --2- 172.21.15.3:0/1580885868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f840ae880 0x7f5f840aecb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009d90 tx=0x7f5f7c01c970 comp rx=0 tx=0).stop 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 -- 172.21.15.3:0/1580885868 shutdown_connections 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 --2- 172.21.15.3:0/1580885868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f840ae880 0x7f5f840aecb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 --2- 172.21.15.3:0/1580885868 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f84073cb0 0x7f5f8415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 -- 172.21.15.3:0/1580885868 >> 172.21.15.3:0/1580885868 conn(0x7f5f84067e00 msgr2=0x7f5f84068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 -- 172.21.15.3:0/1580885868 shutdown_connections 2024-09-12T12:23:21.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.268+0000 7f5f8b144640 1 -- 172.21.15.3:0/1580885868 wait complete. 2024-09-12T12:23:21.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 Processor -- start 2024-09-12T12:23:21.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 -- start start 2024-09-12T12:23:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f84073cb0 0x7f5f840ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f840ae880 0x7f5f8414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f840b30e0 0x7f5f841501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5f840ae4a0 con 0x7f5f840ae880 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f840ae320 con 0x7f5f84073cb0 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5f84168dc0 con 0x7f5f840b30e0 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f840ae880 0x7f5f8414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f840ae880 0x7f5f8414e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38396/0 (socket says 172.21.15.3:38396) 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 -- 172.21.15.3:0/3958385469 learned_addr learned my addr 172.21.15.3:0/3958385469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f896e1640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f840b30e0 0x7f5f841501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f88ee0640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f84073cb0 0x7f5f840ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 -- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f840b30e0 msgr2=0x7f5f841501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f840b30e0 0x7f5f841501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 -- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f84073cb0 msgr2=0x7f5f840ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f84073cb0 0x7f5f840ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 -- 172.21.15.3:0/3958385469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f841511c0 con 0x7f5f840ae880 2024-09-12T12:23:21.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f83fff640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f840ae880 0x7f5f8414e120 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c370 tx=0x7f5f7800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f817fa640 1 -- 172.21.15.3:0/3958385469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78017070 con 0x7f5f840ae880 2024-09-12T12:23:21.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f817fa640 1 -- 172.21.15.3:0/3958385469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f78005cf0 con 0x7f5f840ae880 2024-09-12T12:23:21.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f841514a0 con 0x7f5f840ae880 2024-09-12T12:23:21.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f817fa640 1 -- 172.21.15.3:0/3958385469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f780072f0 con 0x7f5f840ae880 2024-09-12T12:23:21.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f84151a60 con 0x7f5f840ae880 2024-09-12T12:23:21.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8415c710 con 0x7f5f840ae880 2024-09-12T12:23:21.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.272+0000 7f5f817fa640 1 -- 172.21.15.3:0/3958385469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5f7801a070 con 0x7f5f840ae880 2024-09-12T12:23:21.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.276+0000 7f5f817fa640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5f700678c0 0x7f5f70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.276+0000 7f5f817fa640 1 -- 172.21.15.3:0/3958385469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f5f78085330 con 0x7f5f840ae880 2024-09-12T12:23:21.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.276+0000 7f5f88ee0640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5f700678c0 0x7f5f70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.276+0000 7f5f88ee0640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5f700678c0 0x7f5f70069d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f841f83a0 tx=0x7f5f74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.280+0000 7f5f817fa640 1 -- 172.21.15.3:0/3958385469 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f5f78021020 con 0x7f5f840ae880 2024-09-12T12:23:21.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f84151c30 con 0x7f5f840ae880 2024-09-12T12:23:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f817fa640 1 -- 172.21.15.3:0/3958385469 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f5f7805ae20 con 0x7f5f840ae880 2024-09-12T12:23:21.483 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:21.483 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:17.423558+0000","modified":"2024-09-12T12:23:20.482181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/550473013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":550473013},{"type":"v1","addr":"172.21.15.3:6835","nonce":550473013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6012,"qdb_cluster":[6012]},"id":5}]} 2024-09-12T12:23:21.483 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-12T12:23:21.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5f700678c0 msgr2=0x7f5f70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5f700678c0 0x7f5f70069d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5f841f83a0 tx=0x7f5f74008040 comp rx=0 tx=0).stop 2024-09-12T12:23:21.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f840ae880 msgr2=0x7f5f8414e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f840ae880 0x7f5f8414e120 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c370 tx=0x7f5f7800c830 comp rx=0 tx=0).stop 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 shutdown_connections 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5f700678c0 0x7f5f70069d80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5f840b30e0 0x7f5f841501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5f840ae880 0x7f5f8414e120 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 --2- 172.21.15.3:0/3958385469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f84073cb0 0x7f5f840ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.480+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 >> 172.21.15.3:0/3958385469 conn(0x7f5f84067e00 msgr2=0x7f5f8415bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.484+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 shutdown_connections 2024-09-12T12:23:21.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.484+0000 7f5f8b144640 1 -- 172.21.15.3:0/3958385469 wait complete. 2024-09-12T12:23:21.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:21.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.624+0000 7f1dd9054640 1 Processor -- start 2024-09-12T12:23:21.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.624+0000 7f1dd9054640 1 -- start start 2024-09-12T12:23:21.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd9054640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1dd4073cb0 0x7f1dd415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd9054640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dd40b30e0 0x7f1dd40adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd9054640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40ae150 0x7f1dd40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd9054640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd4074090 con 0x7f1dd40b30e0 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd9054640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd40b5000 con 0x7f1dd4073cb0 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd9054640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd40b3530 con 0x7f1dd40ae150 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dd40b30e0 0x7f1dd40adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1dd40b30e0 0x7f1dd40adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38404/0 (socket says 172.21.15.3:38404) 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 -- 172.21.15.3:0/3412862917 learned_addr learned my addr 172.21.15.3:0/3412862917 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd359e640 1 --2- 172.21.15.3:0/3412862917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40ae150 0x7f1dd40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd2d9d640 1 --2- 172.21.15.3:0/3412862917 >> v2:172.21.15.142:3300/0 conn(0x7f1dd4073cb0 0x7f1dd415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 -- 172.21.15.3:0/3412862917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40ae150 msgr2=0x7f1dd40ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 --2- 172.21.15.3:0/3412862917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40ae150 0x7f1dd40ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 -- 172.21.15.3:0/3412862917 >> v2:172.21.15.142:3300/0 conn(0x7f1dd4073cb0 msgr2=0x7f1dd415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:21.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 --2- 172.21.15.3:0/3412862917 >> v2:172.21.15.142:3300/0 conn(0x7f1dd4073cb0 0x7f1dd415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 -- 172.21.15.3:0/3412862917 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd40b37c0 con 0x7f1dd40b30e0 2024-09-12T12:23:21.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd2d9d640 1 --2- 172.21.15.3:0/3412862917 >> v2:172.21.15.142:3300/0 conn(0x7f1dd4073cb0 0x7f1dd415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:21.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd259c640 1 --2- 172.21.15.3:0/3412862917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40b30e0 0x7f1dd40adc10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40d1170 tx=0x7f1dc401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b893ac06ecffc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd1d9b640 1 -- 172.21.15.3:0/3412862917 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40a5670 con 0x7f1dd40b30e0 2024-09-12T12:23:21.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd1d9b640 1 -- 172.21.15.3:0/3412862917 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc40a5810 con 0x7f1dd40b30e0 2024-09-12T12:23:21.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.628+0000 7f1dd1d9b640 1 -- 172.21.15.3:0/3412862917 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40a5be0 con 0x7f1dd40b30e0 2024-09-12T12:23:21.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- 172.21.15.3:0/3412862917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40b30e0 msgr2=0x7f1dd40adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3412862917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40b30e0 0x7f1dd40adc10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40d1170 tx=0x7f1dc401c9f0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- 172.21.15.3:0/3412862917 shutdown_connections 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3412862917 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40ae150 0x7f1dd40ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3412862917 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40b30e0 0x7f1dd40adc10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3412862917 >> v2:172.21.15.142:3300/0 conn(0x7f1dd4073cb0 0x7f1dd415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- 172.21.15.3:0/3412862917 >> 172.21.15.3:0/3412862917 conn(0x7f1dd4067e00 msgr2=0x7f1dd4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- 172.21.15.3:0/3412862917 shutdown_connections 2024-09-12T12:23:21.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- 172.21.15.3:0/3412862917 wait complete. 2024-09-12T12:23:21.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 Processor -- start 2024-09-12T12:23:21.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- start start 2024-09-12T12:23:21.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4073cb0 0x7f1dd41475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40ae150 0x7f1dd409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40b30e0 0x7f1dd409af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd415fe80 con 0x7f1dd40ae150 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd415fd00 con 0x7f1dd4073cb0 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd4160000 con 0x7f1dd40b30e0 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40ae150 0x7f1dd409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40ae150 0x7f1dd409a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38414/0 (socket says 172.21.15.3:38414) 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 -- 172.21.15.3:0/3737587866 learned_addr learned my addr 172.21.15.3:0/3737587866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd359e640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40b30e0 0x7f1dd409af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd2d9d640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4073cb0 0x7f1dd41475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 -- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40b30e0 msgr2=0x7f1dd409af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40b30e0 0x7f1dd409af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 -- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4073cb0 msgr2=0x7f1dd41475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4073cb0 0x7f1dd41475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 -- 172.21.15.3:0/3737587866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd4147af0 con 0x7f1dd40ae150 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd259c640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40ae150 0x7f1dd409a9e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40af040 tx=0x7f1dc4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dcb7fe640 1 -- 172.21.15.3:0/3737587866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40b8670 con 0x7f1dd40ae150 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dcb7fe640 1 -- 172.21.15.3:0/3737587866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc40be070 con 0x7f1dd40ae150 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd41f60a0 con 0x7f1dd40ae150 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dcb7fe640 1 -- 172.21.15.3:0/3737587866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40b8a30 con 0x7f1dd40ae150 2024-09-12T12:23:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.632+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd41f6660 con 0x7f1dd40ae150 2024-09-12T12:23:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.636+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd40ae880 con 0x7f1dd40ae150 2024-09-12T12:23:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.636+0000 7f1dcb7fe640 1 -- 172.21.15.3:0/3737587866 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1dc40b8bd0 con 0x7f1dd40ae150 2024-09-12T12:23:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.636+0000 7f1dcb7fe640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1db80678c0 0x7f1db8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.636+0000 7f1dcb7fe640 1 -- 172.21.15.3:0/3737587866 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f1dc4120e70 con 0x7f1dd40ae150 2024-09-12T12:23:21.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.636+0000 7f1dd2d9d640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1db80678c0 0x7f1db8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.636+0000 7f1dd2d9d640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1db80678c0 0x7f1db8069d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00a9a0 tx=0x7f1dbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.640+0000 7f1dcb7fe640 1 -- 172.21.15.3:0/3737587866 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1dc40165e0 con 0x7f1dd40ae150 2024-09-12T12:23:21.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.804+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dd406bd70 con 0x7f1dd40ae150 2024-09-12T12:23:21.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dcb7fe640 1 -- 172.21.15.3:0/3737587866 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16668 (secure 0 0 0) 0x7f1dc40f1ad0 con 0x7f1dd40ae150 2024-09-12T12:23:21.810 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:21.810 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":66,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:19.099081+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:15.490182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:16.430656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1db80678c0 msgr2=0x7f1db8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1db80678c0 0x7f1db8069d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00a9a0 tx=0x7f1dbc008040 comp rx=0 tx=0).stop 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40ae150 msgr2=0x7f1dd409a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40ae150 0x7f1dd409a9e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40af040 tx=0x7f1dc4019790 comp rx=0 tx=0).stop 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 shutdown_connections 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1db80678c0 0x7f1db8069d80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1dd40b30e0 0x7f1dd409af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dd40ae150 0x7f1dd409a9e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 --2- 172.21.15.3:0/3737587866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4073cb0 0x7f1dd41475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 >> 172.21.15.3:0/3737587866 conn(0x7f1dd4067e00 msgr2=0x7f1dd4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:21.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 shutdown_connections 2024-09-12T12:23:21.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.808+0000 7f1dd9054640 1 -- 172.21.15.3:0/3737587866 wait complete. 2024-09-12T12:23:21.831 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:23:21.831 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:23:21.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd92243640 1 Processor -- start 2024-09-12T12:23:21.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd92243640 1 -- start start 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd92243640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdd8c0d05b0 0x7fdd8c0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd92243640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd8c15f160 con 0x7fdd8c15c000 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd92243640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd8c15f430 con 0x7fdd8c0af410 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd92243640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdd8c15f9f0 con 0x7fdd8c0d05b0 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdd8c15c000 0x7fdd8c0b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47122/0 (socket says 172.21.15.3:47122) 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8b7fe640 1 -- 172.21.15.3:0/1181090023 learned_addr learned my addr 172.21.15.3:0/1181090023 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8bfff640 1 --2- 172.21.15.3:0/1181090023 >> v2:172.21.15.142:3300/0 conn(0x7fdd8c0d05b0 0x7fdd8c0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785390278 0 0) 0x7fdd8c15f160 con 0x7fdd8c15c000 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd78003530 con 0x7fdd8c15c000 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4059408493 0 0) 0x7fdd78003530 con 0x7fdd8c15c000 2024-09-12T12:23:21.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 >> v2:172.21.15.142:3300/0 conn(0x7fdd8c0d05b0 msgr2=0x7fdd8c0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 --2- 172.21.15.3:0/1181090023 >> v2:172.21.15.142:3300/0 conn(0x7fdd8c0d05b0 0x7fdd8c0d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 >> v1:172.21.15.142:6789/0 conn(0x7fdd8c0af410 legacy=0x7fdd8c15ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c15f710 con 0x7fdd8c15c000 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fdd7c001a00 con 0x7fdd8c15c000 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 >> v1:172.21.15.3:6789/0 conn(0x7fdd8c15c000 legacy=0x7fdd8c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780039c0 0x7fdd78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78004810 0x7fdd78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd780051d0 0x7fdd7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c15f9f0 con 0x7fdd780051d0 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7800c170 con 0x7fdd78004810 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7800c370 con 0x7fdd780039c0 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8bfff640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd780051d0 0x7fdd7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8affd640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78004810 0x7fdd78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8b7fe640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780039c0 0x7fdd78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8bfff640 1 -- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780039c0 msgr2=0x7fdd78003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8bfff640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780039c0 0x7fdd78003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8bfff640 1 -- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78004810 msgr2=0x7fdd78004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8bfff640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd78004810 0x7fdd78004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.948+0000 7fdd8bfff640 1 -- 172.21.15.3:0/1181090023 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd78003530 con 0x7fdd780051d0 2024-09-12T12:23:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd8bfff640 1 -- 172.21.15.3:0/1181090023 reap_dead start 2024-09-12T12:23:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd8bfff640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd780051d0 0x7fdd7800b9d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009ae0 tx=0x7fdd8401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9a60aff01ac249 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840096b0 con 0x7fdd780051d0 2024-09-12T12:23:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd84004020 con 0x7fdd780051d0 2024-09-12T12:23:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd8a7fc640 1 -- 172.21.15.3:0/1181090023 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84004410 con 0x7fdd780051d0 2024-09-12T12:23:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 -- 172.21.15.3:0/1181090023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd780051d0 msgr2=0x7fdd7800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd780051d0 0x7fdd7800b9d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009ae0 tx=0x7fdd8401cab0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 -- 172.21.15.3:0/1181090023 shutdown_connections 2024-09-12T12:23:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 --2- 172.21.15.3:0/1181090023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd780051d0 0x7fdd7800b9d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 -- 172.21.15.3:0/1181090023 >> 172.21.15.3:0/1181090023 conn(0x7fdd8c067e00 msgr2=0x7fdd8c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 -- 172.21.15.3:0/1181090023 shutdown_connections 2024-09-12T12:23:21.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 -- 172.21.15.3:0/1181090023 wait complete. 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 Processor -- start 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.952+0000 7fdd92243640 1 -- start start 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780051d0 0x7fdd8c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c09bcb0 0x7fdd8c09c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd8c0b1f20 0x7fdd8c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c15c3e0 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c0b34f0 con 0x7fdd8c0b1f20 2024-09-12T12:23:21.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8c0d0a40 con 0x7fdd780051d0 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c09bcb0 0x7fdd8c09c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c09bcb0 0x7fdd8c09c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38438/0 (socket says 172.21.15.3:38438) 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 -- 172.21.15.3:0/77683873 learned_addr learned my addr 172.21.15.3:0/77683873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8bfff640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd8c0b1f20 0x7fdd8c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8b7fe640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780051d0 0x7fdd8c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 -- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780051d0 msgr2=0x7fdd8c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780051d0 0x7fdd8c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 -- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd8c0b1f20 msgr2=0x7fdd8c0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd8c0b1f20 0x7fdd8c0b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 -- 172.21.15.3:0/77683873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c09c770 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8b7fe640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780051d0 0x7fdd8c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8affd640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c09bcb0 0x7fdd8c09c130 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400e470 tx=0x7fdd7400e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd83fff640 1 -- 172.21.15.3:0/77683873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74019070 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd83fff640 1 -- 172.21.15.3:0/77683873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd740052e0 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c09ca50 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd83fff640 1 -- 172.21.15.3:0/77683873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74003820 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c09d010 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c0d0dd0 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd83fff640 1 -- 172.21.15.3:0/77683873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdd7401c070 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd83fff640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdd6c0678c0 0x7fdd6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd83fff640 1 -- 172.21.15.3:0/77683873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fdd74085210 con 0x7fdd8c09bcb0 2024-09-12T12:23:21.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.956+0000 7fdd8b7fe640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdd6c0678c0 0x7fdd6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.960+0000 7fdd8b7fe640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdd6c0678c0 0x7fdd6c069d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c004c00 tx=0x7fdd7c001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:21.960+0000 7fdd83fff640 1 -- 172.21.15.3:0/77683873 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdd74055670 con 0x7fdd8c09bcb0 2024-09-12T12:23:22.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.116+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd8c073e90 con 0x7fdd6c0678c0 2024-09-12T12:23:22.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.136+0000 7fdd83fff640 1 -- 172.21.15.3:0/77683873 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247331 (secure 0 0 0) 0x7fdd8c073e90 con 0x7fdd6c0678c0 2024-09-12T12:23:22.141 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:22.143 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdd6c0678c0 msgr2=0x7fdd6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdd6c0678c0 0x7fdd6c069d80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c004c00 tx=0x7fdd7c001520 comp rx=0 tx=0).stop 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c09bcb0 msgr2=0x7fdd8c09c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c09bcb0 0x7fdd8c09c130 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400e470 tx=0x7fdd7400e870 comp rx=0 tx=0).stop 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 shutdown_connections 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdd6c0678c0 0x7fdd6c069d80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd8c0b1f20 0x7fdd8c0b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c09bcb0 0x7fdd8c09c130 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 --2- 172.21.15.3:0/77683873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdd780051d0 0x7fdd8c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 >> 172.21.15.3:0/77683873 conn(0x7fdd8c067e00 msgr2=0x7fdd8c15c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 shutdown_connections 2024-09-12T12:23:22.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.140+0000 7fdd92243640 1 -- 172.21.15.3:0/77683873 wait complete. 2024-09-12T12:23:22.158 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-12T12:23:21.729012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235964,"kb_used_data":7216,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738756,"statfs":{"total":773094113280,"available":772852486144,"internally_reserved":0,"allocated":7389184,"data_stored":3743636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.681431"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074113+0000","last_change":"2024-09-12T12:23:17.074113+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074113+0000","last_undegraded":"2024-09-12T12:23:17.074113+0000","last_fullsized":"2024-09-12T12:23:17.074113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063091+0000","last_change":"2024-09-12T12:23:17.063091+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063091+0000","last_undegraded":"2024-09-12T12:23:17.063091+0000","last_fullsized":"2024-09-12T12:23:17.063091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112255+0000","last_change":"2024-09-12T12:23:18.107885+0000","last_active":"2024-09-12T12:23:18.112255+0000","last_peered":"2024-09-12T12:23:18.112255+0000","last_clean":"2024-09-12T12:23:18.112255+0000","last_became_active":"2024-09-12T12:23:18.106854+0000","last_became_peered":"2024-09-12T12:23:18.106854+0000","last_unstale":"2024-09-12T12:23:18.112255+0000","last_undegraded":"2024-09-12T12:23:18.112255+0000","last_fullsized":"2024-09-12T12:23:18.112255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:08:46.225249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066675+0000","last_change":"2024-09-12T12:23:17.066675+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066675+0000","last_undegraded":"2024-09-12T12:23:17.066675+0000","last_fullsized":"2024-09-12T12:23:17.066675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.078213+0000","last_change":"2024-09-12T12:23:17.078213+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.078213+0000","last_undegraded":"2024-09-12T12:23:17.078213+0000","last_fullsized":"2024-09-12T12:23:17.078213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061604+0000","last_change":"2024-09-12T12:23:17.061604+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061604+0000","last_undegraded":"2024-09-12T12:23:17.061604+0000","last_fullsized":"2024-09-12T12:23:17.061604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073184+0000","last_change":"2024-09-12T12:23:17.073184+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073184+0000","last_undegraded":"2024-09-12T12:23:17.073184+0000","last_fullsized":"2024-09-12T12:23:17.073184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.062242+0000","last_change":"2024-09-12T12:23:17.062242+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.062242+0000","last_undegraded":"2024-09-12T12:23:17.062242+0000","last_fullsized":"2024-09-12T12:23:17.062242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.079571+0000","last_change":"2024-09-12T12:23:17.079571+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.079571+0000","last_undegraded":"2024-09-12T12:23:17.079571+0000","last_fullsized":"2024-09-12T12:23:17.079571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066729+0000","last_change":"2024-09-12T12:23:17.066729+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066729+0000","last_undegraded":"2024-09-12T12:23:17.066729+0000","last_fullsized":"2024-09-12T12:23:17.066729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072015+0000","last_change":"2024-09-12T12:23:17.072015+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072015+0000","last_undegraded":"2024-09-12T12:23:17.072015+0000","last_fullsized":"2024-09-12T12:23:17.072015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066185+0000","last_change":"2024-09-12T12:23:17.066185+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066185+0000","last_undegraded":"2024-09-12T12:23:17.066185+0000","last_fullsized":"2024-09-12T12:23:17.066185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076783+0000","last_change":"2024-09-12T12:23:17.076783+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076783+0000","last_undegraded":"2024-09-12T12:23:17.076783+0000","last_fullsized":"2024-09-12T12:23:17.076783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075430+0000","last_change":"2024-09-12T12:23:17.075430+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075430+0000","last_undegraded":"2024-09-12T12:23:17.075430+0000","last_fullsized":"2024-09-12T12:23:17.075430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074111+0000","last_change":"2024-09-12T12:23:17.074111+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074111+0000","last_undegraded":"2024-09-12T12:23:17.074111+0000","last_fullsized":"2024-09-12T12:23:17.074111+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.072096+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077433+0000","last_change":"2024-09-12T12:23:17.077433+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077433+0000","last_undegraded":"2024-09-12T12:23:17.077433+0000","last_fullsized":"2024-09-12T12:23:17.077433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063509+0000","last_change":"2024-09-12T12:23:17.063509+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063509+0000","last_undegraded":"2024-09-12T12:23:17.063509+0000","last_fullsized":"2024-09-12T12:23:17.063509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061634+0000","last_change":"2024-09-12T12:23:17.061634+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061634+0000","last_undegraded":"2024-09-12T12:23:17.061634+0000","last_fullsized":"2024-09-12T12:23:17.061634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072954+0000","last_change":"2024-09-12T12:23:17.072954+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072954+0000","last_undegraded":"2024-09-12T12:23:17.072954+0000","last_fullsized":"2024-09-12T12:23:17.072954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.083851+0000","last_change":"2024-09-12T12:23:17.083851+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.083851+0000","last_undegraded":"2024-09-12T12:23:17.083851+0000","last_fullsized":"2024-09-12T12:23:17.083851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064962+0000","last_change":"2024-09-12T12:23:17.064962+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064962+0000","last_undegraded":"2024-09-12T12:23:17.064962+0000","last_fullsized":"2024-09-12T12:23:17.064962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064538+0000","last_change":"2024-09-12T12:23:17.064538+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064538+0000","last_undegraded":"2024-09-12T12:23:17.064538+0000","last_fullsized":"2024-09-12T12:23:17.064538+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064093+0000","last_change":"2024-09-12T12:23:17.064093+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064093+0000","last_undegraded":"2024-09-12T12:23:17.064093+0000","last_fullsized":"2024-09-12T12:23:17.064093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066150+0000","last_change":"2024-09-12T12:23:17.066150+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066150+0000","last_undegraded":"2024-09-12T12:23:17.066150+0000","last_fullsized":"2024-09-12T12:23:17.066150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073226+0000","last_change":"2024-09-12T12:23:17.073226+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073226+0000","last_undegraded":"2024-09-12T12:23:17.073226+0000","last_fullsized":"2024-09-12T12:23:17.073226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112796+0000","last_change":"2024-09-12T12:23:18.108157+0000","last_active":"2024-09-12T12:23:18.112796+0000","last_peered":"2024-09-12T12:23:18.112796+0000","last_clean":"2024-09-12T12:23:18.112796+0000","last_became_active":"2024-09-12T12:23:18.107527+0000","last_became_peered":"2024-09-12T12:23:18.107527+0000","last_unstale":"2024-09-12T12:23:18.112796+0000","last_undegraded":"2024-09-12T12:23:18.112796+0000","last_fullsized":"2024-09-12T12:23:18.112796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:50.600298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065397+0000","last_change":"2024-09-12T12:23:17.065397+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065397+0000","last_undegraded":"2024-09-12T12:23:17.065397+0000","last_fullsized":"2024-09-12T12:23:17.065397+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112045+0000","last_change":"2024-09-12T12:23:18.107716+0000","last_active":"2024-09-12T12:23:18.112045+0000","last_peered":"2024-09-12T12:23:18.112045+0000","last_clean":"2024-09-12T12:23:18.112045+0000","last_became_active":"2024-09-12T12:23:18.107421+0000","last_became_peered":"2024-09-12T12:23:18.107421+0000","last_unstale":"2024-09-12T12:23:18.112045+0000","last_undegraded":"2024-09-12T12:23:18.112045+0000","last_fullsized":"2024-09-12T12:23:18.112045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:17:59.491002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.070958+0000","last_change":"2024-09-12T12:23:17.070958+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.070958+0000","last_undegraded":"2024-09-12T12:23:17.070958+0000","last_fullsized":"2024-09-12T12:23:17.070958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068727+0000","last_change":"2024-09-12T12:23:17.068727+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068727+0000","last_undegraded":"2024-09-12T12:23:17.068727+0000","last_fullsized":"2024-09-12T12:23:17.068727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061569+0000","last_change":"2024-09-12T12:23:17.061569+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061569+0000","last_undegraded":"2024-09-12T12:23:17.061569+0000","last_fullsized":"2024-09-12T12:23:17.061569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065393+0000","last_change":"2024-09-12T12:23:17.065393+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065393+0000","last_undegraded":"2024-09-12T12:23:17.065393+0000","last_fullsized":"2024-09-12T12:23:17.065393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095301+0000","last_change":"2024-09-12T12:23:17.089339+0000","last_active":"2024-09-12T12:23:17.095301+0000","last_peered":"2024-09-12T12:23:17.095301+0000","last_clean":"2024-09-12T12:23:17.095301+0000","last_became_active":"2024-09-12T12:23:17.088424+0000","last_became_peered":"2024-09-12T12:23:17.088424+0000","last_unstale":"2024-09-12T12:23:17.095301+0000","last_undegraded":"2024-09-12T12:23:17.095301+0000","last_fullsized":"2024-09-12T12:23:17.095301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:03:50.538591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075945+0000","last_change":"2024-09-12T12:23:17.075945+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075945+0000","last_undegraded":"2024-09-12T12:23:17.075945+0000","last_fullsized":"2024-09-12T12:23:17.075945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085927+0000","last_change":"2024-09-12T12:23:17.082761+0000","last_active":"2024-09-12T12:23:17.085927+0000","last_peered":"2024-09-12T12:23:17.085927+0000","last_clean":"2024-09-12T12:23:17.085927+0000","last_became_active":"2024-09-12T12:23:17.082201+0000","last_became_peered":"2024-09-12T12:23:17.082201+0000","last_unstale":"2024-09-12T12:23:17.085927+0000","last_undegraded":"2024-09-12T12:23:17.085927+0000","last_fullsized":"2024-09-12T12:23:17.085927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:36:22.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061131+0000","last_change":"2024-09-12T12:23:17.061131+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061131+0000","last_undegraded":"2024-09-12T12:23:17.061131+0000","last_fullsized":"2024-09-12T12:23:17.061131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096386+0000","last_change":"2024-09-12T12:23:17.093647+0000","last_active":"2024-09-12T12:23:17.096386+0000","last_peered":"2024-09-12T12:23:17.096386+0000","last_clean":"2024-09-12T12:23:17.096386+0000","last_became_active":"2024-09-12T12:23:17.093229+0000","last_became_peered":"2024-09-12T12:23:17.093229+0000","last_unstale":"2024-09-12T12:23:17.096386+0000","last_undegraded":"2024-09-12T12:23:17.096386+0000","last_fullsized":"2024-09-12T12:23:17.096386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:37.847639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069525+0000","last_change":"2024-09-12T12:23:17.069525+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069525+0000","last_undegraded":"2024-09-12T12:23:17.069525+0000","last_fullsized":"2024-09-12T12:23:17.069525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092413+0000","last_change":"2024-09-12T12:23:17.076844+0000","last_active":"2024-09-12T12:23:17.092413+0000","last_peered":"2024-09-12T12:23:17.092413+0000","last_clean":"2024-09-12T12:23:17.092413+0000","last_became_active":"2024-09-12T12:23:17.076096+0000","last_became_peered":"2024-09-12T12:23:17.076096+0000","last_unstale":"2024-09-12T12:23:17.092413+0000","last_undegraded":"2024-09-12T12:23:17.092413+0000","last_fullsized":"2024-09-12T12:23:17.092413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:43.866517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068997+0000","last_change":"2024-09-12T12:23:17.068997+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068997+0000","last_undegraded":"2024-09-12T12:23:17.068997+0000","last_fullsized":"2024-09-12T12:23:17.068997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.127064+0000","last_change":"2024-09-12T12:23:17.088893+0000","last_active":"2024-09-12T12:23:18.127064+0000","last_peered":"2024-09-12T12:23:18.127064+0000","last_clean":"2024-09-12T12:23:18.127064+0000","last_became_active":"2024-09-12T12:23:17.088633+0000","last_became_peered":"2024-09-12T12:23:17.088633+0000","last_unstale":"2024-09-12T12:23:18.127064+0000","last_undegraded":"2024-09-12T12:23:18.127064+0000","last_fullsized":"2024-09-12T12:23:18.127064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:32.158937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073809+0000","last_change":"2024-09-12T12:23:17.073809+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073809+0000","last_undegraded":"2024-09-12T12:23:17.073809+0000","last_fullsized":"2024-09-12T12:23:17.073809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087116+0000","last_change":"2024-09-12T12:23:17.076621+0000","last_active":"2024-09-12T12:23:17.087116+0000","last_peered":"2024-09-12T12:23:17.087116+0000","last_clean":"2024-09-12T12:23:17.087116+0000","last_became_active":"2024-09-12T12:23:17.076227+0000","last_became_peered":"2024-09-12T12:23:17.076227+0000","last_unstale":"2024-09-12T12:23:17.087116+0000","last_undegraded":"2024-09-12T12:23:17.087116+0000","last_fullsized":"2024-09-12T12:23:17.087116+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:54.855620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077266+0000","last_change":"2024-09-12T12:23:17.077266+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077266+0000","last_undegraded":"2024-09-12T12:23:17.077266+0000","last_fullsized":"2024-09-12T12:23:17.077266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096205+0000","last_change":"2024-09-12T12:23:17.091844+0000","last_active":"2024-09-12T12:23:17.096205+0000","last_peered":"2024-09-12T12:23:17.096205+0000","last_clean":"2024-09-12T12:23:17.096205+0000","last_became_active":"2024-09-12T12:23:17.091637+0000","last_became_peered":"2024-09-12T12:23:17.091637+0000","last_unstale":"2024-09-12T12:23:17.096205+0000","last_undegraded":"2024-09-12T12:23:17.096205+0000","last_fullsized":"2024-09-12T12:23:17.096205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073001+0000","last_change":"2024-09-12T12:23:17.073001+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073001+0000","last_undegraded":"2024-09-12T12:23:17.073001+0000","last_fullsized":"2024-09-12T12:23:17.073001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085978+0000","last_change":"2024-09-12T12:23:17.077058+0000","last_active":"2024-09-12T12:23:17.085978+0000","last_peered":"2024-09-12T12:23:17.085978+0000","last_clean":"2024-09-12T12:23:17.085978+0000","last_became_active":"2024-09-12T12:23:17.073114+0000","last_became_peered":"2024-09-12T12:23:17.073114+0000","last_unstale":"2024-09-12T12:23:17.085978+0000","last_undegraded":"2024-09-12T12:23:17.085978+0000","last_fullsized":"2024-09-12T12:23:17.085978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:11:14.125443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069859+0000","last_change":"2024-09-12T12:23:17.069859+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069859+0000","last_undegraded":"2024-09-12T12:23:17.069859+0000","last_fullsized":"2024-09-12T12:23:17.069859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095516+0000","last_change":"2024-09-12T12:23:17.091345+0000","last_active":"2024-09-12T12:23:17.095516+0000","last_peered":"2024-09-12T12:23:17.095516+0000","last_clean":"2024-09-12T12:23:17.095516+0000","last_became_active":"2024-09-12T12:23:17.090644+0000","last_became_peered":"2024-09-12T12:23:17.090644+0000","last_unstale":"2024-09-12T12:23:17.095516+0000","last_undegraded":"2024-09-12T12:23:17.095516+0000","last_fullsized":"2024-09-12T12:23:17.095516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:23.229208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073061+0000","last_change":"2024-09-12T12:23:17.073061+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073061+0000","last_undegraded":"2024-09-12T12:23:17.073061+0000","last_fullsized":"2024-09-12T12:23:17.073061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.068692+0000","last_change":"2024-09-12T12:23:17.064861+0000","last_active":"2024-09-12T12:23:17.068692+0000","last_peered":"2024-09-12T12:23:17.068692+0000","last_clean":"2024-09-12T12:23:17.068692+0000","last_became_active":"2024-09-12T12:23:17.064157+0000","last_became_peered":"2024-09-12T12:23:17.064157+0000","last_unstale":"2024-09-12T12:23:17.068692+0000","last_undegraded":"2024-09-12T12:23:17.068692+0000","last_fullsized":"2024-09-12T12:23:17.068692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:10.504610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.060157+0000","last_change":"2024-09-12T12:23:17.060157+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.060157+0000","last_undegraded":"2024-09-12T12:23:17.060157+0000","last_fullsized":"2024-09-12T12:23:17.060157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.124324+0000","last_change":"2024-09-12T12:23:17.087099+0000","last_active":"2024-09-12T12:23:18.124324+0000","last_peered":"2024-09-12T12:23:18.124324+0000","last_clean":"2024-09-12T12:23:18.124324+0000","last_became_active":"2024-09-12T12:23:17.086732+0000","last_became_peered":"2024-09-12T12:23:17.086732+0000","last_unstale":"2024-09-12T12:23:18.124324+0000","last_undegraded":"2024-09-12T12:23:18.124324+0000","last_fullsized":"2024-09-12T12:23:18.124324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:57:30.109152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075555+0000","last_change":"2024-09-12T12:23:17.075555+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075555+0000","last_undegraded":"2024-09-12T12:23:17.075555+0000","last_fullsized":"2024-09-12T12:23:17.075555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.082993+0000","last_change":"2024-09-12T12:23:17.075577+0000","last_active":"2024-09-12T12:23:17.082993+0000","last_peered":"2024-09-12T12:23:17.082993+0000","last_clean":"2024-09-12T12:23:17.082993+0000","last_became_active":"2024-09-12T12:23:17.074914+0000","last_became_peered":"2024-09-12T12:23:17.074914+0000","last_unstale":"2024-09-12T12:23:17.082993+0000","last_undegraded":"2024-09-12T12:23:17.082993+0000","last_fullsized":"2024-09-12T12:23:17.082993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.067325+0000","last_change":"2024-09-12T12:23:17.067325+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.067325+0000","last_undegraded":"2024-09-12T12:23:17.067325+0000","last_fullsized":"2024-09-12T12:23:17.067325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085808+0000","last_change":"2024-09-12T12:23:17.081531+0000","last_active":"2024-09-12T12:23:17.085808+0000","last_peered":"2024-09-12T12:23:17.085808+0000","last_clean":"2024-09-12T12:23:17.085808+0000","last_became_active":"2024-09-12T12:23:17.081094+0000","last_became_peered":"2024-09-12T12:23:17.081094+0000","last_unstale":"2024-09-12T12:23:17.085808+0000","last_undegraded":"2024-09-12T12:23:17.085808+0000","last_fullsized":"2024-09-12T12:23:17.085808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068665+0000","last_change":"2024-09-12T12:23:17.068665+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068665+0000","last_undegraded":"2024-09-12T12:23:17.068665+0000","last_fullsized":"2024-09-12T12:23:17.068665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.071029+0000","last_change":"2024-09-12T12:23:17.066884+0000","last_active":"2024-09-12T12:23:17.071029+0000","last_peered":"2024-09-12T12:23:17.071029+0000","last_clean":"2024-09-12T12:23:17.071029+0000","last_became_active":"2024-09-12T12:23:17.066181+0000","last_became_peered":"2024-09-12T12:23:17.066181+0000","last_unstale":"2024-09-12T12:23:17.071029+0000","last_undegraded":"2024-09-12T12:23:17.071029+0000","last_fullsized":"2024-09-12T12:23:17.071029+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:55.642549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074022+0000","last_change":"2024-09-12T12:23:17.074022+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074022+0000","last_undegraded":"2024-09-12T12:23:17.074022+0000","last_fullsized":"2024-09-12T12:23:17.074022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.088023+0000","last_change":"2024-09-12T12:23:17.080316+0000","last_active":"2024-09-12T12:23:17.088023+0000","last_peered":"2024-09-12T12:23:17.088023+0000","last_clean":"2024-09-12T12:23:17.088023+0000","last_became_active":"2024-09-12T12:23:17.079639+0000","last_became_peered":"2024-09-12T12:23:17.079639+0000","last_unstale":"2024-09-12T12:23:17.088023+0000","last_undegraded":"2024-09-12T12:23:17.088023+0000","last_fullsized":"2024-09-12T12:23:17.088023+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:46:49.444919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076154+0000","last_change":"2024-09-12T12:23:17.076154+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076154+0000","last_undegraded":"2024-09-12T12:23:17.076154+0000","last_fullsized":"2024-09-12T12:23:17.076154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095475+0000","last_change":"2024-09-12T12:23:17.091123+0000","last_active":"2024-09-12T12:23:17.095475+0000","last_peered":"2024-09-12T12:23:17.095475+0000","last_clean":"2024-09-12T12:23:17.095475+0000","last_became_active":"2024-09-12T12:23:17.090081+0000","last_became_peered":"2024-09-12T12:23:17.090081+0000","last_unstale":"2024-09-12T12:23:17.095475+0000","last_undegraded":"2024-09-12T12:23:17.095475+0000","last_fullsized":"2024-09-12T12:23:17.095475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:23.664767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066148+0000","last_change":"2024-09-12T12:23:17.066148+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066148+0000","last_undegraded":"2024-09-12T12:23:17.066148+0000","last_fullsized":"2024-09-12T12:23:17.066148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092348+0000","last_change":"2024-09-12T12:23:17.078894+0000","last_active":"2024-09-12T12:23:17.092348+0000","last_peered":"2024-09-12T12:23:17.092348+0000","last_clean":"2024-09-12T12:23:17.092348+0000","last_became_active":"2024-09-12T12:23:17.077021+0000","last_became_peered":"2024-09-12T12:23:17.077021+0000","last_unstale":"2024-09-12T12:23:17.092348+0000","last_undegraded":"2024-09-12T12:23:17.092348+0000","last_fullsized":"2024-09-12T12:23:17.092348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068120+0000","last_change":"2024-09-12T12:23:17.068120+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068120+0000","last_undegraded":"2024-09-12T12:23:17.068120+0000","last_fullsized":"2024-09-12T12:23:17.068120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095201+0000","last_change":"2024-09-12T12:23:17.090669+0000","last_active":"2024-09-12T12:23:17.095201+0000","last_peered":"2024-09-12T12:23:17.095201+0000","last_clean":"2024-09-12T12:23:17.095201+0000","last_became_active":"2024-09-12T12:23:17.089910+0000","last_became_peered":"2024-09-12T12:23:17.089910+0000","last_unstale":"2024-09-12T12:23:17.095201+0000","last_undegraded":"2024-09-12T12:23:17.095201+0000","last_fullsized":"2024-09-12T12:23:17.095201+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:08.414320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065359+0000","last_change":"2024-09-12T12:23:17.065359+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065359+0000","last_undegraded":"2024-09-12T12:23:17.065359+0000","last_fullsized":"2024-09-12T12:23:17.065359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096006+0000","last_change":"2024-09-12T12:23:17.093031+0000","last_active":"2024-09-12T12:23:17.096006+0000","last_peered":"2024-09-12T12:23:17.096006+0000","last_clean":"2024-09-12T12:23:17.096006+0000","last_became_active":"2024-09-12T12:23:17.092495+0000","last_became_peered":"2024-09-12T12:23:17.092495+0000","last_unstale":"2024-09-12T12:23:17.096006+0000","last_undegraded":"2024-09-12T12:23:17.096006+0000","last_fullsized":"2024-09-12T12:23:17.096006+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073936+0000","last_change":"2024-09-12T12:23:17.073936+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073936+0000","last_undegraded":"2024-09-12T12:23:17.073936+0000","last_fullsized":"2024-09-12T12:23:17.073936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.097434+0000","last_change":"2024-09-12T12:23:17.096451+0000","last_active":"2024-09-12T12:23:17.097434+0000","last_peered":"2024-09-12T12:23:17.097434+0000","last_clean":"2024-09-12T12:23:17.097434+0000","last_became_active":"2024-09-12T12:23:17.095796+0000","last_became_peered":"2024-09-12T12:23:17.095796+0000","last_unstale":"2024-09-12T12:23:17.097434+0000","last_undegraded":"2024-09-12T12:23:17.097434+0000","last_fullsized":"2024-09-12T12:23:17.097434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:17.993115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.117537+0000","last_change":"2024-09-12T12:23:18.114079+0000","last_active":"2024-09-12T12:23:18.117537+0000","last_peered":"2024-09-12T12:23:18.117537+0000","last_clean":"2024-09-12T12:23:18.117537+0000","last_became_active":"2024-09-12T12:23:18.113666+0000","last_became_peered":"2024-09-12T12:23:18.113666+0000","last_unstale":"2024-09-12T12:23:18.117537+0000","last_undegraded":"2024-09-12T12:23:18.117537+0000","last_fullsized":"2024-09-12T12:23:18.117537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:50.568191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092344+0000","last_change":"2024-09-12T12:23:17.078879+0000","last_active":"2024-09-12T12:23:17.092344+0000","last_peered":"2024-09-12T12:23:17.092344+0000","last_clean":"2024-09-12T12:23:17.092344+0000","last_became_active":"2024-09-12T12:23:17.076836+0000","last_became_peered":"2024-09-12T12:23:17.076836+0000","last_unstale":"2024-09-12T12:23:17.092344+0000","last_undegraded":"2024-09-12T12:23:17.092344+0000","last_fullsized":"2024-09-12T12:23:17.092344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.134210+0000","last_change":"2024-09-12T12:23:18.133120+0000","last_active":"2024-09-12T12:23:18.134210+0000","last_peered":"2024-09-12T12:23:18.134210+0000","last_clean":"2024-09-12T12:23:18.134210+0000","last_became_active":"2024-09-12T12:23:18.132824+0000","last_became_peered":"2024-09-12T12:23:18.132824+0000","last_unstale":"2024-09-12T12:23:18.134210+0000","last_undegraded":"2024-09-12T12:23:18.134210+0000","last_fullsized":"2024-09-12T12:23:18.134210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:09:56.962893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.067441+0000","last_active":"2024-09-12T12:23:17.072096+0000","last_peered":"2024-09-12T12:23:17.072096+0000","last_clean":"2024-09-12T12:23:17.072096+0000","last_became_active":"2024-09-12T12:23:17.066728+0000","last_became_peered":"2024-09-12T12:23:17.066728+0000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:21:59.864966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133456+0000","last_change":"2024-09-12T12:23:18.126379+0000","last_active":"2024-09-12T12:23:18.133456+0000","last_peered":"2024-09-12T12:23:18.133456+0000","last_clean":"2024-09-12T12:23:18.133456+0000","last_became_active":"2024-09-12T12:23:18.126047+0000","last_became_peered":"2024-09-12T12:23:18.126047+0000","last_unstale":"2024-09-12T12:23:18.133456+0000","last_undegraded":"2024-09-12T12:23:18.133456+0000","last_fullsized":"2024-09-12T12:23:18.133456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:46.711808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.083289+0000","last_change":"2024-09-12T12:23:17.078895+0000","last_active":"2024-09-12T12:23:17.083289+0000","last_peered":"2024-09-12T12:23:17.083289+0000","last_clean":"2024-09-12T12:23:17.083289+0000","last_became_active":"2024-09-12T12:23:17.078633+0000","last_became_peered":"2024-09-12T12:23:17.078633+0000","last_unstale":"2024-09-12T12:23:17.083289+0000","last_undegraded":"2024-09-12T12:23:17.083289+0000","last_fullsized":"2024-09-12T12:23:17.083289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:16.522351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072631+0000","last_change":"2024-09-12T12:23:17.072631+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072631+0000","last_undegraded":"2024-09-12T12:23:17.072631+0000","last_fullsized":"2024-09-12T12:23:17.072631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.086997+0000","last_change":"2024-09-12T12:23:17.077229+0000","last_active":"2024-09-12T12:23:17.086997+0000","last_peered":"2024-09-12T12:23:17.086997+0000","last_clean":"2024-09-12T12:23:17.086997+0000","last_became_active":"2024-09-12T12:23:17.076051+0000","last_became_peered":"2024-09-12T12:23:17.076051+0000","last_unstale":"2024-09-12T12:23:17.086997+0000","last_undegraded":"2024-09-12T12:23:17.086997+0000","last_fullsized":"2024-09-12T12:23:17.086997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064090+0000","last_change":"2024-09-12T12:23:17.064090+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064090+0000","last_undegraded":"2024-09-12T12:23:17.064090+0000","last_fullsized":"2024-09-12T12:23:17.064090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087914+0000","last_change":"2024-09-12T12:23:17.079856+0000","last_active":"2024-09-12T12:23:17.087914+0000","last_peered":"2024-09-12T12:23:17.087914+0000","last_clean":"2024-09-12T12:23:17.087914+0000","last_became_active":"2024-09-12T12:23:17.079508+0000","last_became_peered":"2024-09-12T12:23:17.079508+0000","last_unstale":"2024-09-12T12:23:17.087914+0000","last_undegraded":"2024-09-12T12:23:17.087914+0000","last_fullsized":"2024-09-12T12:23:17.087914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:36.840747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066075+0000","last_change":"2024-09-12T12:23:17.066075+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066075+0000","last_undegraded":"2024-09-12T12:23:17.066075+0000","last_fullsized":"2024-09-12T12:23:17.066075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085756+0000","last_change":"2024-09-12T12:23:17.076124+0000","last_active":"2024-09-12T12:23:17.085756+0000","last_peered":"2024-09-12T12:23:17.085756+0000","last_clean":"2024-09-12T12:23:17.085756+0000","last_became_active":"2024-09-12T12:23:17.072753+0000","last_became_peered":"2024-09-12T12:23:17.072753+0000","last_unstale":"2024-09-12T12:23:17.085756+0000","last_undegraded":"2024-09-12T12:23:17.085756+0000","last_fullsized":"2024-09-12T12:23:17.085756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:24.793616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064680+0000","last_change":"2024-09-12T12:23:17.064680+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064680+0000","last_undegraded":"2024-09-12T12:23:17.064680+0000","last_fullsized":"2024-09-12T12:23:17.064680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093479+0000","last_change":"2024-09-12T12:23:17.090571+0000","last_active":"2024-09-12T12:23:17.093479+0000","last_peered":"2024-09-12T12:23:17.093479+0000","last_clean":"2024-09-12T12:23:17.093479+0000","last_became_active":"2024-09-12T12:23:17.090230+0000","last_became_peered":"2024-09-12T12:23:17.090230+0000","last_unstale":"2024-09-12T12:23:17.093479+0000","last_undegraded":"2024-09-12T12:23:17.093479+0000","last_fullsized":"2024-09-12T12:23:17.093479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:25:22.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072543+0000","last_change":"2024-09-12T12:23:17.072543+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072543+0000","last_undegraded":"2024-09-12T12:23:17.072543+0000","last_fullsized":"2024-09-12T12:23:17.072543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085405+0000","last_change":"2024-09-12T12:23:17.073326+0000","last_active":"2024-09-12T12:23:17.085405+0000","last_peered":"2024-09-12T12:23:17.085405+0000","last_clean":"2024-09-12T12:23:17.085405+0000","last_became_active":"2024-09-12T12:23:17.072707+0000","last_became_peered":"2024-09-12T12:23:17.072707+0000","last_unstale":"2024-09-12T12:23:17.085405+0000","last_undegraded":"2024-09-12T12:23:17.085405+0000","last_fullsized":"2024-09-12T12:23:17.085405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:17.187446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.059949+0000","last_change":"2024-09-12T12:23:17.059949+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.059949+0000","last_undegraded":"2024-09-12T12:23:17.059949+0000","last_fullsized":"2024-09-12T12:23:17.059949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093511+0000","last_change":"2024-09-12T12:23:17.089441+0000","last_active":"2024-09-12T12:23:17.093511+0000","last_peered":"2024-09-12T12:23:17.093511+0000","last_clean":"2024-09-12T12:23:17.093511+0000","last_became_active":"2024-09-12T12:23:17.089223+0000","last_became_peered":"2024-09-12T12:23:17.089223+0000","last_unstale":"2024-09-12T12:23:17.093511+0000","last_undegraded":"2024-09-12T12:23:17.093511+0000","last_fullsized":"2024-09-12T12:23:17.093511+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066023+0000","last_change":"2024-09-12T12:23:17.066023+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066023+0000","last_undegraded":"2024-09-12T12:23:17.066023+0000","last_fullsized":"2024-09-12T12:23:17.066023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095568+0000","last_change":"2024-09-12T12:23:17.091347+0000","last_active":"2024-09-12T12:23:17.095568+0000","last_peered":"2024-09-12T12:23:17.095568+0000","last_clean":"2024-09-12T12:23:17.095568+0000","last_became_active":"2024-09-12T12:23:17.090341+0000","last_became_peered":"2024-09-12T12:23:17.090341+0000","last_unstale":"2024-09-12T12:23:17.095568+0000","last_undegraded":"2024-09-12T12:23:17.095568+0000","last_fullsized":"2024-09-12T12:23:17.095568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:38.673027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.057553+0000","last_change":"2024-09-12T12:23:17.057553+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.057553+0000","last_undegraded":"2024-09-12T12:23:17.057553+0000","last_fullsized":"2024-09-12T12:23:17.057553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087239+0000","last_change":"2024-09-12T12:23:17.076822+0000","last_active":"2024-09-12T12:23:17.087239+0000","last_peered":"2024-09-12T12:23:17.087239+0000","last_clean":"2024-09-12T12:23:17.087239+0000","last_became_active":"2024-09-12T12:23:17.075087+0000","last_became_peered":"2024-09-12T12:23:17.075087+0000","last_unstale":"2024-09-12T12:23:17.087239+0000","last_undegraded":"2024-09-12T12:23:17.087239+0000","last_fullsized":"2024-09-12T12:23:17.087239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:29.472329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133487+0000","last_change":"2024-09-12T12:23:18.126382+0000","last_active":"2024-09-12T12:23:18.133487+0000","last_peered":"2024-09-12T12:23:18.133487+0000","last_clean":"2024-09-12T12:23:18.133487+0000","last_became_active":"2024-09-12T12:23:18.126018+0000","last_became_peered":"2024-09-12T12:23:18.126018+0000","last_unstale":"2024-09-12T12:23:18.133487+0000","last_undegraded":"2024-09-12T12:23:18.133487+0000","last_fullsized":"2024-09-12T12:23:18.133487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:57.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087158+0000","last_change":"2024-09-12T12:23:17.076005+0000","last_active":"2024-09-12T12:23:17.087158+0000","last_peered":"2024-09-12T12:23:17.087158+0000","last_clean":"2024-09-12T12:23:17.087158+0000","last_became_active":"2024-09-12T12:23:17.074864+0000","last_became_peered":"2024-09-12T12:23:17.074864+0000","last_unstale":"2024-09-12T12:23:17.087158+0000","last_undegraded":"2024-09-12T12:23:17.087158+0000","last_fullsized":"2024-09-12T12:23:17.087158+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:50.444388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.074838+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:17.074838+0000","last_peered":"2024-09-12T12:23:17.074838+0000","last_clean":"2024-09-12T12:23:17.074838+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:17.074838+0000","last_undegraded":"2024-09-12T12:23:17.074838+0000","last_fullsized":"2024-09-12T12:23:17.074838+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":794624,"data_stored":327173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":904379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1368064,"data_stored":910654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:23:22.163 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:23:22.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8ff581e640 1 Processor -- start 2024-09-12T12:23:22.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8ff581e640 1 -- start start 2024-09-12T12:23:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8ff581e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8ff0073cb0 0x7f8ff015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8ff581e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ff00ae880 0x7f8ff00aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8ff581e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff00b5000 con 0x7f8ff00b30e0 2024-09-12T12:23:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8ff581e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff0074090 con 0x7f8ff00ae880 2024-09-12T12:23:22.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8ff581e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff00b3530 con 0x7f8ff0073cb0 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fef7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ff00ae880 0x7f8ff00aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fee7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8ff00b30e0 0x7f8ff00adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47130/0 (socket says 172.21.15.3:47130) 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fee7fc640 1 -- 172.21.15.3:0/2541032542 learned_addr learned my addr 172.21.15.3:0/2541032542 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8feeffd640 1 --2- 172.21.15.3:0/2541032542 >> v2:172.21.15.142:3300/0 conn(0x7f8ff0073cb0 0x7f8ff015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fef7fe640 1 -- 172.21.15.3:0/2541032542 >> v2:172.21.15.142:3300/0 conn(0x7f8ff0073cb0 msgr2=0x7f8ff015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fef7fe640 1 --2- 172.21.15.3:0/2541032542 >> v2:172.21.15.142:3300/0 conn(0x7f8ff0073cb0 0x7f8ff015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fef7fe640 1 -- 172.21.15.3:0/2541032542 >> v1:172.21.15.3:6789/0 conn(0x7f8ff00b30e0 legacy=0x7f8ff00adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fef7fe640 1 -- 172.21.15.3:0/2541032542 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff00b37c0 con 0x7f8ff00ae880 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fedffb640 1 -- 172.21.15.3:0/2541032542 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530355801 0 0) 0x7f8ff00b5000 con 0x7f8ff00b30e0 2024-09-12T12:23:22.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8feeffd640 1 --2- 172.21.15.3:0/2541032542 >> v2:172.21.15.142:3300/0 conn(0x7f8ff0073cb0 0x7f8ff015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fef7fe640 1 --2- 172.21.15.3:0/2541032542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00aecb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00d1170 tx=0x7f8fe801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9924455953fb4100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fedffb640 1 -- 172.21.15.3:0/2541032542 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80b2070 con 0x7f8ff00ae880 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fedffb640 1 -- 172.21.15.3:0/2541032542 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe80a58e0 con 0x7f8ff00ae880 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.312+0000 7f8fedffb640 1 -- 172.21.15.3:0/2541032542 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80a5cb0 con 0x7f8ff00ae880 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- 172.21.15.3:0/2541032542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 msgr2=0x7f8ff00aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2541032542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00aecb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00d1170 tx=0x7f8fe801c930 comp rx=0 tx=0).stop 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- 172.21.15.3:0/2541032542 shutdown_connections 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2541032542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00aecb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2541032542 >> v2:172.21.15.142:3300/0 conn(0x7f8ff0073cb0 0x7f8ff015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- 172.21.15.3:0/2541032542 >> 172.21.15.3:0/2541032542 conn(0x7f8ff0067e00 msgr2=0x7f8ff0068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- 172.21.15.3:0/2541032542 shutdown_connections 2024-09-12T12:23:22.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- 172.21.15.3:0/2541032542 wait complete. 2024-09-12T12:23:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 Processor -- start 2024-09-12T12:23:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- start start 2024-09-12T12:23:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ff0073cb0 0x7f8ff0145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff00b30e0 0x7f8ff00b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff00ae4a0 con 0x7f8ff00ae880 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff00ae320 con 0x7f8ff00b30e0 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff0168dc0 con 0x7f8ff0073cb0 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38462/0 (socket says 172.21.15.3:38462) 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 -- 172.21.15.3:0/2779521928 learned_addr learned my addr 172.21.15.3:0/2779521928 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8feeffd640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ff0073cb0 0x7f8ff0145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 -- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ff0073cb0 msgr2=0x7f8ff0145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fef7fe640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff00b30e0 0x7f8ff00b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ff0073cb0 0x7f8ff0145690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 -- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff00b30e0 msgr2=0x7f8ff00b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff00b30e0 0x7f8ff00b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 -- 172.21.15.3:0/2779521928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff00b0980 con 0x7f8ff00ae880 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fee7fc640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00b20d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c4b0 tx=0x7f8fe400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fd7fff640 1 -- 172.21.15.3:0/2779521928 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4018070 con 0x7f8ff00ae880 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fd7fff640 1 -- 172.21.15.3:0/2779521928 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe4007500 con 0x7f8ff00ae880 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff00b0c60 con 0x7f8ff00ae880 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8fd7fff640 1 -- 172.21.15.3:0/2779521928 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004a30 con 0x7f8ff00ae880 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.316+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff00b1220 con 0x7f8ff00ae880 2024-09-12T12:23:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.320+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff00aecb0 con 0x7f8ff00ae880 2024-09-12T12:23:22.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.320+0000 7f8fd7fff640 1 -- 172.21.15.3:0/2779521928 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8fe401b070 con 0x7f8ff00ae880 2024-09-12T12:23:22.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.320+0000 7f8fd7fff640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8fdc0678c0 0x7f8fdc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.320+0000 7f8fd7fff640 1 -- 172.21.15.3:0/2779521928 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe4084ed0 con 0x7f8ff00ae880 2024-09-12T12:23:22.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.320+0000 7f8feeffd640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8fdc0678c0 0x7f8fdc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.320+0000 7f8feeffd640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8fdc0678c0 0x7f8fdc069d80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0006fd0 tx=0x7f8fe0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.324+0000 7f8fd7fff640 1 -- 172.21.15.3:0/2779521928 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8fe4055aa0 con 0x7f8ff00ae880 2024-09-12T12:23:22.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.480+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ff0159700 con 0x7f8fdc0678c0 2024-09-12T12:23:22.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.496+0000 7f8fd7fff640 1 -- 172.21.15.3:0/2779521928 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247331 (secure 0 0 0) 0x7f8ff0159700 con 0x7f8fdc0678c0 2024-09-12T12:23:22.503 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:22.504 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:23:22.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8fdc0678c0 msgr2=0x7f8fdc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8fdc0678c0 0x7f8fdc069d80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0006fd0 tx=0x7f8fe0008040 comp rx=0 tx=0).stop 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 msgr2=0x7f8ff00b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00b20d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c4b0 tx=0x7f8fe400c970 comp rx=0 tx=0).stop 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 shutdown_connections 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8fdc0678c0 0x7f8fdc069d80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff00b30e0 0x7f8ff00b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ff00ae880 0x7f8ff00b20d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 --2- 172.21.15.3:0/2779521928 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ff0073cb0 0x7f8ff0145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 >> 172.21.15.3:0/2779521928 conn(0x7f8ff0067e00 msgr2=0x7f8ff006e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 shutdown_connections 2024-09-12T12:23:22.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.500+0000 7f8ff581e640 1 -- 172.21.15.3:0/2779521928 wait complete. 2024-09-12T12:23:22.519 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-12T12:23:21.729012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235964,"kb_used_data":7216,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738756,"statfs":{"total":773094113280,"available":772852486144,"internally_reserved":0,"allocated":7389184,"data_stored":3743636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.681431"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074113+0000","last_change":"2024-09-12T12:23:17.074113+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074113+0000","last_undegraded":"2024-09-12T12:23:17.074113+0000","last_fullsized":"2024-09-12T12:23:17.074113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063091+0000","last_change":"2024-09-12T12:23:17.063091+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063091+0000","last_undegraded":"2024-09-12T12:23:17.063091+0000","last_fullsized":"2024-09-12T12:23:17.063091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112255+0000","last_change":"2024-09-12T12:23:18.107885+0000","last_active":"2024-09-12T12:23:18.112255+0000","last_peered":"2024-09-12T12:23:18.112255+0000","last_clean":"2024-09-12T12:23:18.112255+0000","last_became_active":"2024-09-12T12:23:18.106854+0000","last_became_peered":"2024-09-12T12:23:18.106854+0000","last_unstale":"2024-09-12T12:23:18.112255+0000","last_undegraded":"2024-09-12T12:23:18.112255+0000","last_fullsized":"2024-09-12T12:23:18.112255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:08:46.225249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066675+0000","last_change":"2024-09-12T12:23:17.066675+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066675+0000","last_undegraded":"2024-09-12T12:23:17.066675+0000","last_fullsized":"2024-09-12T12:23:17.066675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.078213+0000","last_change":"2024-09-12T12:23:17.078213+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.078213+0000","last_undegraded":"2024-09-12T12:23:17.078213+0000","last_fullsized":"2024-09-12T12:23:17.078213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061604+0000","last_change":"2024-09-12T12:23:17.061604+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061604+0000","last_undegraded":"2024-09-12T12:23:17.061604+0000","last_fullsized":"2024-09-12T12:23:17.061604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073184+0000","last_change":"2024-09-12T12:23:17.073184+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073184+0000","last_undegraded":"2024-09-12T12:23:17.073184+0000","last_fullsized":"2024-09-12T12:23:17.073184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.062242+0000","last_change":"2024-09-12T12:23:17.062242+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.062242+0000","last_undegraded":"2024-09-12T12:23:17.062242+0000","last_fullsized":"2024-09-12T12:23:17.062242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.079571+0000","last_change":"2024-09-12T12:23:17.079571+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.079571+0000","last_undegraded":"2024-09-12T12:23:17.079571+0000","last_fullsized":"2024-09-12T12:23:17.079571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066729+0000","last_change":"2024-09-12T12:23:17.066729+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066729+0000","last_undegraded":"2024-09-12T12:23:17.066729+0000","last_fullsized":"2024-09-12T12:23:17.066729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072015+0000","last_change":"2024-09-12T12:23:17.072015+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072015+0000","last_undegraded":"2024-09-12T12:23:17.072015+0000","last_fullsized":"2024-09-12T12:23:17.072015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066185+0000","last_change":"2024-09-12T12:23:17.066185+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066185+0000","last_undegraded":"2024-09-12T12:23:17.066185+0000","last_fullsized":"2024-09-12T12:23:17.066185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076783+0000","last_change":"2024-09-12T12:23:17.076783+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076783+0000","last_undegraded":"2024-09-12T12:23:17.076783+0000","last_fullsized":"2024-09-12T12:23:17.076783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075430+0000","last_change":"2024-09-12T12:23:17.075430+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075430+0000","last_undegraded":"2024-09-12T12:23:17.075430+0000","last_fullsized":"2024-09-12T12:23:17.075430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074111+0000","last_change":"2024-09-12T12:23:17.074111+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074111+0000","last_undegraded":"2024-09-12T12:23:17.074111+0000","last_fullsized":"2024-09-12T12:23:17.074111+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.072096+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077433+0000","last_change":"2024-09-12T12:23:17.077433+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077433+0000","last_undegraded":"2024-09-12T12:23:17.077433+0000","last_fullsized":"2024-09-12T12:23:17.077433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063509+0000","last_change":"2024-09-12T12:23:17.063509+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063509+0000","last_undegraded":"2024-09-12T12:23:17.063509+0000","last_fullsized":"2024-09-12T12:23:17.063509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061634+0000","last_change":"2024-09-12T12:23:17.061634+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061634+0000","last_undegraded":"2024-09-12T12:23:17.061634+0000","last_fullsized":"2024-09-12T12:23:17.061634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072954+0000","last_change":"2024-09-12T12:23:17.072954+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072954+0000","last_undegraded":"2024-09-12T12:23:17.072954+0000","last_fullsized":"2024-09-12T12:23:17.072954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.083851+0000","last_change":"2024-09-12T12:23:17.083851+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.083851+0000","last_undegraded":"2024-09-12T12:23:17.083851+0000","last_fullsized":"2024-09-12T12:23:17.083851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064962+0000","last_change":"2024-09-12T12:23:17.064962+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064962+0000","last_undegraded":"2024-09-12T12:23:17.064962+0000","last_fullsized":"2024-09-12T12:23:17.064962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064538+0000","last_change":"2024-09-12T12:23:17.064538+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064538+0000","last_undegraded":"2024-09-12T12:23:17.064538+0000","last_fullsized":"2024-09-12T12:23:17.064538+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064093+0000","last_change":"2024-09-12T12:23:17.064093+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064093+0000","last_undegraded":"2024-09-12T12:23:17.064093+0000","last_fullsized":"2024-09-12T12:23:17.064093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066150+0000","last_change":"2024-09-12T12:23:17.066150+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066150+0000","last_undegraded":"2024-09-12T12:23:17.066150+0000","last_fullsized":"2024-09-12T12:23:17.066150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073226+0000","last_change":"2024-09-12T12:23:17.073226+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073226+0000","last_undegraded":"2024-09-12T12:23:17.073226+0000","last_fullsized":"2024-09-12T12:23:17.073226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112796+0000","last_change":"2024-09-12T12:23:18.108157+0000","last_active":"2024-09-12T12:23:18.112796+0000","last_peered":"2024-09-12T12:23:18.112796+0000","last_clean":"2024-09-12T12:23:18.112796+0000","last_became_active":"2024-09-12T12:23:18.107527+0000","last_became_peered":"2024-09-12T12:23:18.107527+0000","last_unstale":"2024-09-12T12:23:18.112796+0000","last_undegraded":"2024-09-12T12:23:18.112796+0000","last_fullsized":"2024-09-12T12:23:18.112796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:50.600298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065397+0000","last_change":"2024-09-12T12:23:17.065397+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065397+0000","last_undegraded":"2024-09-12T12:23:17.065397+0000","last_fullsized":"2024-09-12T12:23:17.065397+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112045+0000","last_change":"2024-09-12T12:23:18.107716+0000","last_active":"2024-09-12T12:23:18.112045+0000","last_peered":"2024-09-12T12:23:18.112045+0000","last_clean":"2024-09-12T12:23:18.112045+0000","last_became_active":"2024-09-12T12:23:18.107421+0000","last_became_peered":"2024-09-12T12:23:18.107421+0000","last_unstale":"2024-09-12T12:23:18.112045+0000","last_undegraded":"2024-09-12T12:23:18.112045+0000","last_fullsized":"2024-09-12T12:23:18.112045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:17:59.491002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.070958+0000","last_change":"2024-09-12T12:23:17.070958+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.070958+0000","last_undegraded":"2024-09-12T12:23:17.070958+0000","last_fullsized":"2024-09-12T12:23:17.070958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068727+0000","last_change":"2024-09-12T12:23:17.068727+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068727+0000","last_undegraded":"2024-09-12T12:23:17.068727+0000","last_fullsized":"2024-09-12T12:23:17.068727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061569+0000","last_change":"2024-09-12T12:23:17.061569+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061569+0000","last_undegraded":"2024-09-12T12:23:17.061569+0000","last_fullsized":"2024-09-12T12:23:17.061569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065393+0000","last_change":"2024-09-12T12:23:17.065393+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065393+0000","last_undegraded":"2024-09-12T12:23:17.065393+0000","last_fullsized":"2024-09-12T12:23:17.065393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095301+0000","last_change":"2024-09-12T12:23:17.089339+0000","last_active":"2024-09-12T12:23:17.095301+0000","last_peered":"2024-09-12T12:23:17.095301+0000","last_clean":"2024-09-12T12:23:17.095301+0000","last_became_active":"2024-09-12T12:23:17.088424+0000","last_became_peered":"2024-09-12T12:23:17.088424+0000","last_unstale":"2024-09-12T12:23:17.095301+0000","last_undegraded":"2024-09-12T12:23:17.095301+0000","last_fullsized":"2024-09-12T12:23:17.095301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:03:50.538591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075945+0000","last_change":"2024-09-12T12:23:17.075945+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075945+0000","last_undegraded":"2024-09-12T12:23:17.075945+0000","last_fullsized":"2024-09-12T12:23:17.075945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085927+0000","last_change":"2024-09-12T12:23:17.082761+0000","last_active":"2024-09-12T12:23:17.085927+0000","last_peered":"2024-09-12T12:23:17.085927+0000","last_clean":"2024-09-12T12:23:17.085927+0000","last_became_active":"2024-09-12T12:23:17.082201+0000","last_became_peered":"2024-09-12T12:23:17.082201+0000","last_unstale":"2024-09-12T12:23:17.085927+0000","last_undegraded":"2024-09-12T12:23:17.085927+0000","last_fullsized":"2024-09-12T12:23:17.085927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:36:22.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061131+0000","last_change":"2024-09-12T12:23:17.061131+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061131+0000","last_undegraded":"2024-09-12T12:23:17.061131+0000","last_fullsized":"2024-09-12T12:23:17.061131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096386+0000","last_change":"2024-09-12T12:23:17.093647+0000","last_active":"2024-09-12T12:23:17.096386+0000","last_peered":"2024-09-12T12:23:17.096386+0000","last_clean":"2024-09-12T12:23:17.096386+0000","last_became_active":"2024-09-12T12:23:17.093229+0000","last_became_peered":"2024-09-12T12:23:17.093229+0000","last_unstale":"2024-09-12T12:23:17.096386+0000","last_undegraded":"2024-09-12T12:23:17.096386+0000","last_fullsized":"2024-09-12T12:23:17.096386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:37.847639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069525+0000","last_change":"2024-09-12T12:23:17.069525+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069525+0000","last_undegraded":"2024-09-12T12:23:17.069525+0000","last_fullsized":"2024-09-12T12:23:17.069525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092413+0000","last_change":"2024-09-12T12:23:17.076844+0000","last_active":"2024-09-12T12:23:17.092413+0000","last_peered":"2024-09-12T12:23:17.092413+0000","last_clean":"2024-09-12T12:23:17.092413+0000","last_became_active":"2024-09-12T12:23:17.076096+0000","last_became_peered":"2024-09-12T12:23:17.076096+0000","last_unstale":"2024-09-12T12:23:17.092413+0000","last_undegraded":"2024-09-12T12:23:17.092413+0000","last_fullsized":"2024-09-12T12:23:17.092413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:43.866517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068997+0000","last_change":"2024-09-12T12:23:17.068997+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068997+0000","last_undegraded":"2024-09-12T12:23:17.068997+0000","last_fullsized":"2024-09-12T12:23:17.068997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.127064+0000","last_change":"2024-09-12T12:23:17.088893+0000","last_active":"2024-09-12T12:23:18.127064+0000","last_peered":"2024-09-12T12:23:18.127064+0000","last_clean":"2024-09-12T12:23:18.127064+0000","last_became_active":"2024-09-12T12:23:17.088633+0000","last_became_peered":"2024-09-12T12:23:17.088633+0000","last_unstale":"2024-09-12T12:23:18.127064+0000","last_undegraded":"2024-09-12T12:23:18.127064+0000","last_fullsized":"2024-09-12T12:23:18.127064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:32.158937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073809+0000","last_change":"2024-09-12T12:23:17.073809+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073809+0000","last_undegraded":"2024-09-12T12:23:17.073809+0000","last_fullsized":"2024-09-12T12:23:17.073809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087116+0000","last_change":"2024-09-12T12:23:17.076621+0000","last_active":"2024-09-12T12:23:17.087116+0000","last_peered":"2024-09-12T12:23:17.087116+0000","last_clean":"2024-09-12T12:23:17.087116+0000","last_became_active":"2024-09-12T12:23:17.076227+0000","last_became_peered":"2024-09-12T12:23:17.076227+0000","last_unstale":"2024-09-12T12:23:17.087116+0000","last_undegraded":"2024-09-12T12:23:17.087116+0000","last_fullsized":"2024-09-12T12:23:17.087116+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:54.855620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077266+0000","last_change":"2024-09-12T12:23:17.077266+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077266+0000","last_undegraded":"2024-09-12T12:23:17.077266+0000","last_fullsized":"2024-09-12T12:23:17.077266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096205+0000","last_change":"2024-09-12T12:23:17.091844+0000","last_active":"2024-09-12T12:23:17.096205+0000","last_peered":"2024-09-12T12:23:17.096205+0000","last_clean":"2024-09-12T12:23:17.096205+0000","last_became_active":"2024-09-12T12:23:17.091637+0000","last_became_peered":"2024-09-12T12:23:17.091637+0000","last_unstale":"2024-09-12T12:23:17.096205+0000","last_undegraded":"2024-09-12T12:23:17.096205+0000","last_fullsized":"2024-09-12T12:23:17.096205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073001+0000","last_change":"2024-09-12T12:23:17.073001+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073001+0000","last_undegraded":"2024-09-12T12:23:17.073001+0000","last_fullsized":"2024-09-12T12:23:17.073001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085978+0000","last_change":"2024-09-12T12:23:17.077058+0000","last_active":"2024-09-12T12:23:17.085978+0000","last_peered":"2024-09-12T12:23:17.085978+0000","last_clean":"2024-09-12T12:23:17.085978+0000","last_became_active":"2024-09-12T12:23:17.073114+0000","last_became_peered":"2024-09-12T12:23:17.073114+0000","last_unstale":"2024-09-12T12:23:17.085978+0000","last_undegraded":"2024-09-12T12:23:17.085978+0000","last_fullsized":"2024-09-12T12:23:17.085978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:11:14.125443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069859+0000","last_change":"2024-09-12T12:23:17.069859+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069859+0000","last_undegraded":"2024-09-12T12:23:17.069859+0000","last_fullsized":"2024-09-12T12:23:17.069859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095516+0000","last_change":"2024-09-12T12:23:17.091345+0000","last_active":"2024-09-12T12:23:17.095516+0000","last_peered":"2024-09-12T12:23:17.095516+0000","last_clean":"2024-09-12T12:23:17.095516+0000","last_became_active":"2024-09-12T12:23:17.090644+0000","last_became_peered":"2024-09-12T12:23:17.090644+0000","last_unstale":"2024-09-12T12:23:17.095516+0000","last_undegraded":"2024-09-12T12:23:17.095516+0000","last_fullsized":"2024-09-12T12:23:17.095516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:23.229208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073061+0000","last_change":"2024-09-12T12:23:17.073061+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073061+0000","last_undegraded":"2024-09-12T12:23:17.073061+0000","last_fullsized":"2024-09-12T12:23:17.073061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.068692+0000","last_change":"2024-09-12T12:23:17.064861+0000","last_active":"2024-09-12T12:23:17.068692+0000","last_peered":"2024-09-12T12:23:17.068692+0000","last_clean":"2024-09-12T12:23:17.068692+0000","last_became_active":"2024-09-12T12:23:17.064157+0000","last_became_peered":"2024-09-12T12:23:17.064157+0000","last_unstale":"2024-09-12T12:23:17.068692+0000","last_undegraded":"2024-09-12T12:23:17.068692+0000","last_fullsized":"2024-09-12T12:23:17.068692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:10.504610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.060157+0000","last_change":"2024-09-12T12:23:17.060157+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.060157+0000","last_undegraded":"2024-09-12T12:23:17.060157+0000","last_fullsized":"2024-09-12T12:23:17.060157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.124324+0000","last_change":"2024-09-12T12:23:17.087099+0000","last_active":"2024-09-12T12:23:18.124324+0000","last_peered":"2024-09-12T12:23:18.124324+0000","last_clean":"2024-09-12T12:23:18.124324+0000","last_became_active":"2024-09-12T12:23:17.086732+0000","last_became_peered":"2024-09-12T12:23:17.086732+0000","last_unstale":"2024-09-12T12:23:18.124324+0000","last_undegraded":"2024-09-12T12:23:18.124324+0000","last_fullsized":"2024-09-12T12:23:18.124324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:57:30.109152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075555+0000","last_change":"2024-09-12T12:23:17.075555+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075555+0000","last_undegraded":"2024-09-12T12:23:17.075555+0000","last_fullsized":"2024-09-12T12:23:17.075555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.082993+0000","last_change":"2024-09-12T12:23:17.075577+0000","last_active":"2024-09-12T12:23:17.082993+0000","last_peered":"2024-09-12T12:23:17.082993+0000","last_clean":"2024-09-12T12:23:17.082993+0000","last_became_active":"2024-09-12T12:23:17.074914+0000","last_became_peered":"2024-09-12T12:23:17.074914+0000","last_unstale":"2024-09-12T12:23:17.082993+0000","last_undegraded":"2024-09-12T12:23:17.082993+0000","last_fullsized":"2024-09-12T12:23:17.082993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.067325+0000","last_change":"2024-09-12T12:23:17.067325+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.067325+0000","last_undegraded":"2024-09-12T12:23:17.067325+0000","last_fullsized":"2024-09-12T12:23:17.067325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085808+0000","last_change":"2024-09-12T12:23:17.081531+0000","last_active":"2024-09-12T12:23:17.085808+0000","last_peered":"2024-09-12T12:23:17.085808+0000","last_clean":"2024-09-12T12:23:17.085808+0000","last_became_active":"2024-09-12T12:23:17.081094+0000","last_became_peered":"2024-09-12T12:23:17.081094+0000","last_unstale":"2024-09-12T12:23:17.085808+0000","last_undegraded":"2024-09-12T12:23:17.085808+0000","last_fullsized":"2024-09-12T12:23:17.085808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068665+0000","last_change":"2024-09-12T12:23:17.068665+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068665+0000","last_undegraded":"2024-09-12T12:23:17.068665+0000","last_fullsized":"2024-09-12T12:23:17.068665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.071029+0000","last_change":"2024-09-12T12:23:17.066884+0000","last_active":"2024-09-12T12:23:17.071029+0000","last_peered":"2024-09-12T12:23:17.071029+0000","last_clean":"2024-09-12T12:23:17.071029+0000","last_became_active":"2024-09-12T12:23:17.066181+0000","last_became_peered":"2024-09-12T12:23:17.066181+0000","last_unstale":"2024-09-12T12:23:17.071029+0000","last_undegraded":"2024-09-12T12:23:17.071029+0000","last_fullsized":"2024-09-12T12:23:17.071029+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:55.642549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074022+0000","last_change":"2024-09-12T12:23:17.074022+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074022+0000","last_undegraded":"2024-09-12T12:23:17.074022+0000","last_fullsized":"2024-09-12T12:23:17.074022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.088023+0000","last_change":"2024-09-12T12:23:17.080316+0000","last_active":"2024-09-12T12:23:17.088023+0000","last_peered":"2024-09-12T12:23:17.088023+0000","last_clean":"2024-09-12T12:23:17.088023+0000","last_became_active":"2024-09-12T12:23:17.079639+0000","last_became_peered":"2024-09-12T12:23:17.079639+0000","last_unstale":"2024-09-12T12:23:17.088023+0000","last_undegraded":"2024-09-12T12:23:17.088023+0000","last_fullsized":"2024-09-12T12:23:17.088023+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:46:49.444919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076154+0000","last_change":"2024-09-12T12:23:17.076154+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076154+0000","last_undegraded":"2024-09-12T12:23:17.076154+0000","last_fullsized":"2024-09-12T12:23:17.076154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095475+0000","last_change":"2024-09-12T12:23:17.091123+0000","last_active":"2024-09-12T12:23:17.095475+0000","last_peered":"2024-09-12T12:23:17.095475+0000","last_clean":"2024-09-12T12:23:17.095475+0000","last_became_active":"2024-09-12T12:23:17.090081+0000","last_became_peered":"2024-09-12T12:23:17.090081+0000","last_unstale":"2024-09-12T12:23:17.095475+0000","last_undegraded":"2024-09-12T12:23:17.095475+0000","last_fullsized":"2024-09-12T12:23:17.095475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:23.664767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066148+0000","last_change":"2024-09-12T12:23:17.066148+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066148+0000","last_undegraded":"2024-09-12T12:23:17.066148+0000","last_fullsized":"2024-09-12T12:23:17.066148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092348+0000","last_change":"2024-09-12T12:23:17.078894+0000","last_active":"2024-09-12T12:23:17.092348+0000","last_peered":"2024-09-12T12:23:17.092348+0000","last_clean":"2024-09-12T12:23:17.092348+0000","last_became_active":"2024-09-12T12:23:17.077021+0000","last_became_peered":"2024-09-12T12:23:17.077021+0000","last_unstale":"2024-09-12T12:23:17.092348+0000","last_undegraded":"2024-09-12T12:23:17.092348+0000","last_fullsized":"2024-09-12T12:23:17.092348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068120+0000","last_change":"2024-09-12T12:23:17.068120+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068120+0000","last_undegraded":"2024-09-12T12:23:17.068120+0000","last_fullsized":"2024-09-12T12:23:17.068120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095201+0000","last_change":"2024-09-12T12:23:17.090669+0000","last_active":"2024-09-12T12:23:17.095201+0000","last_peered":"2024-09-12T12:23:17.095201+0000","last_clean":"2024-09-12T12:23:17.095201+0000","last_became_active":"2024-09-12T12:23:17.089910+0000","last_became_peered":"2024-09-12T12:23:17.089910+0000","last_unstale":"2024-09-12T12:23:17.095201+0000","last_undegraded":"2024-09-12T12:23:17.095201+0000","last_fullsized":"2024-09-12T12:23:17.095201+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:08.414320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065359+0000","last_change":"2024-09-12T12:23:17.065359+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065359+0000","last_undegraded":"2024-09-12T12:23:17.065359+0000","last_fullsized":"2024-09-12T12:23:17.065359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096006+0000","last_change":"2024-09-12T12:23:17.093031+0000","last_active":"2024-09-12T12:23:17.096006+0000","last_peered":"2024-09-12T12:23:17.096006+0000","last_clean":"2024-09-12T12:23:17.096006+0000","last_became_active":"2024-09-12T12:23:17.092495+0000","last_became_peered":"2024-09-12T12:23:17.092495+0000","last_unstale":"2024-09-12T12:23:17.096006+0000","last_undegraded":"2024-09-12T12:23:17.096006+0000","last_fullsized":"2024-09-12T12:23:17.096006+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073936+0000","last_change":"2024-09-12T12:23:17.073936+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073936+0000","last_undegraded":"2024-09-12T12:23:17.073936+0000","last_fullsized":"2024-09-12T12:23:17.073936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.097434+0000","last_change":"2024-09-12T12:23:17.096451+0000","last_active":"2024-09-12T12:23:17.097434+0000","last_peered":"2024-09-12T12:23:17.097434+0000","last_clean":"2024-09-12T12:23:17.097434+0000","last_became_active":"2024-09-12T12:23:17.095796+0000","last_became_peered":"2024-09-12T12:23:17.095796+0000","last_unstale":"2024-09-12T12:23:17.097434+0000","last_undegraded":"2024-09-12T12:23:17.097434+0000","last_fullsized":"2024-09-12T12:23:17.097434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:17.993115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.117537+0000","last_change":"2024-09-12T12:23:18.114079+0000","last_active":"2024-09-12T12:23:18.117537+0000","last_peered":"2024-09-12T12:23:18.117537+0000","last_clean":"2024-09-12T12:23:18.117537+0000","last_became_active":"2024-09-12T12:23:18.113666+0000","last_became_peered":"2024-09-12T12:23:18.113666+0000","last_unstale":"2024-09-12T12:23:18.117537+0000","last_undegraded":"2024-09-12T12:23:18.117537+0000","last_fullsized":"2024-09-12T12:23:18.117537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:50.568191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092344+0000","last_change":"2024-09-12T12:23:17.078879+0000","last_active":"2024-09-12T12:23:17.092344+0000","last_peered":"2024-09-12T12:23:17.092344+0000","last_clean":"2024-09-12T12:23:17.092344+0000","last_became_active":"2024-09-12T12:23:17.076836+0000","last_became_peered":"2024-09-12T12:23:17.076836+0000","last_unstale":"2024-09-12T12:23:17.092344+0000","last_undegraded":"2024-09-12T12:23:17.092344+0000","last_fullsized":"2024-09-12T12:23:17.092344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.134210+0000","last_change":"2024-09-12T12:23:18.133120+0000","last_active":"2024-09-12T12:23:18.134210+0000","last_peered":"2024-09-12T12:23:18.134210+0000","last_clean":"2024-09-12T12:23:18.134210+0000","last_became_active":"2024-09-12T12:23:18.132824+0000","last_became_peered":"2024-09-12T12:23:18.132824+0000","last_unstale":"2024-09-12T12:23:18.134210+0000","last_undegraded":"2024-09-12T12:23:18.134210+0000","last_fullsized":"2024-09-12T12:23:18.134210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:09:56.962893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.067441+0000","last_active":"2024-09-12T12:23:17.072096+0000","last_peered":"2024-09-12T12:23:17.072096+0000","last_clean":"2024-09-12T12:23:17.072096+0000","last_became_active":"2024-09-12T12:23:17.066728+0000","last_became_peered":"2024-09-12T12:23:17.066728+0000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:21:59.864966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133456+0000","last_change":"2024-09-12T12:23:18.126379+0000","last_active":"2024-09-12T12:23:18.133456+0000","last_peered":"2024-09-12T12:23:18.133456+0000","last_clean":"2024-09-12T12:23:18.133456+0000","last_became_active":"2024-09-12T12:23:18.126047+0000","last_became_peered":"2024-09-12T12:23:18.126047+0000","last_unstale":"2024-09-12T12:23:18.133456+0000","last_undegraded":"2024-09-12T12:23:18.133456+0000","last_fullsized":"2024-09-12T12:23:18.133456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:46.711808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.083289+0000","last_change":"2024-09-12T12:23:17.078895+0000","last_active":"2024-09-12T12:23:17.083289+0000","last_peered":"2024-09-12T12:23:17.083289+0000","last_clean":"2024-09-12T12:23:17.083289+0000","last_became_active":"2024-09-12T12:23:17.078633+0000","last_became_peered":"2024-09-12T12:23:17.078633+0000","last_unstale":"2024-09-12T12:23:17.083289+0000","last_undegraded":"2024-09-12T12:23:17.083289+0000","last_fullsized":"2024-09-12T12:23:17.083289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:16.522351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072631+0000","last_change":"2024-09-12T12:23:17.072631+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072631+0000","last_undegraded":"2024-09-12T12:23:17.072631+0000","last_fullsized":"2024-09-12T12:23:17.072631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.086997+0000","last_change":"2024-09-12T12:23:17.077229+0000","last_active":"2024-09-12T12:23:17.086997+0000","last_peered":"2024-09-12T12:23:17.086997+0000","last_clean":"2024-09-12T12:23:17.086997+0000","last_became_active":"2024-09-12T12:23:17.076051+0000","last_became_peered":"2024-09-12T12:23:17.076051+0000","last_unstale":"2024-09-12T12:23:17.086997+0000","last_undegraded":"2024-09-12T12:23:17.086997+0000","last_fullsized":"2024-09-12T12:23:17.086997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064090+0000","last_change":"2024-09-12T12:23:17.064090+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064090+0000","last_undegraded":"2024-09-12T12:23:17.064090+0000","last_fullsized":"2024-09-12T12:23:17.064090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087914+0000","last_change":"2024-09-12T12:23:17.079856+0000","last_active":"2024-09-12T12:23:17.087914+0000","last_peered":"2024-09-12T12:23:17.087914+0000","last_clean":"2024-09-12T12:23:17.087914+0000","last_became_active":"2024-09-12T12:23:17.079508+0000","last_became_peered":"2024-09-12T12:23:17.079508+0000","last_unstale":"2024-09-12T12:23:17.087914+0000","last_undegraded":"2024-09-12T12:23:17.087914+0000","last_fullsized":"2024-09-12T12:23:17.087914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:36.840747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066075+0000","last_change":"2024-09-12T12:23:17.066075+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066075+0000","last_undegraded":"2024-09-12T12:23:17.066075+0000","last_fullsized":"2024-09-12T12:23:17.066075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085756+0000","last_change":"2024-09-12T12:23:17.076124+0000","last_active":"2024-09-12T12:23:17.085756+0000","last_peered":"2024-09-12T12:23:17.085756+0000","last_clean":"2024-09-12T12:23:17.085756+0000","last_became_active":"2024-09-12T12:23:17.072753+0000","last_became_peered":"2024-09-12T12:23:17.072753+0000","last_unstale":"2024-09-12T12:23:17.085756+0000","last_undegraded":"2024-09-12T12:23:17.085756+0000","last_fullsized":"2024-09-12T12:23:17.085756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:24.793616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064680+0000","last_change":"2024-09-12T12:23:17.064680+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064680+0000","last_undegraded":"2024-09-12T12:23:17.064680+0000","last_fullsized":"2024-09-12T12:23:17.064680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093479+0000","last_change":"2024-09-12T12:23:17.090571+0000","last_active":"2024-09-12T12:23:17.093479+0000","last_peered":"2024-09-12T12:23:17.093479+0000","last_clean":"2024-09-12T12:23:17.093479+0000","last_became_active":"2024-09-12T12:23:17.090230+0000","last_became_peered":"2024-09-12T12:23:17.090230+0000","last_unstale":"2024-09-12T12:23:17.093479+0000","last_undegraded":"2024-09-12T12:23:17.093479+0000","last_fullsized":"2024-09-12T12:23:17.093479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:25:22.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072543+0000","last_change":"2024-09-12T12:23:17.072543+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072543+0000","last_undegraded":"2024-09-12T12:23:17.072543+0000","last_fullsized":"2024-09-12T12:23:17.072543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085405+0000","last_change":"2024-09-12T12:23:17.073326+0000","last_active":"2024-09-12T12:23:17.085405+0000","last_peered":"2024-09-12T12:23:17.085405+0000","last_clean":"2024-09-12T12:23:17.085405+0000","last_became_active":"2024-09-12T12:23:17.072707+0000","last_became_peered":"2024-09-12T12:23:17.072707+0000","last_unstale":"2024-09-12T12:23:17.085405+0000","last_undegraded":"2024-09-12T12:23:17.085405+0000","last_fullsized":"2024-09-12T12:23:17.085405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:17.187446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.059949+0000","last_change":"2024-09-12T12:23:17.059949+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.059949+0000","last_undegraded":"2024-09-12T12:23:17.059949+0000","last_fullsized":"2024-09-12T12:23:17.059949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093511+0000","last_change":"2024-09-12T12:23:17.089441+0000","last_active":"2024-09-12T12:23:17.093511+0000","last_peered":"2024-09-12T12:23:17.093511+0000","last_clean":"2024-09-12T12:23:17.093511+0000","last_became_active":"2024-09-12T12:23:17.089223+0000","last_became_peered":"2024-09-12T12:23:17.089223+0000","last_unstale":"2024-09-12T12:23:17.093511+0000","last_undegraded":"2024-09-12T12:23:17.093511+0000","last_fullsized":"2024-09-12T12:23:17.093511+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066023+0000","last_change":"2024-09-12T12:23:17.066023+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066023+0000","last_undegraded":"2024-09-12T12:23:17.066023+0000","last_fullsized":"2024-09-12T12:23:17.066023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095568+0000","last_change":"2024-09-12T12:23:17.091347+0000","last_active":"2024-09-12T12:23:17.095568+0000","last_peered":"2024-09-12T12:23:17.095568+0000","last_clean":"2024-09-12T12:23:17.095568+0000","last_became_active":"2024-09-12T12:23:17.090341+0000","last_became_peered":"2024-09-12T12:23:17.090341+0000","last_unstale":"2024-09-12T12:23:17.095568+0000","last_undegraded":"2024-09-12T12:23:17.095568+0000","last_fullsized":"2024-09-12T12:23:17.095568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:38.673027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.057553+0000","last_change":"2024-09-12T12:23:17.057553+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.057553+0000","last_undegraded":"2024-09-12T12:23:17.057553+0000","last_fullsized":"2024-09-12T12:23:17.057553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087239+0000","last_change":"2024-09-12T12:23:17.076822+0000","last_active":"2024-09-12T12:23:17.087239+0000","last_peered":"2024-09-12T12:23:17.087239+0000","last_clean":"2024-09-12T12:23:17.087239+0000","last_became_active":"2024-09-12T12:23:17.075087+0000","last_became_peered":"2024-09-12T12:23:17.075087+0000","last_unstale":"2024-09-12T12:23:17.087239+0000","last_undegraded":"2024-09-12T12:23:17.087239+0000","last_fullsized":"2024-09-12T12:23:17.087239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:29.472329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133487+0000","last_change":"2024-09-12T12:23:18.126382+0000","last_active":"2024-09-12T12:23:18.133487+0000","last_peered":"2024-09-12T12:23:18.133487+0000","last_clean":"2024-09-12T12:23:18.133487+0000","last_became_active":"2024-09-12T12:23:18.126018+0000","last_became_peered":"2024-09-12T12:23:18.126018+0000","last_unstale":"2024-09-12T12:23:18.133487+0000","last_undegraded":"2024-09-12T12:23:18.133487+0000","last_fullsized":"2024-09-12T12:23:18.133487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:57.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087158+0000","last_change":"2024-09-12T12:23:17.076005+0000","last_active":"2024-09-12T12:23:17.087158+0000","last_peered":"2024-09-12T12:23:17.087158+0000","last_clean":"2024-09-12T12:23:17.087158+0000","last_became_active":"2024-09-12T12:23:17.074864+0000","last_became_peered":"2024-09-12T12:23:17.074864+0000","last_unstale":"2024-09-12T12:23:17.087158+0000","last_undegraded":"2024-09-12T12:23:17.087158+0000","last_fullsized":"2024-09-12T12:23:17.087158+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:50.444388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.074838+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:17.074838+0000","last_peered":"2024-09-12T12:23:17.074838+0000","last_clean":"2024-09-12T12:23:17.074838+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:17.074838+0000","last_undegraded":"2024-09-12T12:23:17.074838+0000","last_fullsized":"2024-09-12T12:23:17.074838+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":794624,"data_stored":327173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":904379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1368064,"data_stored":910654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:23:22.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:23:22.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedfb97640 1 Processor -- start 2024-09-12T12:23:22.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedfb97640 1 -- start start 2024-09-12T12:23:22.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedfb97640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feed8073cb0 0x7feed815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedfb97640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80ae880 0x7feed80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedfb97640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed80b5000 con 0x7feed80b30e0 2024-09-12T12:23:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedfb97640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7feed8074090 con 0x7feed8073cb0 2024-09-12T12:23:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedfb97640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feed80b3530 con 0x7feed80ae880 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedd933640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feed8073cb0 0x7feed815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feede134640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80ae880 0x7feed80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedd132640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7feed80b30e0 0x7feed80adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:34774/0 (socket says 172.21.15.3:34774) 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedd132640 1 -- 172.21.15.3:0/194623428 learned_addr learned my addr 172.21.15.3:0/194623428 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedd933640 1 -- 172.21.15.3:0/194623428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80ae880 msgr2=0x7feed80aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedd933640 1 --2- 172.21.15.3:0/194623428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80ae880 0x7feed80aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedd933640 1 -- 172.21.15.3:0/194623428 >> v1:172.21.15.142:6789/0 conn(0x7feed80b30e0 legacy=0x7feed80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.684+0000 7feedd933640 1 -- 172.21.15.3:0/194623428 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed80b37c0 con 0x7feed8073cb0 2024-09-12T12:23:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedd933640 1 --2- 172.21.15.3:0/194623428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 0x7feed815c710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feed40097c0 tx=0x7feed401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bcd27ede4beafbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedc931640 1 -- 172.21.15.3:0/194623428 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40b2070 con 0x7feed8073cb0 2024-09-12T12:23:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedc931640 1 -- 172.21.15.3:0/194623428 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed40ad640 con 0x7feed8073cb0 2024-09-12T12:23:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedc931640 1 -- 172.21.15.3:0/194623428 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40ada10 con 0x7feed8073cb0 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 -- 172.21.15.3:0/194623428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 msgr2=0x7feed815c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 --2- 172.21.15.3:0/194623428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 0x7feed815c710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feed40097c0 tx=0x7feed401c4a0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 -- 172.21.15.3:0/194623428 shutdown_connections 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 --2- 172.21.15.3:0/194623428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80ae880 0x7feed80aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 --2- 172.21.15.3:0/194623428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 0x7feed815c710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 -- 172.21.15.3:0/194623428 >> 172.21.15.3:0/194623428 conn(0x7feed8067e00 msgr2=0x7feed8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 -- 172.21.15.3:0/194623428 shutdown_connections 2024-09-12T12:23:22.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.688+0000 7feedfb97640 1 -- 172.21.15.3:0/194623428 wait complete. 2024-09-12T12:23:22.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 Processor -- start 2024-09-12T12:23:22.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 -- start start 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 0x7feed80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feed80ae880 0x7feed809cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80b30e0 0x7feed809b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feed80ae4a0 con 0x7feed80ae880 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feed80ae320 con 0x7feed8073cb0 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feed8168dc0 con 0x7feed80b30e0 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feed80ae880 0x7feed809cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feed80ae880 0x7feed809cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38474/0 (socket says 172.21.15.3:38474) 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 -- 172.21.15.3:0/3097030796 learned_addr learned my addr 172.21.15.3:0/3097030796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd933640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 0x7feed80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feede134640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80b30e0 0x7feed809b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 -- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80b30e0 msgr2=0x7feed809b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80b30e0 0x7feed809b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 -- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 msgr2=0x7feed80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 0x7feed80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 -- 172.21.15.3:0/3097030796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed80cf0f0 con 0x7feed80ae880 2024-09-12T12:23:22.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedd132640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feed80ae880 0x7feed809cfd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7feec800c370 tx=0x7feec800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feece7fc640 1 -- 172.21.15.3:0/3097030796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8017070 con 0x7feed80ae880 2024-09-12T12:23:22.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feece7fc640 1 -- 172.21.15.3:0/3097030796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8005cf0 con 0x7feed80ae880 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feece7fc640 1 -- 172.21.15.3:0/3097030796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80072f0 con 0x7feed80ae880 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed80cf3d0 con 0x7feed80ae880 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed809aa90 con 0x7feed80ae880 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed815c710 con 0x7feed80ae880 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.692+0000 7feece7fc640 1 -- 172.21.15.3:0/3097030796 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feec801a070 con 0x7feed80ae880 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.696+0000 7feece7fc640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feebc0678c0 0x7feebc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.696+0000 7feece7fc640 1 -- 172.21.15.3:0/3097030796 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7feec8084f60 con 0x7feed80ae880 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.696+0000 7feedd933640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feebc0678c0 0x7feebc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.696+0000 7feedd933640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feebc0678c0 0x7feebc069d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feed40af000 tx=0x7feed4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.700+0000 7feece7fc640 1 -- 172.21.15.3:0/3097030796 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feec80553d0 con 0x7feed80ae880 2024-09-12T12:23:22.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.856+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feed8159730 con 0x7feebc0678c0 2024-09-12T12:23:22.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.876+0000 7feece7fc640 1 -- 172.21.15.3:0/3097030796 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247331 (secure 0 0 0) 0x7feed8159730 con 0x7feebc0678c0 2024-09-12T12:23:22.880 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:22.882 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:23:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feebc0678c0 msgr2=0x7feebc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feebc0678c0 0x7feebc069d80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feed40af000 tx=0x7feed4008040 comp rx=0 tx=0).stop 2024-09-12T12:23:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feed80ae880 msgr2=0x7feed809cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feed80ae880 0x7feed809cfd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7feec800c370 tx=0x7feec800c830 comp rx=0 tx=0).stop 2024-09-12T12:23:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 shutdown_connections 2024-09-12T12:23:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feebc0678c0 0x7feebc069d80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feed80b30e0 0x7feed809b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feed80ae880 0x7feed809cfd0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 --2- 172.21.15.3:0/3097030796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feed8073cb0 0x7feed80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 >> 172.21.15.3:0/3097030796 conn(0x7feed8067e00 msgr2=0x7feed815aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 shutdown_connections 2024-09-12T12:23:22.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:22.880+0000 7feedfb97640 1 -- 172.21.15.3:0/3097030796 wait complete. 2024-09-12T12:23:22.897 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-12T12:23:21.729012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235964,"kb_used_data":7216,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738756,"statfs":{"total":773094113280,"available":772852486144,"internally_reserved":0,"allocated":7389184,"data_stored":3743636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.681431"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074113+0000","last_change":"2024-09-12T12:23:17.074113+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074113+0000","last_undegraded":"2024-09-12T12:23:17.074113+0000","last_fullsized":"2024-09-12T12:23:17.074113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063091+0000","last_change":"2024-09-12T12:23:17.063091+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063091+0000","last_undegraded":"2024-09-12T12:23:17.063091+0000","last_fullsized":"2024-09-12T12:23:17.063091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112255+0000","last_change":"2024-09-12T12:23:18.107885+0000","last_active":"2024-09-12T12:23:18.112255+0000","last_peered":"2024-09-12T12:23:18.112255+0000","last_clean":"2024-09-12T12:23:18.112255+0000","last_became_active":"2024-09-12T12:23:18.106854+0000","last_became_peered":"2024-09-12T12:23:18.106854+0000","last_unstale":"2024-09-12T12:23:18.112255+0000","last_undegraded":"2024-09-12T12:23:18.112255+0000","last_fullsized":"2024-09-12T12:23:18.112255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:08:46.225249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066675+0000","last_change":"2024-09-12T12:23:17.066675+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066675+0000","last_undegraded":"2024-09-12T12:23:17.066675+0000","last_fullsized":"2024-09-12T12:23:17.066675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.078213+0000","last_change":"2024-09-12T12:23:17.078213+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.078213+0000","last_undegraded":"2024-09-12T12:23:17.078213+0000","last_fullsized":"2024-09-12T12:23:17.078213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061604+0000","last_change":"2024-09-12T12:23:17.061604+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061604+0000","last_undegraded":"2024-09-12T12:23:17.061604+0000","last_fullsized":"2024-09-12T12:23:17.061604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073184+0000","last_change":"2024-09-12T12:23:17.073184+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073184+0000","last_undegraded":"2024-09-12T12:23:17.073184+0000","last_fullsized":"2024-09-12T12:23:17.073184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.062242+0000","last_change":"2024-09-12T12:23:17.062242+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.062242+0000","last_undegraded":"2024-09-12T12:23:17.062242+0000","last_fullsized":"2024-09-12T12:23:17.062242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.079571+0000","last_change":"2024-09-12T12:23:17.079571+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.079571+0000","last_undegraded":"2024-09-12T12:23:17.079571+0000","last_fullsized":"2024-09-12T12:23:17.079571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066729+0000","last_change":"2024-09-12T12:23:17.066729+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066729+0000","last_undegraded":"2024-09-12T12:23:17.066729+0000","last_fullsized":"2024-09-12T12:23:17.066729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072015+0000","last_change":"2024-09-12T12:23:17.072015+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072015+0000","last_undegraded":"2024-09-12T12:23:17.072015+0000","last_fullsized":"2024-09-12T12:23:17.072015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066185+0000","last_change":"2024-09-12T12:23:17.066185+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066185+0000","last_undegraded":"2024-09-12T12:23:17.066185+0000","last_fullsized":"2024-09-12T12:23:17.066185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076783+0000","last_change":"2024-09-12T12:23:17.076783+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076783+0000","last_undegraded":"2024-09-12T12:23:17.076783+0000","last_fullsized":"2024-09-12T12:23:17.076783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075430+0000","last_change":"2024-09-12T12:23:17.075430+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075430+0000","last_undegraded":"2024-09-12T12:23:17.075430+0000","last_fullsized":"2024-09-12T12:23:17.075430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074111+0000","last_change":"2024-09-12T12:23:17.074111+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074111+0000","last_undegraded":"2024-09-12T12:23:17.074111+0000","last_fullsized":"2024-09-12T12:23:17.074111+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.072096+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077433+0000","last_change":"2024-09-12T12:23:17.077433+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077433+0000","last_undegraded":"2024-09-12T12:23:17.077433+0000","last_fullsized":"2024-09-12T12:23:17.077433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063509+0000","last_change":"2024-09-12T12:23:17.063509+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063509+0000","last_undegraded":"2024-09-12T12:23:17.063509+0000","last_fullsized":"2024-09-12T12:23:17.063509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061634+0000","last_change":"2024-09-12T12:23:17.061634+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061634+0000","last_undegraded":"2024-09-12T12:23:17.061634+0000","last_fullsized":"2024-09-12T12:23:17.061634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072954+0000","last_change":"2024-09-12T12:23:17.072954+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072954+0000","last_undegraded":"2024-09-12T12:23:17.072954+0000","last_fullsized":"2024-09-12T12:23:17.072954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.083851+0000","last_change":"2024-09-12T12:23:17.083851+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.083851+0000","last_undegraded":"2024-09-12T12:23:17.083851+0000","last_fullsized":"2024-09-12T12:23:17.083851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064962+0000","last_change":"2024-09-12T12:23:17.064962+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064962+0000","last_undegraded":"2024-09-12T12:23:17.064962+0000","last_fullsized":"2024-09-12T12:23:17.064962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064538+0000","last_change":"2024-09-12T12:23:17.064538+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064538+0000","last_undegraded":"2024-09-12T12:23:17.064538+0000","last_fullsized":"2024-09-12T12:23:17.064538+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064093+0000","last_change":"2024-09-12T12:23:17.064093+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064093+0000","last_undegraded":"2024-09-12T12:23:17.064093+0000","last_fullsized":"2024-09-12T12:23:17.064093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066150+0000","last_change":"2024-09-12T12:23:17.066150+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066150+0000","last_undegraded":"2024-09-12T12:23:17.066150+0000","last_fullsized":"2024-09-12T12:23:17.066150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073226+0000","last_change":"2024-09-12T12:23:17.073226+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073226+0000","last_undegraded":"2024-09-12T12:23:17.073226+0000","last_fullsized":"2024-09-12T12:23:17.073226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112796+0000","last_change":"2024-09-12T12:23:18.108157+0000","last_active":"2024-09-12T12:23:18.112796+0000","last_peered":"2024-09-12T12:23:18.112796+0000","last_clean":"2024-09-12T12:23:18.112796+0000","last_became_active":"2024-09-12T12:23:18.107527+0000","last_became_peered":"2024-09-12T12:23:18.107527+0000","last_unstale":"2024-09-12T12:23:18.112796+0000","last_undegraded":"2024-09-12T12:23:18.112796+0000","last_fullsized":"2024-09-12T12:23:18.112796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:50.600298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065397+0000","last_change":"2024-09-12T12:23:17.065397+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065397+0000","last_undegraded":"2024-09-12T12:23:17.065397+0000","last_fullsized":"2024-09-12T12:23:17.065397+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112045+0000","last_change":"2024-09-12T12:23:18.107716+0000","last_active":"2024-09-12T12:23:18.112045+0000","last_peered":"2024-09-12T12:23:18.112045+0000","last_clean":"2024-09-12T12:23:18.112045+0000","last_became_active":"2024-09-12T12:23:18.107421+0000","last_became_peered":"2024-09-12T12:23:18.107421+0000","last_unstale":"2024-09-12T12:23:18.112045+0000","last_undegraded":"2024-09-12T12:23:18.112045+0000","last_fullsized":"2024-09-12T12:23:18.112045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:17:59.491002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.070958+0000","last_change":"2024-09-12T12:23:17.070958+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.070958+0000","last_undegraded":"2024-09-12T12:23:17.070958+0000","last_fullsized":"2024-09-12T12:23:17.070958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068727+0000","last_change":"2024-09-12T12:23:17.068727+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068727+0000","last_undegraded":"2024-09-12T12:23:17.068727+0000","last_fullsized":"2024-09-12T12:23:17.068727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061569+0000","last_change":"2024-09-12T12:23:17.061569+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061569+0000","last_undegraded":"2024-09-12T12:23:17.061569+0000","last_fullsized":"2024-09-12T12:23:17.061569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065393+0000","last_change":"2024-09-12T12:23:17.065393+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065393+0000","last_undegraded":"2024-09-12T12:23:17.065393+0000","last_fullsized":"2024-09-12T12:23:17.065393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095301+0000","last_change":"2024-09-12T12:23:17.089339+0000","last_active":"2024-09-12T12:23:17.095301+0000","last_peered":"2024-09-12T12:23:17.095301+0000","last_clean":"2024-09-12T12:23:17.095301+0000","last_became_active":"2024-09-12T12:23:17.088424+0000","last_became_peered":"2024-09-12T12:23:17.088424+0000","last_unstale":"2024-09-12T12:23:17.095301+0000","last_undegraded":"2024-09-12T12:23:17.095301+0000","last_fullsized":"2024-09-12T12:23:17.095301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:03:50.538591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075945+0000","last_change":"2024-09-12T12:23:17.075945+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075945+0000","last_undegraded":"2024-09-12T12:23:17.075945+0000","last_fullsized":"2024-09-12T12:23:17.075945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085927+0000","last_change":"2024-09-12T12:23:17.082761+0000","last_active":"2024-09-12T12:23:17.085927+0000","last_peered":"2024-09-12T12:23:17.085927+0000","last_clean":"2024-09-12T12:23:17.085927+0000","last_became_active":"2024-09-12T12:23:17.082201+0000","last_became_peered":"2024-09-12T12:23:17.082201+0000","last_unstale":"2024-09-12T12:23:17.085927+0000","last_undegraded":"2024-09-12T12:23:17.085927+0000","last_fullsized":"2024-09-12T12:23:17.085927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:36:22.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061131+0000","last_change":"2024-09-12T12:23:17.061131+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061131+0000","last_undegraded":"2024-09-12T12:23:17.061131+0000","last_fullsized":"2024-09-12T12:23:17.061131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096386+0000","last_change":"2024-09-12T12:23:17.093647+0000","last_active":"2024-09-12T12:23:17.096386+0000","last_peered":"2024-09-12T12:23:17.096386+0000","last_clean":"2024-09-12T12:23:17.096386+0000","last_became_active":"2024-09-12T12:23:17.093229+0000","last_became_peered":"2024-09-12T12:23:17.093229+0000","last_unstale":"2024-09-12T12:23:17.096386+0000","last_undegraded":"2024-09-12T12:23:17.096386+0000","last_fullsized":"2024-09-12T12:23:17.096386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:37.847639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069525+0000","last_change":"2024-09-12T12:23:17.069525+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069525+0000","last_undegraded":"2024-09-12T12:23:17.069525+0000","last_fullsized":"2024-09-12T12:23:17.069525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092413+0000","last_change":"2024-09-12T12:23:17.076844+0000","last_active":"2024-09-12T12:23:17.092413+0000","last_peered":"2024-09-12T12:23:17.092413+0000","last_clean":"2024-09-12T12:23:17.092413+0000","last_became_active":"2024-09-12T12:23:17.076096+0000","last_became_peered":"2024-09-12T12:23:17.076096+0000","last_unstale":"2024-09-12T12:23:17.092413+0000","last_undegraded":"2024-09-12T12:23:17.092413+0000","last_fullsized":"2024-09-12T12:23:17.092413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:43.866517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068997+0000","last_change":"2024-09-12T12:23:17.068997+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068997+0000","last_undegraded":"2024-09-12T12:23:17.068997+0000","last_fullsized":"2024-09-12T12:23:17.068997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.127064+0000","last_change":"2024-09-12T12:23:17.088893+0000","last_active":"2024-09-12T12:23:18.127064+0000","last_peered":"2024-09-12T12:23:18.127064+0000","last_clean":"2024-09-12T12:23:18.127064+0000","last_became_active":"2024-09-12T12:23:17.088633+0000","last_became_peered":"2024-09-12T12:23:17.088633+0000","last_unstale":"2024-09-12T12:23:18.127064+0000","last_undegraded":"2024-09-12T12:23:18.127064+0000","last_fullsized":"2024-09-12T12:23:18.127064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:32.158937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073809+0000","last_change":"2024-09-12T12:23:17.073809+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073809+0000","last_undegraded":"2024-09-12T12:23:17.073809+0000","last_fullsized":"2024-09-12T12:23:17.073809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087116+0000","last_change":"2024-09-12T12:23:17.076621+0000","last_active":"2024-09-12T12:23:17.087116+0000","last_peered":"2024-09-12T12:23:17.087116+0000","last_clean":"2024-09-12T12:23:17.087116+0000","last_became_active":"2024-09-12T12:23:17.076227+0000","last_became_peered":"2024-09-12T12:23:17.076227+0000","last_unstale":"2024-09-12T12:23:17.087116+0000","last_undegraded":"2024-09-12T12:23:17.087116+0000","last_fullsized":"2024-09-12T12:23:17.087116+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:54.855620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077266+0000","last_change":"2024-09-12T12:23:17.077266+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077266+0000","last_undegraded":"2024-09-12T12:23:17.077266+0000","last_fullsized":"2024-09-12T12:23:17.077266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096205+0000","last_change":"2024-09-12T12:23:17.091844+0000","last_active":"2024-09-12T12:23:17.096205+0000","last_peered":"2024-09-12T12:23:17.096205+0000","last_clean":"2024-09-12T12:23:17.096205+0000","last_became_active":"2024-09-12T12:23:17.091637+0000","last_became_peered":"2024-09-12T12:23:17.091637+0000","last_unstale":"2024-09-12T12:23:17.096205+0000","last_undegraded":"2024-09-12T12:23:17.096205+0000","last_fullsized":"2024-09-12T12:23:17.096205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073001+0000","last_change":"2024-09-12T12:23:17.073001+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073001+0000","last_undegraded":"2024-09-12T12:23:17.073001+0000","last_fullsized":"2024-09-12T12:23:17.073001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085978+0000","last_change":"2024-09-12T12:23:17.077058+0000","last_active":"2024-09-12T12:23:17.085978+0000","last_peered":"2024-09-12T12:23:17.085978+0000","last_clean":"2024-09-12T12:23:17.085978+0000","last_became_active":"2024-09-12T12:23:17.073114+0000","last_became_peered":"2024-09-12T12:23:17.073114+0000","last_unstale":"2024-09-12T12:23:17.085978+0000","last_undegraded":"2024-09-12T12:23:17.085978+0000","last_fullsized":"2024-09-12T12:23:17.085978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:11:14.125443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069859+0000","last_change":"2024-09-12T12:23:17.069859+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069859+0000","last_undegraded":"2024-09-12T12:23:17.069859+0000","last_fullsized":"2024-09-12T12:23:17.069859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095516+0000","last_change":"2024-09-12T12:23:17.091345+0000","last_active":"2024-09-12T12:23:17.095516+0000","last_peered":"2024-09-12T12:23:17.095516+0000","last_clean":"2024-09-12T12:23:17.095516+0000","last_became_active":"2024-09-12T12:23:17.090644+0000","last_became_peered":"2024-09-12T12:23:17.090644+0000","last_unstale":"2024-09-12T12:23:17.095516+0000","last_undegraded":"2024-09-12T12:23:17.095516+0000","last_fullsized":"2024-09-12T12:23:17.095516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:23.229208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073061+0000","last_change":"2024-09-12T12:23:17.073061+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073061+0000","last_undegraded":"2024-09-12T12:23:17.073061+0000","last_fullsized":"2024-09-12T12:23:17.073061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.068692+0000","last_change":"2024-09-12T12:23:17.064861+0000","last_active":"2024-09-12T12:23:17.068692+0000","last_peered":"2024-09-12T12:23:17.068692+0000","last_clean":"2024-09-12T12:23:17.068692+0000","last_became_active":"2024-09-12T12:23:17.064157+0000","last_became_peered":"2024-09-12T12:23:17.064157+0000","last_unstale":"2024-09-12T12:23:17.068692+0000","last_undegraded":"2024-09-12T12:23:17.068692+0000","last_fullsized":"2024-09-12T12:23:17.068692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:10.504610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.060157+0000","last_change":"2024-09-12T12:23:17.060157+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.060157+0000","last_undegraded":"2024-09-12T12:23:17.060157+0000","last_fullsized":"2024-09-12T12:23:17.060157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.124324+0000","last_change":"2024-09-12T12:23:17.087099+0000","last_active":"2024-09-12T12:23:18.124324+0000","last_peered":"2024-09-12T12:23:18.124324+0000","last_clean":"2024-09-12T12:23:18.124324+0000","last_became_active":"2024-09-12T12:23:17.086732+0000","last_became_peered":"2024-09-12T12:23:17.086732+0000","last_unstale":"2024-09-12T12:23:18.124324+0000","last_undegraded":"2024-09-12T12:23:18.124324+0000","last_fullsized":"2024-09-12T12:23:18.124324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:57:30.109152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075555+0000","last_change":"2024-09-12T12:23:17.075555+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075555+0000","last_undegraded":"2024-09-12T12:23:17.075555+0000","last_fullsized":"2024-09-12T12:23:17.075555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.082993+0000","last_change":"2024-09-12T12:23:17.075577+0000","last_active":"2024-09-12T12:23:17.082993+0000","last_peered":"2024-09-12T12:23:17.082993+0000","last_clean":"2024-09-12T12:23:17.082993+0000","last_became_active":"2024-09-12T12:23:17.074914+0000","last_became_peered":"2024-09-12T12:23:17.074914+0000","last_unstale":"2024-09-12T12:23:17.082993+0000","last_undegraded":"2024-09-12T12:23:17.082993+0000","last_fullsized":"2024-09-12T12:23:17.082993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.067325+0000","last_change":"2024-09-12T12:23:17.067325+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.067325+0000","last_undegraded":"2024-09-12T12:23:17.067325+0000","last_fullsized":"2024-09-12T12:23:17.067325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085808+0000","last_change":"2024-09-12T12:23:17.081531+0000","last_active":"2024-09-12T12:23:17.085808+0000","last_peered":"2024-09-12T12:23:17.085808+0000","last_clean":"2024-09-12T12:23:17.085808+0000","last_became_active":"2024-09-12T12:23:17.081094+0000","last_became_peered":"2024-09-12T12:23:17.081094+0000","last_unstale":"2024-09-12T12:23:17.085808+0000","last_undegraded":"2024-09-12T12:23:17.085808+0000","last_fullsized":"2024-09-12T12:23:17.085808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068665+0000","last_change":"2024-09-12T12:23:17.068665+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068665+0000","last_undegraded":"2024-09-12T12:23:17.068665+0000","last_fullsized":"2024-09-12T12:23:17.068665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.071029+0000","last_change":"2024-09-12T12:23:17.066884+0000","last_active":"2024-09-12T12:23:17.071029+0000","last_peered":"2024-09-12T12:23:17.071029+0000","last_clean":"2024-09-12T12:23:17.071029+0000","last_became_active":"2024-09-12T12:23:17.066181+0000","last_became_peered":"2024-09-12T12:23:17.066181+0000","last_unstale":"2024-09-12T12:23:17.071029+0000","last_undegraded":"2024-09-12T12:23:17.071029+0000","last_fullsized":"2024-09-12T12:23:17.071029+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:55.642549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074022+0000","last_change":"2024-09-12T12:23:17.074022+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074022+0000","last_undegraded":"2024-09-12T12:23:17.074022+0000","last_fullsized":"2024-09-12T12:23:17.074022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.088023+0000","last_change":"2024-09-12T12:23:17.080316+0000","last_active":"2024-09-12T12:23:17.088023+0000","last_peered":"2024-09-12T12:23:17.088023+0000","last_clean":"2024-09-12T12:23:17.088023+0000","last_became_active":"2024-09-12T12:23:17.079639+0000","last_became_peered":"2024-09-12T12:23:17.079639+0000","last_unstale":"2024-09-12T12:23:17.088023+0000","last_undegraded":"2024-09-12T12:23:17.088023+0000","last_fullsized":"2024-09-12T12:23:17.088023+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:46:49.444919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076154+0000","last_change":"2024-09-12T12:23:17.076154+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076154+0000","last_undegraded":"2024-09-12T12:23:17.076154+0000","last_fullsized":"2024-09-12T12:23:17.076154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095475+0000","last_change":"2024-09-12T12:23:17.091123+0000","last_active":"2024-09-12T12:23:17.095475+0000","last_peered":"2024-09-12T12:23:17.095475+0000","last_clean":"2024-09-12T12:23:17.095475+0000","last_became_active":"2024-09-12T12:23:17.090081+0000","last_became_peered":"2024-09-12T12:23:17.090081+0000","last_unstale":"2024-09-12T12:23:17.095475+0000","last_undegraded":"2024-09-12T12:23:17.095475+0000","last_fullsized":"2024-09-12T12:23:17.095475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:23.664767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066148+0000","last_change":"2024-09-12T12:23:17.066148+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066148+0000","last_undegraded":"2024-09-12T12:23:17.066148+0000","last_fullsized":"2024-09-12T12:23:17.066148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092348+0000","last_change":"2024-09-12T12:23:17.078894+0000","last_active":"2024-09-12T12:23:17.092348+0000","last_peered":"2024-09-12T12:23:17.092348+0000","last_clean":"2024-09-12T12:23:17.092348+0000","last_became_active":"2024-09-12T12:23:17.077021+0000","last_became_peered":"2024-09-12T12:23:17.077021+0000","last_unstale":"2024-09-12T12:23:17.092348+0000","last_undegraded":"2024-09-12T12:23:17.092348+0000","last_fullsized":"2024-09-12T12:23:17.092348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068120+0000","last_change":"2024-09-12T12:23:17.068120+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068120+0000","last_undegraded":"2024-09-12T12:23:17.068120+0000","last_fullsized":"2024-09-12T12:23:17.068120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095201+0000","last_change":"2024-09-12T12:23:17.090669+0000","last_active":"2024-09-12T12:23:17.095201+0000","last_peered":"2024-09-12T12:23:17.095201+0000","last_clean":"2024-09-12T12:23:17.095201+0000","last_became_active":"2024-09-12T12:23:17.089910+0000","last_became_peered":"2024-09-12T12:23:17.089910+0000","last_unstale":"2024-09-12T12:23:17.095201+0000","last_undegraded":"2024-09-12T12:23:17.095201+0000","last_fullsized":"2024-09-12T12:23:17.095201+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:08.414320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065359+0000","last_change":"2024-09-12T12:23:17.065359+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065359+0000","last_undegraded":"2024-09-12T12:23:17.065359+0000","last_fullsized":"2024-09-12T12:23:17.065359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096006+0000","last_change":"2024-09-12T12:23:17.093031+0000","last_active":"2024-09-12T12:23:17.096006+0000","last_peered":"2024-09-12T12:23:17.096006+0000","last_clean":"2024-09-12T12:23:17.096006+0000","last_became_active":"2024-09-12T12:23:17.092495+0000","last_became_peered":"2024-09-12T12:23:17.092495+0000","last_unstale":"2024-09-12T12:23:17.096006+0000","last_undegraded":"2024-09-12T12:23:17.096006+0000","last_fullsized":"2024-09-12T12:23:17.096006+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073936+0000","last_change":"2024-09-12T12:23:17.073936+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073936+0000","last_undegraded":"2024-09-12T12:23:17.073936+0000","last_fullsized":"2024-09-12T12:23:17.073936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.097434+0000","last_change":"2024-09-12T12:23:17.096451+0000","last_active":"2024-09-12T12:23:17.097434+0000","last_peered":"2024-09-12T12:23:17.097434+0000","last_clean":"2024-09-12T12:23:17.097434+0000","last_became_active":"2024-09-12T12:23:17.095796+0000","last_became_peered":"2024-09-12T12:23:17.095796+0000","last_unstale":"2024-09-12T12:23:17.097434+0000","last_undegraded":"2024-09-12T12:23:17.097434+0000","last_fullsized":"2024-09-12T12:23:17.097434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:17.993115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.117537+0000","last_change":"2024-09-12T12:23:18.114079+0000","last_active":"2024-09-12T12:23:18.117537+0000","last_peered":"2024-09-12T12:23:18.117537+0000","last_clean":"2024-09-12T12:23:18.117537+0000","last_became_active":"2024-09-12T12:23:18.113666+0000","last_became_peered":"2024-09-12T12:23:18.113666+0000","last_unstale":"2024-09-12T12:23:18.117537+0000","last_undegraded":"2024-09-12T12:23:18.117537+0000","last_fullsized":"2024-09-12T12:23:18.117537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:50.568191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092344+0000","last_change":"2024-09-12T12:23:17.078879+0000","last_active":"2024-09-12T12:23:17.092344+0000","last_peered":"2024-09-12T12:23:17.092344+0000","last_clean":"2024-09-12T12:23:17.092344+0000","last_became_active":"2024-09-12T12:23:17.076836+0000","last_became_peered":"2024-09-12T12:23:17.076836+0000","last_unstale":"2024-09-12T12:23:17.092344+0000","last_undegraded":"2024-09-12T12:23:17.092344+0000","last_fullsized":"2024-09-12T12:23:17.092344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.134210+0000","last_change":"2024-09-12T12:23:18.133120+0000","last_active":"2024-09-12T12:23:18.134210+0000","last_peered":"2024-09-12T12:23:18.134210+0000","last_clean":"2024-09-12T12:23:18.134210+0000","last_became_active":"2024-09-12T12:23:18.132824+0000","last_became_peered":"2024-09-12T12:23:18.132824+0000","last_unstale":"2024-09-12T12:23:18.134210+0000","last_undegraded":"2024-09-12T12:23:18.134210+0000","last_fullsized":"2024-09-12T12:23:18.134210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:09:56.962893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.067441+0000","last_active":"2024-09-12T12:23:17.072096+0000","last_peered":"2024-09-12T12:23:17.072096+0000","last_clean":"2024-09-12T12:23:17.072096+0000","last_became_active":"2024-09-12T12:23:17.066728+0000","last_became_peered":"2024-09-12T12:23:17.066728+0000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:21:59.864966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133456+0000","last_change":"2024-09-12T12:23:18.126379+0000","last_active":"2024-09-12T12:23:18.133456+0000","last_peered":"2024-09-12T12:23:18.133456+0000","last_clean":"2024-09-12T12:23:18.133456+0000","last_became_active":"2024-09-12T12:23:18.126047+0000","last_became_peered":"2024-09-12T12:23:18.126047+0000","last_unstale":"2024-09-12T12:23:18.133456+0000","last_undegraded":"2024-09-12T12:23:18.133456+0000","last_fullsized":"2024-09-12T12:23:18.133456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:46.711808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.083289+0000","last_change":"2024-09-12T12:23:17.078895+0000","last_active":"2024-09-12T12:23:17.083289+0000","last_peered":"2024-09-12T12:23:17.083289+0000","last_clean":"2024-09-12T12:23:17.083289+0000","last_became_active":"2024-09-12T12:23:17.078633+0000","last_became_peered":"2024-09-12T12:23:17.078633+0000","last_unstale":"2024-09-12T12:23:17.083289+0000","last_undegraded":"2024-09-12T12:23:17.083289+0000","last_fullsized":"2024-09-12T12:23:17.083289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:16.522351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072631+0000","last_change":"2024-09-12T12:23:17.072631+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072631+0000","last_undegraded":"2024-09-12T12:23:17.072631+0000","last_fullsized":"2024-09-12T12:23:17.072631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.086997+0000","last_change":"2024-09-12T12:23:17.077229+0000","last_active":"2024-09-12T12:23:17.086997+0000","last_peered":"2024-09-12T12:23:17.086997+0000","last_clean":"2024-09-12T12:23:17.086997+0000","last_became_active":"2024-09-12T12:23:17.076051+0000","last_became_peered":"2024-09-12T12:23:17.076051+0000","last_unstale":"2024-09-12T12:23:17.086997+0000","last_undegraded":"2024-09-12T12:23:17.086997+0000","last_fullsized":"2024-09-12T12:23:17.086997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064090+0000","last_change":"2024-09-12T12:23:17.064090+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064090+0000","last_undegraded":"2024-09-12T12:23:17.064090+0000","last_fullsized":"2024-09-12T12:23:17.064090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087914+0000","last_change":"2024-09-12T12:23:17.079856+0000","last_active":"2024-09-12T12:23:17.087914+0000","last_peered":"2024-09-12T12:23:17.087914+0000","last_clean":"2024-09-12T12:23:17.087914+0000","last_became_active":"2024-09-12T12:23:17.079508+0000","last_became_peered":"2024-09-12T12:23:17.079508+0000","last_unstale":"2024-09-12T12:23:17.087914+0000","last_undegraded":"2024-09-12T12:23:17.087914+0000","last_fullsized":"2024-09-12T12:23:17.087914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:36.840747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066075+0000","last_change":"2024-09-12T12:23:17.066075+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066075+0000","last_undegraded":"2024-09-12T12:23:17.066075+0000","last_fullsized":"2024-09-12T12:23:17.066075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085756+0000","last_change":"2024-09-12T12:23:17.076124+0000","last_active":"2024-09-12T12:23:17.085756+0000","last_peered":"2024-09-12T12:23:17.085756+0000","last_clean":"2024-09-12T12:23:17.085756+0000","last_became_active":"2024-09-12T12:23:17.072753+0000","last_became_peered":"2024-09-12T12:23:17.072753+0000","last_unstale":"2024-09-12T12:23:17.085756+0000","last_undegraded":"2024-09-12T12:23:17.085756+0000","last_fullsized":"2024-09-12T12:23:17.085756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:24.793616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064680+0000","last_change":"2024-09-12T12:23:17.064680+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064680+0000","last_undegraded":"2024-09-12T12:23:17.064680+0000","last_fullsized":"2024-09-12T12:23:17.064680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093479+0000","last_change":"2024-09-12T12:23:17.090571+0000","last_active":"2024-09-12T12:23:17.093479+0000","last_peered":"2024-09-12T12:23:17.093479+0000","last_clean":"2024-09-12T12:23:17.093479+0000","last_became_active":"2024-09-12T12:23:17.090230+0000","last_became_peered":"2024-09-12T12:23:17.090230+0000","last_unstale":"2024-09-12T12:23:17.093479+0000","last_undegraded":"2024-09-12T12:23:17.093479+0000","last_fullsized":"2024-09-12T12:23:17.093479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:25:22.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072543+0000","last_change":"2024-09-12T12:23:17.072543+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072543+0000","last_undegraded":"2024-09-12T12:23:17.072543+0000","last_fullsized":"2024-09-12T12:23:17.072543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085405+0000","last_change":"2024-09-12T12:23:17.073326+0000","last_active":"2024-09-12T12:23:17.085405+0000","last_peered":"2024-09-12T12:23:17.085405+0000","last_clean":"2024-09-12T12:23:17.085405+0000","last_became_active":"2024-09-12T12:23:17.072707+0000","last_became_peered":"2024-09-12T12:23:17.072707+0000","last_unstale":"2024-09-12T12:23:17.085405+0000","last_undegraded":"2024-09-12T12:23:17.085405+0000","last_fullsized":"2024-09-12T12:23:17.085405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:17.187446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.059949+0000","last_change":"2024-09-12T12:23:17.059949+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.059949+0000","last_undegraded":"2024-09-12T12:23:17.059949+0000","last_fullsized":"2024-09-12T12:23:17.059949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093511+0000","last_change":"2024-09-12T12:23:17.089441+0000","last_active":"2024-09-12T12:23:17.093511+0000","last_peered":"2024-09-12T12:23:17.093511+0000","last_clean":"2024-09-12T12:23:17.093511+0000","last_became_active":"2024-09-12T12:23:17.089223+0000","last_became_peered":"2024-09-12T12:23:17.089223+0000","last_unstale":"2024-09-12T12:23:17.093511+0000","last_undegraded":"2024-09-12T12:23:17.093511+0000","last_fullsized":"2024-09-12T12:23:17.093511+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066023+0000","last_change":"2024-09-12T12:23:17.066023+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066023+0000","last_undegraded":"2024-09-12T12:23:17.066023+0000","last_fullsized":"2024-09-12T12:23:17.066023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095568+0000","last_change":"2024-09-12T12:23:17.091347+0000","last_active":"2024-09-12T12:23:17.095568+0000","last_peered":"2024-09-12T12:23:17.095568+0000","last_clean":"2024-09-12T12:23:17.095568+0000","last_became_active":"2024-09-12T12:23:17.090341+0000","last_became_peered":"2024-09-12T12:23:17.090341+0000","last_unstale":"2024-09-12T12:23:17.095568+0000","last_undegraded":"2024-09-12T12:23:17.095568+0000","last_fullsized":"2024-09-12T12:23:17.095568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:38.673027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.057553+0000","last_change":"2024-09-12T12:23:17.057553+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.057553+0000","last_undegraded":"2024-09-12T12:23:17.057553+0000","last_fullsized":"2024-09-12T12:23:17.057553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087239+0000","last_change":"2024-09-12T12:23:17.076822+0000","last_active":"2024-09-12T12:23:17.087239+0000","last_peered":"2024-09-12T12:23:17.087239+0000","last_clean":"2024-09-12T12:23:17.087239+0000","last_became_active":"2024-09-12T12:23:17.075087+0000","last_became_peered":"2024-09-12T12:23:17.075087+0000","last_unstale":"2024-09-12T12:23:17.087239+0000","last_undegraded":"2024-09-12T12:23:17.087239+0000","last_fullsized":"2024-09-12T12:23:17.087239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:29.472329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133487+0000","last_change":"2024-09-12T12:23:18.126382+0000","last_active":"2024-09-12T12:23:18.133487+0000","last_peered":"2024-09-12T12:23:18.133487+0000","last_clean":"2024-09-12T12:23:18.133487+0000","last_became_active":"2024-09-12T12:23:18.126018+0000","last_became_peered":"2024-09-12T12:23:18.126018+0000","last_unstale":"2024-09-12T12:23:18.133487+0000","last_undegraded":"2024-09-12T12:23:18.133487+0000","last_fullsized":"2024-09-12T12:23:18.133487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:57.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087158+0000","last_change":"2024-09-12T12:23:17.076005+0000","last_active":"2024-09-12T12:23:17.087158+0000","last_peered":"2024-09-12T12:23:17.087158+0000","last_clean":"2024-09-12T12:23:17.087158+0000","last_became_active":"2024-09-12T12:23:17.074864+0000","last_became_peered":"2024-09-12T12:23:17.074864+0000","last_unstale":"2024-09-12T12:23:17.087158+0000","last_undegraded":"2024-09-12T12:23:17.087158+0000","last_fullsized":"2024-09-12T12:23:17.087158+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:50.444388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.074838+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:17.074838+0000","last_peered":"2024-09-12T12:23:17.074838+0000","last_clean":"2024-09-12T12:23:17.074838+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:17.074838+0000","last_undegraded":"2024-09-12T12:23:17.074838+0000","last_fullsized":"2024-09-12T12:23:17.074838+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":794624,"data_stored":327173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":904379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1368064,"data_stored":910654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:23:22.901 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3f is not active+clean 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.074113+0000', 'last_change': '2024-09-12T12:23:17.074113+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.074113+0000', 'last_undegraded': '2024-09-12T12:23:17.074113+0000', 'last_fullsized': '2024-09-12T12:23:17.074113+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3e is not active+clean 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.063091+0000', 'last_change': '2024-09-12T12:23:17.063091+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.063091+0000', 'last_undegraded': '2024-09-12T12:23:17.063091+0000', 'last_fullsized': '2024-09-12T12:23:17.063091+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3c is not active+clean 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.066675+0000', 'last_change': '2024-09-12T12:23:17.066675+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.066675+0000', 'last_undegraded': '2024-09-12T12:23:17.066675+0000', 'last_fullsized': '2024-09-12T12:23:17.066675+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3b is not active+clean 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.078213+0000', 'last_change': '2024-09-12T12:23:17.078213+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.078213+0000', 'last_undegraded': '2024-09-12T12:23:17.078213+0000', 'last_fullsized': '2024-09-12T12:23:17.078213+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3a is not active+clean 2024-09-12T12:23:22.902 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.061604+0000', 'last_change': '2024-09-12T12:23:17.061604+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.061604+0000', 'last_undegraded': '2024-09-12T12:23:17.061604+0000', 'last_fullsized': '2024-09-12T12:23:17.061604+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:PG 11.39 is not active+clean 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.073184+0000', 'last_change': '2024-09-12T12:23:17.073184+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.073184+0000', 'last_undegraded': '2024-09-12T12:23:17.073184+0000', 'last_fullsized': '2024-09-12T12:23:17.073184+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:PG 11.38 is not active+clean 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.062242+0000', 'last_change': '2024-09-12T12:23:17.062242+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.062242+0000', 'last_undegraded': '2024-09-12T12:23:17.062242+0000', 'last_fullsized': '2024-09-12T12:23:17.062242+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:PG 11.37 is not active+clean 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.079571+0000', 'last_change': '2024-09-12T12:23:17.079571+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.079571+0000', 'last_undegraded': '2024-09-12T12:23:17.079571+0000', 'last_fullsized': '2024-09-12T12:23:17.079571+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:PG 11.36 is not active+clean 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.066729+0000', 'last_change': '2024-09-12T12:23:17.066729+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.066729+0000', 'last_undegraded': '2024-09-12T12:23:17.066729+0000', 'last_fullsized': '2024-09-12T12:23:17.066729+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:PG 11.35 is not active+clean 2024-09-12T12:23:22.903 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.072015+0000', 'last_change': '2024-09-12T12:23:17.072015+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.072015+0000', 'last_undegraded': '2024-09-12T12:23:17.072015+0000', 'last_fullsized': '2024-09-12T12:23:17.072015+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:PG 11.34 is not active+clean 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.066185+0000', 'last_change': '2024-09-12T12:23:17.066185+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.066185+0000', 'last_undegraded': '2024-09-12T12:23:17.066185+0000', 'last_fullsized': '2024-09-12T12:23:17.066185+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:PG 11.33 is not active+clean 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.076783+0000', 'last_change': '2024-09-12T12:23:17.076783+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.076783+0000', 'last_undegraded': '2024-09-12T12:23:17.076783+0000', 'last_fullsized': '2024-09-12T12:23:17.076783+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:PG 11.32 is not active+clean 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.075430+0000', 'last_change': '2024-09-12T12:23:17.075430+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.075430+0000', 'last_undegraded': '2024-09-12T12:23:17.075430+0000', 'last_fullsized': '2024-09-12T12:23:17.075430+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:PG 11.31 is not active+clean 2024-09-12T12:23:22.904 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.074111+0000', 'last_change': '2024-09-12T12:23:17.074111+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.074111+0000', 'last_undegraded': '2024-09-12T12:23:17.074111+0000', 'last_fullsized': '2024-09-12T12:23:17.074111+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:PG 11.30 is not active+clean 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.072096+0000', 'last_change': '2024-09-12T12:23:17.072096+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.072096+0000', 'last_undegraded': '2024-09-12T12:23:17.072096+0000', 'last_fullsized': '2024-09-12T12:23:17.072096+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2f is not active+clean 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.077433+0000', 'last_change': '2024-09-12T12:23:17.077433+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.077433+0000', 'last_undegraded': '2024-09-12T12:23:17.077433+0000', 'last_fullsized': '2024-09-12T12:23:17.077433+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2e is not active+clean 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.063509+0000', 'last_change': '2024-09-12T12:23:17.063509+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.063509+0000', 'last_undegraded': '2024-09-12T12:23:17.063509+0000', 'last_fullsized': '2024-09-12T12:23:17.063509+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2d is not active+clean 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.061634+0000', 'last_change': '2024-09-12T12:23:17.061634+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.061634+0000', 'last_undegraded': '2024-09-12T12:23:17.061634+0000', 'last_fullsized': '2024-09-12T12:23:17.061634+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2c is not active+clean 2024-09-12T12:23:22.905 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.072954+0000', 'last_change': '2024-09-12T12:23:17.072954+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.072954+0000', 'last_undegraded': '2024-09-12T12:23:17.072954+0000', 'last_fullsized': '2024-09-12T12:23:17.072954+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2b is not active+clean 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.083851+0000', 'last_change': '2024-09-12T12:23:17.083851+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.083851+0000', 'last_undegraded': '2024-09-12T12:23:17.083851+0000', 'last_fullsized': '2024-09-12T12:23:17.083851+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2a is not active+clean 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.064962+0000', 'last_change': '2024-09-12T12:23:17.064962+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.064962+0000', 'last_undegraded': '2024-09-12T12:23:17.064962+0000', 'last_fullsized': '2024-09-12T12:23:17.064962+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:PG 11.29 is not active+clean 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.064538+0000', 'last_change': '2024-09-12T12:23:17.064538+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.064538+0000', 'last_undegraded': '2024-09-12T12:23:17.064538+0000', 'last_fullsized': '2024-09-12T12:23:17.064538+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:PG 11.28 is not active+clean 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.064093+0000', 'last_change': '2024-09-12T12:23:17.064093+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.064093+0000', 'last_undegraded': '2024-09-12T12:23:17.064093+0000', 'last_fullsized': '2024-09-12T12:23:17.064093+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:PG 11.27 is not active+clean 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.066150+0000', 'last_change': '2024-09-12T12:23:17.066150+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.066150+0000', 'last_undegraded': '2024-09-12T12:23:17.066150+0000', 'last_fullsized': '2024-09-12T12:23:17.066150+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.906 INFO:tasks.ceph.ceph_manager.ceph:PG 11.26 is not active+clean 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.073226+0000', 'last_change': '2024-09-12T12:23:17.073226+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.073226+0000', 'last_undegraded': '2024-09-12T12:23:17.073226+0000', 'last_fullsized': '2024-09-12T12:23:17.073226+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:PG 11.24 is not active+clean 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.065397+0000', 'last_change': '2024-09-12T12:23:17.065397+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.065397+0000', 'last_undegraded': '2024-09-12T12:23:17.065397+0000', 'last_fullsized': '2024-09-12T12:23:17.065397+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:PG 11.22 is not active+clean 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.070958+0000', 'last_change': '2024-09-12T12:23:17.070958+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.070958+0000', 'last_undegraded': '2024-09-12T12:23:17.070958+0000', 'last_fullsized': '2024-09-12T12:23:17.070958+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:PG 11.21 is not active+clean 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.068727+0000', 'last_change': '2024-09-12T12:23:17.068727+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.068727+0000', 'last_undegraded': '2024-09-12T12:23:17.068727+0000', 'last_fullsized': '2024-09-12T12:23:17.068727+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:PG 11.20 is not active+clean 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.061569+0000', 'last_change': '2024-09-12T12:23:17.061569+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.061569+0000', 'last_undegraded': '2024-09-12T12:23:17.061569+0000', 'last_fullsized': '2024-09-12T12:23:17.061569+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1e is not active+clean 2024-09-12T12:23:22.907 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.065393+0000', 'last_change': '2024-09-12T12:23:17.065393+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.065393+0000', 'last_undegraded': '2024-09-12T12:23:17.065393+0000', 'last_fullsized': '2024-09-12T12:23:17.065393+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1f is not active+clean 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.075945+0000', 'last_change': '2024-09-12T12:23:17.075945+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.075945+0000', 'last_undegraded': '2024-09-12T12:23:17.075945+0000', 'last_fullsized': '2024-09-12T12:23:17.075945+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1c is not active+clean 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.061131+0000', 'last_change': '2024-09-12T12:23:17.061131+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.061131+0000', 'last_undegraded': '2024-09-12T12:23:17.061131+0000', 'last_fullsized': '2024-09-12T12:23:17.061131+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1d is not active+clean 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.069525+0000', 'last_change': '2024-09-12T12:23:17.069525+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.069525+0000', 'last_undegraded': '2024-09-12T12:23:17.069525+0000', 'last_fullsized': '2024-09-12T12:23:17.069525+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1a is not active+clean 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.068997+0000', 'last_change': '2024-09-12T12:23:17.068997+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.068997+0000', 'last_undegraded': '2024-09-12T12:23:17.068997+0000', 'last_fullsized': '2024-09-12T12:23:17.068997+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1b is not active+clean 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.073809+0000', 'last_change': '2024-09-12T12:23:17.073809+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.073809+0000', 'last_undegraded': '2024-09-12T12:23:17.073809+0000', 'last_fullsized': '2024-09-12T12:23:17.073809+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.908 INFO:tasks.ceph.ceph_manager.ceph:PG 11.18 is not active+clean 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.077266+0000', 'last_change': '2024-09-12T12:23:17.077266+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.077266+0000', 'last_undegraded': '2024-09-12T12:23:17.077266+0000', 'last_fullsized': '2024-09-12T12:23:17.077266+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:PG 11.19 is not active+clean 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.073001+0000', 'last_change': '2024-09-12T12:23:17.073001+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.073001+0000', 'last_undegraded': '2024-09-12T12:23:17.073001+0000', 'last_fullsized': '2024-09-12T12:23:17.073001+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:PG 11.16 is not active+clean 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.069859+0000', 'last_change': '2024-09-12T12:23:17.069859+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.069859+0000', 'last_undegraded': '2024-09-12T12:23:17.069859+0000', 'last_fullsized': '2024-09-12T12:23:17.069859+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:PG 11.17 is not active+clean 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.073061+0000', 'last_change': '2024-09-12T12:23:17.073061+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.073061+0000', 'last_undegraded': '2024-09-12T12:23:17.073061+0000', 'last_fullsized': '2024-09-12T12:23:17.073061+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:PG 11.14 is not active+clean 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.060157+0000', 'last_change': '2024-09-12T12:23:17.060157+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.060157+0000', 'last_undegraded': '2024-09-12T12:23:17.060157+0000', 'last_fullsized': '2024-09-12T12:23:17.060157+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:PG 11.15 is not active+clean 2024-09-12T12:23:22.909 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.075555+0000', 'last_change': '2024-09-12T12:23:17.075555+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.075555+0000', 'last_undegraded': '2024-09-12T12:23:17.075555+0000', 'last_fullsized': '2024-09-12T12:23:17.075555+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:PG 11.12 is not active+clean 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.067325+0000', 'last_change': '2024-09-12T12:23:17.067325+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.067325+0000', 'last_undegraded': '2024-09-12T12:23:17.067325+0000', 'last_fullsized': '2024-09-12T12:23:17.067325+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:PG 11.13 is not active+clean 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.068665+0000', 'last_change': '2024-09-12T12:23:17.068665+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.068665+0000', 'last_undegraded': '2024-09-12T12:23:17.068665+0000', 'last_fullsized': '2024-09-12T12:23:17.068665+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:PG 11.10 is not active+clean 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.074022+0000', 'last_change': '2024-09-12T12:23:17.074022+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.074022+0000', 'last_undegraded': '2024-09-12T12:23:17.074022+0000', 'last_fullsized': '2024-09-12T12:23:17.074022+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:PG 11.11 is not active+clean 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.076154+0000', 'last_change': '2024-09-12T12:23:17.076154+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.076154+0000', 'last_undegraded': '2024-09-12T12:23:17.076154+0000', 'last_fullsized': '2024-09-12T12:23:17.076154+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:PG 11.e is not active+clean 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.066148+0000', 'last_change': '2024-09-12T12:23:17.066148+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.066148+0000', 'last_undegraded': '2024-09-12T12:23:17.066148+0000', 'last_fullsized': '2024-09-12T12:23:17.066148+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:23:22.910 INFO:tasks.ceph.ceph_manager.ceph:PG 11.f is not active+clean 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.068120+0000', 'last_change': '2024-09-12T12:23:17.068120+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.068120+0000', 'last_undegraded': '2024-09-12T12:23:17.068120+0000', 'last_fullsized': '2024-09-12T12:23:17.068120+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:PG 11.c is not active+clean 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.065359+0000', 'last_change': '2024-09-12T12:23:17.065359+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.065359+0000', 'last_undegraded': '2024-09-12T12:23:17.065359+0000', 'last_fullsized': '2024-09-12T12:23:17.065359+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:PG 11.d is not active+clean 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.073936+0000', 'last_change': '2024-09-12T12:23:17.073936+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.073936+0000', 'last_undegraded': '2024-09-12T12:23:17.073936+0000', 'last_fullsized': '2024-09-12T12:23:17.073936+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:PG 11.6 is not active+clean 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.072631+0000', 'last_change': '2024-09-12T12:23:17.072631+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.072631+0000', 'last_undegraded': '2024-09-12T12:23:17.072631+0000', 'last_fullsized': '2024-09-12T12:23:17.072631+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:PG 11.7 is not active+clean 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.064090+0000', 'last_change': '2024-09-12T12:23:17.064090+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.064090+0000', 'last_undegraded': '2024-09-12T12:23:17.064090+0000', 'last_fullsized': '2024-09-12T12:23:17.064090+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:PG 11.4 is not active+clean 2024-09-12T12:23:22.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.066075+0000', 'last_change': '2024-09-12T12:23:17.066075+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.066075+0000', 'last_undegraded': '2024-09-12T12:23:17.066075+0000', 'last_fullsized': '2024-09-12T12:23:17.066075+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:PG 11.5 is not active+clean 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.064680+0000', 'last_change': '2024-09-12T12:23:17.064680+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.064680+0000', 'last_undegraded': '2024-09-12T12:23:17.064680+0000', 'last_fullsized': '2024-09-12T12:23:17.064680+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2 is not active+clean 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.072543+0000', 'last_change': '2024-09-12T12:23:17.072543+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.072543+0000', 'last_undegraded': '2024-09-12T12:23:17.072543+0000', 'last_fullsized': '2024-09-12T12:23:17.072543+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3 is not active+clean 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.059949+0000', 'last_change': '2024-09-12T12:23:17.059949+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.059949+0000', 'last_undegraded': '2024-09-12T12:23:17.059949+0000', 'last_fullsized': '2024-09-12T12:23:17.059949+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:PG 11.0 is not active+clean 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.066023+0000', 'last_change': '2024-09-12T12:23:17.066023+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.066023+0000', 'last_undegraded': '2024-09-12T12:23:17.066023+0000', 'last_fullsized': '2024-09-12T12:23:17.066023+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1 is not active+clean 2024-09-12T12:23:22.912 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 64, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:17.057553+0000', 'last_change': '2024-09-12T12:23:17.057553+0000', 'last_active': '2024-09-12T12:23:17.044485+0000', 'last_peered': '2024-09-12T12:23:17.044485+0000', 'last_clean': '2024-09-12T12:23:17.044485+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:17.057553+0000', 'last_undegraded': '2024-09-12T12:23:17.057553+0000', 'last_fullsized': '2024-09-12T12:23:17.057553+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:17.044485+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:23:22.913 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T12:23:23.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.048+0000 7fe9e0f63640 1 Processor -- start 2024-09-12T12:23:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.048+0000 7fe9e0f63640 1 -- start start 2024-09-12T12:23:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9dc163c90 0x7fe9dc166080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9dc15c630 con 0x7fe9dc0ae250 2024-09-12T12:23:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9dc0ad0c0 con 0x7fe9dc0d05b0 2024-09-12T12:23:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe9dc0ad3a0 con 0x7fe9dc163c90 2024-09-12T12:23:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9dad9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe9dc163c90 0x7fe9dc166080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9d9d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe9dc0ae250 0x7fe9dc0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47138/0 (socket says 172.21.15.3:47138) 2024-09-12T12:23:23.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9d9d9b640 1 -- 172.21.15.3:0/3944793669 learned_addr learned my addr 172.21.15.3:0/3944793669 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9dad9d640 1 -- 172.21.15.3:0/3944793669 >> v1:172.21.15.142:6789/0 conn(0x7fe9dc0d05b0 legacy=0x7fe9dc0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9dad9d640 1 -- 172.21.15.3:0/3944793669 >> v1:172.21.15.3:6789/0 conn(0x7fe9dc0ae250 legacy=0x7fe9dc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9d959a640 1 -- 172.21.15.3:0/3944793669 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645911964 0 0) 0x7fe9d0001120 con 0x7fe9dc0ae250 2024-09-12T12:23:23.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9dad9d640 1 -- 172.21.15.3:0/3944793669 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc0ad680 con 0x7fe9dc163c90 2024-09-12T12:23:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9dad9d640 1 --2- 172.21.15.3:0/3944793669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc166080 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009dc0 tx=0x7fe9d401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0f4a9ea39c6674 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9d959a640 1 -- 172.21.15.3:0/3944793669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40b2070 con 0x7fe9dc163c90 2024-09-12T12:23:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9d959a640 1 -- 172.21.15.3:0/3944793669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d40a58c0 con 0x7fe9dc163c90 2024-09-12T12:23:23.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9d959a640 1 -- 172.21.15.3:0/3944793669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40a5c90 con 0x7fe9dc163c90 2024-09-12T12:23:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- 172.21.15.3:0/3944793669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 msgr2=0x7fe9dc166080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/3944793669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc166080 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4009dc0 tx=0x7fe9d401c910 comp rx=0 tx=0).stop 2024-09-12T12:23:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- 172.21.15.3:0/3944793669 shutdown_connections 2024-09-12T12:23:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/3944793669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc166080 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- 172.21.15.3:0/3944793669 >> 172.21.15.3:0/3944793669 conn(0x7fe9dc067e00 msgr2=0x7fe9dc068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- 172.21.15.3:0/3944793669 shutdown_connections 2024-09-12T12:23:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.052+0000 7fe9e0f63640 1 -- 172.21.15.3:0/3944793669 wait complete. 2024-09-12T12:23:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 Processor -- start 2024-09-12T12:23:23.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 -- start start 2024-09-12T12:23:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9dc0ae250 0x7fe9dc099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9dc0d05b0 0x7fe9dc0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc15c9b0 con 0x7fe9dc163c90 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc15c830 con 0x7fe9dc0ae250 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe9dc0b31e0 con 0x7fe9dc0d05b0 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38496/0 (socket says 172.21.15.3:38496) 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 -- 172.21.15.3:0/2079698341 learned_addr learned my addr 172.21.15.3:0/2079698341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9da59c640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9dc0ae250 0x7fe9dc099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9d9d9b640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9dc0d05b0 0x7fe9dc0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 -- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9dc0d05b0 msgr2=0x7fe9dc0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9dc0d05b0 0x7fe9dc0b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 -- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9dc0ae250 msgr2=0x7fe9dc099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9dc0ae250 0x7fe9dc099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 -- 172.21.15.3:0/2079698341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc0987c0 con 0x7fe9dc163c90 2024-09-12T12:23:23.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9dad9d640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc149470 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40b0040 tx=0x7fe9d40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9caffd640 1 -- 172.21.15.3:0/2079698341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40b2040 con 0x7fe9dc163c90 2024-09-12T12:23:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9caffd640 1 -- 172.21.15.3:0/2079698341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d40bf070 con 0x7fe9dc163c90 2024-09-12T12:23:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc098aa0 con 0x7fe9dc163c90 2024-09-12T12:23:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9caffd640 1 -- 172.21.15.3:0/2079698341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d40b88d0 con 0x7fe9dc163c90 2024-09-12T12:23:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc099060 con 0x7fe9dc163c90 2024-09-12T12:23:23.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.056+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc0ae6e0 con 0x7fe9dc163c90 2024-09-12T12:23:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.060+0000 7fe9caffd640 1 -- 172.21.15.3:0/2079698341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe9d40b8a70 con 0x7fe9dc163c90 2024-09-12T12:23:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.060+0000 7fe9caffd640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9bc0678c0 0x7fe9bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.060+0000 7fe9caffd640 1 -- 172.21.15.3:0/2079698341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fe9d41218b0 con 0x7fe9dc163c90 2024-09-12T12:23:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.060+0000 7fe9da59c640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9bc0678c0 0x7fe9bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.060+0000 7fe9da59c640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9bc0678c0 0x7fe9bc069d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc004e30 tx=0x7fe9cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:23.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.064+0000 7fe9caffd640 1 -- 172.21.15.3:0/2079698341 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe9d4016500 con 0x7fe9dc163c90 2024-09-12T12:23:23.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe9dc0d0b30 con 0x7fe9dc163c90 2024-09-12T12:23:23.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9caffd640 1 -- 172.21.15.3:0/2079698341 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1558 (secure 0 0 0) 0x7fe9d40f24d0 con 0x7fe9dc163c90 2024-09-12T12:23:23.287 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:23.287 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":238,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1726143578,"num_in_osds":8,"osd_in_since":1726143567,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":57},{"state_name":"active+clean","count":40}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590458,"bytes_used":241627136,"bytes_avail":772852486144,"bytes_total":773094113280,"inactive_pgs_ratio":0.58762884140014648,"write_bytes_sec":180,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6012}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-12T12:22:27.710844+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9bc0678c0 msgr2=0x7fe9bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9bc0678c0 0x7fe9bc069d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc004e30 tx=0x7fe9cc009040 comp rx=0 tx=0).stop 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 msgr2=0x7fe9dc149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc149470 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40b0040 tx=0x7fe9d40196b0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 shutdown_connections 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe9bc0678c0 0x7fe9bc069d80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9dc163c90 0x7fe9dc149470 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe9dc0d05b0 0x7fe9dc0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 --2- 172.21.15.3:0/2079698341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe9dc0ae250 0x7fe9dc099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 >> 172.21.15.3:0/2079698341 conn(0x7fe9dc067e00 msgr2=0x7fe9dc0681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:23.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 shutdown_connections 2024-09-12T12:23:23.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.284+0000 7fe9e0f63640 1 -- 172.21.15.3:0/2079698341 wait complete. 2024-09-12T12:23:23.304 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T12:23:23.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:23:23.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.420+0000 7efd5fdc9640 1 Processor -- start 2024-09-12T12:23:23.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.420+0000 7efd5fdc9640 1 -- start start 2024-09-12T12:23:23.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5fdc9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efd580cedf0 0x7efd580d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5fdc9640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd580d2a60 con 0x7efd580b79a0 2024-09-12T12:23:23.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5fdc9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd580d2d40 con 0x7efd580cf580 2024-09-12T12:23:23.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5fdc9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efd580d3670 con 0x7efd580cedf0 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5e366640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efd580b79a0 0x7efd580b7df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47142/0 (socket says 172.21.15.3:47142) 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5e366640 1 -- 172.21.15.3:0/3507491347 learned_addr learned my addr 172.21.15.3:0/3507491347 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5d364640 1 --2- 172.21.15.3:0/3507491347 >> v2:172.21.15.142:3300/0 conn(0x7efd580cedf0 0x7efd580d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509020110 0 0) 0x7efd580d2a60 con 0x7efd580b79a0 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd44003530 con 0x7efd580b79a0 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3212577126 0 0) 0x7efd44003530 con 0x7efd580b79a0 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 >> v2:172.21.15.142:3300/0 conn(0x7efd580cedf0 msgr2=0x7efd580d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 --2- 172.21.15.3:0/3507491347 >> v2:172.21.15.142:3300/0 conn(0x7efd580cedf0 0x7efd580d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 >> v1:172.21.15.142:6789/0 conn(0x7efd580cf580 legacy=0x7efd580cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd580d3020 con 0x7efd580b79a0 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7efd50001a00 con 0x7efd580b79a0 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 >> v1:172.21.15.3:6789/0 conn(0x7efd580b79a0 legacy=0x7efd580b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd44003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd44004810 0x7efd44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd440051d0 0x7efd4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efd580d3670 con 0x7efd44003a10 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd4400c100 con 0x7efd440051d0 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efd4400c300 con 0x7efd44004810 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5db65640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd44003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5e366640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd440051d0 0x7efd4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5d364640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd44004810 0x7efd44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5db65640 1 -- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd44004810 msgr2=0x7efd44004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5db65640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd44004810 0x7efd44004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5db65640 1 -- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd440051d0 msgr2=0x7efd4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5db65640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd440051d0 0x7efd4400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5e366640 1 -- 172.21.15.3:0/3507491347 reap_dead start 2024-09-12T12:23:23.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.424+0000 7efd5db65640 1 -- 172.21.15.3:0/3507491347 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd44003530 con 0x7efd44003a10 2024-09-12T12:23:23.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd44003e50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efd54001780 tx=0x7efd5401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f891538237d225e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:23.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540013a0 con 0x7efd44003a10 2024-09-12T12:23:23.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd540052f0 con 0x7efd44003a10 2024-09-12T12:23:23.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5cb63640 1 -- 172.21.15.3:0/3507491347 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54003840 con 0x7efd44003a10 2024-09-12T12:23:23.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- 172.21.15.3:0/3507491347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 msgr2=0x7efd44003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd44003e50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efd54001780 tx=0x7efd5401c3d0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- 172.21.15.3:0/3507491347 shutdown_connections 2024-09-12T12:23:23.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/3507491347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd44003e50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- 172.21.15.3:0/3507491347 >> 172.21.15.3:0/3507491347 conn(0x7efd58072e20 msgr2=0x7efd58073200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:23.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- 172.21.15.3:0/3507491347 shutdown_connections 2024-09-12T12:23:23.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- 172.21.15.3:0/3507491347 wait complete. 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 Processor -- start 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- start start 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd580b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd580ca220 0x7efd580b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5815f030 0x7efd5815f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efd5815ebd0 con 0x7efd44003a10 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efd5815ea50 con 0x7efd5815f030 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5fdc9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efd580b67b0 con 0x7efd580ca220 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd580b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd580b7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38524/0 (socket says 172.21.15.3:38524) 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 -- 172.21.15.3:0/26083903 learned_addr learned my addr 172.21.15.3:0/26083903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5d364640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd580ca220 0x7efd580b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5e366640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5815f030 0x7efd5815f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 -- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd580ca220 msgr2=0x7efd580b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd580ca220 0x7efd580b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 -- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5815f030 msgr2=0x7efd5815f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5815f030 0x7efd5815f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 -- 172.21.15.3:0/26083903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd581fb740 con 0x7efd44003a10 2024-09-12T12:23:23.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5d364640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd580ca220 0x7efd580b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd5db65640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd580b7d80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efd54005040 tx=0x7efd44005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.432+0000 7efd4e7fc640 1 -- 172.21.15.3:0/26083903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54007a40 con 0x7efd44003a10 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd4e7fc640 1 -- 172.21.15.3:0/26083903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd540bc070 con 0x7efd44003a10 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd581fba20 con 0x7efd44003a10 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd4e7fc640 1 -- 172.21.15.3:0/26083903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540013a0 con 0x7efd44003a10 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd581fbfe0 con 0x7efd44003a10 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd580b6b40 con 0x7efd44003a10 2024-09-12T12:23:23.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd4e7fc640 1 -- 172.21.15.3:0/26083903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efd540b4040 con 0x7efd44003a10 2024-09-12T12:23:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd4e7fc640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd3c0678f0 0x7efd3c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd4e7fc640 1 -- 172.21.15.3:0/26083903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7efd5411fc60 con 0x7efd44003a10 2024-09-12T12:23:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd5d364640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd3c0678f0 0x7efd3c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:23.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.436+0000 7efd5d364640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd3c0678f0 0x7efd3c069db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd580d46d0 tx=0x7efd48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:23.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.440+0000 7efd4e7fc640 1 -- 172.21.15.3:0/26083903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efd54016110 con 0x7efd44003a10 2024-09-12T12:23:23.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.596+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd580cf360 con 0x7efd3c0678f0 2024-09-12T12:23:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.612+0000 7efd4e7fc640 1 -- 172.21.15.3:0/26083903 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247331 (secure 0 0 0) 0x7efd580cf360 con 0x7efd3c0678f0 2024-09-12T12:23:23.618 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:23.619 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:23:23.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd3c0678f0 msgr2=0x7efd3c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd3c0678f0 0x7efd3c069db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd580d46d0 tx=0x7efd48008040 comp rx=0 tx=0).stop 2024-09-12T12:23:23.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 msgr2=0x7efd580b7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:23.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd580b7d80 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efd54005040 tx=0x7efd44005840 comp rx=0 tx=0).stop 2024-09-12T12:23:23.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 shutdown_connections 2024-09-12T12:23:23.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efd3c0678f0 0x7efd3c069db0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd5815f030 0x7efd5815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efd580ca220 0x7efd580b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 --2- 172.21.15.3:0/26083903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd44003a10 0x7efd580b7d80 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:23.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 >> 172.21.15.3:0/26083903 conn(0x7efd58072e20 msgr2=0x7efd580b8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:23.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 shutdown_connections 2024-09-12T12:23:23.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:23.616+0000 7efd5fdc9640 1 -- 172.21.15.3:0/26083903 wait complete. 2024-09-12T12:23:23.635 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-12T12:23:21.729012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235964,"kb_used_data":7216,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738756,"statfs":{"total":773094113280,"available":772852486144,"internally_reserved":0,"allocated":7389184,"data_stored":3743636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.681431"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074113+0000","last_change":"2024-09-12T12:23:17.074113+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074113+0000","last_undegraded":"2024-09-12T12:23:17.074113+0000","last_fullsized":"2024-09-12T12:23:17.074113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063091+0000","last_change":"2024-09-12T12:23:17.063091+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063091+0000","last_undegraded":"2024-09-12T12:23:17.063091+0000","last_fullsized":"2024-09-12T12:23:17.063091+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112255+0000","last_change":"2024-09-12T12:23:18.107885+0000","last_active":"2024-09-12T12:23:18.112255+0000","last_peered":"2024-09-12T12:23:18.112255+0000","last_clean":"2024-09-12T12:23:18.112255+0000","last_became_active":"2024-09-12T12:23:18.106854+0000","last_became_peered":"2024-09-12T12:23:18.106854+0000","last_unstale":"2024-09-12T12:23:18.112255+0000","last_undegraded":"2024-09-12T12:23:18.112255+0000","last_fullsized":"2024-09-12T12:23:18.112255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:08:46.225249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066675+0000","last_change":"2024-09-12T12:23:17.066675+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066675+0000","last_undegraded":"2024-09-12T12:23:17.066675+0000","last_fullsized":"2024-09-12T12:23:17.066675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.078213+0000","last_change":"2024-09-12T12:23:17.078213+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.078213+0000","last_undegraded":"2024-09-12T12:23:17.078213+0000","last_fullsized":"2024-09-12T12:23:17.078213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061604+0000","last_change":"2024-09-12T12:23:17.061604+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061604+0000","last_undegraded":"2024-09-12T12:23:17.061604+0000","last_fullsized":"2024-09-12T12:23:17.061604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073184+0000","last_change":"2024-09-12T12:23:17.073184+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073184+0000","last_undegraded":"2024-09-12T12:23:17.073184+0000","last_fullsized":"2024-09-12T12:23:17.073184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.062242+0000","last_change":"2024-09-12T12:23:17.062242+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.062242+0000","last_undegraded":"2024-09-12T12:23:17.062242+0000","last_fullsized":"2024-09-12T12:23:17.062242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.079571+0000","last_change":"2024-09-12T12:23:17.079571+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.079571+0000","last_undegraded":"2024-09-12T12:23:17.079571+0000","last_fullsized":"2024-09-12T12:23:17.079571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066729+0000","last_change":"2024-09-12T12:23:17.066729+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066729+0000","last_undegraded":"2024-09-12T12:23:17.066729+0000","last_fullsized":"2024-09-12T12:23:17.066729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072015+0000","last_change":"2024-09-12T12:23:17.072015+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072015+0000","last_undegraded":"2024-09-12T12:23:17.072015+0000","last_fullsized":"2024-09-12T12:23:17.072015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066185+0000","last_change":"2024-09-12T12:23:17.066185+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066185+0000","last_undegraded":"2024-09-12T12:23:17.066185+0000","last_fullsized":"2024-09-12T12:23:17.066185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076783+0000","last_change":"2024-09-12T12:23:17.076783+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076783+0000","last_undegraded":"2024-09-12T12:23:17.076783+0000","last_fullsized":"2024-09-12T12:23:17.076783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075430+0000","last_change":"2024-09-12T12:23:17.075430+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075430+0000","last_undegraded":"2024-09-12T12:23:17.075430+0000","last_fullsized":"2024-09-12T12:23:17.075430+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074111+0000","last_change":"2024-09-12T12:23:17.074111+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074111+0000","last_undegraded":"2024-09-12T12:23:17.074111+0000","last_fullsized":"2024-09-12T12:23:17.074111+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.072096+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077433+0000","last_change":"2024-09-12T12:23:17.077433+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077433+0000","last_undegraded":"2024-09-12T12:23:17.077433+0000","last_fullsized":"2024-09-12T12:23:17.077433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.063509+0000","last_change":"2024-09-12T12:23:17.063509+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.063509+0000","last_undegraded":"2024-09-12T12:23:17.063509+0000","last_fullsized":"2024-09-12T12:23:17.063509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061634+0000","last_change":"2024-09-12T12:23:17.061634+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061634+0000","last_undegraded":"2024-09-12T12:23:17.061634+0000","last_fullsized":"2024-09-12T12:23:17.061634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072954+0000","last_change":"2024-09-12T12:23:17.072954+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072954+0000","last_undegraded":"2024-09-12T12:23:17.072954+0000","last_fullsized":"2024-09-12T12:23:17.072954+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.083851+0000","last_change":"2024-09-12T12:23:17.083851+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.083851+0000","last_undegraded":"2024-09-12T12:23:17.083851+0000","last_fullsized":"2024-09-12T12:23:17.083851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064962+0000","last_change":"2024-09-12T12:23:17.064962+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064962+0000","last_undegraded":"2024-09-12T12:23:17.064962+0000","last_fullsized":"2024-09-12T12:23:17.064962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064538+0000","last_change":"2024-09-12T12:23:17.064538+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064538+0000","last_undegraded":"2024-09-12T12:23:17.064538+0000","last_fullsized":"2024-09-12T12:23:17.064538+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064093+0000","last_change":"2024-09-12T12:23:17.064093+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064093+0000","last_undegraded":"2024-09-12T12:23:17.064093+0000","last_fullsized":"2024-09-12T12:23:17.064093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066150+0000","last_change":"2024-09-12T12:23:17.066150+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066150+0000","last_undegraded":"2024-09-12T12:23:17.066150+0000","last_fullsized":"2024-09-12T12:23:17.066150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073226+0000","last_change":"2024-09-12T12:23:17.073226+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073226+0000","last_undegraded":"2024-09-12T12:23:17.073226+0000","last_fullsized":"2024-09-12T12:23:17.073226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112796+0000","last_change":"2024-09-12T12:23:18.108157+0000","last_active":"2024-09-12T12:23:18.112796+0000","last_peered":"2024-09-12T12:23:18.112796+0000","last_clean":"2024-09-12T12:23:18.112796+0000","last_became_active":"2024-09-12T12:23:18.107527+0000","last_became_peered":"2024-09-12T12:23:18.107527+0000","last_unstale":"2024-09-12T12:23:18.112796+0000","last_undegraded":"2024-09-12T12:23:18.112796+0000","last_fullsized":"2024-09-12T12:23:18.112796+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:50.600298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065397+0000","last_change":"2024-09-12T12:23:17.065397+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065397+0000","last_undegraded":"2024-09-12T12:23:17.065397+0000","last_fullsized":"2024-09-12T12:23:17.065397+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.112045+0000","last_change":"2024-09-12T12:23:18.107716+0000","last_active":"2024-09-12T12:23:18.112045+0000","last_peered":"2024-09-12T12:23:18.112045+0000","last_clean":"2024-09-12T12:23:18.112045+0000","last_became_active":"2024-09-12T12:23:18.107421+0000","last_became_peered":"2024-09-12T12:23:18.107421+0000","last_unstale":"2024-09-12T12:23:18.112045+0000","last_undegraded":"2024-09-12T12:23:18.112045+0000","last_fullsized":"2024-09-12T12:23:18.112045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:17:59.491002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.070958+0000","last_change":"2024-09-12T12:23:17.070958+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.070958+0000","last_undegraded":"2024-09-12T12:23:17.070958+0000","last_fullsized":"2024-09-12T12:23:17.070958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068727+0000","last_change":"2024-09-12T12:23:17.068727+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068727+0000","last_undegraded":"2024-09-12T12:23:17.068727+0000","last_fullsized":"2024-09-12T12:23:17.068727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061569+0000","last_change":"2024-09-12T12:23:17.061569+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061569+0000","last_undegraded":"2024-09-12T12:23:17.061569+0000","last_fullsized":"2024-09-12T12:23:17.061569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065393+0000","last_change":"2024-09-12T12:23:17.065393+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065393+0000","last_undegraded":"2024-09-12T12:23:17.065393+0000","last_fullsized":"2024-09-12T12:23:17.065393+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095301+0000","last_change":"2024-09-12T12:23:17.089339+0000","last_active":"2024-09-12T12:23:17.095301+0000","last_peered":"2024-09-12T12:23:17.095301+0000","last_clean":"2024-09-12T12:23:17.095301+0000","last_became_active":"2024-09-12T12:23:17.088424+0000","last_became_peered":"2024-09-12T12:23:17.088424+0000","last_unstale":"2024-09-12T12:23:17.095301+0000","last_undegraded":"2024-09-12T12:23:17.095301+0000","last_fullsized":"2024-09-12T12:23:17.095301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:03:50.538591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075945+0000","last_change":"2024-09-12T12:23:17.075945+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075945+0000","last_undegraded":"2024-09-12T12:23:17.075945+0000","last_fullsized":"2024-09-12T12:23:17.075945+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085927+0000","last_change":"2024-09-12T12:23:17.082761+0000","last_active":"2024-09-12T12:23:17.085927+0000","last_peered":"2024-09-12T12:23:17.085927+0000","last_clean":"2024-09-12T12:23:17.085927+0000","last_became_active":"2024-09-12T12:23:17.082201+0000","last_became_peered":"2024-09-12T12:23:17.082201+0000","last_unstale":"2024-09-12T12:23:17.085927+0000","last_undegraded":"2024-09-12T12:23:17.085927+0000","last_fullsized":"2024-09-12T12:23:17.085927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:36:22.904667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.061131+0000","last_change":"2024-09-12T12:23:17.061131+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.061131+0000","last_undegraded":"2024-09-12T12:23:17.061131+0000","last_fullsized":"2024-09-12T12:23:17.061131+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096386+0000","last_change":"2024-09-12T12:23:17.093647+0000","last_active":"2024-09-12T12:23:17.096386+0000","last_peered":"2024-09-12T12:23:17.096386+0000","last_clean":"2024-09-12T12:23:17.096386+0000","last_became_active":"2024-09-12T12:23:17.093229+0000","last_became_peered":"2024-09-12T12:23:17.093229+0000","last_unstale":"2024-09-12T12:23:17.096386+0000","last_undegraded":"2024-09-12T12:23:17.096386+0000","last_fullsized":"2024-09-12T12:23:17.096386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:37.847639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069525+0000","last_change":"2024-09-12T12:23:17.069525+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069525+0000","last_undegraded":"2024-09-12T12:23:17.069525+0000","last_fullsized":"2024-09-12T12:23:17.069525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092413+0000","last_change":"2024-09-12T12:23:17.076844+0000","last_active":"2024-09-12T12:23:17.092413+0000","last_peered":"2024-09-12T12:23:17.092413+0000","last_clean":"2024-09-12T12:23:17.092413+0000","last_became_active":"2024-09-12T12:23:17.076096+0000","last_became_peered":"2024-09-12T12:23:17.076096+0000","last_unstale":"2024-09-12T12:23:17.092413+0000","last_undegraded":"2024-09-12T12:23:17.092413+0000","last_fullsized":"2024-09-12T12:23:17.092413+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:43.866517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068997+0000","last_change":"2024-09-12T12:23:17.068997+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068997+0000","last_undegraded":"2024-09-12T12:23:17.068997+0000","last_fullsized":"2024-09-12T12:23:17.068997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.127064+0000","last_change":"2024-09-12T12:23:17.088893+0000","last_active":"2024-09-12T12:23:18.127064+0000","last_peered":"2024-09-12T12:23:18.127064+0000","last_clean":"2024-09-12T12:23:18.127064+0000","last_became_active":"2024-09-12T12:23:17.088633+0000","last_became_peered":"2024-09-12T12:23:17.088633+0000","last_unstale":"2024-09-12T12:23:18.127064+0000","last_undegraded":"2024-09-12T12:23:18.127064+0000","last_fullsized":"2024-09-12T12:23:18.127064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:32.158937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073809+0000","last_change":"2024-09-12T12:23:17.073809+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073809+0000","last_undegraded":"2024-09-12T12:23:17.073809+0000","last_fullsized":"2024-09-12T12:23:17.073809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087116+0000","last_change":"2024-09-12T12:23:17.076621+0000","last_active":"2024-09-12T12:23:17.087116+0000","last_peered":"2024-09-12T12:23:17.087116+0000","last_clean":"2024-09-12T12:23:17.087116+0000","last_became_active":"2024-09-12T12:23:17.076227+0000","last_became_peered":"2024-09-12T12:23:17.076227+0000","last_unstale":"2024-09-12T12:23:17.087116+0000","last_undegraded":"2024-09-12T12:23:17.087116+0000","last_fullsized":"2024-09-12T12:23:17.087116+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:54.855620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.077266+0000","last_change":"2024-09-12T12:23:17.077266+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.077266+0000","last_undegraded":"2024-09-12T12:23:17.077266+0000","last_fullsized":"2024-09-12T12:23:17.077266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096205+0000","last_change":"2024-09-12T12:23:17.091844+0000","last_active":"2024-09-12T12:23:17.096205+0000","last_peered":"2024-09-12T12:23:17.096205+0000","last_clean":"2024-09-12T12:23:17.096205+0000","last_became_active":"2024-09-12T12:23:17.091637+0000","last_became_peered":"2024-09-12T12:23:17.091637+0000","last_unstale":"2024-09-12T12:23:17.096205+0000","last_undegraded":"2024-09-12T12:23:17.096205+0000","last_fullsized":"2024-09-12T12:23:17.096205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073001+0000","last_change":"2024-09-12T12:23:17.073001+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073001+0000","last_undegraded":"2024-09-12T12:23:17.073001+0000","last_fullsized":"2024-09-12T12:23:17.073001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085978+0000","last_change":"2024-09-12T12:23:17.077058+0000","last_active":"2024-09-12T12:23:17.085978+0000","last_peered":"2024-09-12T12:23:17.085978+0000","last_clean":"2024-09-12T12:23:17.085978+0000","last_became_active":"2024-09-12T12:23:17.073114+0000","last_became_peered":"2024-09-12T12:23:17.073114+0000","last_unstale":"2024-09-12T12:23:17.085978+0000","last_undegraded":"2024-09-12T12:23:17.085978+0000","last_fullsized":"2024-09-12T12:23:17.085978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:11:14.125443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.069859+0000","last_change":"2024-09-12T12:23:17.069859+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.069859+0000","last_undegraded":"2024-09-12T12:23:17.069859+0000","last_fullsized":"2024-09-12T12:23:17.069859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095516+0000","last_change":"2024-09-12T12:23:17.091345+0000","last_active":"2024-09-12T12:23:17.095516+0000","last_peered":"2024-09-12T12:23:17.095516+0000","last_clean":"2024-09-12T12:23:17.095516+0000","last_became_active":"2024-09-12T12:23:17.090644+0000","last_became_peered":"2024-09-12T12:23:17.090644+0000","last_unstale":"2024-09-12T12:23:17.095516+0000","last_undegraded":"2024-09-12T12:23:17.095516+0000","last_fullsized":"2024-09-12T12:23:17.095516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:23.229208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073061+0000","last_change":"2024-09-12T12:23:17.073061+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073061+0000","last_undegraded":"2024-09-12T12:23:17.073061+0000","last_fullsized":"2024-09-12T12:23:17.073061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.068692+0000","last_change":"2024-09-12T12:23:17.064861+0000","last_active":"2024-09-12T12:23:17.068692+0000","last_peered":"2024-09-12T12:23:17.068692+0000","last_clean":"2024-09-12T12:23:17.068692+0000","last_became_active":"2024-09-12T12:23:17.064157+0000","last_became_peered":"2024-09-12T12:23:17.064157+0000","last_unstale":"2024-09-12T12:23:17.068692+0000","last_undegraded":"2024-09-12T12:23:17.068692+0000","last_fullsized":"2024-09-12T12:23:17.068692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:10.504610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.060157+0000","last_change":"2024-09-12T12:23:17.060157+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.060157+0000","last_undegraded":"2024-09-12T12:23:17.060157+0000","last_fullsized":"2024-09-12T12:23:17.060157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.124324+0000","last_change":"2024-09-12T12:23:17.087099+0000","last_active":"2024-09-12T12:23:18.124324+0000","last_peered":"2024-09-12T12:23:18.124324+0000","last_clean":"2024-09-12T12:23:18.124324+0000","last_became_active":"2024-09-12T12:23:17.086732+0000","last_became_peered":"2024-09-12T12:23:17.086732+0000","last_unstale":"2024-09-12T12:23:18.124324+0000","last_undegraded":"2024-09-12T12:23:18.124324+0000","last_fullsized":"2024-09-12T12:23:18.124324+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:57:30.109152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.075555+0000","last_change":"2024-09-12T12:23:17.075555+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.075555+0000","last_undegraded":"2024-09-12T12:23:17.075555+0000","last_fullsized":"2024-09-12T12:23:17.075555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.082993+0000","last_change":"2024-09-12T12:23:17.075577+0000","last_active":"2024-09-12T12:23:17.082993+0000","last_peered":"2024-09-12T12:23:17.082993+0000","last_clean":"2024-09-12T12:23:17.082993+0000","last_became_active":"2024-09-12T12:23:17.074914+0000","last_became_peered":"2024-09-12T12:23:17.074914+0000","last_unstale":"2024-09-12T12:23:17.082993+0000","last_undegraded":"2024-09-12T12:23:17.082993+0000","last_fullsized":"2024-09-12T12:23:17.082993+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.067325+0000","last_change":"2024-09-12T12:23:17.067325+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.067325+0000","last_undegraded":"2024-09-12T12:23:17.067325+0000","last_fullsized":"2024-09-12T12:23:17.067325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085808+0000","last_change":"2024-09-12T12:23:17.081531+0000","last_active":"2024-09-12T12:23:17.085808+0000","last_peered":"2024-09-12T12:23:17.085808+0000","last_clean":"2024-09-12T12:23:17.085808+0000","last_became_active":"2024-09-12T12:23:17.081094+0000","last_became_peered":"2024-09-12T12:23:17.081094+0000","last_unstale":"2024-09-12T12:23:17.085808+0000","last_undegraded":"2024-09-12T12:23:17.085808+0000","last_fullsized":"2024-09-12T12:23:17.085808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068665+0000","last_change":"2024-09-12T12:23:17.068665+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068665+0000","last_undegraded":"2024-09-12T12:23:17.068665+0000","last_fullsized":"2024-09-12T12:23:17.068665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.071029+0000","last_change":"2024-09-12T12:23:17.066884+0000","last_active":"2024-09-12T12:23:17.071029+0000","last_peered":"2024-09-12T12:23:17.071029+0000","last_clean":"2024-09-12T12:23:17.071029+0000","last_became_active":"2024-09-12T12:23:17.066181+0000","last_became_peered":"2024-09-12T12:23:17.066181+0000","last_unstale":"2024-09-12T12:23:17.071029+0000","last_undegraded":"2024-09-12T12:23:17.071029+0000","last_fullsized":"2024-09-12T12:23:17.071029+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:55.642549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.074022+0000","last_change":"2024-09-12T12:23:17.074022+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.074022+0000","last_undegraded":"2024-09-12T12:23:17.074022+0000","last_fullsized":"2024-09-12T12:23:17.074022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.088023+0000","last_change":"2024-09-12T12:23:17.080316+0000","last_active":"2024-09-12T12:23:17.088023+0000","last_peered":"2024-09-12T12:23:17.088023+0000","last_clean":"2024-09-12T12:23:17.088023+0000","last_became_active":"2024-09-12T12:23:17.079639+0000","last_became_peered":"2024-09-12T12:23:17.079639+0000","last_unstale":"2024-09-12T12:23:17.088023+0000","last_undegraded":"2024-09-12T12:23:17.088023+0000","last_fullsized":"2024-09-12T12:23:17.088023+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:46:49.444919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.076154+0000","last_change":"2024-09-12T12:23:17.076154+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.076154+0000","last_undegraded":"2024-09-12T12:23:17.076154+0000","last_fullsized":"2024-09-12T12:23:17.076154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095475+0000","last_change":"2024-09-12T12:23:17.091123+0000","last_active":"2024-09-12T12:23:17.095475+0000","last_peered":"2024-09-12T12:23:17.095475+0000","last_clean":"2024-09-12T12:23:17.095475+0000","last_became_active":"2024-09-12T12:23:17.090081+0000","last_became_peered":"2024-09-12T12:23:17.090081+0000","last_unstale":"2024-09-12T12:23:17.095475+0000","last_undegraded":"2024-09-12T12:23:17.095475+0000","last_fullsized":"2024-09-12T12:23:17.095475+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:23.664767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066148+0000","last_change":"2024-09-12T12:23:17.066148+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066148+0000","last_undegraded":"2024-09-12T12:23:17.066148+0000","last_fullsized":"2024-09-12T12:23:17.066148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092348+0000","last_change":"2024-09-12T12:23:17.078894+0000","last_active":"2024-09-12T12:23:17.092348+0000","last_peered":"2024-09-12T12:23:17.092348+0000","last_clean":"2024-09-12T12:23:17.092348+0000","last_became_active":"2024-09-12T12:23:17.077021+0000","last_became_peered":"2024-09-12T12:23:17.077021+0000","last_unstale":"2024-09-12T12:23:17.092348+0000","last_undegraded":"2024-09-12T12:23:17.092348+0000","last_fullsized":"2024-09-12T12:23:17.092348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.068120+0000","last_change":"2024-09-12T12:23:17.068120+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.068120+0000","last_undegraded":"2024-09-12T12:23:17.068120+0000","last_fullsized":"2024-09-12T12:23:17.068120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095201+0000","last_change":"2024-09-12T12:23:17.090669+0000","last_active":"2024-09-12T12:23:17.095201+0000","last_peered":"2024-09-12T12:23:17.095201+0000","last_clean":"2024-09-12T12:23:17.095201+0000","last_became_active":"2024-09-12T12:23:17.089910+0000","last_became_peered":"2024-09-12T12:23:17.089910+0000","last_unstale":"2024-09-12T12:23:17.095201+0000","last_undegraded":"2024-09-12T12:23:17.095201+0000","last_fullsized":"2024-09-12T12:23:17.095201+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:08.414320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.065359+0000","last_change":"2024-09-12T12:23:17.065359+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.065359+0000","last_undegraded":"2024-09-12T12:23:17.065359+0000","last_fullsized":"2024-09-12T12:23:17.065359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.096006+0000","last_change":"2024-09-12T12:23:17.093031+0000","last_active":"2024-09-12T12:23:17.096006+0000","last_peered":"2024-09-12T12:23:17.096006+0000","last_clean":"2024-09-12T12:23:17.096006+0000","last_became_active":"2024-09-12T12:23:17.092495+0000","last_became_peered":"2024-09-12T12:23:17.092495+0000","last_unstale":"2024-09-12T12:23:17.096006+0000","last_undegraded":"2024-09-12T12:23:17.096006+0000","last_fullsized":"2024-09-12T12:23:17.096006+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:23.615741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.073936+0000","last_change":"2024-09-12T12:23:17.073936+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.073936+0000","last_undegraded":"2024-09-12T12:23:17.073936+0000","last_fullsized":"2024-09-12T12:23:17.073936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.097434+0000","last_change":"2024-09-12T12:23:17.096451+0000","last_active":"2024-09-12T12:23:17.097434+0000","last_peered":"2024-09-12T12:23:17.097434+0000","last_clean":"2024-09-12T12:23:17.097434+0000","last_became_active":"2024-09-12T12:23:17.095796+0000","last_became_peered":"2024-09-12T12:23:17.095796+0000","last_unstale":"2024-09-12T12:23:17.097434+0000","last_undegraded":"2024-09-12T12:23:17.097434+0000","last_fullsized":"2024-09-12T12:23:17.097434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:17.993115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.117537+0000","last_change":"2024-09-12T12:23:18.114079+0000","last_active":"2024-09-12T12:23:18.117537+0000","last_peered":"2024-09-12T12:23:18.117537+0000","last_clean":"2024-09-12T12:23:18.117537+0000","last_became_active":"2024-09-12T12:23:18.113666+0000","last_became_peered":"2024-09-12T12:23:18.113666+0000","last_unstale":"2024-09-12T12:23:18.117537+0000","last_undegraded":"2024-09-12T12:23:18.117537+0000","last_fullsized":"2024-09-12T12:23:18.117537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:50.568191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.092344+0000","last_change":"2024-09-12T12:23:17.078879+0000","last_active":"2024-09-12T12:23:17.092344+0000","last_peered":"2024-09-12T12:23:17.092344+0000","last_clean":"2024-09-12T12:23:17.092344+0000","last_became_active":"2024-09-12T12:23:17.076836+0000","last_became_peered":"2024-09-12T12:23:17.076836+0000","last_unstale":"2024-09-12T12:23:17.092344+0000","last_undegraded":"2024-09-12T12:23:17.092344+0000","last_fullsized":"2024-09-12T12:23:17.092344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:49.543106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.134210+0000","last_change":"2024-09-12T12:23:18.133120+0000","last_active":"2024-09-12T12:23:18.134210+0000","last_peered":"2024-09-12T12:23:18.134210+0000","last_clean":"2024-09-12T12:23:18.134210+0000","last_became_active":"2024-09-12T12:23:18.132824+0000","last_became_peered":"2024-09-12T12:23:18.132824+0000","last_unstale":"2024-09-12T12:23:18.134210+0000","last_undegraded":"2024-09-12T12:23:18.134210+0000","last_fullsized":"2024-09-12T12:23:18.134210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:09:56.962893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.072096+0000","last_change":"2024-09-12T12:23:17.067441+0000","last_active":"2024-09-12T12:23:17.072096+0000","last_peered":"2024-09-12T12:23:17.072096+0000","last_clean":"2024-09-12T12:23:17.072096+0000","last_became_active":"2024-09-12T12:23:17.066728+0000","last_became_peered":"2024-09-12T12:23:17.066728+0000","last_unstale":"2024-09-12T12:23:17.072096+0000","last_undegraded":"2024-09-12T12:23:17.072096+0000","last_fullsized":"2024-09-12T12:23:17.072096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:21:59.864966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133456+0000","last_change":"2024-09-12T12:23:18.126379+0000","last_active":"2024-09-12T12:23:18.133456+0000","last_peered":"2024-09-12T12:23:18.133456+0000","last_clean":"2024-09-12T12:23:18.133456+0000","last_became_active":"2024-09-12T12:23:18.126047+0000","last_became_peered":"2024-09-12T12:23:18.126047+0000","last_unstale":"2024-09-12T12:23:18.133456+0000","last_undegraded":"2024-09-12T12:23:18.133456+0000","last_fullsized":"2024-09-12T12:23:18.133456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:46.711808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.083289+0000","last_change":"2024-09-12T12:23:17.078895+0000","last_active":"2024-09-12T12:23:17.083289+0000","last_peered":"2024-09-12T12:23:17.083289+0000","last_clean":"2024-09-12T12:23:17.083289+0000","last_became_active":"2024-09-12T12:23:17.078633+0000","last_became_peered":"2024-09-12T12:23:17.078633+0000","last_unstale":"2024-09-12T12:23:17.083289+0000","last_undegraded":"2024-09-12T12:23:17.083289+0000","last_fullsized":"2024-09-12T12:23:17.083289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:16.522351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072631+0000","last_change":"2024-09-12T12:23:17.072631+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072631+0000","last_undegraded":"2024-09-12T12:23:17.072631+0000","last_fullsized":"2024-09-12T12:23:17.072631+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.086997+0000","last_change":"2024-09-12T12:23:17.077229+0000","last_active":"2024-09-12T12:23:17.086997+0000","last_peered":"2024-09-12T12:23:17.086997+0000","last_clean":"2024-09-12T12:23:17.086997+0000","last_became_active":"2024-09-12T12:23:17.076051+0000","last_became_peered":"2024-09-12T12:23:17.076051+0000","last_unstale":"2024-09-12T12:23:17.086997+0000","last_undegraded":"2024-09-12T12:23:17.086997+0000","last_fullsized":"2024-09-12T12:23:17.086997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:03.593086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064090+0000","last_change":"2024-09-12T12:23:17.064090+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064090+0000","last_undegraded":"2024-09-12T12:23:17.064090+0000","last_fullsized":"2024-09-12T12:23:17.064090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087914+0000","last_change":"2024-09-12T12:23:17.079856+0000","last_active":"2024-09-12T12:23:17.087914+0000","last_peered":"2024-09-12T12:23:17.087914+0000","last_clean":"2024-09-12T12:23:17.087914+0000","last_became_active":"2024-09-12T12:23:17.079508+0000","last_became_peered":"2024-09-12T12:23:17.079508+0000","last_unstale":"2024-09-12T12:23:17.087914+0000","last_undegraded":"2024-09-12T12:23:17.087914+0000","last_fullsized":"2024-09-12T12:23:17.087914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:36.840747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066075+0000","last_change":"2024-09-12T12:23:17.066075+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066075+0000","last_undegraded":"2024-09-12T12:23:17.066075+0000","last_fullsized":"2024-09-12T12:23:17.066075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085756+0000","last_change":"2024-09-12T12:23:17.076124+0000","last_active":"2024-09-12T12:23:17.085756+0000","last_peered":"2024-09-12T12:23:17.085756+0000","last_clean":"2024-09-12T12:23:17.085756+0000","last_became_active":"2024-09-12T12:23:17.072753+0000","last_became_peered":"2024-09-12T12:23:17.072753+0000","last_unstale":"2024-09-12T12:23:17.085756+0000","last_undegraded":"2024-09-12T12:23:17.085756+0000","last_fullsized":"2024-09-12T12:23:17.085756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:24.793616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.064680+0000","last_change":"2024-09-12T12:23:17.064680+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.064680+0000","last_undegraded":"2024-09-12T12:23:17.064680+0000","last_fullsized":"2024-09-12T12:23:17.064680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093479+0000","last_change":"2024-09-12T12:23:17.090571+0000","last_active":"2024-09-12T12:23:17.093479+0000","last_peered":"2024-09-12T12:23:17.093479+0000","last_clean":"2024-09-12T12:23:17.093479+0000","last_became_active":"2024-09-12T12:23:17.090230+0000","last_became_peered":"2024-09-12T12:23:17.090230+0000","last_unstale":"2024-09-12T12:23:17.093479+0000","last_undegraded":"2024-09-12T12:23:17.093479+0000","last_fullsized":"2024-09-12T12:23:17.093479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:25:22.430473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.072543+0000","last_change":"2024-09-12T12:23:17.072543+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.072543+0000","last_undegraded":"2024-09-12T12:23:17.072543+0000","last_fullsized":"2024-09-12T12:23:17.072543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.085405+0000","last_change":"2024-09-12T12:23:17.073326+0000","last_active":"2024-09-12T12:23:17.085405+0000","last_peered":"2024-09-12T12:23:17.085405+0000","last_clean":"2024-09-12T12:23:17.085405+0000","last_became_active":"2024-09-12T12:23:17.072707+0000","last_became_peered":"2024-09-12T12:23:17.072707+0000","last_unstale":"2024-09-12T12:23:17.085405+0000","last_undegraded":"2024-09-12T12:23:17.085405+0000","last_fullsized":"2024-09-12T12:23:17.085405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:14:17.187446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.059949+0000","last_change":"2024-09-12T12:23:17.059949+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.059949+0000","last_undegraded":"2024-09-12T12:23:17.059949+0000","last_fullsized":"2024-09-12T12:23:17.059949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.093511+0000","last_change":"2024-09-12T12:23:17.089441+0000","last_active":"2024-09-12T12:23:17.093511+0000","last_peered":"2024-09-12T12:23:17.093511+0000","last_clean":"2024-09-12T12:23:17.093511+0000","last_became_active":"2024-09-12T12:23:17.089223+0000","last_became_peered":"2024-09-12T12:23:17.089223+0000","last_unstale":"2024-09-12T12:23:17.093511+0000","last_undegraded":"2024-09-12T12:23:17.093511+0000","last_fullsized":"2024-09-12T12:23:17.093511+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:04:45.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.066023+0000","last_change":"2024-09-12T12:23:17.066023+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.066023+0000","last_undegraded":"2024-09-12T12:23:17.066023+0000","last_fullsized":"2024-09-12T12:23:17.066023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.095568+0000","last_change":"2024-09-12T12:23:17.091347+0000","last_active":"2024-09-12T12:23:17.095568+0000","last_peered":"2024-09-12T12:23:17.095568+0000","last_clean":"2024-09-12T12:23:17.095568+0000","last_became_active":"2024-09-12T12:23:17.090341+0000","last_became_peered":"2024-09-12T12:23:17.090341+0000","last_unstale":"2024-09-12T12:23:17.095568+0000","last_undegraded":"2024-09-12T12:23:17.095568+0000","last_fullsized":"2024-09-12T12:23:17.095568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:38.673027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":64,"state":"creating+peering","last_fresh":"2024-09-12T12:23:17.057553+0000","last_change":"2024-09-12T12:23:17.057553+0000","last_active":"2024-09-12T12:23:17.044485+0000","last_peered":"2024-09-12T12:23:17.044485+0000","last_clean":"2024-09-12T12:23:17.044485+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:17.057553+0000","last_undegraded":"2024-09-12T12:23:17.057553+0000","last_fullsized":"2024-09-12T12:23:17.057553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087239+0000","last_change":"2024-09-12T12:23:17.076822+0000","last_active":"2024-09-12T12:23:17.087239+0000","last_peered":"2024-09-12T12:23:17.087239+0000","last_clean":"2024-09-12T12:23:17.087239+0000","last_became_active":"2024-09-12T12:23:17.075087+0000","last_became_peered":"2024-09-12T12:23:17.075087+0000","last_unstale":"2024-09-12T12:23:17.087239+0000","last_undegraded":"2024-09-12T12:23:17.087239+0000","last_fullsized":"2024-09-12T12:23:17.087239+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:29.472329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-12T12:23:18.133487+0000","last_change":"2024-09-12T12:23:18.126382+0000","last_active":"2024-09-12T12:23:18.133487+0000","last_peered":"2024-09-12T12:23:18.133487+0000","last_clean":"2024-09-12T12:23:18.133487+0000","last_became_active":"2024-09-12T12:23:18.126018+0000","last_became_peered":"2024-09-12T12:23:18.126018+0000","last_unstale":"2024-09-12T12:23:18.133487+0000","last_undegraded":"2024-09-12T12:23:18.133487+0000","last_fullsized":"2024-09-12T12:23:18.133487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:57.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.087158+0000","last_change":"2024-09-12T12:23:17.076005+0000","last_active":"2024-09-12T12:23:17.087158+0000","last_peered":"2024-09-12T12:23:17.087158+0000","last_clean":"2024-09-12T12:23:17.087158+0000","last_became_active":"2024-09-12T12:23:17.074864+0000","last_became_peered":"2024-09-12T12:23:17.074864+0000","last_unstale":"2024-09-12T12:23:17.087158+0000","last_undegraded":"2024-09-12T12:23:17.087158+0000","last_fullsized":"2024-09-12T12:23:17.087158+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:50.444388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-12T12:23:17.074838+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:17.074838+0000","last_peered":"2024-09-12T12:23:17.074838+0000","last_clean":"2024-09-12T12:23:17.074838+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:17.074838+0000","last_undegraded":"2024-09-12T12:23:17.074838+0000","last_fullsized":"2024-09-12T12:23:17.074838+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":794624,"data_stored":327173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":904379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":774144,"data_stored":320286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1368064,"data_stored":910654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:23:24.042 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-09-12T12:23:24.040+0000 7fdfbb622640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:23:26.640 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:23:26.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f13931640 1 Processor -- start 2024-09-12T12:23:26.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f13931640 1 -- start start 2024-09-12T12:23:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f13931640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f0c0d05b0 0x7f8f0c0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f13931640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c15b620 0x7f8f0c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f13931640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0c0d0ed0 con 0x7f8f0c0ae250 2024-09-12T12:23:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f13931640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8f0c0ad0c0 con 0x7f8f0c0d05b0 2024-09-12T12:23:26.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f13931640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0c0ad3a0 con 0x7f8f0c15b620 2024-09-12T12:23:26.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f116cd640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8f0c0d05b0 0x7f8f0c0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:26.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f11ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c15b620 0x7f8f0c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:26.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f11ece640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c15b620 0x7f8f0c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:51082/0 (socket says 172.21.15.3:51082) 2024-09-12T12:23:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f10ecc640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8f0c0ae250 0x7f8f0c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:34796/0 (socket says 172.21.15.3:34796) 2024-09-12T12:23:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f11ece640 1 -- 172.21.15.3:0/1615086751 learned_addr learned my addr 172.21.15.3:0/1615086751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f116cd640 1 -- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c15b620 msgr2=0x7f8f0c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f116cd640 1 --2- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c15b620 0x7f8f0c15ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f116cd640 1 -- 172.21.15.3:0/1615086751 >> v1:172.21.15.142:6789/0 conn(0x7f8f0c0ae250 legacy=0x7f8f0c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f116cd640 1 -- 172.21.15.3:0/1615086751 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c0ad680 con 0x7f8f0c0d05b0 2024-09-12T12:23:26.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.772+0000 7f8f11ece640 1 --2- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c15b620 0x7f8f0c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f116cd640 1 --2- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c0d05b0 0x7f8f0c0d0990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0097c0 tx=0x7f8efc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a53b3a6192017b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:26.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f03fff640 1 -- 172.21.15.3:0/1615086751 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0b2070 con 0x7f8f0c0d05b0 2024-09-12T12:23:26.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f03fff640 1 -- 172.21.15.3:0/1615086751 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8efc0a9070 con 0x7f8f0c0d05b0 2024-09-12T12:23:26.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f03fff640 1 -- 172.21.15.3:0/1615086751 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0ad880 con 0x7f8f0c0d05b0 2024-09-12T12:23:26.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 -- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c0d05b0 msgr2=0x7f8f0c0d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:26.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 --2- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c0d05b0 0x7f8f0c0d0990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0097c0 tx=0x7f8efc01c470 comp rx=0 tx=0).stop 2024-09-12T12:23:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 -- 172.21.15.3:0/1615086751 shutdown_connections 2024-09-12T12:23:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 --2- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c15b620 0x7f8f0c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 --2- 172.21.15.3:0/1615086751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c0d05b0 0x7f8f0c0d0990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 -- 172.21.15.3:0/1615086751 >> 172.21.15.3:0/1615086751 conn(0x7f8f0c067e00 msgr2=0x7f8f0c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 -- 172.21.15.3:0/1615086751 shutdown_connections 2024-09-12T12:23:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 -- 172.21.15.3:0/1615086751 wait complete. 2024-09-12T12:23:26.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 Processor -- start 2024-09-12T12:23:26.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.776+0000 7f8f13931640 1 -- start start 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f0c0ae250 0x7f8f0c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c0d05b0 0x7f8f0c0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c15b620 0x7f8f0c150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c15c9b0 con 0x7f8f0c0ae250 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c15c830 con 0x7f8f0c15b620 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0c167cb0 con 0x7f8f0c0d05b0 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f0c0ae250 0x7f8f0c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f0c0ae250 0x7f8f0c0ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38532/0 (socket says 172.21.15.3:38532) 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 -- 172.21.15.3:0/3281794327 learned_addr learned my addr 172.21.15.3:0/3281794327 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f10ecc640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c0d05b0 0x7f8f0c0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:26.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f11ece640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c15b620 0x7f8f0c150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 -- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c0d05b0 msgr2=0x7f8f0c0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c0d05b0 0x7f8f0c0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 -- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c15b620 msgr2=0x7f8f0c150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c15b620 0x7f8f0c150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 -- 172.21.15.3:0/3281794327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c0b0840 con 0x7f8f0c0ae250 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f10ecc640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c0d05b0 0x7f8f0c0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f116cd640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f0c0ae250 0x7f8f0c0ca4b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b0040 tx=0x7f8efc019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f01ffb640 1 -- 172.21.15.3:0/3281794327 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0b2040 con 0x7f8f0c0ae250 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f01ffb640 1 -- 172.21.15.3:0/3281794327 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8efc0bc070 con 0x7f8f0c0ae250 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c0b0b20 con 0x7f8f0c0ae250 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f01ffb640 1 -- 172.21.15.3:0/3281794327 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0b6410 con 0x7f8f0c0ae250 2024-09-12T12:23:26.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c0b0e00 con 0x7f8f0c0ae250 2024-09-12T12:23:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0c0d0990 con 0x7f8f0c0ae250 2024-09-12T12:23:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f01ffb640 1 -- 172.21.15.3:0/3281794327 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8efc0ab070 con 0x7f8f0c0ae250 2024-09-12T12:23:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f01ffb640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ef00678c0 0x7f8ef0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f01ffb640 1 -- 172.21.15.3:0/3281794327 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f8efc120200 con 0x7f8f0c0ae250 2024-09-12T12:23:26.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.780+0000 7f8f10ecc640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ef00678c0 0x7f8ef0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:26.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.784+0000 7f8f10ecc640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ef00678c0 0x7f8ef0069d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0b2e70 tx=0x7f8f08007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:26.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.788+0000 7f8f01ffb640 1 -- 172.21.15.3:0/3281794327 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8efc016090 con 0x7f8f0c0ae250 2024-09-12T12:23:26.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.948+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f0c0b05f0 con 0x7f8ef00678c0 2024-09-12T12:23:26.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.964+0000 7f8f01ffb640 1 -- 172.21.15.3:0/3281794327 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252077 (secure 0 0 0) 0x7f8f0c0b05f0 con 0x7f8ef00678c0 2024-09-12T12:23:26.971 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:26.972 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:23:26.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ef00678c0 msgr2=0x7f8ef0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:26.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ef00678c0 0x7f8ef0069d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0b2e70 tx=0x7f8f08007d50 comp rx=0 tx=0).stop 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f0c0ae250 msgr2=0x7f8f0c0ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f0c0ae250 0x7f8f0c0ca4b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b0040 tx=0x7f8efc019210 comp rx=0 tx=0).stop 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 shutdown_connections 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ef00678c0 0x7f8ef0069d80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f0c15b620 0x7f8f0c150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8f0c0d05b0 0x7f8f0c0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 --2- 172.21.15.3:0/3281794327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8f0c0ae250 0x7f8f0c0ca4b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 >> 172.21.15.3:0/3281794327 conn(0x7f8f0c067e00 msgr2=0x7f8f0c152360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:26.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 shutdown_connections 2024-09-12T12:23:26.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:26.968+0000 7f8f13931640 1 -- 172.21.15.3:0/3281794327 wait complete. 2024-09-12T12:23:26.990 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-12T12:23:25.730509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240340,"kb_used_data":7464,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734380,"statfs":{"total":773094113280,"available":772848005120,"internally_reserved":0,"allocated":7643136,"data_stored":3852444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002884"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.086820+0000","last_change":"2024-09-12T12:23:18.137642+0000","last_active":"2024-09-12T12:23:20.086820+0000","last_peered":"2024-09-12T12:23:20.086820+0000","last_clean":"2024-09-12T12:23:20.086820+0000","last_became_active":"2024-09-12T12:23:18.134686+0000","last_became_peered":"2024-09-12T12:23:18.134686+0000","last_unstale":"2024-09-12T12:23:20.086820+0000","last_undegraded":"2024-09-12T12:23:20.086820+0000","last_fullsized":"2024-09-12T12:23:20.086820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:44.061791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088710+0000","last_change":"2024-09-12T12:23:18.124548+0000","last_active":"2024-09-12T12:23:20.088710+0000","last_peered":"2024-09-12T12:23:20.088710+0000","last_clean":"2024-09-12T12:23:20.088710+0000","last_became_active":"2024-09-12T12:23:18.124025+0000","last_became_peered":"2024-09-12T12:23:18.124025+0000","last_unstale":"2024-09-12T12:23:20.088710+0000","last_undegraded":"2024-09-12T12:23:20.088710+0000","last_fullsized":"2024-09-12T12:23:20.088710+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:26:27.250819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638565+0000","last_change":"2024-09-12T12:23:18.107885+0000","last_active":"2024-09-12T12:23:19.638565+0000","last_peered":"2024-09-12T12:23:19.638565+0000","last_clean":"2024-09-12T12:23:19.638565+0000","last_became_active":"2024-09-12T12:23:18.106854+0000","last_became_peered":"2024-09-12T12:23:18.106854+0000","last_unstale":"2024-09-12T12:23:19.638565+0000","last_undegraded":"2024-09-12T12:23:19.638565+0000","last_fullsized":"2024-09-12T12:23:19.638565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:39:33.336868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084558+0000","last_change":"2024-09-12T12:23:18.125328+0000","last_active":"2024-09-12T12:23:20.084558+0000","last_peered":"2024-09-12T12:23:20.084558+0000","last_clean":"2024-09-12T12:23:20.084558+0000","last_became_active":"2024-09-12T12:23:18.125050+0000","last_became_peered":"2024-09-12T12:23:18.125050+0000","last_unstale":"2024-09-12T12:23:20.084558+0000","last_undegraded":"2024-09-12T12:23:20.084558+0000","last_fullsized":"2024-09-12T12:23:20.084558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:32.629326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.109020+0000","last_change":"2024-09-12T12:23:18.111868+0000","last_active":"2024-09-12T12:23:19.109020+0000","last_peered":"2024-09-12T12:23:19.109020+0000","last_clean":"2024-09-12T12:23:19.109020+0000","last_became_active":"2024-09-12T12:23:18.111499+0000","last_became_peered":"2024-09-12T12:23:18.111499+0000","last_unstale":"2024-09-12T12:23:19.109020+0000","last_undegraded":"2024-09-12T12:23:19.109020+0000","last_fullsized":"2024-09-12T12:23:19.109020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:12:11.513323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087661+0000","last_change":"2024-09-12T12:23:18.135493+0000","last_active":"2024-09-12T12:23:20.087661+0000","last_peered":"2024-09-12T12:23:20.087661+0000","last_clean":"2024-09-12T12:23:20.087661+0000","last_became_active":"2024-09-12T12:23:18.132846+0000","last_became_peered":"2024-09-12T12:23:18.132846+0000","last_unstale":"2024-09-12T12:23:20.087661+0000","last_undegraded":"2024-09-12T12:23:20.087661+0000","last_fullsized":"2024-09-12T12:23:20.087661+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:34:16.045016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.118434+0000","last_change":"2024-09-12T12:23:18.116578+0000","last_active":"2024-09-12T12:23:19.118434+0000","last_peered":"2024-09-12T12:23:19.118434+0000","last_clean":"2024-09-12T12:23:19.118434+0000","last_became_active":"2024-09-12T12:23:18.115461+0000","last_became_peered":"2024-09-12T12:23:18.115461+0000","last_unstale":"2024-09-12T12:23:19.118434+0000","last_undegraded":"2024-09-12T12:23:19.118434+0000","last_fullsized":"2024-09-12T12:23:19.118434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:15:58.797574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087821+0000","last_change":"2024-09-12T12:23:18.135547+0000","last_active":"2024-09-12T12:23:20.087821+0000","last_peered":"2024-09-12T12:23:20.087821+0000","last_clean":"2024-09-12T12:23:20.087821+0000","last_became_active":"2024-09-12T12:23:18.127055+0000","last_became_peered":"2024-09-12T12:23:18.127055+0000","last_unstale":"2024-09-12T12:23:20.087821+0000","last_undegraded":"2024-09-12T12:23:20.087821+0000","last_fullsized":"2024-09-12T12:23:20.087821+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.110466+0000","last_change":"2024-09-12T12:23:18.121626+0000","last_active":"2024-09-12T12:23:19.110466+0000","last_peered":"2024-09-12T12:23:19.110466+0000","last_clean":"2024-09-12T12:23:19.110466+0000","last_became_active":"2024-09-12T12:23:18.121307+0000","last_became_peered":"2024-09-12T12:23:18.121307+0000","last_unstale":"2024-09-12T12:23:19.110466+0000","last_undegraded":"2024-09-12T12:23:19.110466+0000","last_fullsized":"2024-09-12T12:23:19.110466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:50.551818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.114528+0000","last_change":"2024-09-12T12:23:18.114073+0000","last_active":"2024-09-12T12:23:19.114528+0000","last_peered":"2024-09-12T12:23:19.114528+0000","last_clean":"2024-09-12T12:23:19.114528+0000","last_became_active":"2024-09-12T12:23:18.113227+0000","last_became_peered":"2024-09-12T12:23:18.113227+0000","last_unstale":"2024-09-12T12:23:19.114528+0000","last_undegraded":"2024-09-12T12:23:19.114528+0000","last_fullsized":"2024-09-12T12:23:19.114528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:41:24.793350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.086321+0000","last_change":"2024-09-12T12:23:18.132183+0000","last_active":"2024-09-12T12:23:20.086321+0000","last_peered":"2024-09-12T12:23:20.086321+0000","last_clean":"2024-09-12T12:23:20.086321+0000","last_became_active":"2024-09-12T12:23:18.130359+0000","last_became_peered":"2024-09-12T12:23:18.130359+0000","last_unstale":"2024-09-12T12:23:20.086321+0000","last_undegraded":"2024-09-12T12:23:20.086321+0000","last_fullsized":"2024-09-12T12:23:20.086321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:39.851919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084809+0000","last_change":"2024-09-12T12:23:18.118403+0000","last_active":"2024-09-12T12:23:20.084809+0000","last_peered":"2024-09-12T12:23:20.084809+0000","last_clean":"2024-09-12T12:23:20.084809+0000","last_became_active":"2024-09-12T12:23:18.118043+0000","last_became_peered":"2024-09-12T12:23:18.118043+0000","last_unstale":"2024-09-12T12:23:20.084809+0000","last_undegraded":"2024-09-12T12:23:20.084809+0000","last_fullsized":"2024-09-12T12:23:20.084809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:23.796146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.121764+0000","last_change":"2024-09-12T12:23:18.111552+0000","last_active":"2024-09-12T12:23:19.121764+0000","last_peered":"2024-09-12T12:23:19.121764+0000","last_clean":"2024-09-12T12:23:19.121764+0000","last_became_active":"2024-09-12T12:23:18.110816+0000","last_became_peered":"2024-09-12T12:23:18.110816+0000","last_unstale":"2024-09-12T12:23:19.121764+0000","last_undegraded":"2024-09-12T12:23:19.121764+0000","last_fullsized":"2024-09-12T12:23:19.121764+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:05.823907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.109368+0000","last_change":"2024-09-12T12:23:18.122429+0000","last_active":"2024-09-12T12:23:19.109368+0000","last_peered":"2024-09-12T12:23:19.109368+0000","last_clean":"2024-09-12T12:23:19.109368+0000","last_became_active":"2024-09-12T12:23:18.122106+0000","last_became_peered":"2024-09-12T12:23:18.122106+0000","last_unstale":"2024-09-12T12:23:19.109368+0000","last_undegraded":"2024-09-12T12:23:19.109368+0000","last_fullsized":"2024-09-12T12:23:19.109368+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:56.651262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.121714+0000","last_change":"2024-09-12T12:23:18.121261+0000","last_active":"2024-09-12T12:23:19.121714+0000","last_peered":"2024-09-12T12:23:19.121714+0000","last_clean":"2024-09-12T12:23:19.121714+0000","last_became_active":"2024-09-12T12:23:18.120928+0000","last_became_peered":"2024-09-12T12:23:18.120928+0000","last_unstale":"2024-09-12T12:23:19.121714+0000","last_undegraded":"2024-09-12T12:23:19.121714+0000","last_fullsized":"2024-09-12T12:23:19.121714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:17.505436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.112905+0000","last_change":"2024-09-12T12:23:18.145741+0000","last_active":"2024-09-12T12:23:19.112905+0000","last_peered":"2024-09-12T12:23:19.112905+0000","last_clean":"2024-09-12T12:23:19.112905+0000","last_became_active":"2024-09-12T12:23:18.144874+0000","last_became_peered":"2024-09-12T12:23:18.144874+0000","last_unstale":"2024-09-12T12:23:19.112905+0000","last_undegraded":"2024-09-12T12:23:19.112905+0000","last_fullsized":"2024-09-12T12:23:19.112905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:46:50.453632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.110368+0000","last_change":"2024-09-12T12:23:18.123269+0000","last_active":"2024-09-12T12:23:19.110368+0000","last_peered":"2024-09-12T12:23:19.110368+0000","last_clean":"2024-09-12T12:23:19.110368+0000","last_became_active":"2024-09-12T12:23:18.120059+0000","last_became_peered":"2024-09-12T12:23:18.120059+0000","last_unstale":"2024-09-12T12:23:19.110368+0000","last_undegraded":"2024-09-12T12:23:19.110368+0000","last_fullsized":"2024-09-12T12:23:19.110368+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:31:44.875230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084925+0000","last_change":"2024-09-12T12:23:18.104566+0000","last_active":"2024-09-12T12:23:20.084925+0000","last_peered":"2024-09-12T12:23:20.084925+0000","last_clean":"2024-09-12T12:23:20.084925+0000","last_became_active":"2024-09-12T12:23:18.103784+0000","last_became_peered":"2024-09-12T12:23:18.103784+0000","last_unstale":"2024-09-12T12:23:20.084925+0000","last_undegraded":"2024-09-12T12:23:20.084925+0000","last_fullsized":"2024-09-12T12:23:20.084925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:17.663471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088137+0000","last_change":"2024-09-12T12:23:18.130875+0000","last_active":"2024-09-12T12:23:20.088137+0000","last_peered":"2024-09-12T12:23:20.088137+0000","last_clean":"2024-09-12T12:23:20.088137+0000","last_became_active":"2024-09-12T12:23:18.126980+0000","last_became_peered":"2024-09-12T12:23:18.126980+0000","last_unstale":"2024-09-12T12:23:20.088137+0000","last_undegraded":"2024-09-12T12:23:20.088137+0000","last_fullsized":"2024-09-12T12:23:20.088137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:37.849459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.122472+0000","last_change":"2024-09-12T12:23:18.107399+0000","last_active":"2024-09-12T12:23:19.122472+0000","last_peered":"2024-09-12T12:23:19.122472+0000","last_clean":"2024-09-12T12:23:19.122472+0000","last_became_active":"2024-09-12T12:23:18.106988+0000","last_became_peered":"2024-09-12T12:23:18.106988+0000","last_unstale":"2024-09-12T12:23:19.122472+0000","last_undegraded":"2024-09-12T12:23:19.122472+0000","last_fullsized":"2024-09-12T12:23:19.122472+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:09.443805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.113481+0000","last_change":"2024-09-12T12:23:18.100446+0000","last_active":"2024-09-12T12:23:19.113481+0000","last_peered":"2024-09-12T12:23:19.113481+0000","last_clean":"2024-09-12T12:23:19.113481+0000","last_became_active":"2024-09-12T12:23:18.099672+0000","last_became_peered":"2024-09-12T12:23:18.099672+0000","last_unstale":"2024-09-12T12:23:19.113481+0000","last_undegraded":"2024-09-12T12:23:19.113481+0000","last_fullsized":"2024-09-12T12:23:19.113481+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:39.693506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.113570+0000","last_change":"2024-09-12T12:23:18.114177+0000","last_active":"2024-09-12T12:23:19.113570+0000","last_peered":"2024-09-12T12:23:19.113570+0000","last_clean":"2024-09-12T12:23:19.113570+0000","last_became_active":"2024-09-12T12:23:18.113433+0000","last_became_peered":"2024-09-12T12:23:18.113433+0000","last_unstale":"2024-09-12T12:23:19.113570+0000","last_undegraded":"2024-09-12T12:23:19.113570+0000","last_fullsized":"2024-09-12T12:23:19.113570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:13:51.286037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.085702+0000","last_change":"2024-09-12T12:23:18.130346+0000","last_active":"2024-09-12T12:23:20.085702+0000","last_peered":"2024-09-12T12:23:20.085702+0000","last_clean":"2024-09-12T12:23:20.085702+0000","last_became_active":"2024-09-12T12:23:18.129672+0000","last_became_peered":"2024-09-12T12:23:18.129672+0000","last_unstale":"2024-09-12T12:23:20.085702+0000","last_undegraded":"2024-09-12T12:23:20.085702+0000","last_fullsized":"2024-09-12T12:23:20.085702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:11.652133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.085467+0000","last_change":"2024-09-12T12:23:18.146877+0000","last_active":"2024-09-12T12:23:20.085467+0000","last_peered":"2024-09-12T12:23:20.085467+0000","last_clean":"2024-09-12T12:23:20.085467+0000","last_became_active":"2024-09-12T12:23:18.146362+0000","last_became_peered":"2024-09-12T12:23:18.146362+0000","last_unstale":"2024-09-12T12:23:20.085467+0000","last_undegraded":"2024-09-12T12:23:20.085467+0000","last_fullsized":"2024-09-12T12:23:20.085467+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:04:52.843272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.114697+0000","last_change":"2024-09-12T12:23:18.126734+0000","last_active":"2024-09-12T12:23:19.114697+0000","last_peered":"2024-09-12T12:23:19.114697+0000","last_clean":"2024-09-12T12:23:19.114697+0000","last_became_active":"2024-09-12T12:23:18.126102+0000","last_became_peered":"2024-09-12T12:23:18.126102+0000","last_unstale":"2024-09-12T12:23:19.114697+0000","last_undegraded":"2024-09-12T12:23:19.114697+0000","last_fullsized":"2024-09-12T12:23:19.114697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:33:34.830768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.119079+0000","last_change":"2024-09-12T12:23:18.117669+0000","last_active":"2024-09-12T12:23:19.119079+0000","last_peered":"2024-09-12T12:23:19.119079+0000","last_clean":"2024-09-12T12:23:19.119079+0000","last_became_active":"2024-09-12T12:23:18.115378+0000","last_became_peered":"2024-09-12T12:23:18.115378+0000","last_unstale":"2024-09-12T12:23:19.119079+0000","last_undegraded":"2024-09-12T12:23:19.119079+0000","last_fullsized":"2024-09-12T12:23:19.119079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:02.145300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638680+0000","last_change":"2024-09-12T12:23:18.108157+0000","last_active":"2024-09-12T12:23:19.638680+0000","last_peered":"2024-09-12T12:23:19.638680+0000","last_clean":"2024-09-12T12:23:19.638680+0000","last_became_active":"2024-09-12T12:23:18.107527+0000","last_became_peered":"2024-09-12T12:23:18.107527+0000","last_unstale":"2024-09-12T12:23:19.638680+0000","last_undegraded":"2024-09-12T12:23:19.638680+0000","last_fullsized":"2024-09-12T12:23:19.638680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:30.305172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.113999+0000","last_change":"2024-09-12T12:23:18.126192+0000","last_active":"2024-09-12T12:23:19.113999+0000","last_peered":"2024-09-12T12:23:19.113999+0000","last_clean":"2024-09-12T12:23:19.113999+0000","last_became_active":"2024-09-12T12:23:18.120336+0000","last_became_peered":"2024-09-12T12:23:18.120336+0000","last_unstale":"2024-09-12T12:23:19.113999+0000","last_undegraded":"2024-09-12T12:23:19.113999+0000","last_fullsized":"2024-09-12T12:23:19.113999+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:26:27.250819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638579+0000","last_change":"2024-09-12T12:23:18.107716+0000","last_active":"2024-09-12T12:23:19.638579+0000","last_peered":"2024-09-12T12:23:19.638579+0000","last_clean":"2024-09-12T12:23:19.638579+0000","last_became_active":"2024-09-12T12:23:18.107421+0000","last_became_peered":"2024-09-12T12:23:18.107421+0000","last_unstale":"2024-09-12T12:23:19.638579+0000","last_undegraded":"2024-09-12T12:23:19.638579+0000","last_fullsized":"2024-09-12T12:23:19.638579+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:14.179968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.118703+0000","last_change":"2024-09-12T12:23:18.143621+0000","last_active":"2024-09-12T12:23:19.118703+0000","last_peered":"2024-09-12T12:23:19.118703+0000","last_clean":"2024-09-12T12:23:19.118703+0000","last_became_active":"2024-09-12T12:23:18.143048+0000","last_became_peered":"2024-09-12T12:23:18.143048+0000","last_unstale":"2024-09-12T12:23:19.118703+0000","last_undegraded":"2024-09-12T12:23:19.118703+0000","last_fullsized":"2024-09-12T12:23:19.118703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:36:39.028144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.118798+0000","last_change":"2024-09-12T12:23:18.106993+0000","last_active":"2024-09-12T12:23:19.118798+0000","last_peered":"2024-09-12T12:23:19.118798+0000","last_clean":"2024-09-12T12:23:19.118798+0000","last_became_active":"2024-09-12T12:23:18.106255+0000","last_became_peered":"2024-09-12T12:23:18.106255+0000","last_unstale":"2024-09-12T12:23:19.118798+0000","last_undegraded":"2024-09-12T12:23:19.118798+0000","last_fullsized":"2024-09-12T12:23:19.118798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:18.573792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088176+0000","last_change":"2024-09-12T12:23:18.135527+0000","last_active":"2024-09-12T12:23:20.088176+0000","last_peered":"2024-09-12T12:23:20.088176+0000","last_clean":"2024-09-12T12:23:20.088176+0000","last_became_active":"2024-09-12T12:23:18.132821+0000","last_became_peered":"2024-09-12T12:23:18.132821+0000","last_unstale":"2024-09-12T12:23:20.088176+0000","last_undegraded":"2024-09-12T12:23:20.088176+0000","last_fullsized":"2024-09-12T12:23:20.088176+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:39.693506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.086783+0000","last_change":"2024-09-12T12:23:18.146006+0000","last_active":"2024-09-12T12:23:20.086783+0000","last_peered":"2024-09-12T12:23:20.086783+0000","last_clean":"2024-09-12T12:23:20.086783+0000","last_became_active":"2024-09-12T12:23:18.145414+0000","last_became_peered":"2024-09-12T12:23:18.145414+0000","last_unstale":"2024-09-12T12:23:20.086783+0000","last_undegraded":"2024-09-12T12:23:20.086783+0000","last_fullsized":"2024-09-12T12:23:20.086783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:00.656914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087510+0000","last_change":"2024-09-12T12:23:17.089339+0000","last_active":"2024-09-12T12:23:20.087510+0000","last_peered":"2024-09-12T12:23:20.087510+0000","last_clean":"2024-09-12T12:23:20.087510+0000","last_became_active":"2024-09-12T12:23:17.088424+0000","last_became_peered":"2024-09-12T12:23:17.088424+0000","last_unstale":"2024-09-12T12:23:20.087510+0000","last_undegraded":"2024-09-12T12:23:20.087510+0000","last_fullsized":"2024-09-12T12:23:20.087510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:14:15.382197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.110457+0000","last_change":"2024-09-12T12:23:18.116352+0000","last_active":"2024-09-12T12:23:19.110457+0000","last_peered":"2024-09-12T12:23:19.110457+0000","last_clean":"2024-09-12T12:23:19.110457+0000","last_became_active":"2024-09-12T12:23:18.115730+0000","last_became_peered":"2024-09-12T12:23:18.115730+0000","last_unstale":"2024-09-12T12:23:19.110457+0000","last_undegraded":"2024-09-12T12:23:19.110457+0000","last_fullsized":"2024-09-12T12:23:19.110457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:04.601798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.122101+0000","last_change":"2024-09-12T12:23:17.082761+0000","last_active":"2024-09-12T12:23:19.122101+0000","last_peered":"2024-09-12T12:23:19.122101+0000","last_clean":"2024-09-12T12:23:19.122101+0000","last_became_active":"2024-09-12T12:23:17.082201+0000","last_became_peered":"2024-09-12T12:23:17.082201+0000","last_unstale":"2024-09-12T12:23:19.122101+0000","last_undegraded":"2024-09-12T12:23:19.122101+0000","last_fullsized":"2024-09-12T12:23:19.122101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:12:36.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088392+0000","last_change":"2024-09-12T12:23:18.135986+0000","last_active":"2024-09-12T12:23:20.088392+0000","last_peered":"2024-09-12T12:23:20.088392+0000","last_clean":"2024-09-12T12:23:20.088392+0000","last_became_active":"2024-09-12T12:23:18.133224+0000","last_became_peered":"2024-09-12T12:23:18.133224+0000","last_unstale":"2024-09-12T12:23:20.088392+0000","last_undegraded":"2024-09-12T12:23:20.088392+0000","last_fullsized":"2024-09-12T12:23:20.088392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:46:50.453632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.114369+0000","last_change":"2024-09-12T12:23:17.093647+0000","last_active":"2024-09-12T12:23:19.114369+0000","last_peered":"2024-09-12T12:23:19.114369+0000","last_clean":"2024-09-12T12:23:19.114369+0000","last_became_active":"2024-09-12T12:23:17.093229+0000","last_became_peered":"2024-09-12T12:23:17.093229+0000","last_unstale":"2024-09-12T12:23:19.114369+0000","last_undegraded":"2024-09-12T12:23:19.114369+0000","last_fullsized":"2024-09-12T12:23:19.114369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:16.654758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.086607+0000","last_change":"2024-09-12T12:23:18.132462+0000","last_active":"2024-09-12T12:23:20.086607+0000","last_peered":"2024-09-12T12:23:20.086607+0000","last_clean":"2024-09-12T12:23:20.086607+0000","last_became_active":"2024-09-12T12:23:18.131094+0000","last_became_peered":"2024-09-12T12:23:18.131094+0000","last_unstale":"2024-09-12T12:23:20.086607+0000","last_undegraded":"2024-09-12T12:23:20.086607+0000","last_fullsized":"2024-09-12T12:23:20.086607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:10.652664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088662+0000","last_change":"2024-09-12T12:23:17.076844+0000","last_active":"2024-09-12T12:23:20.088662+0000","last_peered":"2024-09-12T12:23:20.088662+0000","last_clean":"2024-09-12T12:23:20.088662+0000","last_became_active":"2024-09-12T12:23:17.076096+0000","last_became_peered":"2024-09-12T12:23:17.076096+0000","last_unstale":"2024-09-12T12:23:20.088662+0000","last_undegraded":"2024-09-12T12:23:20.088662+0000","last_fullsized":"2024-09-12T12:23:20.088662+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:32.373585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.118897+0000","last_change":"2024-09-12T12:23:18.100829+0000","last_active":"2024-09-12T12:23:19.118897+0000","last_peered":"2024-09-12T12:23:19.118897+0000","last_clean":"2024-09-12T12:23:19.118897+0000","last_became_active":"2024-09-12T12:23:18.100094+0000","last_became_peered":"2024-09-12T12:23:18.100094+0000","last_unstale":"2024-09-12T12:23:19.118897+0000","last_undegraded":"2024-09-12T12:23:19.118897+0000","last_fullsized":"2024-09-12T12:23:19.118897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.609441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638815+0000","last_change":"2024-09-12T12:23:17.088893+0000","last_active":"2024-09-12T12:23:19.638815+0000","last_peered":"2024-09-12T12:23:19.638815+0000","last_clean":"2024-09-12T12:23:19.638815+0000","last_became_active":"2024-09-12T12:23:17.088633+0000","last_became_peered":"2024-09-12T12:23:17.088633+0000","last_unstale":"2024-09-12T12:23:19.638815+0000","last_undegraded":"2024-09-12T12:23:19.638815+0000","last_fullsized":"2024-09-12T12:23:19.638815+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:32.158937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.113688+0000","last_change":"2024-09-12T12:23:18.108271+0000","last_active":"2024-09-12T12:23:19.113688+0000","last_peered":"2024-09-12T12:23:19.113688+0000","last_clean":"2024-09-12T12:23:19.113688+0000","last_became_active":"2024-09-12T12:23:18.107523+0000","last_became_peered":"2024-09-12T12:23:18.107523+0000","last_unstale":"2024-09-12T12:23:19.113688+0000","last_undegraded":"2024-09-12T12:23:19.113688+0000","last_fullsized":"2024-09-12T12:23:19.113688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.284357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087623+0000","last_change":"2024-09-12T12:23:17.076621+0000","last_active":"2024-09-12T12:23:20.087623+0000","last_peered":"2024-09-12T12:23:20.087623+0000","last_clean":"2024-09-12T12:23:20.087623+0000","last_became_active":"2024-09-12T12:23:17.076227+0000","last_became_peered":"2024-09-12T12:23:17.076227+0000","last_unstale":"2024-09-12T12:23:20.087623+0000","last_undegraded":"2024-09-12T12:23:20.087623+0000","last_fullsized":"2024-09-12T12:23:20.087623+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:45:38.673027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.108558+0000","last_change":"2024-09-12T12:23:18.110381+0000","last_active":"2024-09-12T12:23:19.108558+0000","last_peered":"2024-09-12T12:23:19.108558+0000","last_clean":"2024-09-12T12:23:19.108558+0000","last_became_active":"2024-09-12T12:23:18.110021+0000","last_became_peered":"2024-09-12T12:23:18.110021+0000","last_unstale":"2024-09-12T12:23:19.108558+0000","last_undegraded":"2024-09-12T12:23:19.108558+0000","last_fullsized":"2024-09-12T12:23:19.108558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:21:52.096570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084113+0000","last_change":"2024-09-12T12:23:17.091844+0000","last_active":"2024-09-12T12:23:20.084113+0000","last_peered":"2024-09-12T12:23:20.084113+0000","last_clean":"2024-09-12T12:23:20.084113+0000","last_became_active":"2024-09-12T12:23:17.091637+0000","last_became_peered":"2024-09-12T12:23:17.091637+0000","last_unstale":"2024-09-12T12:23:20.084113+0000","last_undegraded":"2024-09-12T12:23:20.084113+0000","last_fullsized":"2024-09-12T12:23:20.084113+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:08.414320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.121745+0000","last_change":"2024-09-12T12:23:18.109087+0000","last_active":"2024-09-12T12:23:19.121745+0000","last_peered":"2024-09-12T12:23:19.121745+0000","last_clean":"2024-09-12T12:23:19.121745+0000","last_became_active":"2024-09-12T12:23:18.108784+0000","last_became_peered":"2024-09-12T12:23:18.108784+0000","last_unstale":"2024-09-12T12:23:19.121745+0000","last_undegraded":"2024-09-12T12:23:19.121745+0000","last_fullsized":"2024-09-12T12:23:19.121745+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:22:06.343407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.108662+0000","last_change":"2024-09-12T12:23:17.077058+0000","last_active":"2024-09-12T12:23:19.108662+0000","last_peered":"2024-09-12T12:23:19.108662+0000","last_clean":"2024-09-12T12:23:19.108662+0000","last_became_active":"2024-09-12T12:23:17.073114+0000","last_became_peered":"2024-09-12T12:23:17.073114+0000","last_unstale":"2024-09-12T12:23:19.108662+0000","last_undegraded":"2024-09-12T12:23:19.108662+0000","last_fullsized":"2024-09-12T12:23:19.108662+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:35:13.829402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.119297+0000","last_change":"2024-09-12T12:23:18.109308+0000","last_active":"2024-09-12T12:23:19.119297+0000","last_peered":"2024-09-12T12:23:19.119297+0000","last_clean":"2024-09-12T12:23:19.119297+0000","last_became_active":"2024-09-12T12:23:18.108346+0000","last_became_peered":"2024-09-12T12:23:18.108346+0000","last_unstale":"2024-09-12T12:23:19.119297+0000","last_undegraded":"2024-09-12T12:23:19.119297+0000","last_fullsized":"2024-09-12T12:23:19.119297+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:37:36.194283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.114527+0000","last_change":"2024-09-12T12:23:17.091345+0000","last_active":"2024-09-12T12:23:19.114527+0000","last_peered":"2024-09-12T12:23:19.114527+0000","last_clean":"2024-09-12T12:23:19.114527+0000","last_became_active":"2024-09-12T12:23:17.090644+0000","last_became_peered":"2024-09-12T12:23:17.090644+0000","last_unstale":"2024-09-12T12:23:19.114527+0000","last_undegraded":"2024-09-12T12:23:19.114527+0000","last_fullsized":"2024-09-12T12:23:19.114527+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:22.787433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087210+0000","last_change":"2024-09-12T12:23:18.146972+0000","last_active":"2024-09-12T12:23:20.087210+0000","last_peered":"2024-09-12T12:23:20.087210+0000","last_clean":"2024-09-12T12:23:20.087210+0000","last_became_active":"2024-09-12T12:23:18.146448+0000","last_became_peered":"2024-09-12T12:23:18.146448+0000","last_unstale":"2024-09-12T12:23:20.087210+0000","last_undegraded":"2024-09-12T12:23:20.087210+0000","last_fullsized":"2024-09-12T12:23:20.087210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:10:48.681822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088971+0000","last_change":"2024-09-12T12:23:17.064861+0000","last_active":"2024-09-12T12:23:20.088971+0000","last_peered":"2024-09-12T12:23:20.088971+0000","last_clean":"2024-09-12T12:23:20.088971+0000","last_became_active":"2024-09-12T12:23:17.064157+0000","last_became_peered":"2024-09-12T12:23:17.064157+0000","last_unstale":"2024-09-12T12:23:20.088971+0000","last_undegraded":"2024-09-12T12:23:20.088971+0000","last_fullsized":"2024-09-12T12:23:20.088971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:55:46.970335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088485+0000","last_change":"2024-09-12T12:23:18.125443+0000","last_active":"2024-09-12T12:23:20.088485+0000","last_peered":"2024-09-12T12:23:20.088485+0000","last_clean":"2024-09-12T12:23:20.088485+0000","last_became_active":"2024-09-12T12:23:18.124857+0000","last_became_peered":"2024-09-12T12:23:18.124857+0000","last_unstale":"2024-09-12T12:23:20.088485+0000","last_undegraded":"2024-09-12T12:23:20.088485+0000","last_fullsized":"2024-09-12T12:23:20.088485+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:13:51.286037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638776+0000","last_change":"2024-09-12T12:23:17.087099+0000","last_active":"2024-09-12T12:23:19.638776+0000","last_peered":"2024-09-12T12:23:19.638776+0000","last_clean":"2024-09-12T12:23:19.638776+0000","last_became_active":"2024-09-12T12:23:17.086732+0000","last_became_peered":"2024-09-12T12:23:17.086732+0000","last_unstale":"2024-09-12T12:23:19.638776+0000","last_undegraded":"2024-09-12T12:23:19.638776+0000","last_fullsized":"2024-09-12T12:23:19.638776+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:57:30.109152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.109907+0000","last_change":"2024-09-12T12:23:18.120713+0000","last_active":"2024-09-12T12:23:19.109907+0000","last_peered":"2024-09-12T12:23:19.109907+0000","last_clean":"2024-09-12T12:23:19.109907+0000","last_became_active":"2024-09-12T12:23:18.120269+0000","last_became_peered":"2024-09-12T12:23:18.120269+0000","last_unstale":"2024-09-12T12:23:19.109907+0000","last_undegraded":"2024-09-12T12:23:19.109907+0000","last_fullsized":"2024-09-12T12:23:19.109907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:55.864333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.085082+0000","last_change":"2024-09-12T12:23:17.075577+0000","last_active":"2024-09-12T12:23:20.085082+0000","last_peered":"2024-09-12T12:23:20.085082+0000","last_clean":"2024-09-12T12:23:20.085082+0000","last_became_active":"2024-09-12T12:23:17.074914+0000","last_became_peered":"2024-09-12T12:23:17.074914+0000","last_unstale":"2024-09-12T12:23:20.085082+0000","last_undegraded":"2024-09-12T12:23:20.085082+0000","last_fullsized":"2024-09-12T12:23:20.085082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:40.879378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.085621+0000","last_change":"2024-09-12T12:23:18.125092+0000","last_active":"2024-09-12T12:23:20.085621+0000","last_peered":"2024-09-12T12:23:20.085621+0000","last_clean":"2024-09-12T12:23:20.085621+0000","last_became_active":"2024-09-12T12:23:18.124384+0000","last_became_peered":"2024-09-12T12:23:18.124384+0000","last_unstale":"2024-09-12T12:23:20.085621+0000","last_undegraded":"2024-09-12T12:23:20.085621+0000","last_fullsized":"2024-09-12T12:23:20.085621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:58.653722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084303+0000","last_change":"2024-09-12T12:23:17.081531+0000","last_active":"2024-09-12T12:23:20.084303+0000","last_peered":"2024-09-12T12:23:20.084303+0000","last_clean":"2024-09-12T12:23:20.084303+0000","last_became_active":"2024-09-12T12:23:17.081094+0000","last_became_peered":"2024-09-12T12:23:17.081094+0000","last_unstale":"2024-09-12T12:23:20.084303+0000","last_undegraded":"2024-09-12T12:23:20.084303+0000","last_fullsized":"2024-09-12T12:23:20.084303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:32.373585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.119014+0000","last_change":"2024-09-12T12:23:18.095356+0000","last_active":"2024-09-12T12:23:19.119014+0000","last_peered":"2024-09-12T12:23:19.119014+0000","last_clean":"2024-09-12T12:23:19.119014+0000","last_became_active":"2024-09-12T12:23:18.094657+0000","last_became_peered":"2024-09-12T12:23:18.094657+0000","last_unstale":"2024-09-12T12:23:19.119014+0000","last_undegraded":"2024-09-12T12:23:19.119014+0000","last_fullsized":"2024-09-12T12:23:19.119014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:52.855363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087961+0000","last_change":"2024-09-12T12:23:17.066884+0000","last_active":"2024-09-12T12:23:20.087961+0000","last_peered":"2024-09-12T12:23:20.087961+0000","last_clean":"2024-09-12T12:23:20.087961+0000","last_became_active":"2024-09-12T12:23:17.066181+0000","last_became_peered":"2024-09-12T12:23:17.066181+0000","last_unstale":"2024-09-12T12:23:20.087961+0000","last_undegraded":"2024-09-12T12:23:20.087961+0000","last_fullsized":"2024-09-12T12:23:20.087961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:37.847639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.121624+0000","last_change":"2024-09-12T12:23:18.141442+0000","last_active":"2024-09-12T12:23:19.121624+0000","last_peered":"2024-09-12T12:23:19.121624+0000","last_clean":"2024-09-12T12:23:19.121624+0000","last_became_active":"2024-09-12T12:23:18.141101+0000","last_became_peered":"2024-09-12T12:23:18.141101+0000","last_unstale":"2024-09-12T12:23:19.121624+0000","last_undegraded":"2024-09-12T12:23:19.121624+0000","last_fullsized":"2024-09-12T12:23:19.121624+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:06:05.138380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.118543+0000","last_change":"2024-09-12T12:23:17.080316+0000","last_active":"2024-09-12T12:23:19.118543+0000","last_peered":"2024-09-12T12:23:19.118543+0000","last_clean":"2024-09-12T12:23:19.118543+0000","last_became_active":"2024-09-12T12:23:17.079639+0000","last_became_peered":"2024-09-12T12:23:17.079639+0000","last_unstale":"2024-09-12T12:23:19.118543+0000","last_undegraded":"2024-09-12T12:23:19.118543+0000","last_fullsized":"2024-09-12T12:23:19.118543+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:38.684793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.108816+0000","last_change":"2024-09-12T12:23:18.116642+0000","last_active":"2024-09-12T12:23:19.108816+0000","last_peered":"2024-09-12T12:23:19.108816+0000","last_clean":"2024-09-12T12:23:19.108816+0000","last_became_active":"2024-09-12T12:23:18.110791+0000","last_became_peered":"2024-09-12T12:23:18.110791+0000","last_unstale":"2024-09-12T12:23:19.108816+0000","last_undegraded":"2024-09-12T12:23:19.108816+0000","last_fullsized":"2024-09-12T12:23:19.108816+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:41.206152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.112923+0000","last_change":"2024-09-12T12:23:17.091123+0000","last_active":"2024-09-12T12:23:19.112923+0000","last_peered":"2024-09-12T12:23:19.112923+0000","last_clean":"2024-09-12T12:23:19.112923+0000","last_became_active":"2024-09-12T12:23:17.090081+0000","last_became_peered":"2024-09-12T12:23:17.090081+0000","last_unstale":"2024-09-12T12:23:19.112923+0000","last_undegraded":"2024-09-12T12:23:19.112923+0000","last_fullsized":"2024-09-12T12:23:19.112923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:09:59.607300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.114664+0000","last_change":"2024-09-12T12:23:18.094943+0000","last_active":"2024-09-12T12:23:19.114664+0000","last_peered":"2024-09-12T12:23:19.114664+0000","last_clean":"2024-09-12T12:23:19.114664+0000","last_became_active":"2024-09-12T12:23:18.094228+0000","last_became_peered":"2024-09-12T12:23:18.094228+0000","last_unstale":"2024-09-12T12:23:19.114664+0000","last_undegraded":"2024-09-12T12:23:19.114664+0000","last_fullsized":"2024-09-12T12:23:19.114664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:48.067245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088787+0000","last_change":"2024-09-12T12:23:17.078894+0000","last_active":"2024-09-12T12:23:20.088787+0000","last_peered":"2024-09-12T12:23:20.088787+0000","last_clean":"2024-09-12T12:23:20.088787+0000","last_became_active":"2024-09-12T12:23:17.077021+0000","last_became_peered":"2024-09-12T12:23:17.077021+0000","last_unstale":"2024-09-12T12:23:20.088787+0000","last_undegraded":"2024-09-12T12:23:20.088787+0000","last_fullsized":"2024-09-12T12:23:20.088787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:08:08.414320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087579+0000","last_change":"2024-09-12T12:23:18.140296+0000","last_active":"2024-09-12T12:23:20.087579+0000","last_peered":"2024-09-12T12:23:20.087579+0000","last_clean":"2024-09-12T12:23:20.087579+0000","last_became_active":"2024-09-12T12:23:18.137244+0000","last_became_peered":"2024-09-12T12:23:18.137244+0000","last_unstale":"2024-09-12T12:23:20.087579+0000","last_undegraded":"2024-09-12T12:23:20.087579+0000","last_fullsized":"2024-09-12T12:23:20.087579+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:29:34.898445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.114708+0000","last_change":"2024-09-12T12:23:17.090669+0000","last_active":"2024-09-12T12:23:19.114708+0000","last_peered":"2024-09-12T12:23:19.114708+0000","last_clean":"2024-09-12T12:23:19.114708+0000","last_became_active":"2024-09-12T12:23:17.089910+0000","last_became_peered":"2024-09-12T12:23:17.089910+0000","last_unstale":"2024-09-12T12:23:19.114708+0000","last_undegraded":"2024-09-12T12:23:19.114708+0000","last_fullsized":"2024-09-12T12:23:19.114708+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:06.398062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084964+0000","last_change":"2024-09-12T12:23:18.118697+0000","last_active":"2024-09-12T12:23:20.084964+0000","last_peered":"2024-09-12T12:23:20.084964+0000","last_clean":"2024-09-12T12:23:20.084964+0000","last_became_active":"2024-09-12T12:23:18.118440+0000","last_became_peered":"2024-09-12T12:23:18.118440+0000","last_unstale":"2024-09-12T12:23:20.084964+0000","last_undegraded":"2024-09-12T12:23:20.084964+0000","last_fullsized":"2024-09-12T12:23:20.084964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:25:07.406774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088094+0000","last_change":"2024-09-12T12:23:17.093031+0000","last_active":"2024-09-12T12:23:20.088094+0000","last_peered":"2024-09-12T12:23:20.088094+0000","last_clean":"2024-09-12T12:23:20.088094+0000","last_became_active":"2024-09-12T12:23:17.092495+0000","last_became_peered":"2024-09-12T12:23:17.092495+0000","last_unstale":"2024-09-12T12:23:20.088094+0000","last_undegraded":"2024-09-12T12:23:20.088094+0000","last_fullsized":"2024-09-12T12:23:20.088094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:33:40.879378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.109923+0000","last_change":"2024-09-12T12:23:18.110240+0000","last_active":"2024-09-12T12:23:19.109923+0000","last_peered":"2024-09-12T12:23:19.109923+0000","last_clean":"2024-09-12T12:23:19.109923+0000","last_became_active":"2024-09-12T12:23:18.109927+0000","last_became_peered":"2024-09-12T12:23:18.109927+0000","last_unstale":"2024-09-12T12:23:19.109923+0000","last_undegraded":"2024-09-12T12:23:19.109923+0000","last_fullsized":"2024-09-12T12:23:19.109923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:30.481042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.086337+0000","last_change":"2024-09-12T12:23:17.096451+0000","last_active":"2024-09-12T12:23:20.086337+0000","last_peered":"2024-09-12T12:23:20.086337+0000","last_clean":"2024-09-12T12:23:20.086337+0000","last_became_active":"2024-09-12T12:23:17.095796+0000","last_became_peered":"2024-09-12T12:23:17.095796+0000","last_unstale":"2024-09-12T12:23:20.086337+0000","last_undegraded":"2024-09-12T12:23:20.086337+0000","last_fullsized":"2024-09-12T12:23:20.086337+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:53:50.159867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638484+0000","last_change":"2024-09-12T12:23:18.114079+0000","last_active":"2024-09-12T12:23:19.638484+0000","last_peered":"2024-09-12T12:23:19.638484+0000","last_clean":"2024-09-12T12:23:19.638484+0000","last_became_active":"2024-09-12T12:23:18.113666+0000","last_became_peered":"2024-09-12T12:23:18.113666+0000","last_unstale":"2024-09-12T12:23:19.638484+0000","last_undegraded":"2024-09-12T12:23:19.638484+0000","last_fullsized":"2024-09-12T12:23:19.638484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:44:27.759988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088217+0000","last_change":"2024-09-12T12:23:17.078879+0000","last_active":"2024-09-12T12:23:20.088217+0000","last_peered":"2024-09-12T12:23:20.088217+0000","last_clean":"2024-09-12T12:23:20.088217+0000","last_became_active":"2024-09-12T12:23:17.076836+0000","last_became_peered":"2024-09-12T12:23:17.076836+0000","last_unstale":"2024-09-12T12:23:20.088217+0000","last_undegraded":"2024-09-12T12:23:20.088217+0000","last_fullsized":"2024-09-12T12:23:20.088217+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:11:29.862991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638284+0000","last_change":"2024-09-12T12:23:18.133120+0000","last_active":"2024-09-12T12:23:19.638284+0000","last_peered":"2024-09-12T12:23:19.638284+0000","last_clean":"2024-09-12T12:23:19.638284+0000","last_became_active":"2024-09-12T12:23:18.132824+0000","last_became_peered":"2024-09-12T12:23:18.132824+0000","last_unstale":"2024-09-12T12:23:19.638284+0000","last_undegraded":"2024-09-12T12:23:19.638284+0000","last_fullsized":"2024-09-12T12:23:19.638284+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:16:38.941861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.118840+0000","last_change":"2024-09-12T12:23:17.067441+0000","last_active":"2024-09-12T12:23:19.118840+0000","last_peered":"2024-09-12T12:23:19.118840+0000","last_clean":"2024-09-12T12:23:19.118840+0000","last_became_active":"2024-09-12T12:23:17.066728+0000","last_became_peered":"2024-09-12T12:23:17.066728+0000","last_unstale":"2024-09-12T12:23:19.118840+0000","last_undegraded":"2024-09-12T12:23:19.118840+0000","last_fullsized":"2024-09-12T12:23:19.118840+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:13:50.277324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638299+0000","last_change":"2024-09-12T12:23:18.126379+0000","last_active":"2024-09-12T12:23:19.638299+0000","last_peered":"2024-09-12T12:23:19.638299+0000","last_clean":"2024-09-12T12:23:19.638299+0000","last_became_active":"2024-09-12T12:23:18.126047+0000","last_became_peered":"2024-09-12T12:23:18.126047+0000","last_unstale":"2024-09-12T12:23:19.638299+0000","last_undegraded":"2024-09-12T12:23:19.638299+0000","last_fullsized":"2024-09-12T12:23:19.638299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:17.097787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.121335+0000","last_change":"2024-09-12T12:23:17.078895+0000","last_active":"2024-09-12T12:23:19.121335+0000","last_peered":"2024-09-12T12:23:19.121335+0000","last_clean":"2024-09-12T12:23:19.121335+0000","last_became_active":"2024-09-12T12:23:17.078633+0000","last_became_peered":"2024-09-12T12:23:17.078633+0000","last_unstale":"2024-09-12T12:23:19.121335+0000","last_undegraded":"2024-09-12T12:23:19.121335+0000","last_fullsized":"2024-09-12T12:23:19.121335+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:04:17.838363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.121874+0000","last_change":"2024-09-12T12:23:18.117656+0000","last_active":"2024-09-12T12:23:19.121874+0000","last_peered":"2024-09-12T12:23:19.121874+0000","last_clean":"2024-09-12T12:23:19.121874+0000","last_became_active":"2024-09-12T12:23:18.117341+0000","last_became_peered":"2024-09-12T12:23:18.117341+0000","last_unstale":"2024-09-12T12:23:19.121874+0000","last_undegraded":"2024-09-12T12:23:19.121874+0000","last_fullsized":"2024-09-12T12:23:19.121874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:52.372886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088994+0000","last_change":"2024-09-12T12:23:17.077229+0000","last_active":"2024-09-12T12:23:20.088994+0000","last_peered":"2024-09-12T12:23:20.088994+0000","last_clean":"2024-09-12T12:23:20.088994+0000","last_became_active":"2024-09-12T12:23:17.076051+0000","last_became_peered":"2024-09-12T12:23:17.076051+0000","last_unstale":"2024-09-12T12:23:20.088994+0000","last_undegraded":"2024-09-12T12:23:20.088994+0000","last_fullsized":"2024-09-12T12:23:20.088994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:23.664767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.119421+0000","last_change":"2024-09-12T12:23:18.117110+0000","last_active":"2024-09-12T12:23:19.119421+0000","last_peered":"2024-09-12T12:23:19.119421+0000","last_clean":"2024-09-12T12:23:19.119421+0000","last_became_active":"2024-09-12T12:23:18.116415+0000","last_became_peered":"2024-09-12T12:23:18.116415+0000","last_unstale":"2024-09-12T12:23:19.119421+0000","last_undegraded":"2024-09-12T12:23:19.119421+0000","last_fullsized":"2024-09-12T12:23:19.119421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:20:59.607004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.119575+0000","last_change":"2024-09-12T12:23:17.079856+0000","last_active":"2024-09-12T12:23:19.119575+0000","last_peered":"2024-09-12T12:23:19.119575+0000","last_clean":"2024-09-12T12:23:19.119575+0000","last_became_active":"2024-09-12T12:23:17.079508+0000","last_became_peered":"2024-09-12T12:23:17.079508+0000","last_unstale":"2024-09-12T12:23:19.119575+0000","last_undegraded":"2024-09-12T12:23:19.119575+0000","last_fullsized":"2024-09-12T12:23:19.119575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:17.275644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.086259+0000","last_change":"2024-09-12T12:23:18.143138+0000","last_active":"2024-09-12T12:23:20.086259+0000","last_peered":"2024-09-12T12:23:20.086259+0000","last_clean":"2024-09-12T12:23:20.086259+0000","last_became_active":"2024-09-12T12:23:18.142610+0000","last_became_peered":"2024-09-12T12:23:18.142610+0000","last_unstale":"2024-09-12T12:23:20.086259+0000","last_undegraded":"2024-09-12T12:23:20.086259+0000","last_fullsized":"2024-09-12T12:23:20.086259+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:14:42.203627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.110025+0000","last_change":"2024-09-12T12:23:17.076124+0000","last_active":"2024-09-12T12:23:19.110025+0000","last_peered":"2024-09-12T12:23:19.110025+0000","last_clean":"2024-09-12T12:23:19.110025+0000","last_became_active":"2024-09-12T12:23:17.072753+0000","last_became_peered":"2024-09-12T12:23:17.072753+0000","last_unstale":"2024-09-12T12:23:19.110025+0000","last_undegraded":"2024-09-12T12:23:19.110025+0000","last_fullsized":"2024-09-12T12:23:19.110025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:30.218645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.086729+0000","last_change":"2024-09-12T12:23:18.132791+0000","last_active":"2024-09-12T12:23:20.086729+0000","last_peered":"2024-09-12T12:23:20.086729+0000","last_clean":"2024-09-12T12:23:20.086729+0000","last_became_active":"2024-09-12T12:23:18.131803+0000","last_became_peered":"2024-09-12T12:23:18.131803+0000","last_unstale":"2024-09-12T12:23:20.086729+0000","last_undegraded":"2024-09-12T12:23:20.086729+0000","last_fullsized":"2024-09-12T12:23:20.086729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:26.032801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.122234+0000","last_change":"2024-09-12T12:23:17.090571+0000","last_active":"2024-09-12T12:23:19.122234+0000","last_peered":"2024-09-12T12:23:19.122234+0000","last_clean":"2024-09-12T12:23:19.122234+0000","last_became_active":"2024-09-12T12:23:17.090230+0000","last_became_peered":"2024-09-12T12:23:17.090230+0000","last_unstale":"2024-09-12T12:23:19.122234+0000","last_undegraded":"2024-09-12T12:23:19.122234+0000","last_fullsized":"2024-09-12T12:23:19.122234+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:27:57.508461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.109605+0000","last_change":"2024-09-12T12:23:18.117121+0000","last_active":"2024-09-12T12:23:19.109605+0000","last_peered":"2024-09-12T12:23:19.109605+0000","last_clean":"2024-09-12T12:23:19.109605+0000","last_became_active":"2024-09-12T12:23:18.116766+0000","last_became_peered":"2024-09-12T12:23:18.116766+0000","last_unstale":"2024-09-12T12:23:19.109605+0000","last_undegraded":"2024-09-12T12:23:19.109605+0000","last_fullsized":"2024-09-12T12:23:19.109605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:51.453100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.109694+0000","last_change":"2024-09-12T12:23:17.073326+0000","last_active":"2024-09-12T12:23:19.109694+0000","last_peered":"2024-09-12T12:23:19.109694+0000","last_clean":"2024-09-12T12:23:19.109694+0000","last_became_active":"2024-09-12T12:23:17.072707+0000","last_became_peered":"2024-09-12T12:23:17.072707+0000","last_unstale":"2024-09-12T12:23:19.109694+0000","last_undegraded":"2024-09-12T12:23:19.109694+0000","last_fullsized":"2024-09-12T12:23:19.109694+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:14:30.850681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088072+0000","last_change":"2024-09-12T12:23:18.135876+0000","last_active":"2024-09-12T12:23:20.088072+0000","last_peered":"2024-09-12T12:23:20.088072+0000","last_clean":"2024-09-12T12:23:20.088072+0000","last_became_active":"2024-09-12T12:23:18.133318+0000","last_became_peered":"2024-09-12T12:23:18.133318+0000","last_unstale":"2024-09-12T12:23:20.088072+0000","last_undegraded":"2024-09-12T12:23:20.088072+0000","last_fullsized":"2024-09-12T12:23:20.088072+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:00.873678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084053+0000","last_change":"2024-09-12T12:23:17.089441+0000","last_active":"2024-09-12T12:23:20.084053+0000","last_peered":"2024-09-12T12:23:20.084053+0000","last_clean":"2024-09-12T12:23:20.084053+0000","last_became_active":"2024-09-12T12:23:17.089223+0000","last_became_peered":"2024-09-12T12:23:17.089223+0000","last_unstale":"2024-09-12T12:23:20.084053+0000","last_undegraded":"2024-09-12T12:23:20.084053+0000","last_fullsized":"2024-09-12T12:23:20.084053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:55:46.970335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.084301+0000","last_change":"2024-09-12T12:23:18.097514+0000","last_active":"2024-09-12T12:23:20.084301+0000","last_peered":"2024-09-12T12:23:20.084301+0000","last_clean":"2024-09-12T12:23:20.084301+0000","last_became_active":"2024-09-12T12:23:18.096767+0000","last_became_peered":"2024-09-12T12:23:18.096767+0000","last_unstale":"2024-09-12T12:23:20.084301+0000","last_undegraded":"2024-09-12T12:23:20.084301+0000","last_fullsized":"2024-09-12T12:23:20.084301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:10:00.616013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.112845+0000","last_change":"2024-09-12T12:23:17.091347+0000","last_active":"2024-09-12T12:23:19.112845+0000","last_peered":"2024-09-12T12:23:19.112845+0000","last_clean":"2024-09-12T12:23:19.112845+0000","last_became_active":"2024-09-12T12:23:17.090341+0000","last_became_peered":"2024-09-12T12:23:17.090341+0000","last_unstale":"2024-09-12T12:23:19.112845+0000","last_undegraded":"2024-09-12T12:23:19.112845+0000","last_fullsized":"2024-09-12T12:23:19.112845+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:31.620614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.087109+0000","last_change":"2024-09-12T12:23:18.129831+0000","last_active":"2024-09-12T12:23:20.087109+0000","last_peered":"2024-09-12T12:23:20.087109+0000","last_clean":"2024-09-12T12:23:20.087109+0000","last_became_active":"2024-09-12T12:23:18.128936+0000","last_became_peered":"2024-09-12T12:23:18.128936+0000","last_unstale":"2024-09-12T12:23:20.087109+0000","last_undegraded":"2024-09-12T12:23:20.087109+0000","last_fullsized":"2024-09-12T12:23:20.087109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:47:58.242703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088721+0000","last_change":"2024-09-12T12:23:17.076822+0000","last_active":"2024-09-12T12:23:20.088721+0000","last_peered":"2024-09-12T12:23:20.088721+0000","last_clean":"2024-09-12T12:23:20.088721+0000","last_became_active":"2024-09-12T12:23:17.075087+0000","last_became_peered":"2024-09-12T12:23:17.075087+0000","last_unstale":"2024-09-12T12:23:20.088721+0000","last_undegraded":"2024-09-12T12:23:20.088721+0000","last_fullsized":"2024-09-12T12:23:20.088721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:23.229208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:19.638586+0000","last_change":"2024-09-12T12:23:18.126382+0000","last_active":"2024-09-12T12:23:19.638586+0000","last_peered":"2024-09-12T12:23:19.638586+0000","last_clean":"2024-09-12T12:23:19.638586+0000","last_became_active":"2024-09-12T12:23:18.126018+0000","last_became_peered":"2024-09-12T12:23:18.126018+0000","last_unstale":"2024-09-12T12:23:19.638586+0000","last_undegraded":"2024-09-12T12:23:19.638586+0000","last_fullsized":"2024-09-12T12:23:19.638586+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:17.044485+0000","last_clean_scrub_stamp":"2024-09-12T12:23:17.044485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:43.460289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.088549+0000","last_change":"2024-09-12T12:23:17.076005+0000","last_active":"2024-09-12T12:23:20.088549+0000","last_peered":"2024-09-12T12:23:20.088549+0000","last_clean":"2024-09-12T12:23:20.088549+0000","last_became_active":"2024-09-12T12:23:17.074864+0000","last_became_peered":"2024-09-12T12:23:17.074864+0000","last_unstale":"2024-09-12T12:23:20.088549+0000","last_undegraded":"2024-09-12T12:23:20.088549+0000","last_fullsized":"2024-09-12T12:23:20.088549+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:16.035772+0000","last_clean_scrub_stamp":"2024-09-12T12:23:16.035772+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:45:42.789727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-12T12:23:20.085472+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:20.085472+0000","last_peered":"2024-09-12T12:23:20.085472+0000","last_clean":"2024-09-12T12:23:20.085472+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:20.085472+0000","last_undegraded":"2024-09-12T12:23:20.085472+0000","last_fullsized":"2024-09-12T12:23:20.085472+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":806912,"data_stored":333473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":924345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":333453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542192,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":802816,"data_stored":333451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":333431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":811008,"data_stored":334973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":335589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1392640,"data_stored":923729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:23:26.994 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:23:26.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T12:23:27.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.168+0000 7f14502bc640 1 Processor -- start 2024-09-12T12:23:27.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.168+0000 7f14502bc640 1 -- start start 2024-09-12T12:23:27.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.168+0000 7f14502bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f144806e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.168+0000 7f14502bc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14480d0cd0 0x7f144815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.168+0000 7f14502bc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1448160600 con 0x7f14480adf60 2024-09-12T12:23:27.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.168+0000 7f14502bc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f14481608e0 con 0x7f14480d0cd0 2024-09-12T12:23:27.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.168+0000 7f14502bc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1448160bc0 con 0x7f144815c210 2024-09-12T12:23:27.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14480d0cd0 0x7f144815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f14480d0cd0 0x7f144815feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38548/0 (socket says 172.21.15.3:38548) 2024-09-12T12:23:27.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 -- 172.21.15.3:0/3846273839 learned_addr learned my addr 172.21.15.3:0/3846273839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:27.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e058640 1 --2- 172.21.15.3:0/3846273839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f144806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 -- 172.21.15.3:0/3846273839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 msgr2=0x7f144806e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:27.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 --2- 172.21.15.3:0/3846273839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f144806e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 -- 172.21.15.3:0/3846273839 >> v1:172.21.15.142:6789/0 conn(0x7f14480adf60 legacy=0x7f14480d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:27.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 -- 172.21.15.3:0/3846273839 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14480cec00 con 0x7f14480d0cd0 2024-09-12T12:23:27.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e058640 1 --2- 172.21.15.3:0/3846273839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f144806e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144e859640 1 --2- 172.21.15.3:0/3846273839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f144815feb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1440009870 tx=0x7f144001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20e6b6359223c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144d056640 1 -- 172.21.15.3:0/3846273839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400b0070 con 0x7f14480d0cd0 2024-09-12T12:23:27.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144d056640 1 -- 172.21.15.3:0/3846273839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14400ab600 con 0x7f14480d0cd0 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.172+0000 7f144d056640 1 -- 172.21.15.3:0/3846273839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14400ab9d0 con 0x7f14480d0cd0 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- 172.21.15.3:0/3846273839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 msgr2=0x7f144815feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 --2- 172.21.15.3:0/3846273839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f144815feb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1440009870 tx=0x7f144001c400 comp rx=0 tx=0).stop 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- 172.21.15.3:0/3846273839 shutdown_connections 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 --2- 172.21.15.3:0/3846273839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f144815feb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 --2- 172.21.15.3:0/3846273839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f144806e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- 172.21.15.3:0/3846273839 >> 172.21.15.3:0/3846273839 conn(0x7f1448067e00 msgr2=0x7f1448068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- 172.21.15.3:0/3846273839 shutdown_connections 2024-09-12T12:23:27.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- 172.21.15.3:0/3846273839 wait complete. 2024-09-12T12:23:27.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 Processor -- start 2024-09-12T12:23:27.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- start start 2024-09-12T12:23:27.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14480adf60 0x7f144815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f1448159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f1448072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14480cfcc0 con 0x7f14480d0cd0 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f14480cfb40 con 0x7f14480adf60 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f14502bc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f14480cfe40 con 0x7f144815c210 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f1448159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f1448159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38560/0 (socket says 172.21.15.3:38560) 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 -- 172.21.15.3:0/217124322 learned_addr learned my addr 172.21.15.3:0/217124322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144e058640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14480adf60 0x7f144815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144e859640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f1448072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 -- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 msgr2=0x7f1448072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f1448072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 -- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14480adf60 msgr2=0x7f144815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14480adf60 0x7f144815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 -- 172.21.15.3:0/217124322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144815a2e0 con 0x7f14480d0cd0 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144e859640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f1448072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144e058640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14480adf60 0x7f144815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f144d857640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f1448159be0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f144400c7a0 tx=0x7f144400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.176+0000 7f143affd640 1 -- 172.21.15.3:0/217124322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444016020 con 0x7f14480d0cd0 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f143affd640 1 -- 172.21.15.3:0/217124322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144400aa50 con 0x7f14480d0cd0 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144815a5c0 con 0x7f14480d0cd0 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f143affd640 1 -- 172.21.15.3:0/217124322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444012420 con 0x7f14480d0cd0 2024-09-12T12:23:27.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144815ab80 con 0x7f14480d0cd0 2024-09-12T12:23:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144806e450 con 0x7f14480d0cd0 2024-09-12T12:23:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f143affd640 1 -- 172.21.15.3:0/217124322 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1444019020 con 0x7f14480d0cd0 2024-09-12T12:23:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f143affd640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f142c0678c0 0x7f142c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f143affd640 1 -- 172.21.15.3:0/217124322 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f1444085e00 con 0x7f14480d0cd0 2024-09-12T12:23:27.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f144e058640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f142c0678c0 0x7f142c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.180+0000 7f144e058640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f142c0678c0 0x7f142c069d80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f143c008890 tx=0x7f143c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:27.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.184+0000 7f143affd640 1 -- 172.21.15.3:0/217124322 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1444020030 con 0x7f14480d0cd0 2024-09-12T12:23:27.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.412+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f14480ce840 con 0x7f14480d0cd0 2024-09-12T12:23:27.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.416+0000 7f143affd640 1 -- 172.21.15.3:0/217124322 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f14440568b0 con 0x7f14480d0cd0 2024-09-12T12:23:27.419 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T12:23:27.419 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:23:27.419 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-12T12:23:27.419 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-12T12:23:27.419 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T12:23:27.420 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-12T12:23:27.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f142c0678c0 msgr2=0x7f142c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f142c0678c0 0x7f142c069d80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f143c008890 tx=0x7f143c008040 comp rx=0 tx=0).stop 2024-09-12T12:23:27.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 msgr2=0x7f1448159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f1448159be0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f144400c7a0 tx=0x7f144400cc60 comp rx=0 tx=0).stop 2024-09-12T12:23:27.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 shutdown_connections 2024-09-12T12:23:27.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f142c0678c0 0x7f142c069d80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f144815c210 0x7f1448072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14480d0cd0 0x7f1448159be0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 --2- 172.21.15.3:0/217124322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14480adf60 0x7f144815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 >> 172.21.15.3:0/217124322 conn(0x7f1448067e00 msgr2=0x7f1448166210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:27.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 shutdown_connections 2024-09-12T12:23:27.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.420+0000 7f14502bc640 1 -- 172.21.15.3:0/217124322 wait complete. 2024-09-12T12:23:27.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:27.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f599b636640 1 Processor -- start 2024-09-12T12:23:27.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f599b636640 1 -- start start 2024-09-12T12:23:27.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f599b636640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 0x7f599415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f599b636640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940b5000 con 0x7f59940ae880 2024-09-12T12:23:27.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f599b636640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59940b3500 con 0x7f59940b30e0 2024-09-12T12:23:27.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f599b636640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5994074090 con 0x7f5994073cb0 2024-09-12T12:23:27.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5999bd3640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59940ae880 0x7f59940d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:47156/0 (socket says 172.21.15.3:47156) 2024-09-12T12:23:27.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5999bd3640 1 -- 172.21.15.3:0/1055328092 learned_addr learned my addr 172.21.15.3:0/1055328092 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:27.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f59993d2640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 0x7f599415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591324385 0 0) 0x7f59940b5000 con 0x7f59940ae880 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5980003530 con 0x7f59940ae880 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 785738120 0 0) 0x7f5980003530 con 0x7f59940ae880 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 msgr2=0x7f599415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 0x7f599415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 >> v1:172.21.15.142:6789/0 conn(0x7f59940b30e0 legacy=0x7f59940adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940b37c0 con 0x7f59940ae880 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f598c001a00 con 0x7f59940ae880 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 >> v1:172.21.15.3:6789/0 conn(0x7f59940ae880 legacy=0x7f59940d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 0x7f59800039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f5980004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5980004d50 0x7f59800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f598000ba80 con 0x7f5980003f70 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f598000bc30 con 0x7f5980004d50 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f598000be30 con 0x7f5994073cb0 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5998bd1640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f5980004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5999bd3640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5980004d50 0x7f59800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f59993d2640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 0x7f59800039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5998bd1640 1 -- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 msgr2=0x7f59800039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5998bd1640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5994073cb0 0x7f59800039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5998bd1640 1 -- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5980004d50 msgr2=0x7f59800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5998bd1640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5980004d50 0x7f59800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.556+0000 7f5998bd1640 1 -- 172.21.15.3:0/1055328092 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5980003530 con 0x7f5980003f70 2024-09-12T12:23:27.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f5998bd1640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f5980004810 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5984001780 tx=0x7f598401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14d132d711b3ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840013a0 con 0x7f5980003f70 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59840052f0 con 0x7f5980003f70 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f598bfff640 1 -- 172.21.15.3:0/1055328092 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984003840 con 0x7f5980003f70 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 -- 172.21.15.3:0/1055328092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 msgr2=0x7f5980004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 --2- 172.21.15.3:0/1055328092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f5980004810 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5984001780 tx=0x7f598401c420 comp rx=0 tx=0).stop 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f5999bd3640 1 -- 172.21.15.3:0/1055328092 reap_dead start 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 -- 172.21.15.3:0/1055328092 shutdown_connections 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 -- 172.21.15.3:0/1055328092 >> 172.21.15.3:0/1055328092 conn(0x7f5994067e00 msgr2=0x7f5994068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 -- 172.21.15.3:0/1055328092 shutdown_connections 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 -- 172.21.15.3:0/1055328092 wait complete. 2024-09-12T12:23:27.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 Processor -- start 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.560+0000 7f599b636640 1 -- start start 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f59941f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f599409a1d0 0x7f599414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59940cec60 0x7f59940cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59940ad6f0 con 0x7f5980003f70 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59940b3d80 con 0x7f599409a1d0 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f59940ae320 con 0x7f59940cec60 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f59941f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f59941f2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38580/0 (socket says 172.21.15.3:38580) 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 -- 172.21.15.3:0/3003589080 learned_addr learned my addr 172.21.15.3:0/3003589080 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5999bd3640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59940cec60 0x7f59940cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5998bd1640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f599409a1d0 0x7f599414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 -- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59940cec60 msgr2=0x7f59940cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59940cec60 0x7f59940cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 -- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f599409a1d0 msgr2=0x7f599414d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f599409a1d0 0x7f599414d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 -- 172.21.15.3:0/3003589080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940b1190 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5999bd3640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59940cec60 0x7f59940cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f59993d2640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f59941f2410 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59900115e0 tx=0x7f5990011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5989ffb640 1 -- 172.21.15.3:0/3003589080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599000f8f0 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5989ffb640 1 -- 172.21.15.3:0/3003589080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599000fa90 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59940af710 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5989ffb640 1 -- 172.21.15.3:0/3003589080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599000fe60 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599409a990 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599415c710 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5989ffb640 1 -- 172.21.15.3:0/3003589080 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5990015460 con 0x7f5980003f70 2024-09-12T12:23:27.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5989ffb640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59780678f0 0x7f5978069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5989ffb640 1 -- 172.21.15.3:0/3003589080 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f59900865a0 con 0x7f5980003f70 2024-09-12T12:23:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.564+0000 7f5998bd1640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59780678f0 0x7f5978069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.568+0000 7f5998bd1640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59780678f0 0x7f5978069db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5984001750 tx=0x7f59840011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:27.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.568+0000 7f5989ffb640 1 -- 172.21.15.3:0/3003589080 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f59940b1190 con 0x7f5980003f70 2024-09-12T12:23:27.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.772+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5994068340 con 0x7f5980003f70 2024-09-12T12:23:27.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.772+0000 7f5989ffb640 1 -- 172.21.15.3:0/3003589080 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f5990019030 con 0x7f5980003f70 2024-09-12T12:23:27.776 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:27.777 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:17.423558+0000","modified":"2024-09-12T12:23:20.482181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/550473013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":550473013},{"type":"v1","addr":"172.21.15.3:6835","nonce":550473013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6012,"qdb_cluster":[6012]},"id":5}]} 2024-09-12T12:23:27.777 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-12T12:23:27.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59780678f0 msgr2=0x7f5978069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59780678f0 0x7f5978069db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5984001750 tx=0x7f59840011a0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 msgr2=0x7f59941f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f59941f2410 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f59900115e0 tx=0x7f5990011aa0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 shutdown_connections 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f59780678f0 0x7f5978069db0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f59940cec60 0x7f59940cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f599409a1d0 0x7f599414d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 --2- 172.21.15.3:0/3003589080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5980003f70 0x7f59941f2410 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 >> 172.21.15.3:0/3003589080 conn(0x7f5994067e00 msgr2=0x7f599415ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 shutdown_connections 2024-09-12T12:23:27.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:27.776+0000 7f599b636640 1 -- 172.21.15.3:0/3003589080 wait complete. 2024-09-12T12:23:28.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:23:17.423558+0000', 'modified': '2024-09-12T12:23:20.482181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6012}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6012': {'gid': 6012, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/550473013', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6833', 'nonce': 550473013}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 550473013}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6012, 'qdb_cluster': [6012]} 2024-09-12T12:23:28.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:23:28.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T12:23:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.332+0000 7f02d3ee7640 1 Processor -- start 2024-09-12T12:23:28.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.332+0000 7f02d3ee7640 1 -- start start 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d3ee7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f02cc0b7740 0x7f02cc0b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d3ee7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02cc0d30b0 con 0x7f02cc0d4150 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d3ee7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02cc0d3390 con 0x7f02cc0d1190 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d3ee7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f02cc0d45e0 con 0x7f02cc0b7740 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1482640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f02cc0d4150 0x7f02cc0cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49682/0 (socket says 172.21.15.3:49682) 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1482640 1 -- 172.21.15.3:0/2445978013 learned_addr learned my addr 172.21.15.3:0/2445978013 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059682300 0 0) 0x7f02cc0d30b0 con 0x7f02cc0d4150 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02b8003530 con 0x7f02cc0d4150 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d2484640 1 --2- 172.21.15.3:0/2445978013 >> v2:172.21.15.142:3300/0 conn(0x7f02cc0b7740 0x7f02cc0b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603746897 0 0) 0x7f02b8003530 con 0x7f02cc0d4150 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 >> v2:172.21.15.142:3300/0 conn(0x7f02cc0b7740 msgr2=0x7f02cc0b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 --2- 172.21.15.3:0/2445978013 >> v2:172.21.15.142:3300/0 conn(0x7f02cc0b7740 0x7f02cc0b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 >> v1:172.21.15.142:6789/0 conn(0x7f02cc0d1190 legacy=0x7f02cc0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc0d3670 con 0x7f02cc0d4150 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f02c8001680 con 0x7f02cc0d4150 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 >> v1:172.21.15.3:6789/0 conn(0x7f02cc0d4150 legacy=0x7f02cc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b80039c0 0x7f02b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b8004810 0x7f02b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80051d0 0x7f02b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc0d45e0 con 0x7f02b80039c0 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02b800c080 con 0x7f02b8004810 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02b800c280 con 0x7f02b80051d0 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b80039c0 0x7f02b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1c83640 1 -- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80051d0 msgr2=0x7f02b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80051d0 0x7f02b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1c83640 1 -- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b8004810 msgr2=0x7f02b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1482640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b8004810 0x7f02b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b8004810 0x7f02b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d2484640 1 -- 172.21.15.3:0/2445978013 reap_dead start 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1c83640 1 -- 172.21.15.3:0/2445978013 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b8003530 con 0x7f02b80039c0 2024-09-12T12:23:28.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.336+0000 7f02d1482640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02b8004810 0x7f02b8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b80039c0 0x7f02b8003de0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f02bc001780 tx=0x7f02bc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da95db3d5e8548fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0013a0 con 0x7f02b80039c0 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02bc0052f0 con 0x7f02b80039c0 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d0c81640 1 -- 172.21.15.3:0/2445978013 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc003840 con 0x7f02b80039c0 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2445978013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b80039c0 msgr2=0x7f02b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b80039c0 0x7f02b8003de0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f02bc001780 tx=0x7f02bc01c400 comp rx=0 tx=0).stop 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2445978013 shutdown_connections 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2445978013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02b80039c0 0x7f02b8003de0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2445978013 >> 172.21.15.3:0/2445978013 conn(0x7f02cc067a00 msgr2=0x7f02cc067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2445978013 shutdown_connections 2024-09-12T12:23:28.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2445978013 wait complete. 2024-09-12T12:23:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 Processor -- start 2024-09-12T12:23:28.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- start start 2024-09-12T12:23:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80039c0 0x7f02cc15f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc15fd20 0x7f02cc1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02cc0b5310 0x7f02cc0b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc0d2cd0 con 0x7f02cc0b5310 2024-09-12T12:23:28.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc0d3cc0 con 0x7f02cc15fd20 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d3ee7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02cc0d2e50 con 0x7f02b80039c0 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02cc0b5310 0x7f02cc0b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02cc0b5310 0x7f02cc0b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37108/0 (socket says 172.21.15.3:37108) 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 -- 172.21.15.3:0/2851893089 learned_addr learned my addr 172.21.15.3:0/2851893089 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80039c0 0x7f02cc15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d1482640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc15fd20 0x7f02cc1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 -- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80039c0 msgr2=0x7f02cc15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80039c0 0x7f02cc15f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 -- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc15fd20 msgr2=0x7f02cc1550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc15fd20 0x7f02cc1550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d2484640 1 -- 172.21.15.3:0/2851893089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02cc1604a0 con 0x7f02cc0b5310 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.340+0000 7f02d1482640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc15fd20 0x7f02cc1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d2484640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02cc0b5310 0x7f02cc0b5790 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0d05e0 tx=0x7f02c400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02c27fc640 1 -- 172.21.15.3:0/2851893089 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400ba80 con 0x7f02cc0b5310 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02c27fc640 1 -- 172.21.15.3:0/2851893089 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c401a070 con 0x7f02cc0b5310 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02c27fc640 1 -- 172.21.15.3:0/2851893089 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c400be40 con 0x7f02cc0b5310 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02cc160780 con 0x7f02cc0b5310 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02cc160d40 con 0x7f02cc0b5310 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02c27fc640 1 -- 172.21.15.3:0/2851893089 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f02c4013070 con 0x7f02cc0b5310 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02c27fc640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f02b00678f0 0x7f02b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02c27fc640 1 -- 172.21.15.3:0/2851893089 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f02c4085d80 con 0x7f02cc0b5310 2024-09-12T12:23:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f02b00678f0 0x7f02b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 Processor -- start 2024-09-12T12:23:28.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 -- start start 2024-09-12T12:23:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f02b00678f0 0x7f02b0069db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02bc004000 tx=0x7f02bc0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28daff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28db0440 0x55cc28d65d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28d66290 0x55cc28d66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cc28c4db60 con 0x55cc28d66290 2024-09-12T12:23:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cc28bb4c70 con 0x55cc28dafb20 2024-09-12T12:23:28.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.344+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55cc28b5aed0 con 0x55cc28db0440 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28d66290 0x55cc28d66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28d66290 0x55cc28d66670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37120/0 (socket says 172.21.15.3:37120) 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 -- 172.21.15.3:0/533908943 learned_addr learned my addr 172.21.15.3:0/533908943 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1482640 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28db0440 0x55cc28d65d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28daff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 -- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28db0440 msgr2=0x55cc28d65d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28db0440 0x55cc28d65d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 -- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 msgr2=0x55cc28daff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28daff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 -- 172.21.15.3:0/533908943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc28b6ce50 con 0x55cc28d66290 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28d66290 0x55cc28d66670 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f02c4002240 tx=0x7f02c4056280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f5d2430da9c8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02af7fe640 1 -- 172.21.15.3:0/533908943 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40544c0 con 0x55cc28d66290 2024-09-12T12:23:28.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02af7fe640 1 -- 172.21.15.3:0/533908943 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c406e120 con 0x55cc28d66290 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02af7fe640 1 -- 172.21.15.3:0/533908943 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c405e910 con 0x55cc28d66290 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- 172.21.15.3:0/533908943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28d66290 msgr2=0x55cc28d66670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28d66290 0x55cc28d66670 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f02c4002240 tx=0x7f02c4056280 comp rx=0 tx=0).stop 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- 172.21.15.3:0/533908943 shutdown_connections 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28d66290 0x55cc28d66670 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28db0440 0x55cc28d65d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/533908943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28daff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- 172.21.15.3:0/533908943 >> 172.21.15.3:0/533908943 conn(0x55cc28d6c600 msgr2=0x55cc28d64630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- 172.21.15.3:0/533908943 shutdown_connections 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- 172.21.15.3:0/533908943 wait complete. 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- start start 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 Processor -- start 2024-09-12T12:23:28.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28dafb20 0x55cc28d6adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28db0440 0x55cc28d70cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28d71230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cc28c16b00 con 0x55cc28dafb20 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cc28b8f360 con 0x55cc28db0440 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55cc28b8c2d0 con 0x55cc28d6b330 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 10 client.0 _ll_register_callbacks cb 0x55cc28d51820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28dafb20 0x55cc28d6adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28dafb20 0x55cc28d6adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37132/0 (socket says 172.21.15.3:37132) 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 -- 172.21.15.3:0/2453971037 learned_addr learned my addr 172.21.15.3:0/2453971037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28d71230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1482640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28db0440 0x55cc28d70cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 -- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 msgr2=0x55cc28d71230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28d71230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 -- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28db0440 msgr2=0x55cc28d70cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28db0440 0x55cc28d70cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 -- 172.21.15.3:0/2453971037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc28b5b120 con 0x55cc28dafb20 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d2484640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28d71230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d1c83640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28dafb20 0x55cc28d6adf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f02bc01c8e0 tx=0x7f02bc01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34532efbb15550c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02acff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02acff9640 1 -- 172.21.15.3:0/2453971037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc007510 con 0x55cc28dafb20 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c406e120 con 0x55cc28dafb20 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02acff9640 1 -- 172.21.15.3:0/2453971037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f02bc0067e0 con 0x55cc28dafb20 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02acff9640 1 -- 172.21.15.3:0/2453971037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f02bc0ce050 con 0x55cc28dafb20 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 10 client.6159 fetch_fsmap learned FSMap version 45 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.348+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55cc28b54450 con 0x55cc28dafb20 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02acff9640 1 -- 172.21.15.3:0/2453971037 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c406e120 con 0x55cc28dafb20 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02acff9640 1 -- 172.21.15.3:0/2453971037 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f02bc0d1020 con 0x55cc28dafb20 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 10 client.6159 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 10 client.6159 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 20 client.6159 populate_metadata read hostname 'smithi003' 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28dbe280 0x55cc28dc0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 4 client.6159 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 --> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc28b4ec00 con 0x55cc28dbe280 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d1482640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28dbe280 0x55cc28dc0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d1482640 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28dbe280 0x55cc28dc0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166af327b5581a36 server_cookie=1f27f945761d0033 in_seq=0 out_seq=0 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02acff9640 10 client.6159 ms_handle_connect on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02acff9640 1 -- 172.21.15.3:0/2453971037 <== mds.0 v2:172.21.15.3:6833/550473013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc28b4ec00 con 0x55cc28dbe280 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02acff9640 10 client.6159 handle_command_reply: tid=0 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 client.6159 shutdown 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28dafb20 msgr2=0x55cc28d6adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28dafb20 0x55cc28d6adf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f02bc01c8e0 tx=0x7f02bc01ac40 comp rx=0 tx=0).stop 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 shutdown_connections 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28dbe280 0x55cc28dc0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28d71230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28db0440 0x55cc28d70cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/2453971037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28dafb20 0x55cc28d6adf0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 >> 172.21.15.3:0/2453971037 conn(0x55cc28d66290 msgr2=0x55cc28d64d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02acff9640 0 client.6159 ms_handle_reset on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 shutdown_connections 2024-09-12T12:23:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 1 -- 172.21.15.3:0/2453971037 wait complete. 2024-09-12T12:23:28.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.352+0000 7f02d6fa3000 20 client.6159 trim_cache size 0 max 16384 2024-09-12T12:23:28.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 Processor -- start 2024-09-12T12:23:28.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- start start 2024-09-12T12:23:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28d6b330 0x55cc28daec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28dafb20 0x55cc28daf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28dc0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc2001a0 con 0x55cc28db0440 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02cc0ccf50 con 0x55cc28d6b330 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f02cc0cd5a0 con 0x55cc28dafb20 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28dc0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28dc0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37142/0 (socket says 172.21.15.3:37142) 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 -- 172.21.15.3:0/3250456124 learned_addr learned my addr 172.21.15.3:0/3250456124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 -- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28dafb20 msgr2=0x55cc28daf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d1482640 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28dafb20 0x55cc28daf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d1c83640 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28d6b330 0x55cc28daec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28dafb20 0x55cc28daf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 -- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28d6b330 msgr2=0x55cc28daec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28d6b330 0x55cc28daec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 -- 172.21.15.3:0/3250456124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc28db0270 con 0x55cc28db0440 2024-09-12T12:23:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d2484640 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28dc0670 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f02c406d040 tx=0x7f02c4055e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31260b8c2ce8b02e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02aeffd640 1 -- 172.21.15.3:0/3250456124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40544c0 con 0x55cc28db0440 2024-09-12T12:23:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02aeffd640 1 -- 172.21.15.3:0/3250456124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c406e120 con 0x55cc28db0440 2024-09-12T12:23:28.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02aeffd640 1 -- 172.21.15.3:0/3250456124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c405e910 con 0x55cc28db0440 2024-09-12T12:23:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3250456124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 msgr2=0x55cc28dc0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28dc0670 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f02c406d040 tx=0x7f02c4055e30 comp rx=0 tx=0).stop 2024-09-12T12:23:28.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3250456124 shutdown_connections 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28dc0670 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28dafb20 0x55cc28daf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3250456124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28d6b330 0x55cc28daec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3250456124 >> 172.21.15.3:0/3250456124 conn(0x55cc28d66290 msgr2=0x55cc28d6a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3250456124 shutdown_connections 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3250456124 wait complete. 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 -- start start 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.372+0000 7f02d6fa3000 1 Processor -- start 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28dc3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28dde960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28ddeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cc28bb3f80 con 0x55cc28db0440 2024-09-12T12:23:28.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cc28b6cca0 con 0x55cc28dafb20 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55cc28bb92a0 con 0x55cc28d6b330 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 10 client.0 _ll_register_callbacks cb 0x7f02b8003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28ddeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28ddeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37156/0 (socket says 172.21.15.3:37156) 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 -- 172.21.15.3:0/3108295501 learned_addr learned my addr 172.21.15.3:0/3108295501 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d1c83640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28dc3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d1482640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28dde960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 -- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 msgr2=0x55cc28dc3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28dc3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 -- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 msgr2=0x55cc28dde960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28dde960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 -- 172.21.15.3:0/3108295501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc28b3bfd0 con 0x55cc28db0440 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d1482640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28dde960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d2484640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28ddeea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f02c405e870 tx=0x7f02c4021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a00c729aa58dda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 1 -- 172.21.15.3:0/3108295501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4072070 con 0x55cc28db0440 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 1 -- 172.21.15.3:0/3108295501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f02c405eb10 con 0x55cc28db0440 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c406e120 con 0x55cc28db0440 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 1 -- 172.21.15.3:0/3108295501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f02c405a0d0 con 0x55cc28db0440 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 10 client.6165 fetch_fsmap learned FSMap version 45 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55cc28c37750 con 0x55cc28db0440 2024-09-12T12:23:28.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 1 -- 172.21.15.3:0/3108295501 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c406e120 con 0x55cc28db0440 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 1 -- 172.21.15.3:0/3108295501 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f02c4055610 con 0x55cc28db0440 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 10 client.6165 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 10 client.6165 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 20 client.6165 populate_metadata read hostname 'smithi003' 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28ddf3e0 0x55cc28e23140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 4 client.6165 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 --> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] -- command(tid 0: {"prefix": "status"}) -- 0x55cc28bc23a0 con 0x55cc28ddf3e0 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d1c83640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28ddf3e0 0x55cc28e23140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 10 client.6165 ms_handle_connect on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d1c83640 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28ddf3e0 0x55cc28e23140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29349fc256f721a4 server_cookie=ccf2047d1644a5d8 in_seq=0 out_seq=0 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 1 -- 172.21.15.3:0/3108295501 <== mds.0 v2:172.21.15.3:6833/550473013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55cc28bc23a0 con 0x55cc28ddf3e0 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 10 client.6165 handle_command_reply: tid=0 2024-09-12T12:23:28.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 client.6165 shutdown 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 msgr2=0x55cc28ddeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28ddeea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f02c405e870 tx=0x7f02c4021040 comp rx=0 tx=0).stop 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 shutdown_connections 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55cc28ddf3e0 0x55cc28e23140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cc28db0440 0x55cc28ddeea0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc28dafb20 0x55cc28dde960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02acff9640 0 client.6165 ms_handle_reset on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 --2- 172.21.15.3:0/3108295501 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55cc28d6b330 0x55cc28dc3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 >> 172.21.15.3:0/3108295501 conn(0x55cc28d66290 msgr2=0x55cc28dadc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 shutdown_connections 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 1 -- 172.21.15.3:0/3108295501 wait complete. 2024-09-12T12:23:28.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d6fa3000 20 client.6165 trim_cache size 0 max 16384 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6012, 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.276271614000001, 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 45, 2024-09-12T12:23:28.388 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 66, 2024-09-12T12:23:28.389 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 66, 2024-09-12T12:23:28.389 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.328118277 2024-09-12T12:23:28.389 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:23:28.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f02b00678f0 msgr2=0x7f02b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f02b00678f0 0x7f02b0069db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f02bc004000 tx=0x7f02bc0011a0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02cc0b5310 msgr2=0x7f02cc0b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02cc0b5310 0x7f02cc0b5790 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0d05e0 tx=0x7f02c400b480 comp rx=0 tx=0).stop 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 shutdown_connections 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f02b00678f0 0x7f02b0069db0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f02cc0b5310 0x7f02cc0b5790 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02cc15fd20 0x7f02cc1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 --2- 172.21.15.3:0/2851893089 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f02b80039c0 0x7f02cc15f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 >> 172.21.15.3:0/2851893089 conn(0x7f02cc067a00 msgr2=0x7f02cc0b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 shutdown_connections 2024-09-12T12:23:28.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.376+0000 7f02d3ee7640 1 -- 172.21.15.3:0/2851893089 wait complete. 2024-09-12T12:23:28.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:23:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee089ba640 1 Processor -- start 2024-09-12T12:23:28.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee089ba640 1 -- start start 2024-09-12T12:23:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee089ba640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fee0415c000 0x7fee040b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee089ba640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0415f160 con 0x7fee040af410 2024-09-12T12:23:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee089ba640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0415f430 con 0x7fee040d05b0 2024-09-12T12:23:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee089ba640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fee0415f9f0 con 0x7fee0415c000 2024-09-12T12:23:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fee040af410 0x7fee0415ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49690/0 (socket says 172.21.15.3:49690) 2024-09-12T12:23:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 -- 172.21.15.3:0/1626416935 learned_addr learned my addr 172.21.15.3:0/1626416935 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0259c640 1 --2- 172.21.15.3:0/1626416935 >> v2:172.21.15.142:3300/0 conn(0x7fee0415c000 0x7fee040b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966369494 0 0) 0x7fee0415f160 con 0x7fee040af410 2024-09-12T12:23:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedec003530 con 0x7fee040af410 2024-09-12T12:23:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4055754960 0 0) 0x7fedec003530 con 0x7fee040af410 2024-09-12T12:23:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 >> v2:172.21.15.142:3300/0 conn(0x7fee0415c000 msgr2=0x7fee040b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 --2- 172.21.15.3:0/1626416935 >> v2:172.21.15.142:3300/0 conn(0x7fee0415c000 0x7fee040b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 >> v1:172.21.15.142:6789/0 conn(0x7fee040d05b0 legacy=0x7fee0406d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0415f710 con 0x7fee040af410 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fedf8001680 con 0x7fee040af410 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 >> v1:172.21.15.3:6789/0 conn(0x7fee040af410 legacy=0x7fee0415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec0039c0 0x7fedec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedec004810 0x7fedec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedec0051d0 0x7fedec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee0415f9f0 con 0x7fedec004810 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fedec00c170 con 0x7fedec0051d0 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fedec00c370 con 0x7fedec0039c0 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedec004810 0x7fedec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0259c640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec0039c0 0x7fedec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee02d9d640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedec0051d0 0x7fedec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 -- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec0039c0 msgr2=0x7fedec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec0039c0 0x7fedec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 -- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedec0051d0 msgr2=0x7fedec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedec0051d0 0x7fedec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee01d9b640 1 -- 172.21.15.3:0/1626416935 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedec003530 con 0x7fedec004810 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee02d9d640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedec0051d0 0x7fedec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee0259c640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec0039c0 0x7fedec003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:28.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.512+0000 7fee02d9d640 1 -- 172.21.15.3:0/1626416935 reap_dead start 2024-09-12T12:23:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.516+0000 7fee01d9b640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedec004810 0x7fedec004c90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c210 tx=0x7fedf801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db863d3b26319e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8007d20 con 0x7fedec004810 2024-09-12T12:23:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf80a9070 con 0x7fedec004810 2024-09-12T12:23:28.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee0159a640 1 -- 172.21.15.3:0/1626416935 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80a5dd0 con 0x7fedec004810 2024-09-12T12:23:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- 172.21.15.3:0/1626416935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedec004810 msgr2=0x7fedec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedec004810 0x7fedec004c90 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c210 tx=0x7fedf801cc30 comp rx=0 tx=0).stop 2024-09-12T12:23:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- 172.21.15.3:0/1626416935 shutdown_connections 2024-09-12T12:23:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 --2- 172.21.15.3:0/1626416935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fedec004810 0x7fedec004c90 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- 172.21.15.3:0/1626416935 >> 172.21.15.3:0/1626416935 conn(0x7fee04067e00 msgr2=0x7fee04068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- 172.21.15.3:0/1626416935 shutdown_connections 2024-09-12T12:23:28.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- 172.21.15.3:0/1626416935 wait complete. 2024-09-12T12:23:28.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 Processor -- start 2024-09-12T12:23:28.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- start start 2024-09-12T12:23:28.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec004810 0x7fee04147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee040b0ef0 0x7fee04071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee040b1370 0x7fee040724c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee0415efe0 con 0x7fee040b0ef0 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fee0415ee60 con 0x7fee040b1370 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fee040d0da0 con 0x7fedec004810 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee040b0ef0 0x7fee04071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee040b0ef0 0x7fee04071f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37172/0 (socket says 172.21.15.3:37172) 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 -- 172.21.15.3:0/3614049133 learned_addr learned my addr 172.21.15.3:0/3614049133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee0259c640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec004810 0x7fee04147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 -- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec004810 msgr2=0x7fee04147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec004810 0x7fee04147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 -- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee040b1370 msgr2=0x7fee040724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee02d9d640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee040b1370 0x7fee040724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee040b1370 0x7fee040724c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 -- 172.21.15.3:0/3614049133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee041477c0 con 0x7fee040b0ef0 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee0259c640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec004810 0x7fee04147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee01d9b640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee040b0ef0 0x7fee04071f80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fedf801ca60 tx=0x7fedec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fedf2ffd640 1 -- 172.21.15.3:0/3614049133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80a5910 con 0x7fee040b0ef0 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fedf2ffd640 1 -- 172.21.15.3:0/3614049133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedf80a5ab0 con 0x7fee040b0ef0 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fedf2ffd640 1 -- 172.21.15.3:0/3614049133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80a5e60 con 0x7fee040b0ef0 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee040b2310 con 0x7fee040b0ef0 2024-09-12T12:23:28.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.520+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee040b28d0 con 0x7fee040b0ef0 2024-09-12T12:23:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.524+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0406d9f0 con 0x7fee040b0ef0 2024-09-12T12:23:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.524+0000 7fedf2ffd640 1 -- 172.21.15.3:0/3614049133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fedf8007180 con 0x7fee040b0ef0 2024-09-12T12:23:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.524+0000 7fedf2ffd640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fede40678f0 0x7fede4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.524+0000 7fedf2ffd640 1 -- 172.21.15.3:0/3614049133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fedf811f6b0 con 0x7fee040b0ef0 2024-09-12T12:23:28.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.524+0000 7fee0259c640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fede40678f0 0x7fede4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:28.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.524+0000 7fee0259c640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fede40678f0 0x7fede4069db0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fee040b0e80 tx=0x7fedf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:28.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.528+0000 7fedf2ffd640 1 -- 172.21.15.3:0/3614049133 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fedf8016970 con 0x7fee040b0ef0 2024-09-12T12:23:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.728+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fee040b2b10 con 0x7fee040b0ef0 2024-09-12T12:23:28.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.728+0000 7fedf2ffd640 1 -- 172.21.15.3:0/3614049133 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fedf80f01f0 con 0x7fee040b0ef0 2024-09-12T12:23:28.731 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:23:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.728+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fede40678f0 msgr2=0x7fede4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.728+0000 7fee089ba640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fede40678f0 0x7fede4069db0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fee040b0e80 tx=0x7fedf4008040 comp rx=0 tx=0).stop 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.728+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee040b0ef0 msgr2=0x7fee04071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.728+0000 7fee089ba640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee040b0ef0 0x7fee04071f80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fedf801ca60 tx=0x7fedec005840 comp rx=0 tx=0).stop 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.728+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 shutdown_connections 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.732+0000 7fee089ba640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fede40678f0 0x7fede4069db0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.732+0000 7fee089ba640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee040b1370 0x7fee040724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.732+0000 7fee089ba640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee040b0ef0 0x7fee04071f80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.732+0000 7fee089ba640 1 --2- 172.21.15.3:0/3614049133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedec004810 0x7fee04147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.732+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 >> 172.21.15.3:0/3614049133 conn(0x7fee04067e00 msgr2=0x7fee040ad0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.732+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 shutdown_connections 2024-09-12T12:23:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28.732+0000 7fee089ba640 1 -- 172.21.15.3:0/3614049133 wait complete. 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:23:28.750 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:23:28.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:28.751 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:28.757 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42278sec preferred_lft 42278sec 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:28.758 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:23:28.759 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:23:28.760 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.760 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:23:28.760 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:23:28.760 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:23:28.760 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:28.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:28.760 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:23:28.812 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:23:28.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:28.813 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:23:28.863 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:23:28.863 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:23:28.863 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:23:28.863 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:23:28.863 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:23:28.863 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:23:28.863 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:23:28.863 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:23:28.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:23:28.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:23:28.864 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:23:28.864 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:23:28.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:23:28.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:23:28.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:28.986 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:23:28.992 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:23:28.992 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:23:28.992 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:23:28.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:28.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:23:29.045 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:23:29.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:23:29.045 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:29.094 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:23:29.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:23:29.094 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:29.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:23:29.201 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:29.251 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:23:29.251 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:23:29.251 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:23:29.251 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:23:29.251 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:23:29.252 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:23:29.253 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:23:29.254 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:23:29.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:29.255 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:23:29.302 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:23:29.302 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:23:29.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:23:29.365 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:23:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:29.415 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:23:29.416 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:23:29.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:29.417 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:23:29.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:23:29.432+0000 7f663a316580 -1 init, newargv = 0x55834d325e70 newargc=15 2024-09-12T12:23:29.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[41558]: starting ceph client 2024-09-12T12:23:29.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[41558]: starting fuse 2024-09-12T12:23:30.540 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:23:30.556 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:30.603 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:23:30.603 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:23:30.603 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:23:30.603 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:23:30.604 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:23:30.605 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:23:30.606 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:23:30.606 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:23:30.606 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:23:30.606 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:30.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:30.607 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:23:30.654 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:23:30.655 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:23:30.655 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:23:30.656 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:23:30.781 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.41558.asok 2024-09-12T12:23:30.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:23:30.794 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41558.asok 2024-09-12T12:23:30.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:30.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41558.asok status 2024-09-12T12:23:30.955 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:23:30.955 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:23:30.955 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "41558", 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6180, 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:23:30.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6180 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3917940345 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3917940345 2024-09-12T12:23:30.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.6180 192.168.0.1:0/3917940345", 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3917940345", 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 45, 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 66, 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:23:30.958 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:23:30.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:30.969 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:23:30.969 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:30.975 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:23:30.975 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:30.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:30.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:31.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:23:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.192+0000 7ff7bece5640 1 Processor -- start 2024-09-12T12:23:31.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.192+0000 7ff7bece5640 1 -- start start 2024-09-12T12:23:31.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bece5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b80add30 0x7ff7b8070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:31.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bece5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff7b80ae110 0x7ff7b806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:31.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bece5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bece5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b80b3280 con 0x7ff7b80add30 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bece5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b80b3560 con 0x7ff7b80ae110 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bece5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80b3840 con 0x7ff7b806e870 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b80add30 0x7ff7b8070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff7b80add30 0x7ff7b8070d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37178/0 (socket says 172.21.15.3:37178) 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 -- 172.21.15.3:0/1795449473 learned_addr learned my addr 172.21.15.3:0/1795449473 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bd282640 1 --2- 172.21.15.3:0/1795449473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 -- 172.21.15.3:0/1795449473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 msgr2=0x7ff7b815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 --2- 172.21.15.3:0/1795449473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7b7fff640 1 --2- 172.21.15.3:0/1795449473 >> v2:172.21.15.142:3300/0 conn(0x7ff7b80ae110 0x7ff7b806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 -- 172.21.15.3:0/1795449473 >> v2:172.21.15.142:3300/0 conn(0x7ff7b80ae110 msgr2=0x7ff7b806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 --2- 172.21.15.3:0/1795449473 >> v2:172.21.15.142:3300/0 conn(0x7ff7b80ae110 0x7ff7b806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 -- 172.21.15.3:0/1795449473 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b80b3b20 con 0x7ff7b80add30 2024-09-12T12:23:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bd282640 1 --2- 172.21.15.3:0/1795449473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7b7fff640 1 --2- 172.21.15.3:0/1795449473 >> v2:172.21.15.142:3300/0 conn(0x7ff7b80ae110 0x7ff7b806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7bca81640 1 --2- 172.21.15.3:0/1795449473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80add30 0x7ff7b8070d30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80af530 tx=0x7ff7a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345dfd1bbbb1371c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.196+0000 7ff7b77fe640 1 -- 172.21.15.3:0/1795449473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80b0070 con 0x7ff7b80add30 2024-09-12T12:23:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7b77fe640 1 -- 172.21.15.3:0/1795449473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a80ab600 con 0x7ff7b80add30 2024-09-12T12:23:31.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7b77fe640 1 -- 172.21.15.3:0/1795449473 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a80ab9d0 con 0x7ff7b80add30 2024-09-12T12:23:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- 172.21.15.3:0/1795449473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80add30 msgr2=0x7ff7b8070d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 --2- 172.21.15.3:0/1795449473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80add30 0x7ff7b8070d30 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80af530 tx=0x7ff7a801c520 comp rx=0 tx=0).stop 2024-09-12T12:23:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- 172.21.15.3:0/1795449473 shutdown_connections 2024-09-12T12:23:31.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 --2- 172.21.15.3:0/1795449473 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 --2- 172.21.15.3:0/1795449473 >> v2:172.21.15.142:3300/0 conn(0x7ff7b80ae110 0x7ff7b806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 --2- 172.21.15.3:0/1795449473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80add30 0x7ff7b8070d30 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- 172.21.15.3:0/1795449473 >> 172.21.15.3:0/1795449473 conn(0x7ff7b8067e00 msgr2=0x7ff7b8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- 172.21.15.3:0/1795449473 shutdown_connections 2024-09-12T12:23:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- 172.21.15.3:0/1795449473 wait complete. 2024-09-12T12:23:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 Processor -- start 2024-09-12T12:23:31.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- start start 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b809d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80add30 0x7ff7b809dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80ae110 0x7ff7b8153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80b41a0 con 0x7ff7b80ae110 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b80b4020 con 0x7ff7b80add30 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bece5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b80b4320 con 0x7ff7b806e870 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bd282640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80ae110 0x7ff7b8153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bd282640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80ae110 0x7ff7b8153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37182/0 (socket says 172.21.15.3:37182) 2024-09-12T12:23:31.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bd282640 1 -- 172.21.15.3:0/4241311111 learned_addr learned my addr 172.21.15.3:0/4241311111 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7b7fff640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80add30 0x7ff7b809dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bca81640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b809d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bd282640 1 -- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 msgr2=0x7ff7b809d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bd282640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b809d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bd282640 1 -- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80add30 msgr2=0x7ff7b809dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.200+0000 7ff7bd282640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80add30 0x7ff7b809dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7bd282640 1 -- 172.21.15.3:0/4241311111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b8154080 con 0x7ff7b80ae110 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7bca81640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b809d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7bd282640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80ae110 0x7ff7b8153320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000c7f0 tx=0x7ff7b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7b57fa640 1 -- 172.21.15.3:0/4241311111 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0015070 con 0x7ff7b80ae110 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7b57fa640 1 -- 172.21.15.3:0/4241311111 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b0007820 con 0x7ff7b80ae110 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7b57fa640 1 -- 172.21.15.3:0/4241311111 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0007c10 con 0x7ff7b80ae110 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b8154360 con 0x7ff7b80ae110 2024-09-12T12:23:31.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b8154920 con 0x7ff7b80ae110 2024-09-12T12:23:31.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b8070d30 con 0x7ff7b80ae110 2024-09-12T12:23:31.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7b57fa640 1 -- 172.21.15.3:0/4241311111 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff7b0018070 con 0x7ff7b80ae110 2024-09-12T12:23:31.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7b57fa640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7980678c0 0x7ff798069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:31.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7b57fa640 1 -- 172.21.15.3:0/4241311111 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7ff7b0085e90 con 0x7ff7b80ae110 2024-09-12T12:23:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.204+0000 7ff7bca81640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7980678c0 0x7ff798069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:31.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.208+0000 7ff7bca81640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7980678c0 0x7ff798069d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80ad000 tx=0x7ff7a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:31.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.208+0000 7ff7b57fa640 1 -- 172.21.15.3:0/4241311111 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff7b00569b0 con 0x7ff7b80ae110 2024-09-12T12:23:31.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.412+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff7b8068560 con 0x7ff7b80ae110 2024-09-12T12:23:31.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7b57fa640 1 -- 172.21.15.3:0/4241311111 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7ff7b00562b0 con 0x7ff7b80ae110 2024-09-12T12:23:31.418 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:23:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7980678c0 msgr2=0x7ff798069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7980678c0 0x7ff798069d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff7a80ad000 tx=0x7ff7a8008040 comp rx=0 tx=0).stop 2024-09-12T12:23:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80ae110 msgr2=0x7ff7b8153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:31.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80ae110 0x7ff7b8153320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000c7f0 tx=0x7ff7b000ccb0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 shutdown_connections 2024-09-12T12:23:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff7980678c0 0x7ff798069d80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b80ae110 0x7ff7b8153320 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7b80add30 0x7ff7b809dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 --2- 172.21.15.3:0/4241311111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff7b806e870 0x7ff7b809d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 >> 172.21.15.3:0/4241311111 conn(0x7ff7b8067e00 msgr2=0x7ff7b809a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:31.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 shutdown_connections 2024-09-12T12:23:31.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:31.416+0000 7ff7bece5640 1 -- 172.21.15.3:0/4241311111 wait complete. 2024-09-12T12:23:31.437 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:23:31.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:23:31.437 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:23:31.438 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:23:31.438 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:31.438 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:23:31.438 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:23:31.438 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:23:31.438 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:23:31.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:31.439 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:23:31.448 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:23:31.448 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:23:31.448 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:23:31.448 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.449 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:23:31.449 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.449 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:31.449 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:31.449 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:23:31.451 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:31.451 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:31.452 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:23:31.452 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:23:31.452 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:31.452 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:23:31.452 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:31.453 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:23:31.453 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42273sec preferred_lft 42273sec 2024-09-12T12:23:31.453 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:23:31.453 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.453 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:23:31.454 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:31.454 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:23:31.454 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.454 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:23:31.454 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:23:31.454 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:23:31.455 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.455 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:23:31.455 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.455 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:23:31.455 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:31.456 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:23:31.456 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.456 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:23:31.456 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:31.456 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:23:31.457 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:23:31.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:31.457 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:23:31.504 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:23:31.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:31.506 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:23:31.556 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:23:31.557 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:23:31.557 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:23:31.557 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:23:31.557 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:23:31.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:23:31.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:23:31.557 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:23:31.557 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:23:31.645 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:23:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:23:31.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:23:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:23:31.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:31.680 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:23:31.686 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:23:31.687 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:23:31.687 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:23:31.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:31.687 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:23:31.747 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:23:31.747 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:23:31.747 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:31.796 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:23:31.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:23:31.797 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:31.845 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:23:31.905 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:23:31.953 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:23:31.954 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:31.955 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:31.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:31.956 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:23:32.007 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:23:32.008 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:23:32.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:23:32.067 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:32.116 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:23:32.116 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:23:32.116 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:23:32.116 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:23:32.116 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:23:32.116 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:23:32.116 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:32.117 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:32.118 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:32.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:32.119 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:23:32.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:23:32.132+0000 7f559a02a580 -1 init, newargv = 0x55d2cb476000 newargc=15 2024-09-12T12:23:32.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36259]: starting ceph client 2024-09-12T12:23:32.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36259]: starting fuse 2024-09-12T12:23:33.237 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:23:33.253 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:23:33.301 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:23:33.301 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:23:33.301 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:23:33.301 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:23:33.301 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:23:33.301 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:23:33.301 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:23:33.302 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:23:33.303 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:23:33.304 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:23:33.304 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:33.304 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:23:33.304 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:33.304 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:23:33.304 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:33.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:33.305 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:23:33.357 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:23:33.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:23:33.358 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:23:33.359 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:23:33.360 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:23:33.360 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:23:33.360 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:23:33.360 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:23:33.360 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:23:33.479 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.36259.asok 2024-09-12T12:23:33.482 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:23:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:23:33.490 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36259.asok 2024-09-12T12:23:33.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:33.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36259.asok status 2024-09-12T12:23:33.642 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:23:33.642 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:23:33.642 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:23:33.642 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:23:33.642 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "36259", 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 5617, 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 5617 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:23:33.643 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1080916064 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1080916064 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.5617 192.168.0.1:0/1080916064", 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/1080916064", 2024-09-12T12:23:33.644 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:23:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 45, 2024-09-12T12:23:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 68, 2024-09-12T12:23:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:23:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:23:33.645 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:23:33.645 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:23:33.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:33.657 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:23:33.657 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:33.663 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:23:33.663 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:33.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:33.663 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:33.724 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds ms_async_reap_threshold 1 2024-09-12T12:23:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581abd2640 1 Processor -- start 2024-09-12T12:23:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581abd2640 1 -- start start 2024-09-12T12:23:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581abd2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f581415f890 0x7f5814142740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581abd2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f58140726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:33.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581abd2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5814074fb0 con 0x7f5814160cc0 2024-09-12T12:23:33.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581abd2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f5814142c80 con 0x7f581415f890 2024-09-12T12:23:33.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581abd2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f58141601e0 con 0x7f58140ccc40 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581916f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f58140726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581916f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f58140726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:60998/0 (socket says 172.21.15.3:60998) 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581916f640 1 -- 172.21.15.3:0/1545264184 learned_addr learned my addr 172.21.15.3:0/1545264184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581896e640 1 --2- 172.21.15.3:0/1545264184 >> v2:172.21.15.142:3300/0 conn(0x7f581415f890 0x7f5814142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581916f640 1 -- 172.21.15.3:0/1545264184 >> v2:172.21.15.142:3300/0 conn(0x7f581415f890 msgr2=0x7f5814142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581916f640 1 --2- 172.21.15.3:0/1545264184 >> v2:172.21.15.142:3300/0 conn(0x7f581415f890 0x7f5814142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581916f640 1 -- 172.21.15.3:0/1545264184 >> v1:172.21.15.142:6789/0 conn(0x7f5814160cc0 legacy=0x7f58140cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f58137fe640 1 -- 172.21.15.3:0/1545264184 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825716264 0 0) 0x7f5814074fb0 con 0x7f5814160cc0 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581916f640 1 -- 172.21.15.3:0/1545264184 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58141604c0 con 0x7f58140ccc40 2024-09-12T12:23:33.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.860+0000 7f581896e640 1 --2- 172.21.15.3:0/1545264184 >> v2:172.21.15.142:3300/0 conn(0x7f581415f890 0x7f5814142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581916f640 1 --2- 172.21.15.3:0/1545264184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f58140726b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=413b35b7279b7568 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f58137fe640 1 -- 172.21.15.3:0/1545264184 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040a7070 con 0x7f58140ccc40 2024-09-12T12:23:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f58137fe640 1 -- 172.21.15.3:0/1545264184 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58040b0070 con 0x7f58140ccc40 2024-09-12T12:23:33.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f58137fe640 1 -- 172.21.15.3:0/1545264184 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040ab890 con 0x7f58140ccc40 2024-09-12T12:23:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 -- 172.21.15.3:0/1545264184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 msgr2=0x7f58140726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 --2- 172.21.15.3:0/1545264184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f58140726b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580401c560 comp rx=0 tx=0).stop 2024-09-12T12:23:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 -- 172.21.15.3:0/1545264184 shutdown_connections 2024-09-12T12:23:33.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 --2- 172.21.15.3:0/1545264184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f58140726b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 --2- 172.21.15.3:0/1545264184 >> v2:172.21.15.142:3300/0 conn(0x7f581415f890 0x7f5814142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 -- 172.21.15.3:0/1545264184 >> 172.21.15.3:0/1545264184 conn(0x7f58140d0a70 msgr2=0x7f58140d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 -- 172.21.15.3:0/1545264184 shutdown_connections 2024-09-12T12:23:33.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.864+0000 7f581abd2640 1 -- 172.21.15.3:0/1545264184 wait complete. 2024-09-12T12:23:33.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 Processor -- start 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 -- start start 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f581413a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581415f890 0x7f5814159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5814160cc0 0x7f581415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5814169560 con 0x7f581415f890 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f58141693e0 con 0x7f5814160cc0 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f58141696e0 con 0x7f58140ccc40 2024-09-12T12:23:33.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581415f890 0x7f5814159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581415f890 0x7f5814159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37184/0 (socket says 172.21.15.3:37184) 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 -- 172.21.15.3:0/726983030 learned_addr learned my addr 172.21.15.3:0/726983030 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581896e640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f581413a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581916f640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5814160cc0 0x7f581415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 -- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 msgr2=0x7f581413a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f581413a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 -- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5814160cc0 msgr2=0x7f581415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5814160cc0 0x7f581415e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 -- 172.21.15.3:0/726983030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5814142fa0 con 0x7f581415f890 2024-09-12T12:23:33.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f5813fff640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581415f890 0x7f5814159090 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c4d0 tx=0x7f57fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f58117fa640 1 -- 172.21.15.3:0/726983030 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc017070 con 0x7f581415f890 2024-09-12T12:23:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f58117fa640 1 -- 172.21.15.3:0/726983030 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc005480 con 0x7f581415f890 2024-09-12T12:23:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5814143280 con 0x7f581415f890 2024-09-12T12:23:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f58117fa640 1 -- 172.21.15.3:0/726983030 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc003840 con 0x7f581415f890 2024-09-12T12:23:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5814143840 con 0x7f581415f890 2024-09-12T12:23:33.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58140726b0 con 0x7f581415f890 2024-09-12T12:23:33.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.868+0000 7f58117fa640 1 -- 172.21.15.3:0/726983030 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f57fc01a070 con 0x7f581415f890 2024-09-12T12:23:33.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.872+0000 7f58117fa640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f57f80678c0 0x7f57f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:33.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.872+0000 7f58117fa640 1 -- 172.21.15.3:0/726983030 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f57fc0854b0 con 0x7f581415f890 2024-09-12T12:23:33.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.872+0000 7f581896e640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f57f80678c0 0x7f57f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:33.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.872+0000 7f581896e640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f57f80678c0 0x7f57f8069d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f580c009bd0 tx=0x7f580c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:33.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:33.872+0000 7f58117fa640 1 -- 172.21.15.3:0/726983030 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f57fc021020 con 0x7f581415f890 2024-09-12T12:23:34.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.040+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=ms_async_reap_threshold}] v 0) -- 0x7f5814145510 con 0x7f581415f890 2024-09-12T12:23:34.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.044+0000 7f58117fa640 1 -- 172.21.15.3:0/726983030 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=ms_async_reap_threshold}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f57fc056150 con 0x7f581415f890 2024-09-12T12:23:34.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f57f80678c0 msgr2=0x7f57f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f57f80678c0 0x7f57f8069d80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f580c009bd0 tx=0x7f580c00b040 comp rx=0 tx=0).stop 2024-09-12T12:23:34.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581415f890 msgr2=0x7f5814159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581415f890 0x7f5814159090 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c4d0 tx=0x7f57fc00c990 comp rx=0 tx=0).stop 2024-09-12T12:23:34.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 shutdown_connections 2024-09-12T12:23:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f57f80678c0 0x7f57f8069d80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5814160cc0 0x7f581415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f581415f890 0x7f5814159090 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 --2- 172.21.15.3:0/726983030 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f58140ccc40 0x7f581413a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 >> 172.21.15.3:0/726983030 conn(0x7f58140d0a70 msgr2=0x7f58141424a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 shutdown_connections 2024-09-12T12:23:34.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.048+0000 7f581abd2640 1 -- 172.21.15.3:0/726983030 wait complete. 2024-09-12T12:23:34.069 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:34.069 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:34.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:23:34.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa9658a7640 1 Processor -- start 2024-09-12T12:23:34.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa9658a7640 1 -- start start 2024-09-12T12:23:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa9658a7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9600ae250 0x7fa9600ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa9658a7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 0x7fa96015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa9658a7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9600ad0c0 con 0x7fa9600d05b0 2024-09-12T12:23:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa9658a7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9600ad3a0 con 0x7fa9600ae250 2024-09-12T12:23:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa9658a7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9600ad680 con 0x7fa96015b620 2024-09-12T12:23:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95effd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa9600d05b0 0x7fa9600d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49698/0 (socket says 172.21.15.3:49698) 2024-09-12T12:23:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95effd640 1 -- 172.21.15.3:0/2568140389 learned_addr learned my addr 172.21.15.3:0/2568140389 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95e7fc640 1 --2- 172.21.15.3:0/2568140389 >> v2:172.21.15.142:3300/0 conn(0x7fa9600ae250 0x7fa9600ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95f7fe640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 0x7fa96015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982198514 0 0) 0x7fa9600ad0c0 con 0x7fa9600d05b0 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa948003530 con 0x7fa9600d05b0 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3649202685 0 0) 0x7fa948003530 con 0x7fa9600d05b0 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 msgr2=0x7fa96015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 0x7fa96015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 >> v2:172.21.15.142:3300/0 conn(0x7fa9600ae250 msgr2=0x7fa9600ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 --2- 172.21.15.3:0/2568140389 >> v2:172.21.15.142:3300/0 conn(0x7fa9600ae250 0x7fa9600ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9600ad960 con 0x7fa9600d05b0 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.240+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fa950001680 con 0x7fa9600d05b0 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 >> v1:172.21.15.3:6789/0 conn(0x7fa9600d05b0 legacy=0x7fa9600d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 0x7fa9480039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948003f40 0x7fa948004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa948004d50 0x7fa9480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600ad3a0 con 0x7fa948004d50 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600ad680 con 0x7fa948003f40 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa94800b9d0 con 0x7fa96015b620 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa948004d50 0x7fa9480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95effd640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 0x7fa9480039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95e7fc640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948003f40 0x7fa948004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 -- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 msgr2=0x7fa9480039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa96015b620 0x7fa9480039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 -- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948003f40 msgr2=0x7fa948004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa948003f40 0x7fa948004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 -- 172.21.15.3:0/2568140389 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa948003530 con 0x7fa948004d50 2024-09-12T12:23:34.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa948004d50 0x7fa9480051d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa958009fa0 tx=0x7fa95801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bd8336f8ee706b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:34.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580b2070 con 0x7fa948004d50 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9580a5d10 con 0x7fa948004d50 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95dffb640 1 -- 172.21.15.3:0/2568140389 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958009bf0 con 0x7fa948004d50 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa9658a7640 1 -- 172.21.15.3:0/2568140389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa948004d50 msgr2=0x7fa9480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa9658a7640 1 --2- 172.21.15.3:0/2568140389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa948004d50 0x7fa9480051d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa958009fa0 tx=0x7fa95801cbe0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa95f7fe640 1 -- 172.21.15.3:0/2568140389 reap_dead start 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa9658a7640 1 -- 172.21.15.3:0/2568140389 shutdown_connections 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa9658a7640 1 -- 172.21.15.3:0/2568140389 >> 172.21.15.3:0/2568140389 conn(0x7fa960067e00 msgr2=0x7fa960068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa9658a7640 1 -- 172.21.15.3:0/2568140389 shutdown_connections 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.244+0000 7fa9658a7640 1 -- 172.21.15.3:0/2568140389 wait complete. 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 Processor -- start 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 -- start start 2024-09-12T12:23:34.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9601f1b50 0x7fa9600b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa960158e40 0x7fa9600b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96015bec0 0x7fa9600b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600aeda0 con 0x7fa960158e40 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600aec20 con 0x7fa96015bec0 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9601559b0 con 0x7fa9601f1b50 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa960158e40 0x7fa9600b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa960158e40 0x7fa9600b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37204/0 (socket says 172.21.15.3:37204) 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 -- 172.21.15.3:0/3733063462 learned_addr learned my addr 172.21.15.3:0/3733063462 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95effd640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9601f1b50 0x7fa9600b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95f7fe640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96015bec0 0x7fa9600b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 -- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9601f1b50 msgr2=0x7fa9600b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9601f1b50 0x7fa9600b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 -- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96015bec0 msgr2=0x7fa9600b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96015bec0 0x7fa9600b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 -- 172.21.15.3:0/3733063462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9600afd50 con 0x7fa960158e40 2024-09-12T12:23:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa95e7fc640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa960158e40 0x7fa9600b0760 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa954011640 tx=0x7fa954011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa947fff640 1 -- 172.21.15.3:0/3733063462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954015870 con 0x7fa960158e40 2024-09-12T12:23:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa947fff640 1 -- 172.21.15.3:0/3733063462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa954015a10 con 0x7fa960158e40 2024-09-12T12:23:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa947fff640 1 -- 172.21.15.3:0/3733063462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954015dc0 con 0x7fa960158e40 2024-09-12T12:23:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa960156760 con 0x7fa960158e40 2024-09-12T12:23:34.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa960156d20 con 0x7fa960158e40 2024-09-12T12:23:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96015c830 con 0x7fa960158e40 2024-09-12T12:23:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.248+0000 7fa947fff640 1 -- 172.21.15.3:0/3733063462 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa95400f450 con 0x7fa960158e40 2024-09-12T12:23:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.252+0000 7fa947fff640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678f0 0x7fa94c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.252+0000 7fa947fff640 1 -- 172.21.15.3:0/3733063462 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fa9540865f0 con 0x7fa960158e40 2024-09-12T12:23:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.252+0000 7fa95effd640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678f0 0x7fa94c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.252+0000 7fa95effd640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678f0 0x7fa94c069db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa950000f70 tx=0x7fa950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:34.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.256+0000 7fa947fff640 1 -- 172.21.15.3:0/3733063462 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa954019030 con 0x7fa960158e40 2024-09-12T12:23:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.432+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa96006de70 con 0x7fa960158e40 2024-09-12T12:23:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.432+0000 7fa947fff640 1 -- 172.21.15.3:0/3733063462 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7fa9540570f0 con 0x7fa960158e40 2024-09-12T12:23:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:23:34.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678f0 msgr2=0x7fa94c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678f0 0x7fa94c069db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa950000f70 tx=0x7fa950009040 comp rx=0 tx=0).stop 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa960158e40 msgr2=0x7fa9600b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa960158e40 0x7fa9600b0760 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa954011640 tx=0x7fa954011b00 comp rx=0 tx=0).stop 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 shutdown_connections 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678f0 0x7fa94c069db0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96015bec0 0x7fa9600b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa960158e40 0x7fa9600b0760 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 --2- 172.21.15.3:0/3733063462 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9601f1b50 0x7fa9600b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 >> 172.21.15.3:0/3733063462 conn(0x7fa960067e00 msgr2=0x7fa960152c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 shutdown_connections 2024-09-12T12:23:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.436+0000 7fa9658a7640 1 -- 172.21.15.3:0/3733063462 wait complete. 2024-09-12T12:23:34.454 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:34.455 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:34.460 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:23:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7ca2ee0640 1 Processor -- start 2024-09-12T12:23:34.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7ca2ee0640 1 -- start start 2024-09-12T12:23:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7ca2ee0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 0x7f7c9c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7ca2ee0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0b5000 con 0x7f7c9c15c710 2024-09-12T12:23:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7ca2ee0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0b3510 con 0x7f7c9c073cb0 2024-09-12T12:23:34.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7ca2ee0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c074090 con 0x7f7c9c0b30e0 2024-09-12T12:23:34.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7c9c15c710 0x7f7c9c0ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49700/0 (socket says 172.21.15.3:49700) 2024-09-12T12:23:34.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9bfff640 1 -- 172.21.15.3:0/4227246134 learned_addr learned my addr 172.21.15.3:0/4227246134 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:34.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7ca147d640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 0x7f7c9c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165490330 0 0) 0x7f7c9c0b5000 con 0x7f7c9c15c710 2024-09-12T12:23:34.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c88003530 con 0x7f7c9c15c710 2024-09-12T12:23:34.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156666779 0 0) 0x7f7c9c0b3510 con 0x7f7c9c073cb0 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c9c0b5000 con 0x7f7c9c073cb0 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2503199129 0 0) 0x7f7c88003530 con 0x7f7c9c15c710 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 msgr2=0x7f7c9c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 0x7f7c9c0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 >> v1:172.21.15.142:6789/0 conn(0x7f7c9c073cb0 legacy=0x7f7c9c0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0b37c0 con 0x7f7c9c15c710 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f7c84001a00 con 0x7f7c9c15c710 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 >> v1:172.21.15.3:6789/0 conn(0x7f7c9c15c710 legacy=0x7f7c9c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88003b80 0x7f7c88003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c88004810 0x7f7c88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 0x7f7c880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8800ba80 con 0x7f7c88004810 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.616+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8800bc00 con 0x7f7c88003b80 2024-09-12T12:23:34.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8800be00 con 0x7f7c9c0b30e0 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9bfff640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c88004810 0x7f7c88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca0c7c640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88003b80 0x7f7c88003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca147d640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 0x7f7c880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9bfff640 1 -- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 msgr2=0x7f7c880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9bfff640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c9c0b30e0 0x7f7c880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9bfff640 1 -- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88003b80 msgr2=0x7f7c88003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9bfff640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88003b80 0x7f7c88003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9bfff640 1 -- 172.21.15.3:0/4227246134 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c88003530 con 0x7f7c88004810 2024-09-12T12:23:34.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca0c7c640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88003b80 0x7f7c88003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9bfff640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c88004810 0x7f7c88004c90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400a7e0 tx=0x7f7c8401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20a2f88ad48615d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8400a430 con 0x7f7c88004810 2024-09-12T12:23:34.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c840ad650 con 0x7f7c88004810 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7c9b7fe640 1 -- 172.21.15.3:0/4227246134 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c840ada40 con 0x7f7c88004810 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/4227246134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c88004810 msgr2=0x7f7c88004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca2ee0640 1 --2- 172.21.15.3:0/4227246134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c88004810 0x7f7c88004c90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400a7e0 tx=0x7f7c8401d2a0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca147d640 1 -- 172.21.15.3:0/4227246134 reap_dead start 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/4227246134 shutdown_connections 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/4227246134 >> 172.21.15.3:0/4227246134 conn(0x7f7c9c067e00 msgr2=0x7f7c9c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/4227246134 shutdown_connections 2024-09-12T12:23:34.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.620+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/4227246134 wait complete. 2024-09-12T12:23:34.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 Processor -- start 2024-09-12T12:23:34.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 -- start start 2024-09-12T12:23:34.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c88004810 0x7f7c9c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c9c0ca870 0x7f7c9c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c06e9d0 0x7f7c9c0b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c0ad6f0 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c0b3d80 con 0x7f7c9c0ca870 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c0ae320 con 0x7f7c88004810 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c06e9d0 0x7f7c9c0b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c06e9d0 0x7f7c9c0b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37224/0 (socket says 172.21.15.3:37224) 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 -- 172.21.15.3:0/3843010788 learned_addr learned my addr 172.21.15.3:0/3843010788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca0c7c640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c88004810 0x7f7c9c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7c9bfff640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c9c0ca870 0x7f7c9c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 -- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c88004810 msgr2=0x7f7c9c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c88004810 0x7f7c9c06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 -- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c9c0ca870 msgr2=0x7f7c9c0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c9c0ca870 0x7f7c9c0b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 -- 172.21.15.3:0/3843010788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0c97d0 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7c9bfff640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c9c0ca870 0x7f7c9c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:34.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca147d640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c06e9d0 0x7f7c9c0b1c30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0afd00 tx=0x7f7c8c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:34.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7c997fa640 1 -- 172.21.15.3:0/3843010788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c003620 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7c997fa640 1 -- 172.21.15.3:0/3843010788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c8c01b070 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7c997fa640 1 -- 172.21.15.3:0/3843010788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c003a40 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c0c9ab0 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.624+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c0ca070 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.628+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c0adc10 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.628+0000 7f7c997fa640 1 -- 172.21.15.3:0/3843010788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c8c003be0 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.628+0000 7f7c997fa640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c7c0678f0 0x7f7c7c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.628+0000 7f7c997fa640 1 -- 172.21.15.3:0/3843010788 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f7c8c085df0 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.628+0000 7f7ca0c7c640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c7c0678f0 0x7f7c7c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:34.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.628+0000 7f7ca0c7c640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c7c0678f0 0x7f7c7c069db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c94005630 tx=0x7f7c94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:34.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.632+0000 7f7c997fa640 1 -- 172.21.15.3:0/3843010788 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7c8c025020 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c9c068420 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7c997fa640 1 -- 172.21.15.3:0/3843010788 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f7c8c01d310 con 0x7f7c9c06e9d0 2024-09-12T12:23:34.810 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c7c0678f0 msgr2=0x7f7c7c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c7c0678f0 0x7f7c7c069db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c94005630 tx=0x7f7c94009040 comp rx=0 tx=0).stop 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c06e9d0 msgr2=0x7f7c9c0b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c06e9d0 0x7f7c9c0b1c30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0afd00 tx=0x7f7c8c011920 comp rx=0 tx=0).stop 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 shutdown_connections 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c7c0678f0 0x7f7c7c069db0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c9c06e9d0 0x7f7c9c0b1c30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c9c0ca870 0x7f7c9c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 --2- 172.21.15.3:0/3843010788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c88004810 0x7f7c9c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 >> 172.21.15.3:0/3843010788 conn(0x7f7c9c067e00 msgr2=0x7f7c9c15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 shutdown_connections 2024-09-12T12:23:34.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:34.808+0000 7f7ca2ee0640 1 -- 172.21.15.3:0/3843010788 wait complete. 2024-09-12T12:23:34.828 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:23:34.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:34.829 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:23:34.840 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[41558]: fuse finished with error 0 and tester_r 0 2024-09-12T12:23:34.842 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:40.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:23:40.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:40.944 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:23:40.952 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:40.952 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:40.959 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:23:41.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa947ca5640 1 Processor -- start 2024-09-12T12:23:41.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa947ca5640 1 -- start start 2024-09-12T12:23:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa947ca5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa94015c710 0x7fa9400b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa947ca5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9400b5000 con 0x7fa940073cb0 2024-09-12T12:23:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa947ca5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9400b37c0 con 0x7fa9400ae880 2024-09-12T12:23:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa947ca5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa940074090 con 0x7fa94015c710 2024-09-12T12:23:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa945240640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa94015c710 0x7fa9400b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa945240640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa94015c710 0x7fa9400b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39752/0 (socket says 172.21.15.3:39752) 2024-09-12T12:23:41.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa945240640 1 -- 172.21.15.3:0/3980620058 learned_addr learned my addr 172.21.15.3:0/3980620058 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa945240640 1 -- 172.21.15.3:0/3980620058 >> v1:172.21.15.142:6789/0 conn(0x7fa9400ae880 legacy=0x7fa9400d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa945240640 1 -- 172.21.15.3:0/3980620058 >> v1:172.21.15.3:6789/0 conn(0x7fa940073cb0 legacy=0x7fa9400adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.128+0000 7fa945240640 1 -- 172.21.15.3:0/3980620058 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9400b3aa0 con 0x7fa94015c710 2024-09-12T12:23:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa945240640 1 --2- 172.21.15.3:0/3980620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94015c710 0x7fa9400b30e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009870 tx=0x7fa93c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830ace0e68930c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa944a3f640 1 -- 172.21.15.3:0/3980620058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0ab610 con 0x7fa94015c710 2024-09-12T12:23:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa944a3f640 1 -- 172.21.15.3:0/3980620058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c0b0070 con 0x7fa94015c710 2024-09-12T12:23:41.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa944a3f640 1 -- 172.21.15.3:0/3980620058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0ab9d0 con 0x7fa94015c710 2024-09-12T12:23:41.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 -- 172.21.15.3:0/3980620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94015c710 msgr2=0x7fa9400b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 --2- 172.21.15.3:0/3980620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94015c710 0x7fa9400b30e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009870 tx=0x7fa93c01c520 comp rx=0 tx=0).stop 2024-09-12T12:23:41.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 -- 172.21.15.3:0/3980620058 shutdown_connections 2024-09-12T12:23:41.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 --2- 172.21.15.3:0/3980620058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa94015c710 0x7fa9400b30e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 -- 172.21.15.3:0/3980620058 >> 172.21.15.3:0/3980620058 conn(0x7fa940067e00 msgr2=0x7fa940068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:41.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 -- 172.21.15.3:0/3980620058 shutdown_connections 2024-09-12T12:23:41.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 -- 172.21.15.3:0/3980620058 wait complete. 2024-09-12T12:23:41.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 Processor -- start 2024-09-12T12:23:41.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.132+0000 7fa947ca5640 1 -- start start 2024-09-12T12:23:41.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa940073cb0 0x7fa940156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9400ae880 0x7fa94006e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94015c710 0x7fa940151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400d0e40 con 0x7fa9400ae880 2024-09-12T12:23:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400d0cc0 con 0x7fa94015c710 2024-09-12T12:23:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9400aecb0 con 0x7fa940073cb0 2024-09-12T12:23:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9400ae880 0x7fa94006e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9400ae880 0x7fa94006e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39760/0 (socket says 172.21.15.3:39760) 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 -- 172.21.15.3:0/320927074 learned_addr learned my addr 172.21.15.3:0/320927074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 -- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa940073cb0 msgr2=0x7fa940156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa946242640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94015c710 0x7fa940151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945a41640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa940073cb0 0x7fa940156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa940073cb0 0x7fa940156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 -- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94015c710 msgr2=0x7fa940151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94015c710 0x7fa940151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 -- 172.21.15.3:0/320927074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94006ded0 con 0x7fa9400ae880 2024-09-12T12:23:41.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945a41640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa940073cb0 0x7fa940156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:41.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa946242640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94015c710 0x7fa940151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:41.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945240640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9400ae880 0x7fa94006e5a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0ad040 tx=0x7fa93c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa9327fc640 1 -- 172.21.15.3:0/320927074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0b4800 con 0x7fa9400ae880 2024-09-12T12:23:41.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa9327fc640 1 -- 172.21.15.3:0/320927074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c0b49a0 con 0x7fa9400ae880 2024-09-12T12:23:41.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94006eae0 con 0x7fa9400ae880 2024-09-12T12:23:41.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa9327fc640 1 -- 172.21.15.3:0/320927074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0b4d90 con 0x7fa9400ae880 2024-09-12T12:23:41.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94006dbf0 con 0x7fa9400ae880 2024-09-12T12:23:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9400af040 con 0x7fa9400ae880 2024-09-12T12:23:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa9327fc640 1 -- 172.21.15.3:0/320927074 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa93c0c8420 con 0x7fa9400ae880 2024-09-12T12:23:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa9327fc640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9240678f0 0x7fa924069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa9327fc640 1 -- 172.21.15.3:0/320927074 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fa93c1204c0 con 0x7fa9400ae880 2024-09-12T12:23:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.136+0000 7fa945a41640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9240678f0 0x7fa924069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.140+0000 7fa945a41640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9240678f0 0x7fa924069db0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9340016f0 tx=0x7fa934009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.140+0000 7fa9327fc640 1 -- 172.21.15.3:0/320927074 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa93c0c4060 con 0x7fa9400ae880 2024-09-12T12:23:41.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.316+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9400b3260 con 0x7fa9400ae880 2024-09-12T12:23:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.316+0000 7fa9327fc640 1 -- 172.21.15.3:0/320927074 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7fa93c0a8050 con 0x7fa9400ae880 2024-09-12T12:23:41.321 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:23:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9240678f0 msgr2=0x7fa924069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9240678f0 0x7fa924069db0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9340016f0 tx=0x7fa934009040 comp rx=0 tx=0).stop 2024-09-12T12:23:41.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9400ae880 msgr2=0x7fa94006e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9400ae880 0x7fa94006e5a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0ad040 tx=0x7fa93c01ad60 comp rx=0 tx=0).stop 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 shutdown_connections 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9240678f0 0x7fa924069db0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa94015c710 0x7fa940151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9400ae880 0x7fa94006e5a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 --2- 172.21.15.3:0/320927074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa940073cb0 0x7fa940156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 >> 172.21.15.3:0/320927074 conn(0x7fa940067e00 msgr2=0x7fa940069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 shutdown_connections 2024-09-12T12:23:41.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.320+0000 7fa947ca5640 1 -- 172.21.15.3:0/320927074 wait complete. 2024-09-12T12:23:41.339 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:41.339 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:41.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:23:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fee54640 1 Processor -- start 2024-09-12T12:23:41.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fee54640 1 -- start start 2024-09-12T12:23:41.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fee54640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 0x7fa5f80d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fee54640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5f80b5000 con 0x7fa5f8073cb0 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fee54640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5f80b3510 con 0x7fa5f815c710 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fee54640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f8074090 con 0x7fa5f80b30e0 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa5f8073cb0 0x7fa5f80adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34750/0 (socket says 172.21.15.3:34750) 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/1151270762 learned_addr learned my addr 172.21.15.3:0/1151270762 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fd3f1640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 0x7fa5f80d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1720430583 0 0) 0x7fa5f80b5000 con 0x7fa5f8073cb0 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5e0003530 con 0x7fa5f8073cb0 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2116665083 0 0) 0x7fa5e0003530 con 0x7fa5f8073cb0 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 msgr2=0x7fa5f80d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 0x7fa5f80d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 >> v1:172.21.15.142:6789/0 conn(0x7fa5f815c710 legacy=0x7fa5f80ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f80b37c0 con 0x7fa5f8073cb0 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fa5e8001680 con 0x7fa5f8073cb0 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 >> v1:172.21.15.3:6789/0 conn(0x7fa5f8073cb0 legacy=0x7fa5f80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0004810 0x7fa5e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 0x7fa5e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e000fe90 con 0x7fa5e00039f0 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0010040 con 0x7fa5e0004810 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e0010240 con 0x7fa5f80b30e0 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5f7fff640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0004810 0x7fa5e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fd3f1640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 0x7fa5e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 msgr2=0x7fa5e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f80b30e0 0x7fa5e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0004810 msgr2=0x7fa5e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5e0004810 0x7fa5e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.460+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/1151270762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e0003530 con 0x7fa5e00039f0 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fcbf0640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5e0003e10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800c210 tx=0x7fa5e801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cf4d10cc5e7090 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8007d20 con 0x7fa5e00039f0 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e80a9070 con 0x7fa5e00039f0 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5f77fe640 1 -- 172.21.15.3:0/1151270762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8005b20 con 0x7fa5e00039f0 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 -- 172.21.15.3:0/1151270762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 msgr2=0x7fa5e0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 --2- 172.21.15.3:0/1151270762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5e0003e10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa5e800c210 tx=0x7fa5e801cc40 comp rx=0 tx=0).stop 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fd3f1640 1 -- 172.21.15.3:0/1151270762 reap_dead start 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 -- 172.21.15.3:0/1151270762 shutdown_connections 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 -- 172.21.15.3:0/1151270762 >> 172.21.15.3:0/1151270762 conn(0x7fa5f8067e00 msgr2=0x7fa5f8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:41.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 -- 172.21.15.3:0/1151270762 shutdown_connections 2024-09-12T12:23:41.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 -- 172.21.15.3:0/1151270762 wait complete. 2024-09-12T12:23:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 Processor -- start 2024-09-12T12:23:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.464+0000 7fa5fee54640 1 -- start start 2024-09-12T12:23:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5f81f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f814f850 0x7fa5f81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5f814fcd0 0x7fa5f815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f80ae4a0 con 0x7fa5e00039f0 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f80ae320 con 0x7fa5f814fcd0 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f80aef90 con 0x7fa5f814f850 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5f81f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5f81f2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39778/0 (socket says 172.21.15.3:39778) 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/2645404003 learned_addr learned my addr 172.21.15.3:0/2645404003 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f7fff640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f814f850 0x7fa5f81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fd3f1640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5f814fcd0 0x7fa5f815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f814f850 msgr2=0x7fa5f81500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f814f850 0x7fa5f81500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5f814fcd0 msgr2=0x7fa5f815e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5f814fcd0 0x7fa5f815e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 -- 172.21.15.3:0/2645404003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f8155bc0 con 0x7fa5e00039f0 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fd3f1640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5f814fcd0 0x7fa5f815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:41.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f7fff640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f814f850 0x7fa5f81500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fcbf0640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5f81f2e90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa5e801ca70 tx=0x7fa5e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f57fa640 1 -- 172.21.15.3:0/2645404003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8007d20 con 0x7fa5e00039f0 2024-09-12T12:23:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f57fa640 1 -- 172.21.15.3:0/2645404003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e8005b00 con 0x7fa5e00039f0 2024-09-12T12:23:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f80ce3a0 con 0x7fa5e00039f0 2024-09-12T12:23:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f57fa640 1 -- 172.21.15.3:0/2645404003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8007110 con 0x7fa5e00039f0 2024-09-12T12:23:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f80ce920 con 0x7fa5e00039f0 2024-09-12T12:23:41.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f80adc10 con 0x7fa5e00039f0 2024-09-12T12:23:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f57fa640 1 -- 172.21.15.3:0/2645404003 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa5e80b7030 con 0x7fa5e00039f0 2024-09-12T12:23:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f57fa640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5e40678c0 0x7fa5e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f57fa640 1 -- 172.21.15.3:0/2645404003 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fa5e811f160 con 0x7fa5e00039f0 2024-09-12T12:23:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.468+0000 7fa5f7fff640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5e40678c0 0x7fa5e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.472+0000 7fa5f7fff640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5e40678c0 0x7fa5e4069d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec000c10 tx=0x7fa5ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.472+0000 7fa5f57fa640 1 -- 172.21.15.3:0/2645404003 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa5e8016950 con 0x7fa5e00039f0 2024-09-12T12:23:41.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.652+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5f80b4a50 con 0x7fa5e00039f0 2024-09-12T12:23:41.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.652+0000 7fa5f57fa640 1 -- 172.21.15.3:0/2645404003 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7fa5e80efdd0 con 0x7fa5e00039f0 2024-09-12T12:23:41.655 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5e40678c0 msgr2=0x7fa5e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5e40678c0 0x7fa5e4069d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec000c10 tx=0x7fa5ec009040 comp rx=0 tx=0).stop 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 msgr2=0x7fa5f81f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5f81f2e90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa5e801ca70 tx=0x7fa5e0005840 comp rx=0 tx=0).stop 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 shutdown_connections 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5e40678c0 0x7fa5e4069d80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5f814fcd0 0x7fa5f815e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5f814f850 0x7fa5f81500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 --2- 172.21.15.3:0/2645404003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5e00039f0 0x7fa5f81f2e90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 >> 172.21.15.3:0/2645404003 conn(0x7fa5f8067e00 msgr2=0x7fa5f815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:41.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 shutdown_connections 2024-09-12T12:23:41.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.656+0000 7fa5fee54640 1 -- 172.21.15.3:0/2645404003 wait complete. 2024-09-12T12:23:41.678 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:23:41.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:41.678 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:23:41.691 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36259]: fuse finished with error 0 and tester_r 0 2024-09-12T12:23:41.693 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:41.738 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:23:41.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:41.738 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:23:41.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:41.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 Processor -- start 2024-09-12T12:23:41.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 -- start start 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d04073cb0 0x7f2d0415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d040b30e0 0x7f2d040adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2d040ae150 0x7f2d040ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2d04074090 con 0x7f2d04073cb0 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2d040b5000 con 0x7f2d040ae150 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0be4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2d040b3530 con 0x7f2d040b30e0 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d04073cb0 0x7f2d0415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2d04073cb0 0x7f2d0415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39794/0 (socket says 172.21.15.3:39794) 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 -- 172.21.15.3:0/2917108747 learned_addr learned my addr 172.21.15.3:0/2917108747 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d0a3eb640 1 --2- 172.21.15.3:0/2917108747 >> v2:172.21.15.142:3300/0 conn(0x7f2d040ae150 0x7f2d040ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d093e9640 1 --2- 172.21.15.3:0/2917108747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d040b30e0 0x7f2d040adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 -- 172.21.15.3:0/2917108747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d040b30e0 msgr2=0x7f2d040adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:41.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 --2- 172.21.15.3:0/2917108747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d040b30e0 0x7f2d040adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 -- 172.21.15.3:0/2917108747 >> v2:172.21.15.142:3300/0 conn(0x7f2d040ae150 msgr2=0x7f2d040ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:41.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 --2- 172.21.15.3:0/2917108747 >> v2:172.21.15.142:3300/0 conn(0x7f2d040ae150 0x7f2d040ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.872+0000 7f2d09bea640 1 -- 172.21.15.3:0/2917108747 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d040b37c0 con 0x7f2d04073cb0 2024-09-12T12:23:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d09bea640 1 --2- 172.21.15.3:0/2917108747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04073cb0 0x7f2d0415c710 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d040b4cd0 tx=0x7f2cfc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df81fa501ead0286 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d08be8640 1 -- 172.21.15.3:0/2917108747 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0b0070 con 0x7f2d04073cb0 2024-09-12T12:23:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d08be8640 1 -- 172.21.15.3:0/2917108747 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cfc0ab600 con 0x7f2d04073cb0 2024-09-12T12:23:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d08be8640 1 -- 172.21.15.3:0/2917108747 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0ab9d0 con 0x7f2d04073cb0 2024-09-12T12:23:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2917108747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04073cb0 msgr2=0x7f2d0415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2917108747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04073cb0 0x7f2d0415c710 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2d040b4cd0 tx=0x7f2cfc01c520 comp rx=0 tx=0).stop 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2917108747 shutdown_connections 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2917108747 >> v2:172.21.15.142:3300/0 conn(0x7f2d040ae150 0x7f2d040ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2917108747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d040b30e0 0x7f2d040adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2917108747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d04073cb0 0x7f2d0415c710 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2917108747 >> 172.21.15.3:0/2917108747 conn(0x7f2d04067e00 msgr2=0x7f2d04068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2917108747 shutdown_connections 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2917108747 wait complete. 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 Processor -- start 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- start start 2024-09-12T12:23:41.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d04073cb0 0x7f2d041f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d040ae150 0x7f2d0415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d040b30e0 0x7f2d0415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0415fe80 con 0x7f2d040b30e0 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0415fd00 con 0x7f2d040ae150 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0be4e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2d04160000 con 0x7f2d04073cb0 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.876+0000 7f2d0a3eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d040b30e0 0x7f2d0415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d040b30e0 0x7f2d0415e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39798/0 (socket says 172.21.15.3:39798) 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 -- 172.21.15.3:0/2932705487 learned_addr learned my addr 172.21.15.3:0/2932705487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d09bea640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d04073cb0 0x7f2d041f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d093e9640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d040ae150 0x7f2d0415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 -- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d04073cb0 msgr2=0x7f2d041f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d04073cb0 0x7f2d041f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 -- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d040ae150 msgr2=0x7f2d0415dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d040ae150 0x7f2d0415dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 -- 172.21.15.3:0/2932705487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d04154710 con 0x7f2d040b30e0 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0a3eb640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d040b30e0 0x7f2d0415e320 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400d800 tx=0x7f2cf400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d027fc640 1 -- 172.21.15.3:0/2932705487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400b6c0 con 0x7f2d040b30e0 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d027fc640 1 -- 172.21.15.3:0/2932705487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf4019070 con 0x7f2d040b30e0 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d027fc640 1 -- 172.21.15.3:0/2932705487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400baa0 con 0x7f2d040b30e0 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d041549f0 con 0x7f2d040b30e0 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0409b4f0 con 0x7f2d040b30e0 2024-09-12T12:23:41.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d040ae880 con 0x7f2d040b30e0 2024-09-12T12:23:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d027fc640 1 -- 172.21.15.3:0/2932705487 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2cf400bc40 con 0x7f2d040b30e0 2024-09-12T12:23:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d027fc640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2cf00678c0 0x7f2cf0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d027fc640 1 -- 172.21.15.3:0/2932705487 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f2cf40860d0 con 0x7f2d040b30e0 2024-09-12T12:23:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d09bea640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2cf00678c0 0x7f2cf0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.880+0000 7f2d09bea640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2cf00678c0 0x7f2cf0069d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0ad000 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:41.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:41.884+0000 7f2d027fc640 1 -- 172.21.15.3:0/2932705487 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2cf4056c60 con 0x7f2d040b30e0 2024-09-12T12:23:42.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.084+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d0415ad60 con 0x7f2d040b30e0 2024-09-12T12:23:42.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.084+0000 7f2d027fc640 1 -- 172.21.15.3:0/2932705487 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f2cf4056560 con 0x7f2d040b30e0 2024-09-12T12:23:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:42.088 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:17.423558+0000","modified":"2024-09-12T12:23:20.482181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/550473013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":550473013},{"type":"v1","addr":"172.21.15.3:6835","nonce":550473013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6012,"qdb_cluster":[6012]},"id":5}]} 2024-09-12T12:23:42.088 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-12T12:23:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2cf00678c0 msgr2=0x7f2cf0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2cf00678c0 0x7f2cf0069d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0ad000 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2024-09-12T12:23:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d040b30e0 msgr2=0x7f2d0415e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d040b30e0 0x7f2d0415e320 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400d800 tx=0x7f2cf400dcc0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 shutdown_connections 2024-09-12T12:23:42.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2cf00678c0 0x7f2cf0069d80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d040b30e0 0x7f2d0415e320 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d040ae150 0x7f2d0415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 --2- 172.21.15.3:0/2932705487 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d04073cb0 0x7f2d041f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 >> 172.21.15.3:0/2932705487 conn(0x7f2d04067e00 msgr2=0x7f2d0415b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 shutdown_connections 2024-09-12T12:23:42.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.088+0000 7f2d0be4e640 1 -- 172.21.15.3:0/2932705487 wait complete. 2024-09-12T12:23:42.108 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-12T12:23:42.252 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":27,"msgr_send_messages":29,"msgr_recv_bytes":7184,"msgr_send_bytes":21032,"msgr_created_connections":7,"msgr_active_connections":2,"msgr_running_total_time":0.004788464,"msgr_running_send_time":0.000602337,"msgr_running_recv_time":0.002388205,"msgr_running_fast_dispatch_time":0.000702065,"msgr_send_messages_queue_lat":{"avgcount":28,"sum":0.059264588,"avgtime":0.002116592},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":31,"msgr_send_messages":33,"msgr_recv_bytes":8934,"msgr_send_bytes":131668,"msgr_created_connections":8,"msgr_active_connections":3,"msgr_running_total_time":0.009357180,"msgr_running_send_time":0.001675123,"msgr_running_recv_time":0.006749340,"msgr_running_fast_dispatch_time":0.000724672,"msgr_send_messages_queue_lat":{"avgcount":32,"sum":0.211164108,"avgtime":0.006598878},"msgr_handle_ack_lat":{"avgcount":19,"sum":0.000004990,"avgtime":0.000000262},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":99568},"AsyncMessenger::Worker-2":{"msgr_recv_messages":75,"msgr_send_messages":61,"msgr_recv_bytes":117264,"msgr_send_bytes":16698,"msgr_created_connections":5,"msgr_active_connections":3,"msgr_running_total_time":0.011266570,"msgr_running_send_time":0.003057593,"msgr_running_recv_time":0.012977643,"msgr_running_fast_dispatch_time":0.000725949,"msgr_send_messages_queue_lat":{"avgcount":60,"sum":0.018363192,"avgtime":0.000306053},"msgr_handle_ack_lat":{"avgcount":18,"sum":0.000004482,"avgtime":0.000000249},"msgr_recv_encrypted_bytes":111120,"msgr_send_encrypted_bytes":13024},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":49,"sum":0.005045260,"avgtime":0.000102964}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000015279,"avgtime":0.000007639}},"mds":{"request":3,"reply":3,"reply_latency":{"avgcount":3,"sum":0.000987512,"avgtime":0.000329170},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":5,"traverse_hit":5,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":7,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197227,"rdpos":0,"jlat":{"avgcount":5,"sum":0.013686093,"avgtime":0.002737218},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":39004,"heap":231732},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":8,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":2,"sum":0.000718317,"avgtime":0.000359158},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000269195,"avgtime":0.000269195},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":2,"average_load":1,"avg_session_uptime":5,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":213411,"buffer_anon_items":61,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5504,"osdmap_items":79,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52712,"mds_co_items":212,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":14133,"op_resend":31,"op_reply":44,"op_latency":{"avgcount":44,"sum":1.051991647,"avgtime":0.023908901},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":68,"map_full":0,"map_inc":3,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":128,"get_sum":117524,"get_or_fail_fail":0,"get_or_fail_success":128,"take":0,"take_sum":0,"put":128,"put_sum":117524,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":14133,"put":44,"put_sum":14133,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5601e13806e0":{"val":0,"max":3758096384,"get_started":0,"get":7,"get_sum":2923,"get_or_fail_fail":0,"get_or_fail_success":7,"take":0,"take_sum":0,"put":5,"put_sum":2923,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-12T12:23:42.254 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 7184, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 27, "msgr_running_fast_dispatch_time": 0.000702065, "msgr_running_recv_time": 0.002388205, "msgr_running_send_time": 0.000602337, "msgr_running_total_time": 0.004788464, "msgr_send_bytes": 21032, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 29, "msgr_send_messages_queue_lat": { "avgcount": 28, "avgtime": 0.002116592, "sum": 0.059264588 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 19, "avgtime": 2.62e-07, "sum": 4.99e-06 }, "msgr_recv_bytes": 8934, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 31, "msgr_running_fast_dispatch_time": 0.000724672, "msgr_running_recv_time": 0.00674934, "msgr_running_send_time": 0.001675123, "msgr_running_total_time": 0.00935718, "msgr_send_bytes": 131668, "msgr_send_encrypted_bytes": 99568, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 32, "avgtime": 0.006598878, "sum": 0.211164108 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 18, "avgtime": 2.49e-07, "sum": 4.482e-06 }, "msgr_recv_bytes": 117264, "msgr_recv_encrypted_bytes": 111120, "msgr_recv_messages": 75, "msgr_running_fast_dispatch_time": 0.000725949, "msgr_running_recv_time": 0.012977643, "msgr_running_send_time": 0.003057593, "msgr_running_total_time": 0.01126657, "msgr_send_bytes": 16698, "msgr_send_encrypted_bytes": 13024, "msgr_send_messages": 61, "msgr_send_messages_queue_lat": { "avgcount": 60, "avgtime": 0.000306053, "sum": 0.018363192 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 49, "avgtime": 0.000102964, "sum": 0.00504526 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 7.639e-06, "sum": 1.5279e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 3, "reply_latency": { "avgcount": 3, "avgtime": 0.00032917, "sum": 0.000987512 }, "request": 3, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 5, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 5, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 7, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.002737218, "sum": 0.013686093 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197227 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 39004 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 8, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 2, "avgtime": 0.000359158, "sum": 0.000718317 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000269195, "sum": 0.000269195 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 5, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 2 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 213411, "buffer_anon_items": 61, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52712, "mds_co_items": 212, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5504, "osdmap_items": 79, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 68, "map_full": 0, "map_inc": 3, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.023908901, "sum": 1.051991647 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 31, "op_rmw": 0, "op_send": 44, "op_send_bytes": 14133, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 128, "get_or_fail_fail": 0, "get_or_fail_success": 128, "get_started": 0, "get_sum": 117524, "max": 104857600, "put": 128, "put_sum": 117524, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 14133, "take": 44, "take_sum": 14133, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5601e13806e0": { "get": 7, "get_or_fail_fail": 0, "get_or_fail_success": 7, "get_started": 0, "get_sum": 2923, "max": 3758096384, "put": 5, "put_sum": 2923, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-12T12:23:42.255 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 session ls 2024-09-12T12:23:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28c738640 1 Processor -- start 2024-09-12T12:23:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28c738640 1 -- start start 2024-09-12T12:23:42.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28c738640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa2840ce0c0 0x7fa2840d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28c738640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2840d0b00 0x7fa2840ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28c738640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840ad1f0 con 0x7fa28415b420 2024-09-12T12:23:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28c738640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa28406e310 con 0x7fa2840d0b00 2024-09-12T12:23:42.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28c738640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa2840d0f50 con 0x7fa2840ce0c0 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.376+0000 7fa28acd5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2840d0b00 0x7fa2840ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28acd5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa2840d0b00 0x7fa2840ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39810/0 (socket says 172.21.15.3:39810) 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28acd5640 1 -- 172.21.15.3:0/4060847132 learned_addr learned my addr 172.21.15.3:0/4060847132 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa289cd3640 1 --2- 172.21.15.3:0/4060847132 >> v2:172.21.15.142:3300/0 conn(0x7fa2840ce0c0 0x7fa2840d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28acd5640 1 -- 172.21.15.3:0/4060847132 >> v2:172.21.15.142:3300/0 conn(0x7fa2840ce0c0 msgr2=0x7fa2840d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28acd5640 1 --2- 172.21.15.3:0/4060847132 >> v2:172.21.15.142:3300/0 conn(0x7fa2840ce0c0 0x7fa2840d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28acd5640 1 -- 172.21.15.3:0/4060847132 >> v1:172.21.15.142:6789/0 conn(0x7fa28415b420 legacy=0x7fa28406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28acd5640 1 -- 172.21.15.3:0/4060847132 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28406da70 con 0x7fa2840d0b00 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28acd5640 1 --2- 172.21.15.3:0/4060847132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa2840ad6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa2840ccc10 tx=0x7fa27c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198da3b6ddb853c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa2894d2640 1 -- 172.21.15.3:0/4060847132 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0ab610 con 0x7fa2840d0b00 2024-09-12T12:23:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa2894d2640 1 -- 172.21.15.3:0/4060847132 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c0b0070 con 0x7fa2840d0b00 2024-09-12T12:23:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa2894d2640 1 -- 172.21.15.3:0/4060847132 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0ab9d0 con 0x7fa2840d0b00 2024-09-12T12:23:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 -- 172.21.15.3:0/4060847132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 msgr2=0x7fa2840ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 --2- 172.21.15.3:0/4060847132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa2840ad6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa2840ccc10 tx=0x7fa27c01c520 comp rx=0 tx=0).stop 2024-09-12T12:23:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 -- 172.21.15.3:0/4060847132 shutdown_connections 2024-09-12T12:23:42.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 --2- 172.21.15.3:0/4060847132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa2840ad6d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 --2- 172.21.15.3:0/4060847132 >> v2:172.21.15.142:3300/0 conn(0x7fa2840ce0c0 0x7fa2840d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 -- 172.21.15.3:0/4060847132 >> 172.21.15.3:0/4060847132 conn(0x7fa284067e00 msgr2=0x7fa284068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 -- 172.21.15.3:0/4060847132 shutdown_connections 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.380+0000 7fa28c738640 1 -- 172.21.15.3:0/4060847132 wait complete. 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 Processor -- start 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 -- start start 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2840ce0c0 0x7fa284150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa284147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28415b420 0x7fa2841483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2840ae7f0 con 0x7fa2840d0b00 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2840ae670 con 0x7fa2840ce0c0 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa2840ae970 con 0x7fa28415b420 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa284147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa284147d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39818/0 (socket says 172.21.15.3:39818) 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 -- 172.21.15.3:0/2631863184 learned_addr learned my addr 172.21.15.3:0/2631863184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28acd5640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28415b420 0x7fa2841483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2840ce0c0 0x7fa284150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 -- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28415b420 msgr2=0x7fa2841483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28415b420 0x7fa2841483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 -- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2840ce0c0 msgr2=0x7fa284150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2840ce0c0 0x7fa284150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 -- 172.21.15.3:0/2631863184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa284068fb0 con 0x7fa2840d0b00 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28acd5640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28415b420 0x7fa2841483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa289cd3640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa284147d30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa27400d800 tx=0x7fa27400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa27affd640 1 -- 172.21.15.3:0/2631863184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27400b6c0 con 0x7fa2840d0b00 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa27affd640 1 -- 172.21.15.3:0/2631863184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa274019070 con 0x7fa2840d0b00 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa284153e40 con 0x7fa2840d0b00 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa27affd640 1 -- 172.21.15.3:0/2631863184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27400bae0 con 0x7fa2840d0b00 2024-09-12T12:23:42.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.384+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa284148f00 con 0x7fa2840d0b00 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa27affd640 1 -- 172.21.15.3:0/2631863184 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa27400bc80 con 0x7fa2840d0b00 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa27affd640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2680678c0 0x7fa268069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa27affd640 1 -- 172.21.15.3:0/2631863184 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fa274086220 con 0x7fa2840d0b00 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2680678c0 0x7fa268069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 Processor -- start 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- start start 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2680678c0 0x7fa268069d80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa280001c20 tx=0x7fa280009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900caab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca61290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca61bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55900c948c40 con 0x55900ca617d0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55900c8afd50 con 0x55900caabca0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55900c86aab0 con 0x55900caab380 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca61bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca61bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39820/0 (socket says 172.21.15.3:39820) 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 -- 172.21.15.3:0/1510583360 learned_addr learned my addr 172.21.15.3:0/1510583360 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900caab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa289cd3640 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca61290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 -- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 msgr2=0x55900caab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900caab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 -- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 msgr2=0x55900ca61290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca61290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 -- 172.21.15.3:0/1510583360 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900c856590 con 0x55900ca617d0 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900caab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa289cd3640 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca61290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28acd5640 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca61bb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa27c01c560 tx=0x7fa27c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1988b85059a5f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa267fff640 1 -- 172.21.15.3:0/1510583360 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0ab950 con 0x55900ca617d0 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa267fff640 1 -- 172.21.15.3:0/1510583360 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c0b0040 con 0x55900ca617d0 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa267fff640 1 -- 172.21.15.3:0/1510583360 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0abcf0 con 0x55900ca617d0 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1510583360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 msgr2=0x55900ca61bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca61bb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa27c01c560 tx=0x7fa27c0a7000 comp rx=0 tx=0).stop 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1510583360 shutdown_connections 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca61bb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca61290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1510583360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900caab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1510583360 >> 172.21.15.3:0/1510583360 conn(0x55900ca58bc0 msgr2=0x55900ca5fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1510583360 shutdown_connections 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.388+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1510583360 wait complete. 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- start start 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 Processor -- start 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900ca955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca95b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55900c86e400 con 0x55900ca617d0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55900c96c3a0 con 0x55900caabca0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55900c8ef260 con 0x55900caab380 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 10 client.0 _ll_register_callbacks cb 0x55900c911020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca66880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39826/0 (socket says 172.21.15.3:39826) 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 -- 172.21.15.3:0/265423244 learned_addr learned my addr 172.21.15.3:0/265423244 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa289cd3640 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900ca955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28acd5640 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca95b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 -- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 msgr2=0x55900ca955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900ca955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 -- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 msgr2=0x55900ca95b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca95b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 -- 172.21.15.3:0/265423244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900c852dd0 con 0x55900ca617d0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca66880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa28000dfe0 tx=0x7fa280000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8d1534fca61689 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 1 -- 172.21.15.3:0/265423244 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2800127e0 con 0x55900ca617d0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 1 -- 172.21.15.3:0/265423244 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fa280012980 con 0x55900ca617d0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27c0b0040 con 0x55900ca617d0 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 1 -- 172.21.15.3:0/265423244 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa28002c050 con 0x55900ca617d0 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 10 client.6237 fetch_fsmap learned FSMap version 45 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55900c84f800 con 0x55900ca617d0 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 1 -- 172.21.15.3:0/265423244 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c0b0040 con 0x55900ca617d0 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 1 -- 172.21.15.3:0/265423244 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7fa28002f020 con 0x55900ca617d0 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 10 client.6237 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 10 client.6237 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 20 client.6237 populate_metadata read hostname 'smithi003' 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900ca66dc0 0x55900cab21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 4 client.6237 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 --> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900c849e20 con 0x55900ca66dc0 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa289cd3640 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900ca66dc0 0x55900cab21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 10 client.6237 ms_handle_connect on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa289cd3640 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900ca66dc0 0x55900cab21c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9484104e0287046e server_cookie=b4d6242bda6bbf67 in_seq=0 out_seq=0 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 1 -- 172.21.15.3:0/265423244 <== mds.0 v2:172.21.15.3:6833/550473013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55900c849e20 con 0x55900ca66dc0 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 10 client.6237 handle_command_reply: tid=0 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 client.6237 shutdown 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 msgr2=0x55900ca66880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca66880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa28000dfe0 tx=0x7fa280000f50 comp rx=0 tx=0).stop 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 shutdown_connections 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900ca66dc0 0x55900cab21c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caabca0 0x55900ca95b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900ca955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa2657fa640 0 client.6237 ms_handle_reset on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/265423244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900ca617d0 0x55900ca66880 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 >> 172.21.15.3:0/265423244 conn(0x55900ca58bc0 msgr2=0x55900ca60280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 shutdown_connections 2024-09-12T12:23:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 1 -- 172.21.15.3:0/265423244 wait complete. 2024-09-12T12:23:42.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.392+0000 7fa28f7f4000 20 client.6237 trim_cache size 0 max 16384 2024-09-12T12:23:42.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 Processor -- start 2024-09-12T12:23:42.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 -- start start 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900ca617d0 0x55900cb1a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caab380 0x55900cb1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cb1adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa2842028e0 con 0x55900caabca0 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2841552c0 con 0x55900caab380 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa284202290 con 0x55900ca617d0 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cb1adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cb1adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39842/0 (socket says 172.21.15.3:39842) 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 -- 172.21.15.3:0/4247296890 learned_addr learned my addr 172.21.15.3:0/4247296890 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900ca617d0 0x55900cb1a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 -- 172.21.15.3:0/4247296890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900ca617d0 msgr2=0x55900cb1a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900ca617d0 0x55900cb1a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 -- 172.21.15.3:0/4247296890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caab380 msgr2=0x55900cb1a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caab380 0x55900cb1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 -- 172.21.15.3:0/4247296890 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900c86d480 con 0x55900caabca0 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa28acd5640 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cb1adf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa27c01c560 tx=0x7fa27c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cdd280783a0257 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa2677fe640 1 -- 172.21.15.3:0/4247296890 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c004300 con 0x55900caabca0 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa2677fe640 1 -- 172.21.15.3:0/4247296890 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c0c4070 con 0x55900caabca0 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.412+0000 7fa2677fe640 1 -- 172.21.15.3:0/4247296890 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0ab990 con 0x55900caabca0 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/4247296890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 msgr2=0x55900cb1adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cb1adf0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa27c01c560 tx=0x7fa27c004740 comp rx=0 tx=0).stop 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/4247296890 shutdown_connections 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cb1adf0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900caab380 0x55900cb1a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/4247296890 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900ca617d0 0x55900cb1a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/4247296890 >> 172.21.15.3:0/4247296890 conn(0x55900ca58bc0 msgr2=0x55900ca94270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/4247296890 shutdown_connections 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/4247296890 wait complete. 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- start start 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 Processor -- start 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900ca617d0 0x55900cb1acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900cac0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cac1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55900c855f30 con 0x55900caabca0 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55900c86e7d0 con 0x55900ca617d0 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55900c8af060 con 0x55900caab380 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 10 client.0 _ll_register_callbacks cb 0x7fa280001920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:42.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cac1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cac1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39844/0 (socket says 172.21.15.3:39844) 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 -- 172.21.15.3:0/1486773304 learned_addr learned my addr 172.21.15.3:0/1486773304 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa289cd3640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900cac0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900ca617d0 0x55900cb1acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 -- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 msgr2=0x55900cac0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900cac0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 -- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900ca617d0 msgr2=0x55900cb1acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900ca617d0 0x55900cb1acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 -- 172.21.15.3:0/1486773304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900c843650 con 0x55900caabca0 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa289cd3640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900cac0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28acd5640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cac1020 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa27c01abd0 tx=0x7fa27c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5236994db463901 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 1 -- 172.21.15.3:0/1486773304 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0cd020 con 0x55900caabca0 2024-09-12T12:23:42.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 1 -- 172.21.15.3:0/1486773304 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fa27c0ab810 con 0x55900caabca0 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27c0c4070 con 0x55900caabca0 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 1 -- 172.21.15.3:0/1486773304 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa27c0d4050 con 0x55900caabca0 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 10 client.6243 fetch_fsmap learned FSMap version 45 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55900c8473c0 con 0x55900caabca0 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 1 -- 172.21.15.3:0/1486773304 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa27c0c4070 con 0x55900caabca0 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 1 -- 172.21.15.3:0/1486773304 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7fa27c0d7020 con 0x55900caabca0 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 10 client.6243 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:23:42.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 10 client.6243 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-12T12:23:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 20 client.6243 populate_metadata read hostname 'smithi003' 2024-09-12T12:23:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900cb1b1f0 0x55900cac1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 4 client.6243 mds_command: new command op to 6012 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-09-12T12:23:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 --> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] -- command(tid 0: {"prefix": "session ls"}) -- 0x55900c8bd480 con 0x55900cb1b1f0 2024-09-12T12:23:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900cb1b1f0 0x55900cac1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 10 client.6243 ms_handle_connect on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:42.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28a4d4640 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900cb1b1f0 0x55900cac1560 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d34172950e90c741 server_cookie=90c9062ab845e7d7 in_seq=0 out_seq=0 2024-09-12T12:23:42.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 1 -- 172.21.15.3:0/1486773304 <== mds.0 v2:172.21.15.3:6833/550473013 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55900c8bd480 con 0x55900cb1b1f0 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 10 client.6243 handle_command_reply: tid=0 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 client.6243 shutdown 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 msgr2=0x55900cac1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cac1020 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa27c01abd0 tx=0x7fa27c01ad60 comp rx=0 tx=0).stop 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 shutdown_connections 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:6833/550473013,v1:172.21.15.3:6835/550473013] conn(0x55900cb1b1f0 0x55900cac1560 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55900caabca0 0x55900cac1020 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55900caab380 0x55900cac0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 --2- 172.21.15.3:0/1486773304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55900ca617d0 0x55900cb1acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa2657fa640 0 client.6243 ms_handle_reset on v2:172.21.15.3:6833/550473013 2024-09-12T12:23:42.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 >> 172.21.15.3:0/1486773304 conn(0x55900ca58bc0 msgr2=0x55900ca94a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 shutdown_connections 2024-09-12T12:23:42.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 1 -- 172.21.15.3:0/1486773304 wait complete. 2024-09-12T12:23:42.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.416+0000 7fa28f7f4000 20 client.6243 trim_cache size 0 max 16384 2024-09-12T12:23:42.434 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2680678c0 msgr2=0x7fa268069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2680678c0 0x7fa268069d80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa280001c20 tx=0x7fa280009040 comp rx=0 tx=0).stop 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 msgr2=0x7fa284147d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa284147d30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa27400d800 tx=0x7fa27400dcc0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 shutdown_connections 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa2680678c0 0x7fa268069d80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28415b420 0x7fa2841483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa2840d0b00 0x7fa284147d30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 --2- 172.21.15.3:0/2631863184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2840ce0c0 0x7fa284150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 >> 172.21.15.3:0/2631863184 conn(0x7fa284067e00 msgr2=0x7fa284157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 shutdown_connections 2024-09-12T12:23:42.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:42.420+0000 7fa28c738640 1 -- 172.21.15.3:0/2631863184 wait complete. 2024-09-12T12:23:42.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-12T12:23:42.568 INFO:teuthology.orchestra.run.smithi003.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":27,"msgr_send_messages":29,"msgr_recv_bytes":7184,"msgr_send_bytes":21032,"msgr_created_connections":7,"msgr_active_connections":2,"msgr_running_total_time":0.004888022,"msgr_running_send_time":0.000602337,"msgr_running_recv_time":0.002388205,"msgr_running_fast_dispatch_time":0.000702065,"msgr_send_messages_queue_lat":{"avgcount":28,"sum":0.059264588,"avgtime":0.002116592},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":33,"msgr_send_messages":35,"msgr_recv_bytes":9224,"msgr_send_bytes":156038,"msgr_created_connections":10,"msgr_active_connections":3,"msgr_running_total_time":0.010155142,"msgr_running_send_time":0.001744333,"msgr_running_recv_time":0.007409574,"msgr_running_fast_dispatch_time":0.000724672,"msgr_send_messages_queue_lat":{"avgcount":34,"sum":0.211191970,"avgtime":0.006211528},"msgr_handle_ack_lat":{"avgcount":21,"sum":0.000005164,"avgtime":0.000000245},"msgr_recv_encrypted_bytes":1728,"msgr_send_encrypted_bytes":99568},"AsyncMessenger::Worker-2":{"msgr_recv_messages":75,"msgr_send_messages":61,"msgr_recv_bytes":117264,"msgr_send_bytes":16698,"msgr_created_connections":5,"msgr_active_connections":3,"msgr_running_total_time":0.011266570,"msgr_running_send_time":0.003057593,"msgr_running_recv_time":0.012977643,"msgr_running_fast_dispatch_time":0.000725949,"msgr_send_messages_queue_lat":{"avgcount":60,"sum":0.018363192,"avgtime":0.000306053},"msgr_handle_ack_lat":{"avgcount":18,"sum":0.000004482,"avgtime":0.000000249},"msgr_recv_encrypted_bytes":111120,"msgr_send_encrypted_bytes":13024},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":49,"sum":0.005045260,"avgtime":0.000102964}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000015279,"avgtime":0.000007639}},"mds":{"request":3,"reply":3,"reply_latency":{"avgcount":3,"sum":0.000987512,"avgtime":0.000329170},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":0,"inodes_bottom":10,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":5,"traverse_hit":5,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":7,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4197227,"rdpos":0,"jlat":{"avgcount":5,"sum":0.013686093,"avgtime":0.002737218},"replayed":0},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":0,"cap+":2,"cap-":2,"rss":39004,"heap":231732},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":8,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":2,"sum":0.000718317,"avgtime":0.000359158},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000269195,"avgtime":0.000269195},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":0,"session_add":2,"session_remove":2,"sessions_open":0,"sessions_stale":0,"total_load":2,"average_load":1,"avg_session_uptime":5,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":213411,"buffer_anon_items":61,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5504,"osdmap_items":79,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":52712,"mds_co_items":212,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":14133,"op_resend":31,"op_reply":44,"op_latency":{"avgcount":44,"sum":1.051991647,"avgtime":0.023908901},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":5,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":68,"map_full":0,"map_inc":3,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":130,"get_sum":117634,"get_or_fail_fail":0,"get_or_fail_success":130,"take":0,"take_sum":0,"put":130,"put_sum":117634,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":14133,"put":44,"put_sum":14133,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":44,"take_sum":44,"put":44,"put_sum":44,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x5601e13806e0":{"val":0,"max":3758096384,"get_started":0,"get":7,"get_sum":2923,"get_or_fail_fail":0,"get_or_fail_success":7,"take":0,"take_sum":0,"put":5,"put_sum":2923,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-12T12:23:42.570 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 7184, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 27, "msgr_running_fast_dispatch_time": 0.000702065, "msgr_running_recv_time": 0.002388205, "msgr_running_send_time": 0.000602337, "msgr_running_total_time": 0.004888022, "msgr_send_bytes": 21032, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 29, "msgr_send_messages_queue_lat": { "avgcount": 28, "avgtime": 0.002116592, "sum": 0.059264588 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 3, "msgr_created_connections": 10, "msgr_handle_ack_lat": { "avgcount": 21, "avgtime": 2.45e-07, "sum": 5.164e-06 }, "msgr_recv_bytes": 9224, "msgr_recv_encrypted_bytes": 1728, "msgr_recv_messages": 33, "msgr_running_fast_dispatch_time": 0.000724672, "msgr_running_recv_time": 0.007409574, "msgr_running_send_time": 0.001744333, "msgr_running_total_time": 0.010155142, "msgr_send_bytes": 156038, "msgr_send_encrypted_bytes": 99568, "msgr_send_messages": 35, "msgr_send_messages_queue_lat": { "avgcount": 34, "avgtime": 0.006211528, "sum": 0.21119197 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 3, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 18, "avgtime": 2.49e-07, "sum": 4.482e-06 }, "msgr_recv_bytes": 117264, "msgr_recv_encrypted_bytes": 111120, "msgr_recv_messages": 75, "msgr_running_fast_dispatch_time": 0.000725949, "msgr_running_recv_time": 0.012977643, "msgr_running_send_time": 0.003057593, "msgr_running_total_time": 0.01126657, "msgr_send_bytes": 16698, "msgr_send_encrypted_bytes": 13024, "msgr_send_messages": 61, "msgr_send_messages_queue_lat": { "avgcount": 60, "avgtime": 0.000306053, "sum": 0.018363192 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 49, "avgtime": 0.000102964, "sum": 0.00504526 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 7.639e-06, "sum": 1.5279e-05 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 10, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 0, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 3, "reply_latency": { "avgcount": 3, "avgtime": 0.00032917, "sum": 0.000987512 }, "request": 3, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 5, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 5, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 7, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.002737218, "sum": 0.013686093 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4197227 }, "mds_mem": { "cap": 0, "cap+": 2, "cap-": 2, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 231732, "ino": 13, "ino+": 13, "ino-": 0, "rss": 39004 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 8, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 2, "avgtime": 0.000359158, "sum": 0.000718317 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000269195, "sum": 0.000269195 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 5, "mdthresh_evicted": 0, "session_add": 2, "session_count": 0, "session_remove": 2, "sessions_open": 0, "sessions_stale": 0, "total_load": 2 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 213411, "buffer_anon_items": 61, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 52712, "mds_co_items": 212, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5504, "osdmap_items": 79, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 68, "map_full": 0, "map_inc": 3, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.023908901, "sum": 1.051991647 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 31, "op_rmw": 0, "op_send": 44, "op_send_bytes": 14133, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 130, "get_or_fail_fail": 0, "get_or_fail_success": 130, "get_started": 0, "get_sum": 117634, "max": 104857600, "put": 130, "put_sum": 117634, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 44, "put_sum": 14133, "take": 44, "take_sum": 14133, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 44, "put_sum": 44, "take": 44, "take_sum": 44, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x5601e13806e0": { "get": 7, "get_or_fail_fail": 0, "get_or_fail_success": 7, "get_started": 0, "get_sum": 2923, "max": 3758096384, "put": 5, "put_sum": 2923, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-12T12:23:42.570 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-12T12:23:42.570 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:23:42.570 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:42.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:42.669 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:23:42.669 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:42.669 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:42.669 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:42.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:42.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:23:42.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:42.675 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:23:42.725 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-12T12:23:42.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:42.726 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:42.727 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:42.727 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:42.733 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:42.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:23:42.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:42.733 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:23:42.780 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-12T12:23:42.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:42.781 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:42.782 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:42.782 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:42.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:42.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:23:42.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:42.789 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T12:23:42.838 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-12T12:23:42.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:42.839 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:23:42.839 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:42.839 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:42.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:42.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:23:42.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:23:42.845 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T12:23:42.892 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-12T12:23:42.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:23:42.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:43.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9f40f640 1 Processor -- start 2024-09-12T12:23:43.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9f40f640 1 -- start start 2024-09-12T12:23:43.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9f40f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 0x7f1b98070d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9f40f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1b980ae110 0x7f1b9806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9f40f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b980b3280 con 0x7f1b9806e870 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9f40f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1b980b3560 con 0x7f1b980ae110 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9f40f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b980b3840 con 0x7f1b980add30 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d9ac640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1b9806e870 0x7f1b9815a3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34766/0 (socket says 172.21.15.3:34766) 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d9ac640 1 -- 172.21.15.3:0/2525602999 learned_addr learned my addr 172.21.15.3:0/2525602999 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 0x7f1b98070d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9c9aa640 1 --2- 172.21.15.3:0/2525602999 >> v2:172.21.15.142:3300/0 conn(0x7f1b980ae110 0x7f1b9806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867013261 0 0) 0x7f1b980b3280 con 0x7f1b9806e870 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b84005740 con 0x7f1b9806e870 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1971690363 0 0) 0x7f1b84005740 con 0x7f1b9806e870 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 msgr2=0x7f1b98070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 0x7f1b98070d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 >> v2:172.21.15.142:3300/0 conn(0x7f1b980ae110 msgr2=0x7f1b9806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 --2- 172.21.15.3:0/2525602999 >> v2:172.21.15.142:3300/0 conn(0x7f1b980ae110 0x7f1b9806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b980b3b20 con 0x7f1b9806e870 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1b90001a00 con 0x7f1b9806e870 2024-09-12T12:23:43.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 >> v1:172.21.15.3:6789/0 conn(0x7f1b9806e870 legacy=0x7f1b9815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b84005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 0x7f1b84006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b84006f60 0x7f1b840073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980b3560 con 0x7f1b84005bd0 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980b3840 con 0x7f1b84006f60 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b8400dbe0 con 0x7f1b980add30 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b84005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d9ac640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b84006f60 0x7f1b840073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9c9aa640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 0x7f1b84006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d1ab640 1 -- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 msgr2=0x7f1b84006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b980add30 0x7f1b84006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d1ab640 1 -- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b84006f60 msgr2=0x7f1b840073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b84006f60 0x7f1b840073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.024+0000 7f1b9d1ab640 1 -- 172.21.15.3:0/2525602999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b84005740 con 0x7f1b84005bd0 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b84005ff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009ae0 tx=0x7f1b9401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10a28e43fdf3048 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940096b0 con 0x7f1b84005bd0 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b940a6450 con 0x7f1b84005bd0 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b8ffff640 1 -- 172.21.15.3:0/2525602999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940a6840 con 0x7f1b84005bd0 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 -- 172.21.15.3:0/2525602999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 msgr2=0x7f1b84005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 --2- 172.21.15.3:0/2525602999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b84005ff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009ae0 tx=0x7f1b9401ca70 comp rx=0 tx=0).stop 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9d9ac640 1 -- 172.21.15.3:0/2525602999 reap_dead start 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 -- 172.21.15.3:0/2525602999 shutdown_connections 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 -- 172.21.15.3:0/2525602999 >> 172.21.15.3:0/2525602999 conn(0x7f1b98067e00 msgr2=0x7f1b98068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:43.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 -- 172.21.15.3:0/2525602999 shutdown_connections 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 -- 172.21.15.3:0/2525602999 wait complete. 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 Processor -- start 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.028+0000 7f1b9f40f640 1 -- start start 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b981fa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b98071c50 0x7f1b980720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b9815cc20 0x7f1b9815d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980b43b0 con 0x7f1b84005bd0 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1b980b4230 con 0x7f1b98071c50 2024-09-12T12:23:43.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1b980b4530 con 0x7f1b9815cc20 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b981fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b981fa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39860/0 (socket says 172.21.15.3:39860) 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 -- 172.21.15.3:0/3686542177 learned_addr learned my addr 172.21.15.3:0/3686542177 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d9ac640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b9815cc20 0x7f1b9815d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9c9aa640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b98071c50 0x7f1b980720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 -- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b9815cc20 msgr2=0x7f1b9815d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b9815cc20 0x7f1b9815d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 -- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b98071c50 msgr2=0x7f1b980720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b98071c50 0x7f1b980720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 -- 172.21.15.3:0/3686542177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9815d9d0 con 0x7f1b84005bd0 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9d1ab640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b981fa880 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b9401cf50 tx=0x7f1b84007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b8dffb640 1 -- 172.21.15.3:0/3686542177 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b940a6e00 con 0x7f1b84005bd0 2024-09-12T12:23:43.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b8dffb640 1 -- 172.21.15.3:0/3686542177 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b94004020 con 0x7f1b84005bd0 2024-09-12T12:23:43.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9815dcb0 con 0x7f1b84005bd0 2024-09-12T12:23:43.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b8dffb640 1 -- 172.21.15.3:0/3686542177 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94004410 con 0x7f1b84005bd0 2024-09-12T12:23:43.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9815e270 con 0x7f1b84005bd0 2024-09-12T12:23:43.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b980b4970 con 0x7f1b84005bd0 2024-09-12T12:23:43.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b8dffb640 1 -- 172.21.15.3:0/3686542177 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1b94003620 con 0x7f1b84005bd0 2024-09-12T12:23:43.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b8dffb640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b7c0678c0 0x7f1b7c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b8dffb640 1 -- 172.21.15.3:0/3686542177 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f1b941203c0 con 0x7f1b84005bd0 2024-09-12T12:23:43.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.032+0000 7f1b9c9aa640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b7c0678c0 0x7f1b7c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.036+0000 7f1b9c9aa640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b7c0678c0 0x7f1b7c069d80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b980731a0 tx=0x7f1b88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.036+0000 7f1b8dffb640 1 -- 172.21.15.3:0/3686542177 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1b940ca080 con 0x7f1b84005bd0 2024-09-12T12:23:43.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.228+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b980b0fe0 con 0x7f1b84005bd0 2024-09-12T12:23:43.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.228+0000 7f1b8dffb640 1 -- 172.21.15.3:0/3686542177 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f1b940ac020 con 0x7f1b84005bd0 2024-09-12T12:23:43.232 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:43.232 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:17.423558+0000","modified":"2024-09-12T12:23:20.482181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/550473013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":550473013},{"type":"v1","addr":"172.21.15.3:6835","nonce":550473013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6012,"qdb_cluster":[6012]},"id":5}]} 2024-09-12T12:23:43.232 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b7c0678c0 msgr2=0x7f1b7c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b7c0678c0 0x7f1b7c069d80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b980731a0 tx=0x7f1b88008040 comp rx=0 tx=0).stop 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 msgr2=0x7f1b981fa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b981fa880 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1b9401cf50 tx=0x7f1b84007a50 comp rx=0 tx=0).stop 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 shutdown_connections 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1b7c0678c0 0x7f1b7c069d80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1b9815cc20 0x7f1b9815d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1b98071c50 0x7f1b980720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 --2- 172.21.15.3:0/3686542177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b84005bd0 0x7f1b981fa880 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 >> 172.21.15.3:0/3686542177 conn(0x7f1b98067e00 msgr2=0x7f1b9806d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 shutdown_connections 2024-09-12T12:23:43.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.232+0000 7f1b9f40f640 1 -- 172.21.15.3:0/3686542177 wait complete. 2024-09-12T12:23:43.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8777e78640 1 Processor -- start 2024-09-12T12:23:43.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8777e78640 1 -- start start 2024-09-12T12:23:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8777e78640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87700d1190 0x7f87700d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8777e78640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700d1570 0x7f87700cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8777e78640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87700d30b0 con 0x7f87700cf950 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8777e78640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f87700d3cc0 con 0x7f87700d1190 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8777e78640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f87700d3390 con 0x7f87700d1570 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775c14640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87700d1190 0x7f87700d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775c14640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f87700d1190 0x7f87700d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39870/0 (socket says 172.21.15.3:39870) 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775c14640 1 -- 172.21.15.3:0/1001204682 learned_addr learned my addr 172.21.15.3:0/1001204682 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775413640 1 --2- 172.21.15.3:0/1001204682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700d1570 0x7f87700cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775c14640 1 -- 172.21.15.3:0/1001204682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700d1570 msgr2=0x7f87700cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775c14640 1 --2- 172.21.15.3:0/1001204682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700d1570 0x7f87700cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775c14640 1 -- 172.21.15.3:0/1001204682 >> v1:172.21.15.142:6789/0 conn(0x7f87700cf950 legacy=0x7f87700b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.368+0000 7f8775c14640 1 -- 172.21.15.3:0/1001204682 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87700d3670 con 0x7f87700d1190 2024-09-12T12:23:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8775c14640 1 --2- 172.21.15.3:0/1001204682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1190 0x7f87700d4150 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f876c009870 tx=0x7f876c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6b45eed1cdef0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8774c12640 1 -- 172.21.15.3:0/1001204682 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c0aa430 con 0x7f87700d1190 2024-09-12T12:23:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8774c12640 1 -- 172.21.15.3:0/1001204682 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876c0af070 con 0x7f87700d1190 2024-09-12T12:23:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8774c12640 1 -- 172.21.15.3:0/1001204682 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c0aa7f0 con 0x7f87700d1190 2024-09-12T12:23:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- 172.21.15.3:0/1001204682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1190 msgr2=0x7f87700d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 --2- 172.21.15.3:0/1001204682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1190 0x7f87700d4150 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f876c009870 tx=0x7f876c01c4f0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- 172.21.15.3:0/1001204682 shutdown_connections 2024-09-12T12:23:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 --2- 172.21.15.3:0/1001204682 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700d1570 0x7f87700cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 --2- 172.21.15.3:0/1001204682 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1190 0x7f87700d4150 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- 172.21.15.3:0/1001204682 >> 172.21.15.3:0/1001204682 conn(0x7f8770067a00 msgr2=0x7f8770067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- 172.21.15.3:0/1001204682 shutdown_connections 2024-09-12T12:23:43.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- 172.21.15.3:0/1001204682 wait complete. 2024-09-12T12:23:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 Processor -- start 2024-09-12T12:23:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- start start 2024-09-12T12:23:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700cf950 0x7f8770156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87700d1190 0x7f8770151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1570 0x7f877015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87700b83e0 con 0x7f87700d1570 2024-09-12T12:23:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87700b8260 con 0x7f87700d1190 2024-09-12T12:23:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f877015e860 con 0x7f87700cf950 2024-09-12T12:23:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1570 0x7f877015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1570 0x7f877015b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39880/0 (socket says 172.21.15.3:39880) 2024-09-12T12:23:43.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 -- 172.21.15.3:0/3800471749 learned_addr learned my addr 172.21.15.3:0/3800471749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8775c14640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700cf950 0x7f8770156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8775413640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87700d1190 0x7f8770151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 -- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700cf950 msgr2=0x7f8770156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700cf950 0x7f8770156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 -- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87700d1190 msgr2=0x7f8770151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87700d1190 0x7f8770151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 -- 172.21.15.3:0/3800471749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87700c8220 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8775c14640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700cf950 0x7f8770156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8776415640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1570 0x7f877015b310 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f876800c780 tx=0x7f876800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f87667fc640 1 -- 172.21.15.3:0/3800471749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768004b70 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f87667fc640 1 -- 172.21.15.3:0/3800471749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768004d10 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87700c84c0 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.372+0000 7f87667fc640 1 -- 172.21.15.3:0/3800471749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876800aa90 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.376+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87700ca150 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.376+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87700cf410 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.376+0000 7f87667fc640 1 -- 172.21.15.3:0/3800471749 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f876800ac30 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.376+0000 7f87667fc640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87540678c0 0x7f8754069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.376+0000 7f87667fc640 1 -- 172.21.15.3:0/3800471749 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f8768085f70 con 0x7f87700d1570 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.376+0000 7f8775c14640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87540678c0 0x7f8754069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.376+0000 7f8775c14640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87540678c0 0x7f8754069d80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f876c0ae000 tx=0x7f876c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.380+0000 7f87667fc640 1 -- 172.21.15.3:0/3800471749 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8768020070 con 0x7f87700d1570 2024-09-12T12:23:43.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.576+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87700ba4c0 con 0x7f87700d1570 2024-09-12T12:23:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.576+0000 7f87667fc640 1 -- 172.21.15.3:0/3800471749 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f8768056a70 con 0x7f87700d1570 2024-09-12T12:23:43.580 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:43.580 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:17.423558+0000","modified":"2024-09-12T12:23:20.482181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/550473013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":550473013},{"type":"v1","addr":"172.21.15.3:6835","nonce":550473013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6012,"qdb_cluster":[6012]},"id":5}]} 2024-09-12T12:23:43.580 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87540678c0 msgr2=0x7f8754069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87540678c0 0x7f8754069d80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f876c0ae000 tx=0x7f876c008040 comp rx=0 tx=0).stop 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1570 msgr2=0x7f877015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1570 0x7f877015b310 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f876800c780 tx=0x7f876800cc40 comp rx=0 tx=0).stop 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 shutdown_connections 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f87540678c0 0x7f8754069d80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87700d1570 0x7f877015b310 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87700d1190 0x7f8770151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 --2- 172.21.15.3:0/3800471749 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87700cf950 0x7f8770156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 >> 172.21.15.3:0/3800471749 conn(0x7f8770067a00 msgr2=0x7f8770067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:43.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 shutdown_connections 2024-09-12T12:23:43.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.580+0000 7f8777e78640 1 -- 172.21.15.3:0/3800471749 wait complete. 2024-09-12T12:23:43.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:43.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f306e6d4640 1 Processor -- start 2024-09-12T12:23:43.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f306e6d4640 1 -- start start 2024-09-12T12:23:43.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f306e6d4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3068073cb0 0x7f306815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f306e6d4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f30680b30e0 0x7f30680adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f306e6d4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30680b5000 con 0x7f30680ae150 2024-09-12T12:23:43.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f306e6d4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3068074090 con 0x7f3068073cb0 2024-09-12T12:23:43.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f306e6d4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f30680b3530 con 0x7f30680b30e0 2024-09-12T12:23:43.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f3067fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3068073cb0 0x7f306815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f3067fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3068073cb0 0x7f306815c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39886/0 (socket says 172.21.15.3:39886) 2024-09-12T12:23:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f3067fff640 1 -- 172.21.15.3:0/2297228838 learned_addr learned my addr 172.21.15.3:0/2297228838 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f30677fe640 1 --2- 172.21.15.3:0/2297228838 >> v2:172.21.15.142:3300/0 conn(0x7f30680b30e0 0x7f30680adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f3067fff640 1 -- 172.21.15.3:0/2297228838 >> v2:172.21.15.142:3300/0 conn(0x7f30680b30e0 msgr2=0x7f30680adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f3067fff640 1 --2- 172.21.15.3:0/2297228838 >> v2:172.21.15.142:3300/0 conn(0x7f30680b30e0 0x7f30680adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f3067fff640 1 -- 172.21.15.3:0/2297228838 >> v1:172.21.15.142:6789/0 conn(0x7f30680ae150 legacy=0x7f30680ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.716+0000 7f3067fff640 1 -- 172.21.15.3:0/2297228838 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30680b37c0 con 0x7f3068073cb0 2024-09-12T12:23:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f3067fff640 1 --2- 172.21.15.3:0/2297228838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3068073cb0 0x7f306815c710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f306806efc0 tx=0x7f305801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93fa3973d1eed91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f3066ffd640 1 -- 172.21.15.3:0/2297228838 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580b0070 con 0x7f3068073cb0 2024-09-12T12:23:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f3066ffd640 1 -- 172.21.15.3:0/2297228838 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30580ab600 con 0x7f3068073cb0 2024-09-12T12:23:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f3066ffd640 1 -- 172.21.15.3:0/2297228838 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30580ab9d0 con 0x7f3068073cb0 2024-09-12T12:23:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 -- 172.21.15.3:0/2297228838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3068073cb0 msgr2=0x7f306815c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 --2- 172.21.15.3:0/2297228838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3068073cb0 0x7f306815c710 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f306806efc0 tx=0x7f305801c520 comp rx=0 tx=0).stop 2024-09-12T12:23:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 -- 172.21.15.3:0/2297228838 shutdown_connections 2024-09-12T12:23:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 --2- 172.21.15.3:0/2297228838 >> v2:172.21.15.142:3300/0 conn(0x7f30680b30e0 0x7f30680adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 --2- 172.21.15.3:0/2297228838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3068073cb0 0x7f306815c710 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 -- 172.21.15.3:0/2297228838 >> 172.21.15.3:0/2297228838 conn(0x7f3068067e00 msgr2=0x7f3068068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 -- 172.21.15.3:0/2297228838 shutdown_connections 2024-09-12T12:23:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 -- 172.21.15.3:0/2297228838 wait complete. 2024-09-12T12:23:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 Processor -- start 2024-09-12T12:23:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.720+0000 7f306e6d4640 1 -- start start 2024-09-12T12:23:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3068073cb0 0x7f30680cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30680ae150 0x7f30680c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30680b30e0 0x7f306806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f30680af110 con 0x7f30680ae150 2024-09-12T12:23:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30680aef90 con 0x7f30680b30e0 2024-09-12T12:23:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3068168dc0 con 0x7f3068073cb0 2024-09-12T12:23:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30680ae150 0x7f30680c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30680ae150 0x7f30680c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39888/0 (socket says 172.21.15.3:39888) 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 -- 172.21.15.3:0/1678260932 learned_addr learned my addr 172.21.15.3:0/1678260932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3067fff640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3068073cb0 0x7f30680cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306cc71640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30680b30e0 0x7f306806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 -- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3068073cb0 msgr2=0x7f30680cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3068073cb0 0x7f30680cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 -- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30680b30e0 msgr2=0x7f306806e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30680b30e0 0x7f306806e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 -- 172.21.15.3:0/1678260932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30680c9cb0 con 0x7f30680ae150 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f30677fe640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30680ae150 0x7f30680c9460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f305c00d5f0 tx=0x7f305c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3064ff9640 1 -- 172.21.15.3:0/1678260932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c0125f0 con 0x7f30680ae150 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3064ff9640 1 -- 172.21.15.3:0/1678260932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305c019070 con 0x7f30680ae150 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30680c9f90 con 0x7f30680ae150 2024-09-12T12:23:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3064ff9640 1 -- 172.21.15.3:0/1678260932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c0129b0 con 0x7f30680ae150 2024-09-12T12:23:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30680ca550 con 0x7f30680ae150 2024-09-12T12:23:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30680adc10 con 0x7f30680ae150 2024-09-12T12:23:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3064ff9640 1 -- 172.21.15.3:0/1678260932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f305c012b50 con 0x7f30680ae150 2024-09-12T12:23:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3064ff9640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30540678c0 0x7f3054069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3064ff9640 1 -- 172.21.15.3:0/1678260932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f305c086170 con 0x7f30680ae150 2024-09-12T12:23:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.724+0000 7f3067fff640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30540678c0 0x7f3054069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.728+0000 7f3067fff640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30540678c0 0x7f3054069d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f30580ad000 tx=0x7f3058008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:43.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.728+0000 7f3064ff9640 1 -- 172.21.15.3:0/1678260932 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f305c056bd0 con 0x7f30680ae150 2024-09-12T12:23:43.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.892+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f306815a670 con 0x7f30680ae150 2024-09-12T12:23:43.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f3064ff9640 1 -- 172.21.15.3:0/1678260932 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16765 (secure 0 0 0) 0x7f305c0564d0 con 0x7f30680ae150 2024-09-12T12:23:43.898 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:43.898 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":68,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:30.801236+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:15.490182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:16.430656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30540678c0 msgr2=0x7f3054069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30540678c0 0x7f3054069d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f30580ad000 tx=0x7f3058008040 comp rx=0 tx=0).stop 2024-09-12T12:23:43.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30680ae150 msgr2=0x7f30680c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30680ae150 0x7f30680c9460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f305c00d5f0 tx=0x7f305c00dab0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 shutdown_connections 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30540678c0 0x7f3054069d80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30680b30e0 0x7f306806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30680ae150 0x7f30680c9460 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 --2- 172.21.15.3:0/1678260932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3068073cb0 0x7f30680cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 >> 172.21.15.3:0/1678260932 conn(0x7f3068067e00 msgr2=0x7f30680ae670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 shutdown_connections 2024-09-12T12:23:43.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:43.896+0000 7f306e6d4640 1 -- 172.21.15.3:0/1678260932 wait complete. 2024-09-12T12:23:43.917 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:23:43.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:44.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.032+0000 7f9de1d9d640 1 Processor -- start 2024-09-12T12:23:44.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.032+0000 7f9de1d9d640 1 -- start start 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9de1d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc0b7740 0x7f9ddc0b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9de1d9d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ddc0d30b0 con 0x7f9ddc0d4150 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9de1d9d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ddc0d3390 con 0x7f9ddc0d1190 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9de1d9d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc0d45e0 con 0x7f9ddc0b7740 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9ddc0d4150 0x7f9ddc0cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34770/0 (socket says 172.21.15.3:34770) 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 -- 172.21.15.3:0/438574544 learned_addr learned my addr 172.21.15.3:0/438574544 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddbfff640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc0b7740 0x7f9ddc0b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2227265895 0 0) 0x7f9ddc0d30b0 con 0x7f9ddc0d4150 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dc4003530 con 0x7f9ddc0d4150 2024-09-12T12:23:44.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 199954210 0 0) 0x7f9dc4003530 con 0x7f9ddc0d4150 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc0b7740 msgr2=0x7f9ddc0b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc0b7740 0x7f9ddc0b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 >> v1:172.21.15.142:6789/0 conn(0x7f9ddc0d1190 legacy=0x7f9ddc0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc0d3670 con 0x7f9ddc0d4150 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f9dd0001680 con 0x7f9ddc0d4150 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 >> v1:172.21.15.3:6789/0 conn(0x7f9ddc0d4150 legacy=0x7f9ddc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc0b7740 0x7f9dc40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc4003f70 0x7f9dc4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc4004d50 0x7f9dc40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0d45e0 con 0x7f9dc4003f70 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc400bb40 con 0x7f9dc4004d50 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc400bd40 con 0x7f9ddc0b7740 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc4003f70 0x7f9dc4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddbfff640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc4004d50 0x7f9dc40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 -- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc0b7740 msgr2=0x7f9dc40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc0b7740 0x7f9dc40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 -- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc4004d50 msgr2=0x7f9dc40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc4004d50 0x7f9dc40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 -- 172.21.15.3:0/438574544 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc4003530 con 0x7f9dc4003f70 2024-09-12T12:23:44.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddbfff640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dc4004d50 0x7f9dc40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9ddaffd640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc4003f70 0x7f9dc4004810 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c210 tx=0x7f9dd001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890dd7e0337a5ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0007d40 con 0x7f9dc4003f70 2024-09-12T12:23:44.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd00a9070 con 0x7f9dc4003f70 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.036+0000 7f9dda7fc640 1 -- 172.21.15.3:0/438574544 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd00a5e40 con 0x7f9dc4003f70 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- 172.21.15.3:0/438574544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc4003f70 msgr2=0x7f9dc4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 --2- 172.21.15.3:0/438574544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9dc4003f70 0x7f9dc4004810 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9dd000c210 tx=0x7f9dd001cca0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddbfff640 1 -- 172.21.15.3:0/438574544 reap_dead start 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- 172.21.15.3:0/438574544 shutdown_connections 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- 172.21.15.3:0/438574544 >> 172.21.15.3:0/438574544 conn(0x7f9ddc067a00 msgr2=0x7f9ddc067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- 172.21.15.3:0/438574544 shutdown_connections 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- 172.21.15.3:0/438574544 wait complete. 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 Processor -- start 2024-09-12T12:23:44.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- start start 2024-09-12T12:23:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc153ee0 0x7f9ddc0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc1553d0 0x7f9ddc0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc15f2f0 0x7f9ddc15f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0d2cd0 con 0x7f9ddc153ee0 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0d3cc0 con 0x7f9ddc1553d0 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc0d2e50 con 0x7f9ddc15f2f0 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc153ee0 0x7f9ddc0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc153ee0 0x7f9ddc0cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39908/0 (socket says 172.21.15.3:39908) 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 -- 172.21.15.3:0/3967762021 learned_addr learned my addr 172.21.15.3:0/3967762021 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddaffd640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc1553d0 0x7f9ddc0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddbfff640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc15f2f0 0x7f9ddc15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 -- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc15f2f0 msgr2=0x7f9ddc15f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc15f2f0 0x7f9ddc15f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 -- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc1553d0 msgr2=0x7f9ddc0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc1553d0 0x7f9ddc0cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 -- 172.21.15.3:0/3967762021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc15ffb0 con 0x7f9ddc153ee0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9ddb7fe640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc153ee0 0x7f9ddc0cefe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c4d0 tx=0x7f9dcc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9dc3fff640 1 -- 172.21.15.3:0/3967762021 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0052d0 con 0x7f9ddc153ee0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9dc3fff640 1 -- 172.21.15.3:0/3967762021 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc005490 con 0x7f9ddc153ee0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc160290 con 0x7f9ddc153ee0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9dc3fff640 1 -- 172.21.15.3:0/3967762021 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc012850 con 0x7f9ddc153ee0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.040+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc0c9de0 con 0x7f9ddc153ee0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.044+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc15e860 con 0x7f9ddc153ee0 2024-09-12T12:23:44.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.044+0000 7f9dc3fff640 1 -- 172.21.15.3:0/3967762021 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9dcc0129f0 con 0x7f9ddc153ee0 2024-09-12T12:23:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.044+0000 7f9dc3fff640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9dc80678f0 0x7f9dc8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.044+0000 7f9dc3fff640 1 -- 172.21.15.3:0/3967762021 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f9dcc085890 con 0x7f9ddc153ee0 2024-09-12T12:23:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.044+0000 7f9ddaffd640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9dc80678f0 0x7f9dc8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.044+0000 7f9ddaffd640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9dc80678f0 0x7f9dc8069db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00ab000 tx=0x7f9dd00b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.048+0000 7f9dc3fff640 1 -- 172.21.15.3:0/3967762021 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9dcc056390 con 0x7f9ddc153ee0 2024-09-12T12:23:44.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.240+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ddc0ba4c0 con 0x7f9ddc153ee0 2024-09-12T12:23:44.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.240+0000 7f9dc3fff640 1 -- 172.21.15.3:0/3967762021 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f9dcc055c90 con 0x7f9ddc153ee0 2024-09-12T12:23:44.245 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:44.245 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:17.423558+0000","modified":"2024-09-12T12:23:20.482181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/550473013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":550473013},{"type":"v1","addr":"172.21.15.3:6835","nonce":550473013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6012,"qdb_cluster":[6012]},"id":5}]} 2024-09-12T12:23:44.245 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-12T12:23:44.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9dc80678f0 msgr2=0x7f9dc8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9dc80678f0 0x7f9dc8069db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00ab000 tx=0x7f9dd00b0040 comp rx=0 tx=0).stop 2024-09-12T12:23:44.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc153ee0 msgr2=0x7f9ddc0cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc153ee0 0x7f9ddc0cefe0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c4d0 tx=0x7f9dcc00c990 comp rx=0 tx=0).stop 2024-09-12T12:23:44.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 shutdown_connections 2024-09-12T12:23:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9dc80678f0 0x7f9dc8069db0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ddc15f2f0 0x7f9ddc15f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ddc1553d0 0x7f9ddc0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 --2- 172.21.15.3:0/3967762021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ddc153ee0 0x7f9ddc0cefe0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 >> 172.21.15.3:0/3967762021 conn(0x7f9ddc067a00 msgr2=0x7f9ddc067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 shutdown_connections 2024-09-12T12:23:44.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.244+0000 7f9de1d9d640 1 -- 172.21.15.3:0/3967762021 wait complete. 2024-09-12T12:23:44.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 Processor -- start 2024-09-12T12:23:44.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 -- start start 2024-09-12T12:23:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c60073cb0 0x7f1c6015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1c600b30e0 0x7f1c600adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c600ae150 0x7f1c600ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1c60074090 con 0x7f1c600ae150 2024-09-12T12:23:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1c600b5000 con 0x7f1c600b30e0 2024-09-12T12:23:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c67ede640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1c600b3500 con 0x7f1c60073cb0 2024-09-12T12:23:44.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c600ae150 0x7f1c600ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1c600ae150 0x7f1c600ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39916/0 (socket says 172.21.15.3:39916) 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 -- 172.21.15.3:0/743968331 learned_addr learned my addr 172.21.15.3:0/743968331 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c65c7a640 1 --2- 172.21.15.3:0/743968331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c60073cb0 0x7f1c6015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c65479640 1 --2- 172.21.15.3:0/743968331 >> v2:172.21.15.142:3300/0 conn(0x7f1c600b30e0 0x7f1c600adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 -- 172.21.15.3:0/743968331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c60073cb0 msgr2=0x7f1c6015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 --2- 172.21.15.3:0/743968331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c60073cb0 0x7f1c6015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 -- 172.21.15.3:0/743968331 >> v2:172.21.15.142:3300/0 conn(0x7f1c600b30e0 msgr2=0x7f1c600adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 --2- 172.21.15.3:0/743968331 >> v2:172.21.15.142:3300/0 conn(0x7f1c600b30e0 0x7f1c600adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.380+0000 7f1c6647b640 1 -- 172.21.15.3:0/743968331 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c600b37c0 con 0x7f1c600ae150 2024-09-12T12:23:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 --2- 172.21.15.3:0/743968331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600ae150 0x7f1c600ae880 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1c6006f160 tx=0x7f1c5801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4fdb20ac3abd6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c64c78640 1 -- 172.21.15.3:0/743968331 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c580a5780 con 0x7f1c600ae150 2024-09-12T12:23:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c64c78640 1 -- 172.21.15.3:0/743968331 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c580a5920 con 0x7f1c600ae150 2024-09-12T12:23:44.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c64c78640 1 -- 172.21.15.3:0/743968331 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c580a5cf0 con 0x7f1c600ae150 2024-09-12T12:23:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- 172.21.15.3:0/743968331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600ae150 msgr2=0x7f1c600ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 --2- 172.21.15.3:0/743968331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600ae150 0x7f1c600ae880 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1c6006f160 tx=0x7f1c5801c9f0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- 172.21.15.3:0/743968331 shutdown_connections 2024-09-12T12:23:44.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 --2- 172.21.15.3:0/743968331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600ae150 0x7f1c600ae880 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 --2- 172.21.15.3:0/743968331 >> v2:172.21.15.142:3300/0 conn(0x7f1c600b30e0 0x7f1c600adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 --2- 172.21.15.3:0/743968331 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c60073cb0 0x7f1c6015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- 172.21.15.3:0/743968331 >> 172.21.15.3:0/743968331 conn(0x7f1c60067e00 msgr2=0x7f1c60068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:44.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- 172.21.15.3:0/743968331 shutdown_connections 2024-09-12T12:23:44.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- 172.21.15.3:0/743968331 wait complete. 2024-09-12T12:23:44.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 Processor -- start 2024-09-12T12:23:44.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- start start 2024-09-12T12:23:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c60073cb0 0x7f1c601553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c600ae150 0x7f1c6009d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600b30e0 0x7f1c6009dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6015fe80 con 0x7f1c600b30e0 2024-09-12T12:23:44.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6015fd00 con 0x7f1c60073cb0 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c67ede640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1c60160000 con 0x7f1c600ae150 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600b30e0 0x7f1c6009dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600b30e0 0x7f1c6009dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39920/0 (socket says 172.21.15.3:39920) 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 -- 172.21.15.3:0/1963925760 learned_addr learned my addr 172.21.15.3:0/1963925760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c65c7a640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c60073cb0 0x7f1c601553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 -- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c600ae150 msgr2=0x7f1c6009d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c600ae150 0x7f1c6009d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 -- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c60073cb0 msgr2=0x7f1c601553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c60073cb0 0x7f1c601553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 -- 172.21.15.3:0/1963925760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c600ccc20 con 0x7f1c600b30e0 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c6647b640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600b30e0 0x7f1c6009dd70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1c5801d290 tx=0x7f1c58019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c527fc640 1 -- 172.21.15.3:0/1963925760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c580b8670 con 0x7f1c600b30e0 2024-09-12T12:23:44.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c527fc640 1 -- 172.21.15.3:0/1963925760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c580be070 con 0x7f1c600b30e0 2024-09-12T12:23:44.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.384+0000 7f1c527fc640 1 -- 172.21.15.3:0/1963925760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c580b8a90 con 0x7f1c600b30e0 2024-09-12T12:23:44.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c601568e0 con 0x7f1c600b30e0 2024-09-12T12:23:44.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c600cd1a0 con 0x7f1c600b30e0 2024-09-12T12:23:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c600ae880 con 0x7f1c600b30e0 2024-09-12T12:23:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c527fc640 1 -- 172.21.15.3:0/1963925760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1c580b8c30 con 0x7f1c600b30e0 2024-09-12T12:23:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c527fc640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1c440678c0 0x7f1c44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c527fc640 1 -- 172.21.15.3:0/1963925760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7f1c58120f30 con 0x7f1c600b30e0 2024-09-12T12:23:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c65c7a640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1c440678c0 0x7f1c44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.388+0000 7f1c65c7a640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1c440678c0 0x7f1c44069d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00a970 tx=0x7f1c5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.392+0000 7f1c527fc640 1 -- 172.21.15.3:0/1963925760 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1c580165e0 con 0x7f1c600b30e0 2024-09-12T12:23:44.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.584+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c6015eeb0 con 0x7f1c600b30e0 2024-09-12T12:23:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.584+0000 7f1c527fc640 1 -- 172.21.15.3:0/1963925760 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5222 (secure 0 0 0) 0x7f1c580f1990 con 0x7f1c600b30e0 2024-09-12T12:23:44.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:44.588 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":45,"btime":"2024-09-12T12:23:21:116168+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:17.423558+0000","modified":"2024-09-12T12:23:20.482181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6012},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6012":{"gid":6012,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/550473013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":550473013},{"type":"v1","addr":"172.21.15.3:6835","nonce":550473013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6012,"qdb_cluster":[6012]},"id":5}]} 2024-09-12T12:23:44.588 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 45 2024-09-12T12:23:44.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1c440678c0 msgr2=0x7f1c44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1c440678c0 0x7f1c44069d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00a970 tx=0x7f1c5c008040 comp rx=0 tx=0).stop 2024-09-12T12:23:44.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600b30e0 msgr2=0x7f1c6009dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600b30e0 0x7f1c6009dd70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1c5801d290 tx=0x7f1c58019790 comp rx=0 tx=0).stop 2024-09-12T12:23:44.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 shutdown_connections 2024-09-12T12:23:44.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1c440678c0 0x7f1c44069d80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c600b30e0 0x7f1c6009dd70 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c600ae150 0x7f1c6009d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 --2- 172.21.15.3:0/1963925760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c60073cb0 0x7f1c601553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 >> 172.21.15.3:0/1963925760 conn(0x7f1c60067e00 msgr2=0x7f1c60157a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:44.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 shutdown_connections 2024-09-12T12:23:44.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.588+0000 7f1c67ede640 1 -- 172.21.15.3:0/1963925760 wait complete. 2024-09-12T12:23:44.607 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.724+0000 7fdf977af640 1 Processor -- start 2024-09-12T12:23:44.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.724+0000 7fdf977af640 1 -- start start 2024-09-12T12:23:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf977af640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf90073cb0 0x7fdf9015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf977af640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf900b30e0 0x7fdf900adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf977af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf900ae150 0x7fdf900ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf977af640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdf90074090 con 0x7fdf90073cb0 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf977af640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdf900b5000 con 0x7fdf900b30e0 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf977af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf900b3530 con 0x7fdf900ae150 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf90073cb0 0x7fdf9015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdf90073cb0 0x7fdf9015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39936/0 (socket says 172.21.15.3:39936) 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 -- 172.21.15.3:0/1516736442 learned_addr learned my addr 172.21.15.3:0/1516736442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf95d4c640 1 --2- 172.21.15.3:0/1516736442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf900ae150 0x7fdf900ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf94d4a640 1 --2- 172.21.15.3:0/1516736442 >> v2:172.21.15.142:3300/0 conn(0x7fdf900b30e0 0x7fdf900adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 -- 172.21.15.3:0/1516736442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf900ae150 msgr2=0x7fdf900ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 --2- 172.21.15.3:0/1516736442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf900ae150 0x7fdf900ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 -- 172.21.15.3:0/1516736442 >> v2:172.21.15.142:3300/0 conn(0x7fdf900b30e0 msgr2=0x7fdf900adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 --2- 172.21.15.3:0/1516736442 >> v2:172.21.15.142:3300/0 conn(0x7fdf900b30e0 0x7fdf900adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 -- 172.21.15.3:0/1516736442 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf900b37c0 con 0x7fdf90073cb0 2024-09-12T12:23:44.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf9554b640 1 --2- 172.21.15.3:0/1516736442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf90073cb0 0x7fdf9015c710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdf9006efc0 tx=0x7fdf8801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd53efd62e00e6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf87fff640 1 -- 172.21.15.3:0/1516736442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880a5670 con 0x7fdf90073cb0 2024-09-12T12:23:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf87fff640 1 -- 172.21.15.3:0/1516736442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf880a5810 con 0x7fdf90073cb0 2024-09-12T12:23:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.728+0000 7fdf87fff640 1 -- 172.21.15.3:0/1516736442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880a5be0 con 0x7fdf90073cb0 2024-09-12T12:23:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- 172.21.15.3:0/1516736442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf90073cb0 msgr2=0x7fdf9015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 --2- 172.21.15.3:0/1516736442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf90073cb0 0x7fdf9015c710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdf9006efc0 tx=0x7fdf8801c9f0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- 172.21.15.3:0/1516736442 shutdown_connections 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 --2- 172.21.15.3:0/1516736442 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf900ae150 0x7fdf900ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 --2- 172.21.15.3:0/1516736442 >> v2:172.21.15.142:3300/0 conn(0x7fdf900b30e0 0x7fdf900adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 --2- 172.21.15.3:0/1516736442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf90073cb0 0x7fdf9015c710 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- 172.21.15.3:0/1516736442 >> 172.21.15.3:0/1516736442 conn(0x7fdf90067e00 msgr2=0x7fdf90068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- 172.21.15.3:0/1516736442 shutdown_connections 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- 172.21.15.3:0/1516736442 wait complete. 2024-09-12T12:23:44.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 Processor -- start 2024-09-12T12:23:44.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- start start 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf90073cb0 0x7fdf9014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf900ae150 0x7fdf90152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf900b30e0 0x7fdf90152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9015fe80 con 0x7fdf900b30e0 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9015fd00 con 0x7fdf900ae150 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf90160000 con 0x7fdf90073cb0 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf900b30e0 0x7fdf90152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf900b30e0 0x7fdf90152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39940/0 (socket says 172.21.15.3:39940) 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 -- 172.21.15.3:0/3025216896 learned_addr learned my addr 172.21.15.3:0/3025216896 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf94d4a640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf900ae150 0x7fdf90152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf9554b640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf90073cb0 0x7fdf9014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 -- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf90073cb0 msgr2=0x7fdf9014f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf90073cb0 0x7fdf9014f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 -- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf900ae150 msgr2=0x7fdf90152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf900ae150 0x7fdf90152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 -- 172.21.15.3:0/3025216896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf901f54b0 con 0x7fdf900b30e0 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf94d4a640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf900ae150 0x7fdf90152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf95d4c640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf900b30e0 0x7fdf90152eb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00d800 tx=0x7fdf8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf85ffb640 1 -- 172.21.15.3:0/3025216896 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00b6c0 con 0x7fdf900b30e0 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf85ffb640 1 -- 172.21.15.3:0/3025216896 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c019070 con 0x7fdf900b30e0 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf901f5720 con 0x7fdf900b30e0 2024-09-12T12:23:44.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf85ffb640 1 -- 172.21.15.3:0/3025216896 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00baa0 con 0x7fdf900b30e0 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.732+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf901f5ce0 con 0x7fdf900b30e0 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.736+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf900ae880 con 0x7fdf900b30e0 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.736+0000 7fdf85ffb640 1 -- 172.21.15.3:0/3025216896 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf8c00bc40 con 0x7fdf900b30e0 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.736+0000 7fdf85ffb640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf740678c0 0x7fdf74069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.736+0000 7fdf85ffb640 1 -- 172.21.15.3:0/3025216896 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fdf8c086150 con 0x7fdf900b30e0 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.736+0000 7fdf9554b640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf740678c0 0x7fdf74069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.736+0000 7fdf9554b640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf740678c0 0x7fdf74069d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ae000 tx=0x7fdf880a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:44.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.740+0000 7fdf85ffb640 1 -- 172.21.15.3:0/3025216896 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdf8c056c50 con 0x7fdf900b30e0 2024-09-12T12:23:44.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.904+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf901f5eb0 con 0x7fdf900b30e0 2024-09-12T12:23:44.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.904+0000 7fdf85ffb640 1 -- 172.21.15.3:0/3025216896 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16765 (secure 0 0 0) 0x7fdf8c056550 con 0x7fdf900b30e0 2024-09-12T12:23:44.908 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:44.908 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":68,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:30.801236+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:15.490182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:16.430656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.3","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf740678c0 msgr2=0x7fdf74069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf740678c0 0x7fdf74069d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf880ae000 tx=0x7fdf880a7070 comp rx=0 tx=0).stop 2024-09-12T12:23:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf900b30e0 msgr2=0x7fdf90152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf900b30e0 0x7fdf90152eb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00d800 tx=0x7fdf8c00dcc0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 shutdown_connections 2024-09-12T12:23:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf740678c0 0x7fdf74069d80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf900b30e0 0x7fdf90152eb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf900ae150 0x7fdf90152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 --2- 172.21.15.3:0/3025216896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf90073cb0 0x7fdf9014f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:44.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 >> 172.21.15.3:0/3025216896 conn(0x7fdf90067e00 msgr2=0x7fdf9015b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:44.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 shutdown_connections 2024-09-12T12:23:44.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:44.908+0000 7fdf977af640 1 -- 172.21.15.3:0/3025216896 wait complete. 2024-09-12T12:23:44.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T12:23:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba99122640 1 Processor -- start 2024-09-12T12:23:45.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba99122640 1 -- start start 2024-09-12T12:23:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba99122640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 0x7fba940d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba99122640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fba940d1570 0x7fba940cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba99122640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba940d30b0 con 0x7fba940cf950 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba99122640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fba940d3cc0 con 0x7fba940d1570 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba99122640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fba940d3390 con 0x7fba940d1190 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba9359e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fba940cf950 0x7fba940b7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34778/0 (socket says 172.21.15.3:34778) 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.052+0000 7fba9359e640 1 -- 172.21.15.3:0/656658225 learned_addr learned my addr 172.21.15.3:0/656658225 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba92d9d640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 0x7fba940d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 --2- 172.21.15.3:0/656658225 >> v2:172.21.15.142:3300/0 conn(0x7fba940d1570 0x7fba940cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099051017 0 0) 0x7fba940d30b0 con 0x7fba940cf950 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba7c003530 con 0x7fba940cf950 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1569920919 0 0) 0x7fba7c003530 con 0x7fba940cf950 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 msgr2=0x7fba940d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 0x7fba940d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 >> v2:172.21.15.142:3300/0 conn(0x7fba940d1570 msgr2=0x7fba940cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:45.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 --2- 172.21.15.3:0/656658225 >> v2:172.21.15.142:3300/0 conn(0x7fba940d1570 0x7fba940cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba940d3670 con 0x7fba940cf950 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fba8c001a00 con 0x7fba940cf950 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 >> v1:172.21.15.3:6789/0 conn(0x7fba940cf950 legacy=0x7fba940b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 0x7fba7c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7c003f40 0x7fba7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c004d50 0x7fba7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fba940d3cc0 con 0x7fba7c003f40 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba940d3390 con 0x7fba7c004d50 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c00b9d0 con 0x7fba940d1190 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7c003f40 0x7fba7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba92d9d640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 0x7fba7c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9359e640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c004d50 0x7fba7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 -- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 msgr2=0x7fba7c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:45.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 0x7fba7c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 -- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c004d50 msgr2=0x7fba7c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:45.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba7c004d50 0x7fba7c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 -- 172.21.15.3:0/656658225 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c003530 con 0x7fba7c003f40 2024-09-12T12:23:45.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba92d9d640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba940d1190 0x7fba7c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9259c640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7c003f40 0x7fba7c004810 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fba88009ab0 tx=0x7fba8801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8f6c7c23f983ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880b2070 con 0x7fba7c003f40 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba880a5680 con 0x7fba7c003f40 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba91d9b640 1 -- 172.21.15.3:0/656658225 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880096b0 con 0x7fba7c003f40 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba99122640 1 -- 172.21.15.3:0/656658225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7c003f40 msgr2=0x7fba7c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba99122640 1 --2- 172.21.15.3:0/656658225 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba7c003f40 0x7fba7c004810 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fba88009ab0 tx=0x7fba8801ca30 comp rx=0 tx=0).stop 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba9359e640 1 -- 172.21.15.3:0/656658225 reap_dead start 2024-09-12T12:23:45.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba99122640 1 -- 172.21.15.3:0/656658225 shutdown_connections 2024-09-12T12:23:45.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba99122640 1 -- 172.21.15.3:0/656658225 >> 172.21.15.3:0/656658225 conn(0x7fba94067a00 msgr2=0x7fba94067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:45.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba99122640 1 -- 172.21.15.3:0/656658225 shutdown_connections 2024-09-12T12:23:45.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.056+0000 7fba99122640 1 -- 172.21.15.3:0/656658225 wait complete. 2024-09-12T12:23:45.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 Processor -- start 2024-09-12T12:23:45.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 -- start start 2024-09-12T12:23:45.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba7c003f40 0x7fba9415b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba9415bc10 0x7fba94160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9415ea70 0x7fba94161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fba940d2cd0 con 0x7fba9415bc10 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fba940d19d0 con 0x7fba9415ea70 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fba940d2e50 con 0x7fba7c003f40 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba9415bc10 0x7fba94160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba9415bc10 0x7fba94160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39946/0 (socket says 172.21.15.3:39946) 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 -- 172.21.15.3:0/347748430 learned_addr learned my addr 172.21.15.3:0/347748430 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba92d9d640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba7c003f40 0x7fba9415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9359e640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9415ea70 0x7fba94161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 -- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba7c003f40 msgr2=0x7fba9415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba7c003f40 0x7fba9415b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 -- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9415ea70 msgr2=0x7fba94161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9415ea70 0x7fba94161210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 -- 172.21.15.3:0/347748430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba941609d0 con 0x7fba9415bc10 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba9259c640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba9415bc10 0x7fba94160490 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c10 tx=0x7fba7c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba7b7fe640 1 -- 172.21.15.3:0/347748430 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880b2040 con 0x7fba9415bc10 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba7b7fe640 1 -- 172.21.15.3:0/347748430 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba880b8440 con 0x7fba9415bc10 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba7b7fe640 1 -- 172.21.15.3:0/347748430 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba880096b0 con 0x7fba9415bc10 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba94160c50 con 0x7fba9415bc10 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba940cd030 con 0x7fba9415bc10 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.060+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9415e860 con 0x7fba9415bc10 2024-09-12T12:23:45.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.064+0000 7fba7b7fe640 1 -- 172.21.15.3:0/347748430 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba880bf060 con 0x7fba9415bc10 2024-09-12T12:23:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.064+0000 7fba7b7fe640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fba800678c0 0x7fba80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.064+0000 7fba7b7fe640 1 -- 172.21.15.3:0/347748430 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5767+0+0 (secure 0 0 0) 0x7fba8811f440 con 0x7fba9415bc10 2024-09-12T12:23:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.064+0000 7fba92d9d640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fba800678c0 0x7fba80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.064+0000 7fba92d9d640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fba800678c0 0x7fba80069d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba840099e0 tx=0x7fba84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:45.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.068+0000 7fba7b7fe640 1 -- 172.21.15.3:0/347748430 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fba941609d0 con 0x7fba9415bc10 2024-09-12T12:23:45.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.260+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fba940ba4c0 con 0x7fba9415bc10 2024-09-12T12:23:45.263 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:23:45.260+0000 7f0a5d435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:23:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba7b7fe640 1 -- 172.21.15.3:0/347748430 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7fba880167a0 con 0x7fba9415bc10 2024-09-12T12:23:45.265 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:23:45.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fba800678c0 msgr2=0x7fba80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fba800678c0 0x7fba80069d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba840099e0 tx=0x7fba84008040 comp rx=0 tx=0).stop 2024-09-12T12:23:45.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba9415bc10 msgr2=0x7fba94160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba9415bc10 0x7fba94160490 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c10 tx=0x7fba7c005840 comp rx=0 tx=0).stop 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 shutdown_connections 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fba800678c0 0x7fba80069d80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba9415ea70 0x7fba94161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba9415bc10 0x7fba94160490 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 --2- 172.21.15.3:0/347748430 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fba7c003f40 0x7fba9415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 >> 172.21.15.3:0/347748430 conn(0x7fba94067a00 msgr2=0x7fba940ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 shutdown_connections 2024-09-12T12:23:45.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.264+0000 7fba99122640 1 -- 172.21.15.3:0/347748430 wait complete. 2024-09-12T12:23:45.271 INFO:tasks.ceph.mds.c.smithi003.stderr: -1334> 2024-09-12T12:23:24.040+0000 7fdfbb622640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:23:45.286 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:23:45.307 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:23:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9f5e1640 1 Processor -- start 2024-09-12T12:23:45.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9f5e1640 1 -- start start 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9f5e1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb980d0cd0 0x7feb9806e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9f5e1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb98160600 con 0x7feb9815c210 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9f5e1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb981608e0 con 0x7feb980adf60 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9f5e1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7feb98160bc0 con 0x7feb980d0cd0 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9cb7c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb980d0cd0 0x7feb9806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9d37d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feb9815c210 0x7feb980d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34798/0 (socket says 172.21.15.3:34798) 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9d37d640 1 -- 172.21.15.3:0/1480260569 learned_addr learned my addr 172.21.15.3:0/1480260569 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:45.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9cb7c640 1 -- 172.21.15.3:0/1480260569 >> v1:172.21.15.142:6789/0 conn(0x7feb980adf60 legacy=0x7feb980cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9cb7c640 1 -- 172.21.15.3:0/1480260569 >> v1:172.21.15.3:6789/0 conn(0x7feb9815c210 legacy=0x7feb980d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb8ffff640 1 -- 172.21.15.3:0/1480260569 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4190990800 0 0) 0x7feb98160600 con 0x7feb9815c210 2024-09-12T12:23:45.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.404+0000 7feb9cb7c640 1 -- 172.21.15.3:0/1480260569 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb980cec00 con 0x7feb980d0cd0 2024-09-12T12:23:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9cb7c640 1 --2- 172.21.15.3:0/1480260569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb9806e450 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7feb880097c0 tx=0x7feb8801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee955572e1e8531b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb8ffff640 1 -- 172.21.15.3:0/1480260569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880b0070 con 0x7feb980d0cd0 2024-09-12T12:23:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb8ffff640 1 -- 172.21.15.3:0/1480260569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb880ab600 con 0x7feb980d0cd0 2024-09-12T12:23:45.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb8ffff640 1 -- 172.21.15.3:0/1480260569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880ab9d0 con 0x7feb980d0cd0 2024-09-12T12:23:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 -- 172.21.15.3:0/1480260569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 msgr2=0x7feb9806e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/1480260569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb9806e450 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7feb880097c0 tx=0x7feb8801c3d0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 -- 172.21.15.3:0/1480260569 shutdown_connections 2024-09-12T12:23:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/1480260569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb9806e450 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 -- 172.21.15.3:0/1480260569 >> 172.21.15.3:0/1480260569 conn(0x7feb98067e00 msgr2=0x7feb98068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 -- 172.21.15.3:0/1480260569 shutdown_connections 2024-09-12T12:23:45.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 -- 172.21.15.3:0/1480260569 wait complete. 2024-09-12T12:23:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 Processor -- start 2024-09-12T12:23:45.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.408+0000 7feb9f5e1640 1 -- start start 2024-09-12T12:23:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb980adf60 0x7feb980ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb98152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb9815c210 0x7feb98153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb9816a5f0 con 0x7feb980d0cd0 2024-09-12T12:23:45.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb9816a470 con 0x7feb980adf60 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb9816a770 con 0x7feb9815c210 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb98152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb98152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39984/0 (socket says 172.21.15.3:39984) 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 -- 172.21.15.3:0/3469893329 learned_addr learned my addr 172.21.15.3:0/3469893329 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9d37d640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb980adf60 0x7feb980ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 -- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb9815c210 msgr2=0x7feb98153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb9815c210 0x7feb98153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 -- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb980adf60 msgr2=0x7feb980ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb980adf60 0x7feb980ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 -- 172.21.15.3:0/3469893329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb981599b0 con 0x7feb980d0cd0 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9cb7c640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb98152cf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7feb880ae040 tx=0x7feb8801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb8dffb640 1 -- 172.21.15.3:0/3469893329 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880b0050 con 0x7feb980d0cd0 2024-09-12T12:23:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb8dffb640 1 -- 172.21.15.3:0/3469893329 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb880b1040 con 0x7feb980d0cd0 2024-09-12T12:23:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb98159c90 con 0x7feb980d0cd0 2024-09-12T12:23:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb8dffb640 1 -- 172.21.15.3:0/3469893329 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004220 con 0x7feb980d0cd0 2024-09-12T12:23:45.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9815a250 con 0x7feb980d0cd0 2024-09-12T12:23:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9806e450 con 0x7feb980d0cd0 2024-09-12T12:23:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb8dffb640 1 -- 172.21.15.3:0/3469893329 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feb880be060 con 0x7feb980d0cd0 2024-09-12T12:23:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb8dffb640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb7c0678c0 0x7feb7c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:45.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb8dffb640 1 -- 172.21.15.3:0/3469893329 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5853+0+0 (secure 0 0 0) 0x7feb8811f730 con 0x7feb980d0cd0 2024-09-12T12:23:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.412+0000 7feb9d37d640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb7c0678c0 0x7feb7c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.416+0000 7feb9d37d640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb7c0678c0 0x7feb7c069d80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb94000f70 tx=0x7feb94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:45.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.420+0000 7feb8dffb640 1 -- 172.21.15.3:0/3469893329 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feb88016050 con 0x7feb980d0cd0 2024-09-12T12:23:45.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:45.608+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feb980ce840 con 0x7feb980d0cd0 2024-09-12T12:23:46.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.264+0000 7feb8dffb640 1 -- 172.21.15.3:0/3469893329 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) ==== 104+0+0 (secure 0 0 0) 0x7feb880f01e0 con 0x7feb980d0cd0 2024-09-12T12:23:46.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb7c0678c0 msgr2=0x7feb7c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:46.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb7c0678c0 0x7feb7c069d80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb94000f70 tx=0x7feb94009040 comp rx=0 tx=0).stop 2024-09-12T12:23:46.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 msgr2=0x7feb98152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:46.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb98152cf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7feb880ae040 tx=0x7feb8801ac10 comp rx=0 tx=0).stop 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 shutdown_connections 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb7c0678c0 0x7feb7c069d80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb9815c210 0x7feb98153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb980d0cd0 0x7feb98152cf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 --2- 172.21.15.3:0/3469893329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb980adf60 0x7feb980ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 >> 172.21.15.3:0/3469893329 conn(0x7feb98067e00 msgr2=0x7feb981559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 shutdown_connections 2024-09-12T12:23:46.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.268+0000 7feb9f5e1640 1 -- 172.21.15.3:0/3469893329 wait complete. 2024-09-12T12:23:46.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:23:46.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 Processor -- start 2024-09-12T12:23:46.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 -- start start 2024-09-12T12:23:46.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8dd41523f0 0x7f8dd40d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:46.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8dd415c910 0x7f8dd40d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:46.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd40ae2c0 0x7f8dd40ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:46.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd40ad160 con 0x7f8dd415c910 2024-09-12T12:23:46.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd40ad440 con 0x7f8dd41523f0 2024-09-12T12:23:46.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd8f87640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd40ad720 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd1d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8dd415c910 0x7f8dd40d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd1d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8dd415c910 0x7f8dd40d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39998/0 (socket says 172.21.15.3:39998) 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd1d9b640 1 -- 172.21.15.3:0/2632237941 learned_addr learned my addr 172.21.15.3:0/2632237941 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd2d9d640 1 --2- 172.21.15.3:0/2632237941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd40ae2c0 0x7f8dd40ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.464+0000 7f8dd259c640 1 --2- 172.21.15.3:0/2632237941 >> v2:172.21.15.142:3300/0 conn(0x7f8dd41523f0 0x7f8dd40d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd1d9b640 1 -- 172.21.15.3:0/2632237941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd40ae2c0 msgr2=0x7f8dd40ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd1d9b640 1 --2- 172.21.15.3:0/2632237941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd40ae2c0 0x7f8dd40ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd1d9b640 1 -- 172.21.15.3:0/2632237941 >> v2:172.21.15.142:3300/0 conn(0x7f8dd41523f0 msgr2=0x7f8dd40d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd1d9b640 1 --2- 172.21.15.3:0/2632237941 >> v2:172.21.15.142:3300/0 conn(0x7f8dd41523f0 0x7f8dd40d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd1d9b640 1 -- 172.21.15.3:0/2632237941 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd40ada00 con 0x7f8dd415c910 2024-09-12T12:23:46.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd1d9b640 1 --2- 172.21.15.3:0/2632237941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd415c910 0x7f8dd40d08e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40d1030 tx=0x7f8dc801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b456349c3dea38d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:46.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd159a640 1 -- 172.21.15.3:0/2632237941 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80aa430 con 0x7f8dd415c910 2024-09-12T12:23:46.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd159a640 1 -- 172.21.15.3:0/2632237941 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc80af070 con 0x7f8dd415c910 2024-09-12T12:23:46.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd159a640 1 -- 172.21.15.3:0/2632237941 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80aa7f0 con 0x7f8dd415c910 2024-09-12T12:23:46.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 -- 172.21.15.3:0/2632237941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd415c910 msgr2=0x7f8dd40d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:46.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/2632237941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd415c910 0x7f8dd40d08e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40d1030 tx=0x7f8dc801c4f0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 -- 172.21.15.3:0/2632237941 shutdown_connections 2024-09-12T12:23:46.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/2632237941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd40ae2c0 0x7f8dd40ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/2632237941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd415c910 0x7f8dd40d08e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/2632237941 >> v2:172.21.15.142:3300/0 conn(0x7f8dd41523f0 0x7f8dd40d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 -- 172.21.15.3:0/2632237941 >> 172.21.15.3:0/2632237941 conn(0x7f8dd4067bd0 msgr2=0x7f8dd4067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:46.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 -- 172.21.15.3:0/2632237941 shutdown_connections 2024-09-12T12:23:46.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.468+0000 7f8dd8f87640 1 -- 172.21.15.3:0/2632237941 wait complete. 2024-09-12T12:23:46.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 Processor -- start 2024-09-12T12:23:46.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 -- start start 2024-09-12T12:23:46.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd40ae2c0 0x7f8dd414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:46.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd41523f0 0x7f8dd4072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:46.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dd415c910 0x7f8dd415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:46.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd415e9a0 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd415e820 con 0x7f8dd415c910 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd415eb20 con 0x7f8dd41523f0 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd40ae2c0 0x7f8dd414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd40ae2c0 0x7f8dd414c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40002/0 (socket says 172.21.15.3:40002) 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 -- 172.21.15.3:0/3180153933 learned_addr learned my addr 172.21.15.3:0/3180153933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd1d9b640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd41523f0 0x7f8dd4072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd2d9d640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dd415c910 0x7f8dd415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 -- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd41523f0 msgr2=0x7f8dd4072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd41523f0 0x7f8dd4072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 -- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dd415c910 msgr2=0x7f8dd415a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dd415c910 0x7f8dd415a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 -- 172.21.15.3:0/3180153933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd41514c0 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd1d9b640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd41523f0 0x7f8dd4072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:46.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd259c640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd40ae2c0 0x7f8dd414c6c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e8a0 tx=0x7f8dc400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:46.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dc2ffd640 1 -- 172.21.15.3:0/3180153933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4017070 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dc2ffd640 1 -- 172.21.15.3:0/3180153933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc400c940 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd41517a0 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dc2ffd640 1 -- 172.21.15.3:0/3180153933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc400ccf0 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd40ce4f0 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd40ae750 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dc2ffd640 1 -- 172.21.15.3:0/3180153933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8dc401a070 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dc2ffd640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8db40678c0 0x7f8db4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:46.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.472+0000 7f8dc2ffd640 1 -- 172.21.15.3:0/3180153933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5853+0+0 (secure 0 0 0) 0x7f8dc4085c10 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.476+0000 7f8dd1d9b640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8db40678c0 0x7f8db4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:46.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.476+0000 7f8dd1d9b640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8db40678c0 0x7f8db4069d80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80ae000 tx=0x7f8dc80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:46.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.480+0000 7f8dc2ffd640 1 -- 172.21.15.3:0/3180153933 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8dc40566c0 con 0x7f8dd40ae2c0 2024-09-12T12:23:46.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:46.644+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8dd40cc380 con 0x7f8dd40ae2c0 2024-09-12T12:23:47.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:23:47.024 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:23:47.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.276+0000 7f8dc2ffd640 1 -- 172.21.15.3:0/3180153933 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) ==== 181+0+0 (secure 0 0 0) 0x7f8dc4055fc0 con 0x7f8dd40ae2c0 2024-09-12T12:23:47.278 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:23:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.276+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8db40678c0 msgr2=0x7f8db4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.276+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8db40678c0 0x7f8db4069d80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80ae000 tx=0x7f8dc80a7000 comp rx=0 tx=0).stop 2024-09-12T12:23:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.276+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd40ae2c0 msgr2=0x7f8dd414c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:47.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.276+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd40ae2c0 0x7f8dd414c6c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e8a0 tx=0x7f8dc400eca0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 shutdown_connections 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8db40678c0 0x7f8db4069d80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dd415c910 0x7f8dd415a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8dd41523f0 0x7f8dd4072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 --2- 172.21.15.3:0/3180153933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8dd40ae2c0 0x7f8dd414c6c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 >> 172.21.15.3:0/3180153933 conn(0x7f8dd4067bd0 msgr2=0x7f8dd40b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 shutdown_connections 2024-09-12T12:23:47.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.280+0000 7f8dd8f87640 1 -- 172.21.15.3:0/3180153933 wait complete. 2024-09-12T12:23:47.304 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:23:47.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.432+0000 7fda593de640 1 Processor -- start 2024-09-12T12:23:47.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.432+0000 7fda593de640 1 -- start start 2024-09-12T12:23:47.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda540ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:47.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda5415c910 0x7fda540ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:47.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda540ad160 con 0x7fda541523f0 2024-09-12T12:23:47.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fda540ad440 con 0x7fda5415c910 2024-09-12T12:23:47.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda540ad720 con 0x7fda540d0ab0 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda537fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda5415c910 0x7fda540ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda52ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fda541523f0 0x7fda540d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:34812/0 (socket says 172.21.15.3:34812) 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda52ffd640 1 -- 172.21.15.3:0/3700766010 learned_addr learned my addr 172.21.15.3:0/3700766010 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda537fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fda5415c910 0x7fda540ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40018/0 (socket says 172.21.15.3:40018) 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda4bfff640 1 --2- 172.21.15.3:0/3700766010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda540ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda537fe640 1 -- 172.21.15.3:0/3700766010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 msgr2=0x7fda540ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda537fe640 1 --2- 172.21.15.3:0/3700766010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda540ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda537fe640 1 -- 172.21.15.3:0/3700766010 >> v1:172.21.15.3:6789/0 conn(0x7fda541523f0 legacy=0x7fda540d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:47.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda537fe640 1 -- 172.21.15.3:0/3700766010 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda540ada00 con 0x7fda5415c910 2024-09-12T12:23:47.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda537fe640 1 --2- 172.21.15.3:0/3700766010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda540ae800 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00bfa0 tx=0x7fda4c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48746f9086c08230 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:47.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda527fc640 1 -- 172.21.15.3:0/3700766010 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00b2d0 con 0x7fda5415c910 2024-09-12T12:23:47.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda527fc640 1 -- 172.21.15.3:0/3700766010 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c0a7a30 con 0x7fda5415c910 2024-09-12T12:23:47.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda527fc640 1 -- 172.21.15.3:0/3700766010 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0a7e00 con 0x7fda5415c910 2024-09-12T12:23:47.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- 172.21.15.3:0/3700766010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 msgr2=0x7fda540ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:47.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 --2- 172.21.15.3:0/3700766010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda540ae800 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00bfa0 tx=0x7fda4c01eb40 comp rx=0 tx=0).stop 2024-09-12T12:23:47.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- 172.21.15.3:0/3700766010 shutdown_connections 2024-09-12T12:23:47.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 --2- 172.21.15.3:0/3700766010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda540ae800 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 --2- 172.21.15.3:0/3700766010 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda540ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- 172.21.15.3:0/3700766010 >> 172.21.15.3:0/3700766010 conn(0x7fda54067bd0 msgr2=0x7fda54067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:47.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- 172.21.15.3:0/3700766010 shutdown_connections 2024-09-12T12:23:47.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.436+0000 7fda593de640 1 -- 172.21.15.3:0/3700766010 wait complete. 2024-09-12T12:23:47.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 Processor -- start 2024-09-12T12:23:47.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 -- start start 2024-09-12T12:23:47.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda5414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:47.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda541523f0 0x7fda54158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:47.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda54150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fda5415e960 con 0x7fda5415c910 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda5415e7e0 con 0x7fda541523f0 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fda5415eae0 con 0x7fda540d0ab0 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda54150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda54150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40028/0 (socket says 172.21.15.3:40028) 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 -- 172.21.15.3:0/568637369 learned_addr learned my addr 172.21.15.3:0/568637369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda52ffd640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda5414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda4bfff640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda541523f0 0x7fda54158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 -- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 msgr2=0x7fda5414a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda5414a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 -- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda541523f0 msgr2=0x7fda54158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda541523f0 0x7fda54158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:47.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 -- 172.21.15.3:0/568637369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda54152870 con 0x7fda5415c910 2024-09-12T12:23:47.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda537fe640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda54150de0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00bfa0 tx=0x7fda4c01b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:47.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda4b7fe640 1 -- 172.21.15.3:0/568637369 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c009950 con 0x7fda5415c910 2024-09-12T12:23:47.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda4b7fe640 1 -- 172.21.15.3:0/568637369 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c0c0070 con 0x7fda5415c910 2024-09-12T12:23:47.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda54159000 con 0x7fda5415c910 2024-09-12T12:23:47.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda4b7fe640 1 -- 172.21.15.3:0/568637369 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c009d50 con 0x7fda5415c910 2024-09-12T12:23:47.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.440+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda54152df0 con 0x7fda5415c910 2024-09-12T12:23:47.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.444+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda540d03a0 con 0x7fda5415c910 2024-09-12T12:23:47.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.444+0000 7fda4b7fe640 1 -- 172.21.15.3:0/568637369 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fda4c0ba470 con 0x7fda5415c910 2024-09-12T12:23:47.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.444+0000 7fda4b7fe640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fda340678f0 0x7fda34069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:47.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.444+0000 7fda4b7fe640 1 -- 172.21.15.3:0/568637369 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5415+0+0 (secure 0 0 0) 0x7fda4c123120 con 0x7fda5415c910 2024-09-12T12:23:47.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.444+0000 7fda52ffd640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fda340678f0 0x7fda34069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:47.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.444+0000 7fda52ffd640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fda340678f0 0x7fda34069db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda44000f70 tx=0x7fda44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:47.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.448+0000 7fda4b7fe640 1 -- 172.21.15.3:0/568637369 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fda4c018760 con 0x7fda5415c910 2024-09-12T12:23:47.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:47.608+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda540cd500 con 0x7fda5415c910 2024-09-12T12:23:48.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.284+0000 7fda4b7fe640 1 -- 172.21.15.3:0/568637369 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) ==== 169+0+0 (secure 0 0 0) 0x7fda4c0f3ce0 con 0x7fda5415c910 2024-09-12T12:23:48.286 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T12:23:48.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fda340678f0 msgr2=0x7fda34069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fda340678f0 0x7fda34069db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda44000f70 tx=0x7fda44009040 comp rx=0 tx=0).stop 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 msgr2=0x7fda54150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda54150de0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00bfa0 tx=0x7fda4c01b8e0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 shutdown_connections 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fda340678f0 0x7fda34069db0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fda5415c910 0x7fda54150de0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda541523f0 0x7fda54158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 --2- 172.21.15.3:0/568637369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fda540d0ab0 0x7fda5414a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 >> 172.21.15.3:0/568637369 conn(0x7fda54067bd0 msgr2=0x7fda5415ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 shutdown_connections 2024-09-12T12:23:48.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.288+0000 7fda593de640 1 -- 172.21.15.3:0/568637369 wait complete. 2024-09-12T12:23:48.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds ms_async_reap_threshold 2024-09-12T12:23:48.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f261c640 1 Processor -- start 2024-09-12T12:23:48.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f261c640 1 -- start start 2024-09-12T12:23:48.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f261c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec15c430 0x7f37ec0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f261c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37ec0cff20 0x7f37ec0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f261c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ec0ccdf0 con 0x7f37ec1600c0 2024-09-12T12:23:48.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f261c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f37ec0cd6c0 con 0x7f37ec0cff20 2024-09-12T12:23:48.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f261c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec0d0420 con 0x7f37ec15c430 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec15c430 0x7f37ec0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37eb7fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37ec0cff20 0x7f37ec0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f0bb9640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f37ec1600c0 0x7f37ec0b1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:49042/0 (socket says 172.21.15.3:49042) 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec15c430 0x7f37ec0b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:55606/0 (socket says 172.21.15.3:55606) 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37f0bb9640 1 -- 172.21.15.3:0/1295691356 learned_addr learned my addr 172.21.15.3:0/1295691356 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37eaffd640 1 -- 172.21.15.3:0/1295691356 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035466819 0 0) 0x7f37e40014a0 con 0x7f37ec1600c0 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37eaffd640 1 -- 172.21.15.3:0/1295691356 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37d8003530 con 0x7f37ec1600c0 2024-09-12T12:23:48.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37ebfff640 1 -- 172.21.15.3:0/1295691356 >> v2:172.21.15.142:3300/0 conn(0x7f37ec0cff20 msgr2=0x7f37ec0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37ebfff640 1 --2- 172.21.15.3:0/1295691356 >> v2:172.21.15.142:3300/0 conn(0x7f37ec0cff20 0x7f37ec0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37ebfff640 1 -- 172.21.15.3:0/1295691356 >> v1:172.21.15.142:6789/0 conn(0x7f37ec1600c0 legacy=0x7f37ec0b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.444+0000 7f37ebfff640 1 -- 172.21.15.3:0/1295691356 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec0cd070 con 0x7f37ec15c430 2024-09-12T12:23:48.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37ebfff640 1 --2- 172.21.15.3:0/1295691356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec15c430 0x7f37ec0b49c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097c0 tx=0x7f37dc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2dec7a0b48fdc35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37eaffd640 1 -- 172.21.15.3:0/1295691356 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0b0070 con 0x7f37ec15c430 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37eaffd640 1 -- 172.21.15.3:0/1295691356 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc0ab600 con 0x7f37ec15c430 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37eaffd640 1 -- 172.21.15.3:0/1295691356 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0ab9b0 con 0x7f37ec15c430 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 -- 172.21.15.3:0/1295691356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec15c430 msgr2=0x7f37ec0b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 --2- 172.21.15.3:0/1295691356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec15c430 0x7f37ec0b49c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097c0 tx=0x7f37dc01c410 comp rx=0 tx=0).stop 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 -- 172.21.15.3:0/1295691356 shutdown_connections 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 --2- 172.21.15.3:0/1295691356 >> v2:172.21.15.142:3300/0 conn(0x7f37ec0cff20 0x7f37ec0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 --2- 172.21.15.3:0/1295691356 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec15c430 0x7f37ec0b49c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 -- 172.21.15.3:0/1295691356 >> 172.21.15.3:0/1295691356 conn(0x7f37ec0b3ea0 msgr2=0x7f37ec0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 -- 172.21.15.3:0/1295691356 shutdown_connections 2024-09-12T12:23:48.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.448+0000 7f37f261c640 1 -- 172.21.15.3:0/1295691356 wait complete. 2024-09-12T12:23:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 Processor -- start 2024-09-12T12:23:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 -- start start 2024-09-12T12:23:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cff20 0x7f37ec1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37ec15c430 0x7f37ec157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec1600c0 0x7f37ec06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1608e0 con 0x7f37ec15c430 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec15c810 con 0x7f37ec1600c0 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec15e620 con 0x7f37ec0cff20 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37ec15c430 0x7f37ec157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37ec15c430 0x7f37ec157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46572/0 (socket says 172.21.15.3:46572) 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 -- 172.21.15.3:0/390025268 learned_addr learned my addr 172.21.15.3:0/390025268 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37ebfff640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cff20 0x7f37ec1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f0bb9640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec1600c0 0x7f37ec06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 -- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cff20 msgr2=0x7f37ec1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cff20 0x7f37ec1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 -- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec1600c0 msgr2=0x7f37ec06fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec1600c0 0x7f37ec06fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 -- 172.21.15.3:0/390025268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec15e870 con 0x7f37ec15c430 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37eb7fe640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37ec15c430 0x7f37ec157420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e930 tx=0x7f37d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37e8ff9640 1 -- 172.21.15.3:0/390025268 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4017070 con 0x7f37ec15c430 2024-09-12T12:23:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37e8ff9640 1 -- 172.21.15.3:0/390025268 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d400ca00 con 0x7f37ec15c430 2024-09-12T12:23:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37e8ff9640 1 -- 172.21.15.3:0/390025268 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d400cdb0 con 0x7f37ec15c430 2024-09-12T12:23:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec15eb50 con 0x7f37ec15c430 2024-09-12T12:23:48.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec151e00 con 0x7f37ec15c430 2024-09-12T12:23:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.452+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec0cdb50 con 0x7f37ec15c430 2024-09-12T12:23:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.456+0000 7f37e8ff9640 1 -- 172.21.15.3:0/390025268 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f37d401a070 con 0x7f37ec15c430 2024-09-12T12:23:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.456+0000 7f37e8ff9640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37cc0678c0 0x7f37cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.456+0000 7f37e8ff9640 1 -- 172.21.15.3:0/390025268 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7f37d4085d90 con 0x7f37ec15c430 2024-09-12T12:23:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.456+0000 7f37ebfff640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37cc0678c0 0x7f37cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.456+0000 7f37ebfff640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37cc0678c0 0x7f37cc069d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ad000 tx=0x7f37dc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.460+0000 7f37e8ff9640 1 -- 172.21.15.3:0/390025268 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f37d40565a0 con 0x7f37ec15c430 2024-09-12T12:23:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.624+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"} v 0) -- 0x7f37ec070920 con 0x7f37ec15c430 2024-09-12T12:23:48.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.628+0000 7f37e8ff9640 1 -- 172.21.15.3:0/390025268 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "ms_async_reap_threshold"}]=0 v20) ==== 106+0+0 (secure 0 0 0) 0x7f37d405a8a0 con 0x7f37ec15c430 2024-09-12T12:23:48.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37cc0678c0 msgr2=0x7f37cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37cc0678c0 0x7f37cc069d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0ad000 tx=0x7f37dc0a6000 comp rx=0 tx=0).stop 2024-09-12T12:23:48.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37ec15c430 msgr2=0x7f37ec157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37ec15c430 0x7f37ec157420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f37d400e930 tx=0x7f37d400edf0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 shutdown_connections 2024-09-12T12:23:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37cc0678c0 0x7f37cc069d80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37ec1600c0 0x7f37ec06fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37ec15c430 0x7f37ec157420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 --2- 172.21.15.3:0/390025268 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f37ec0cff20 0x7f37ec1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 >> 172.21.15.3:0/390025268 conn(0x7f37ec0b3ea0 msgr2=0x7f37ec166cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 shutdown_connections 2024-09-12T12:23:48.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.632+0000 7f37f261c640 1 -- 172.21.15.3:0/390025268 wait complete. 2024-09-12T12:23:48.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T12:23:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.764+0000 7fc12147c640 1 Processor -- start 2024-09-12T12:23:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.764+0000 7fc12147c640 1 -- start start 2024-09-12T12:23:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc12147c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc11c0d0cd0 0x7fc11c06e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc12147c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc12147c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc11c160600 con 0x7fc11c15c210 2024-09-12T12:23:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc12147c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc11c1608e0 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc12147c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c160bc0 con 0x7fc11c0adf60 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11a7fc640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc11c0d0cd0 0x7fc11c06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c15feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:55624/0 (socket says 172.21.15.3:55624) 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11b7fe640 1 -- 172.21.15.3:0/2790816293 learned_addr learned my addr 172.21.15.3:0/2790816293 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11b7fe640 1 -- 172.21.15.3:0/2790816293 >> v2:172.21.15.142:3300/0 conn(0x7fc11c0d0cd0 msgr2=0x7fc11c06e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11b7fe640 1 --2- 172.21.15.3:0/2790816293 >> v2:172.21.15.142:3300/0 conn(0x7fc11c0d0cd0 0x7fc11c06e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11b7fe640 1 -- 172.21.15.3:0/2790816293 >> v1:172.21.15.142:6789/0 conn(0x7fc11c15c210 legacy=0x7fc11c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11b7fe640 1 -- 172.21.15.3:0/2790816293 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c0cec00 con 0x7fc11c0adf60 2024-09-12T12:23:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.768+0000 7fc11a7fc640 1 --2- 172.21.15.3:0/2790816293 >> v2:172.21.15.142:3300/0 conn(0x7fc11c0d0cd0 0x7fc11c06e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc11b7fe640 1 --2- 172.21.15.3:0/2790816293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c15feb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc114009e50 tx=0x7fc11401c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a3895a9a9052b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc119ffb640 1 -- 172.21.15.3:0/2790816293 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140a56e0 con 0x7fc11c0adf60 2024-09-12T12:23:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc119ffb640 1 -- 172.21.15.3:0/2790816293 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1140a5880 con 0x7fc11c0adf60 2024-09-12T12:23:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc119ffb640 1 -- 172.21.15.3:0/2790816293 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140a5c50 con 0x7fc11c0adf60 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- 172.21.15.3:0/2790816293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 msgr2=0x7fc11c15feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 --2- 172.21.15.3:0/2790816293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c15feb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc114009e50 tx=0x7fc11401c910 comp rx=0 tx=0).stop 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- 172.21.15.3:0/2790816293 shutdown_connections 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 --2- 172.21.15.3:0/2790816293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c15feb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 --2- 172.21.15.3:0/2790816293 >> v2:172.21.15.142:3300/0 conn(0x7fc11c0d0cd0 0x7fc11c06e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- 172.21.15.3:0/2790816293 >> 172.21.15.3:0/2790816293 conn(0x7fc11c067e00 msgr2=0x7fc11c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- 172.21.15.3:0/2790816293 shutdown_connections 2024-09-12T12:23:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- 172.21.15.3:0/2790816293 wait complete. 2024-09-12T12:23:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 Processor -- start 2024-09-12T12:23:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- start start 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11c0d0cd0 0x7fc11c070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11c15c210 0x7fc11c0aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0cfcc0 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c0cfb40 con 0x7fc11c15c210 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.772+0000 7fc12147c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc11c0cfe40 con 0x7fc11c0adf60 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11c0d0cd0 0x7fc11c070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11c0d0cd0 0x7fc11c070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46576/0 (socket says 172.21.15.3:46576) 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 -- 172.21.15.3:0/3706087179 learned_addr learned my addr 172.21.15.3:0/3706087179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:48.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11affd640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11b7fe640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11c15c210 0x7fc11c0aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 -- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 msgr2=0x7fc11c147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 -- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11c15c210 msgr2=0x7fc11c0aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11c15c210 0x7fc11c0aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 -- 172.21.15.3:0/3706087179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c148170 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11a7fc640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11c0d0cd0 0x7fc11c070db0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc11000e980 tx=0x7fc11000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc1037fe640 1 -- 172.21.15.3:0/3706087179 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc110017070 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc1037fe640 1 -- 172.21.15.3:0/3706087179 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11000ca00 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c072d50 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc1037fe640 1 -- 172.21.15.3:0/3706087179 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11000cdd0 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c073310 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c06e450 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc1037fe640 1 -- 172.21.15.3:0/3706087179 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc11001a070 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc1037fe640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1080678c0 0x7fc108069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc1037fe640 1 -- 172.21.15.3:0/3706087179 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7fc110085f60 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.776+0000 7fc11affd640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1080678c0 0x7fc108069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.780+0000 7fc11affd640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1080678c0 0x7fc108069d80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc10c001810 tx=0x7fc10c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.780+0000 7fc1037fe640 1 -- 172.21.15.3:0/3706087179 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc1100567c0 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.948+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc11c15a350 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.952+0000 7fc1037fe640 1 -- 172.21.15.3:0/3706087179 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fc11005aac0 con 0x7fc11c0d0cd0 2024-09-12T12:23:48.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1080678c0 msgr2=0x7fc108069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1080678c0 0x7fc108069d80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc10c001810 tx=0x7fc10c009040 comp rx=0 tx=0).stop 2024-09-12T12:23:48.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11c0d0cd0 msgr2=0x7fc11c070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11c0d0cd0 0x7fc11c070db0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc11000e980 tx=0x7fc11000ee40 comp rx=0 tx=0).stop 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 shutdown_connections 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc1080678c0 0x7fc108069d80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc11c15c210 0x7fc11c0aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc11c0d0cd0 0x7fc11c070db0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 --2- 172.21.15.3:0/3706087179 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc11c0adf60 0x7fc11c147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 >> 172.21.15.3:0/3706087179 conn(0x7fc11c067e00 msgr2=0x7fc11c0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 shutdown_connections 2024-09-12T12:23:48.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:48.956+0000 7fc12147c640 1 -- 172.21.15.3:0/3706087179 wait complete. 2024-09-12T12:23:48.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close' 2024-09-12T12:23:49.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dc62a640 1 Processor -- start 2024-09-12T12:23:49.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dc62a640 1 -- start start 2024-09-12T12:23:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dc62a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4d40ae250 0x7ff4d40ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dc62a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 0x7ff4d415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dc62a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d40ad0c0 con 0x7ff4d40d05b0 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dc62a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d40ad3a0 con 0x7ff4d40ae250 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dc62a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d40ad680 con 0x7ff4d415b620 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4da3c6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff4d40d05b0 0x7ff4d40d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56660/0 (socket says 172.21.15.3:56660) 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4da3c6640 1 -- 172.21.15.3:0/3691683608 learned_addr learned my addr 172.21.15.3:0/3691683608 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d9bc5640 1 --2- 172.21.15.3:0/3691683608 >> v2:172.21.15.142:3300/0 conn(0x7ff4d40ae250 0x7ff4d40ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dabc7640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 0x7ff4d415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888477244 0 0) 0x7ff4d40ad0c0 con 0x7ff4d40d05b0 2024-09-12T12:23:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4c0003530 con 0x7ff4d40d05b0 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1811242550 0 0) 0x7ff4c0003530 con 0x7ff4d40d05b0 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 msgr2=0x7ff4d415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 0x7ff4d415bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 >> v2:172.21.15.142:3300/0 conn(0x7ff4d40ae250 msgr2=0x7ff4d40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 --2- 172.21.15.3:0/3691683608 >> v2:172.21.15.142:3300/0 conn(0x7ff4d40ae250 0x7ff4d40ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40ad960 con 0x7ff4d40d05b0 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7ff4d0001680 con 0x7ff4d40d05b0 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 >> v1:172.21.15.3:6789/0 conn(0x7ff4d40d05b0 legacy=0x7ff4d40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 0x7ff4c00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c0003f40 0x7ff4c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c0004d50 0x7ff4c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40ad3a0 con 0x7ff4c0004d50 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40ad680 con 0x7ff4c0003f40 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c000b9d0 con 0x7ff4d415b620 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d9bc5640 1 --2- 172.21.15.3:0/3691683608 >> v2:172.21.15.142:3300/0 conn(0x7ff4d40ae250 0x7ff4d40ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dabc7640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c0004d50 0x7ff4c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4da3c6640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 0x7ff4c00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4d9bc5640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c0003f40 0x7ff4c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dabc7640 1 -- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 msgr2=0x7ff4c00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dabc7640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d415b620 0x7ff4c00039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dabc7640 1 -- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c0003f40 msgr2=0x7ff4c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dabc7640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4c0003f40 0x7ff4c0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.092+0000 7ff4dabc7640 1 -- 172.21.15.3:0/3691683608 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c0003530 con 0x7ff4c0004d50 2024-09-12T12:23:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dabc7640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c0004d50 0x7ff4c00051d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009fc0 tx=0x7ff4cc01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c2b24c2184c4ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0b2070 con 0x7ff4c0004d50 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc0a5c90 con 0x7ff4c0004d50 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4d93c4640 1 -- 172.21.15.3:0/3691683608 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc009c10 con 0x7ff4c0004d50 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3691683608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c0004d50 msgr2=0x7ff4c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 --2- 172.21.15.3:0/3691683608 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4c0004d50 0x7ff4c00051d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009fc0 tx=0x7ff4cc01cc20 comp rx=0 tx=0).stop 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dabc7640 1 -- 172.21.15.3:0/3691683608 reap_dead start 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3691683608 shutdown_connections 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3691683608 >> 172.21.15.3:0/3691683608 conn(0x7ff4d4067e00 msgr2=0x7ff4d4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3691683608 shutdown_connections 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3691683608 wait complete. 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 Processor -- start 2024-09-12T12:23:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.096+0000 7ff4dc62a640 1 -- start start 2024-09-12T12:23:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4d415a550 0x7ff4d4071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d4072f00 0x7ff4d41f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d415a930 0x7ff4d40cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40aeda0 con 0x7ff4d415a550 2024-09-12T12:23:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40aec20 con 0x7ff4d415a930 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d41559b0 con 0x7ff4d4072f00 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4d415a550 0x7ff4d4071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4d415a550 0x7ff4d4071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46594/0 (socket says 172.21.15.3:46594) 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 -- 172.21.15.3:0/3813341459 learned_addr learned my addr 172.21.15.3:0/3813341459 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4d9bc5640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d4072f00 0x7ff4d41f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dabc7640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d415a930 0x7ff4d40cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 -- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d4072f00 msgr2=0x7ff4d41f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d4072f00 0x7ff4d41f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 -- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d415a930 msgr2=0x7ff4d40cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d415a930 0x7ff4d40cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 -- 172.21.15.3:0/3813341459 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d41f5190 con 0x7ff4d415a550 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4d9bc5640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d4072f00 0x7ff4d41f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4da3c6640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4d415a550 0x7ff4d4071c50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000db10 tx=0x7ff4d0001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4caffd640 1 -- 172.21.15.3:0/3813341459 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0019070 con 0x7ff4d415a550 2024-09-12T12:23:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4caffd640 1 -- 172.21.15.3:0/3813341459 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d0004cc0 con 0x7ff4d415a550 2024-09-12T12:23:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d41f5470 con 0x7ff4d415a550 2024-09-12T12:23:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4caffd640 1 -- 172.21.15.3:0/3813341459 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00072f0 con 0x7ff4d415a550 2024-09-12T12:23:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d4071450 con 0x7ff4d415a550 2024-09-12T12:23:49.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d415c830 con 0x7ff4d415a550 2024-09-12T12:23:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4caffd640 1 -- 172.21.15.3:0/3813341459 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff4d001c070 con 0x7ff4d415a550 2024-09-12T12:23:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4caffd640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff4b80678c0 0x7ff4b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4caffd640 1 -- 172.21.15.3:0/3813341459 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7ff4d00854a0 con 0x7ff4d415a550 2024-09-12T12:23:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.100+0000 7ff4d9bc5640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff4b80678c0 0x7ff4b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:49.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.104+0000 7ff4d9bc5640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff4b80678c0 0x7ff4b8069d80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff4d415b0c0 tx=0x7ff4c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.104+0000 7ff4caffd640 1 -- 172.21.15.3:0/3813341459 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff4d00562e0 con 0x7ff4d415a550 2024-09-12T12:23:49.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:49.328+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]} v 0) -- 0x7ff4d4152c00 con 0x7ff4d415a550 2024-09-12T12:23:50.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.296+0000 7ff4caffd640 1 -- 172.21.15.3:0/3813341459 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_conn_close"]}]=0 v216) ==== 143+0+0 (secure 0 0 0) 0x7ff4d0055be0 con 0x7ff4d415a550 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff4b80678c0 msgr2=0x7ff4b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff4b80678c0 0x7ff4b8069d80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff4d415b0c0 tx=0x7ff4c4008040 comp rx=0 tx=0).stop 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4d415a550 msgr2=0x7ff4d4071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4d415a550 0x7ff4d4071c50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000db10 tx=0x7ff4d0001e10 comp rx=0 tx=0).stop 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 shutdown_connections 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff4b80678c0 0x7ff4b8069d80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d415a930 0x7ff4d40cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d4072f00 0x7ff4d41f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 --2- 172.21.15.3:0/3813341459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4d415a550 0x7ff4d4071c50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 >> 172.21.15.3:0/3813341459 conn(0x7ff4d4067e00 msgr2=0x7ff4d4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 shutdown_connections 2024-09-12T12:23:50.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.300+0000 7ff4dc62a640 1 -- 172.21.15.3:0/3813341459 wait complete. 2024-09-12T12:23:50.326 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-09-12T12:23:50.327 INFO:tasks.cephfs_test_runner:Starting test: test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) That when a `tell` command is sent using the python CLI, 2024-09-12T12:23:50.327 INFO:tasks.cephfs_test_runner:test_tell_session_drop (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-12T12:23:50.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:50.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e78f08640 1 Processor -- start 2024-09-12T12:23:50.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e78f08640 1 -- start start 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e78f08640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3e740d1190 0x7f3e740d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e78f08640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e740d1570 0x7f3e740cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e78f08640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e740d30b0 con 0x7f3e740cf950 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e78f08640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3e740d3cc0 con 0x7f3e740d1570 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e78f08640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3e740d3390 con 0x7f3e740d1190 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e71d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e740d1570 0x7f3e740cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e71d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3e740d1570 0x7f3e740cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46596/0 (socket says 172.21.15.3:46596) 2024-09-12T12:23:50.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e71d9b640 1 -- 172.21.15.3:0/1092790569 learned_addr learned my addr 172.21.15.3:0/1092790569 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e7259c640 1 --2- 172.21.15.3:0/1092790569 >> v2:172.21.15.142:3300/0 conn(0x7f3e740d1190 0x7f3e740d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e71d9b640 1 -- 172.21.15.3:0/1092790569 >> v2:172.21.15.142:3300/0 conn(0x7f3e740d1190 msgr2=0x7f3e740d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e71d9b640 1 --2- 172.21.15.3:0/1092790569 >> v2:172.21.15.142:3300/0 conn(0x7f3e740d1190 0x7f3e740d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e71d9b640 1 -- 172.21.15.3:0/1092790569 >> v1:172.21.15.142:6789/0 conn(0x7f3e740cf950 legacy=0x7f3e740b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.456+0000 7f3e71d9b640 1 -- 172.21.15.3:0/1092790569 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740d3670 con 0x7f3e740d1570 2024-09-12T12:23:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 --2- 172.21.15.3:0/1092790569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1570 0x7f3e740cf410 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3e7406d9e0 tx=0x7f3e6801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeecaa2daba562c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e7159a640 1 -- 172.21.15.3:0/1092790569 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680b5070 con 0x7f3e740d1570 2024-09-12T12:23:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e7159a640 1 -- 172.21.15.3:0/1092790569 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e680a8a10 con 0x7f3e740d1570 2024-09-12T12:23:50.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e7159a640 1 -- 172.21.15.3:0/1092790569 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680a8de0 con 0x7f3e740d1570 2024-09-12T12:23:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- 172.21.15.3:0/1092790569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1570 msgr2=0x7f3e740cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 --2- 172.21.15.3:0/1092790569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1570 0x7f3e740cf410 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3e7406d9e0 tx=0x7f3e6801ed10 comp rx=0 tx=0).stop 2024-09-12T12:23:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- 172.21.15.3:0/1092790569 shutdown_connections 2024-09-12T12:23:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 --2- 172.21.15.3:0/1092790569 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1570 0x7f3e740cf410 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 --2- 172.21.15.3:0/1092790569 >> v2:172.21.15.142:3300/0 conn(0x7f3e740d1190 0x7f3e740d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- 172.21.15.3:0/1092790569 >> 172.21.15.3:0/1092790569 conn(0x7f3e74067a00 msgr2=0x7f3e74067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:50.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- 172.21.15.3:0/1092790569 shutdown_connections 2024-09-12T12:23:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- 172.21.15.3:0/1092790569 wait complete. 2024-09-12T12:23:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 Processor -- start 2024-09-12T12:23:50.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- start start 2024-09-12T12:23:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740cf950 0x7f3e74155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1190 0x7f3e7415d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740d1570 0x7f3e7415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740d2cd0 con 0x7f3e740d1190 2024-09-12T12:23:50.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740d1a00 con 0x7f3e740cf950 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740d2e50 con 0x7f3e740d1570 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1190 0x7f3e7415d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1190 0x7f3e7415d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46608/0 (socket says 172.21.15.3:46608) 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 -- 172.21.15.3:0/3681050531 learned_addr learned my addr 172.21.15.3:0/3681050531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e7259c640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740cf950 0x7f3e74155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e72d9d640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740d1570 0x7f3e7415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 -- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740d1570 msgr2=0x7f3e7415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740d1570 0x7f3e7415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 -- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740cf950 msgr2=0x7f3e74155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740cf950 0x7f3e74155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 -- 172.21.15.3:0/3681050531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e741592f0 con 0x7f3e740d1190 2024-09-12T12:23:50.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e71d9b640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1190 0x7f3e7415d860 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3e680b3040 tx=0x7f3e6801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e62ffd640 1 -- 172.21.15.3:0/3681050531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680c0070 con 0x7f3e740d1190 2024-09-12T12:23:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e62ffd640 1 -- 172.21.15.3:0/3681050531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e680b0a10 con 0x7f3e740d1190 2024-09-12T12:23:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e741595d0 con 0x7f3e740d1190 2024-09-12T12:23:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e62ffd640 1 -- 172.21.15.3:0/3681050531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680b5040 con 0x7f3e740d1190 2024-09-12T12:23:50.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.460+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e740c8520 con 0x7f3e740d1190 2024-09-12T12:23:50.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.464+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7415e860 con 0x7f3e740d1190 2024-09-12T12:23:50.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.464+0000 7f3e62ffd640 1 -- 172.21.15.3:0/3681050531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3e680c3070 con 0x7f3e740d1190 2024-09-12T12:23:50.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.464+0000 7f3e62ffd640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3e540678c0 0x7f3e54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.464+0000 7f3e62ffd640 1 -- 172.21.15.3:0/3681050531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7f3e681227a0 con 0x7f3e740d1190 2024-09-12T12:23:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.464+0000 7f3e7259c640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3e540678c0 0x7f3e54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.464+0000 7f3e7259c640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3e540678c0 0x7f3e54069d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006aa0 tx=0x7f3e64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:50.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.468+0000 7f3e62ffd640 1 -- 172.21.15.3:0/3681050531 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3e68018870 con 0x7f3e740d1190 2024-09-12T12:23:50.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.632+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e740ba4c0 con 0x7f3e740d1190 2024-09-12T12:23:50.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.632+0000 7f3e62ffd640 1 -- 172.21.15.3:0/3681050531 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+12969 (secure 0 0 0) 0x7f3e680f3020 con 0x7f3e740d1190 2024-09-12T12:23:50.635 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:50.636 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":71,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:48.278046+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/550473013":"2024-09-13T12:23:45.263242+0000","172.21.15.3:6833/550473013":"2024-09-13T12:23:45.263242+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:50.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3e540678c0 msgr2=0x7f3e54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3e540678c0 0x7f3e54069d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006aa0 tx=0x7f3e64008040 comp rx=0 tx=0).stop 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1190 msgr2=0x7f3e7415d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1190 0x7f3e7415d860 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3e680b3040 tx=0x7f3e6801bab0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 shutdown_connections 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3e540678c0 0x7f3e54069d80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3e740d1570 0x7f3e7415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3e740d1190 0x7f3e7415d860 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 --2- 172.21.15.3:0/3681050531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e740cf950 0x7f3e74155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 >> 172.21.15.3:0/3681050531 conn(0x7f3e74067a00 msgr2=0x7f3e740b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 shutdown_connections 2024-09-12T12:23:50.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.636+0000 7f3e78f08640 1 -- 172.21.15.3:0/3681050531 wait complete. 2024-09-12T12:23:50.655 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:19:37.735209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:23:50.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T12:23:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa629889640 1 Processor -- start 2024-09-12T12:23:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa629889640 1 -- start start 2024-09-12T12:23:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa629889640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa6240b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa629889640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa6240b24c0 0x7fa6240af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa629889640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6240d1790 con 0x7fa6240c7420 2024-09-12T12:23:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa629889640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa6240b3df0 con 0x7fa6240b24c0 2024-09-12T12:23:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa629889640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240c80c0 con 0x7fa62415c6d0 2024-09-12T12:23:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa6240b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa6237fe640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa6240b24c0 0x7fa6240af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa6240b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:55694/0 (socket says 172.21.15.3:55694) 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa622ffd640 1 -- 172.21.15.3:0/2851347929 learned_addr learned my addr 172.21.15.3:0/2851347929 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa622ffd640 1 -- 172.21.15.3:0/2851347929 >> v2:172.21.15.142:3300/0 conn(0x7fa6240b24c0 msgr2=0x7fa6240af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa622ffd640 1 --2- 172.21.15.3:0/2851347929 >> v2:172.21.15.142:3300/0 conn(0x7fa6240b24c0 0x7fa6240af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa622ffd640 1 -- 172.21.15.3:0/2851347929 >> v1:172.21.15.142:6789/0 conn(0x7fa6240c7420 legacy=0x7fa6240b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa622ffd640 1 -- 172.21.15.3:0/2851347929 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6240b3830 con 0x7fa62415c6d0 2024-09-12T12:23:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.780+0000 7fa6237fe640 1 --2- 172.21.15.3:0/2851347929 >> v2:172.21.15.142:3300/0 conn(0x7fa6240b24c0 0x7fa6240af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa622ffd640 1 --2- 172.21.15.3:0/2851347929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa6240b4a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa614009870 tx=0x7fa61401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba30a6e4b78a1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:50.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa621ffb640 1 -- 172.21.15.3:0/2851347929 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140aa600 con 0x7fa62415c6d0 2024-09-12T12:23:50.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa621ffb640 1 -- 172.21.15.3:0/2851347929 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6140af070 con 0x7fa62415c6d0 2024-09-12T12:23:50.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa621ffb640 1 -- 172.21.15.3:0/2851347929 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140aa9c0 con 0x7fa62415c6d0 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 -- 172.21.15.3:0/2851347929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 msgr2=0x7fa6240b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 --2- 172.21.15.3:0/2851347929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa6240b4a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa614009870 tx=0x7fa61401c4a0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 -- 172.21.15.3:0/2851347929 shutdown_connections 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 --2- 172.21.15.3:0/2851347929 >> v2:172.21.15.142:3300/0 conn(0x7fa6240b24c0 0x7fa6240af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 --2- 172.21.15.3:0/2851347929 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa6240b4a90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 -- 172.21.15.3:0/2851347929 >> 172.21.15.3:0/2851347929 conn(0x7fa6240b3fa0 msgr2=0x7fa6240b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 -- 172.21.15.3:0/2851347929 shutdown_connections 2024-09-12T12:23:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 -- 172.21.15.3:0/2851347929 wait complete. 2024-09-12T12:23:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.784+0000 7fa629889640 1 Processor -- start 2024-09-12T12:23:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 -- start start 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6240b24c0 0x7fa62414c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6240c7420 0x7fa624152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa624158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240b0360 con 0x7fa6240b24c0 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240b01e0 con 0x7fa6240c7420 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240b04e0 con 0x7fa62415c6d0 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6240b24c0 0x7fa62414c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6240b24c0 0x7fa62414c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46618/0 (socket says 172.21.15.3:46618) 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 -- 172.21.15.3:0/532803362 learned_addr learned my addr 172.21.15.3:0/532803362 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa6237fe640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa624158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa6227fc640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6240c7420 0x7fa624152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 -- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 msgr2=0x7fa624158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa624158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 -- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6240c7420 msgr2=0x7fa624152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6240c7420 0x7fa624152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 -- 172.21.15.3:0/532803362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624157760 con 0x7fa6240b24c0 2024-09-12T12:23:50.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa622ffd640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6240b24c0 0x7fa62414c9b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa6140ac040 tx=0x7fa614019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:50.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa60bfff640 1 -- 172.21.15.3:0/532803362 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140be610 con 0x7fa6240b24c0 2024-09-12T12:23:50.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa60bfff640 1 -- 172.21.15.3:0/532803362 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6140aae50 con 0x7fa6240b24c0 2024-09-12T12:23:50.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa60bfff640 1 -- 172.21.15.3:0/532803362 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140bea30 con 0x7fa6240b24c0 2024-09-12T12:23:50.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa624157a40 con 0x7fa6240b24c0 2024-09-12T12:23:50.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62414cef0 con 0x7fa6240b24c0 2024-09-12T12:23:50.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6240af280 con 0x7fa6240b24c0 2024-09-12T12:23:50.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.788+0000 7fa60bfff640 1 -- 172.21.15.3:0/532803362 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6140bc020 con 0x7fa6240b24c0 2024-09-12T12:23:50.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.792+0000 7fa60bfff640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa6100678f0 0x7fa610069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:50.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.792+0000 7fa60bfff640 1 -- 172.21.15.3:0/532803362 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7fa61411ff20 con 0x7fa6240b24c0 2024-09-12T12:23:50.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.792+0000 7fa6227fc640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa6100678f0 0x7fa610069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:50.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.792+0000 7fa6227fc640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa6100678f0 0x7fa610069db0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa62414d570 tx=0x7fa618009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.796+0000 7fa60bfff640 1 -- 172.21.15.3:0/532803362 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa614016150 con 0x7fa6240b24c0 2024-09-12T12:23:50.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.960+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6240c7a00 con 0x7fa6240b24c0 2024-09-12T12:23:50.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.960+0000 7fa60bfff640 1 -- 172.21.15.3:0/532803362 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fa6140f0d50 con 0x7fa6240b24c0 2024-09-12T12:23:50.963 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T12:23:50.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.960+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa6100678f0 msgr2=0x7fa610069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.960+0000 7fa629889640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa6100678f0 0x7fa610069db0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa62414d570 tx=0x7fa618009040 comp rx=0 tx=0).stop 2024-09-12T12:23:50.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.960+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6240b24c0 msgr2=0x7fa62414c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.960+0000 7fa629889640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6240b24c0 0x7fa62414c9b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa6140ac040 tx=0x7fa614019240 comp rx=0 tx=0).stop 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 shutdown_connections 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa6100678f0 0x7fa610069db0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa62415c6d0 0x7fa624158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6240c7420 0x7fa624152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 --2- 172.21.15.3:0/532803362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6240b24c0 0x7fa62414c9b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 >> 172.21.15.3:0/532803362 conn(0x7fa6240b3fa0 msgr2=0x7fa6240d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 shutdown_connections 2024-09-12T12:23:50.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:50.964+0000 7fa629889640 1 -- 172.21.15.3:0/532803362 wait complete. 2024-09-12T12:23:50.981 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-09-12T12:23:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.104+0000 7f7054556640 1 Processor -- start 2024-09-12T12:23:51.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.104+0000 7f7054556640 1 -- start start 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.104+0000 7f7054556640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f704c0b30e0 0x7f704c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.104+0000 7f7054556640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704c0b5000 con 0x7f704c15c710 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.104+0000 7f7054556640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704c0b3510 con 0x7f704c073cb0 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7054556640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f704c074090 con 0x7f704c0b30e0 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7051af1640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f704c15c710 0x7f704c0ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56674/0 (socket says 172.21.15.3:56674) 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7051af1640 1 -- 172.21.15.3:0/1191272166 learned_addr learned my addr 172.21.15.3:0/1191272166 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7052af3640 1 --2- 172.21.15.3:0/1191272166 >> v2:172.21.15.142:3300/0 conn(0x7f704c0b30e0 0x7f704c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1368344340 0 0) 0x7f704c0b5000 con 0x7f704c15c710 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7034003530 con 0x7f704c15c710 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4090906456 0 0) 0x7f704c0b3510 con 0x7f704c073cb0 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f704c0b5000 con 0x7f704c073cb0 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2700987161 0 0) 0x7f7034003530 con 0x7f704c15c710 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 >> v2:172.21.15.142:3300/0 conn(0x7f704c0b30e0 msgr2=0x7f704c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 --2- 172.21.15.3:0/1191272166 >> v2:172.21.15.142:3300/0 conn(0x7f704c0b30e0 0x7f704c0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 >> v1:172.21.15.142:6789/0 conn(0x7f704c073cb0 legacy=0x7f704c0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c0b37c0 con 0x7f704c15c710 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f7040001680 con 0x7f704c15c710 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 >> v1:172.21.15.3:6789/0 conn(0x7f704c15c710 legacy=0x7f704c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034003b50 0x7f7034003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034004810 0x7f7034004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70340051d0 0x7f703400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f703400c020 con 0x7f7034003b50 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f703400c1d0 con 0x7f7034004810 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f703400c3d0 con 0x7f70340051d0 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70522f2640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034003b50 0x7f7034003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7051af1640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034004810 0x7f7034004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7052af3640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70340051d0 0x7f703400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70522f2640 1 -- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70340051d0 msgr2=0x7f703400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70522f2640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70340051d0 0x7f703400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70522f2640 1 -- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034004810 msgr2=0x7f7034004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70522f2640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034004810 0x7f7034004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:51.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70522f2640 1 -- 172.21.15.3:0/1191272166 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7034003530 con 0x7f7034003b50 2024-09-12T12:23:51.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7052af3640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70340051d0 0x7f703400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:51.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f7052af3640 1 -- 172.21.15.3:0/1191272166 reap_dead start 2024-09-12T12:23:51.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70522f2640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034003b50 0x7f7034003f50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f703c0018d0 tx=0x7f703c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6218c2498695729d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c001520 con 0x7f7034003b50 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703c0052f0 con 0x7f7034003b50 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.108+0000 7f70512f0640 1 -- 172.21.15.3:0/1191272166 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c003840 con 0x7f7034003b50 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- 172.21.15.3:0/1191272166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034003b50 msgr2=0x7f7034003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034003b50 0x7f7034003f50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f703c0018d0 tx=0x7f703c01c520 comp rx=0 tx=0).stop 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- 172.21.15.3:0/1191272166 shutdown_connections 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 --2- 172.21.15.3:0/1191272166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7034003b50 0x7f7034003f50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- 172.21.15.3:0/1191272166 >> 172.21.15.3:0/1191272166 conn(0x7f704c067e00 msgr2=0x7f704c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- 172.21.15.3:0/1191272166 shutdown_connections 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- 172.21.15.3:0/1191272166 wait complete. 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 Processor -- start 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- start start 2024-09-12T12:23:51.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034003b50 0x7f704c1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f704c06ccd0 0x7f704c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f704c0ce720 0x7f704c0c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f704c0aea00 con 0x7f704c06ccd0 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f704c0ae880 con 0x7f7034003b50 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f704c0aeb80 con 0x7f704c0ce720 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f704c06ccd0 0x7f704c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f704c06ccd0 0x7f704c0afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46644/0 (socket says 172.21.15.3:46644) 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 -- 172.21.15.3:0/2697995143 learned_addr learned my addr 172.21.15.3:0/2697995143 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7052af3640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f704c0ce720 0x7f704c0c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f70522f2640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034003b50 0x7f704c1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 -- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f704c0ce720 msgr2=0x7f704c0c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f704c0ce720 0x7f704c0c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:51.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 -- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034003b50 msgr2=0x7f704c1569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034003b50 0x7f704c1569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 -- 172.21.15.3:0/2697995143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c06d1d0 con 0x7f704c06ccd0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7051af1640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f704c06ccd0 0x7f704c0afec0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f704000e7b0 tx=0x7f704000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f704affd640 1 -- 172.21.15.3:0/2697995143 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704000ee60 con 0x7f704c06ccd0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f704affd640 1 -- 172.21.15.3:0/2697995143 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7040018070 con 0x7f704c06ccd0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c06d4b0 con 0x7f704c06ccd0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f704affd640 1 -- 172.21.15.3:0/2697995143 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704000cb10 con 0x7f704c06ccd0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.112+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c06da70 con 0x7f704c06ccd0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.116+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c0b34c0 con 0x7f704c06ccd0 2024-09-12T12:23:51.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.116+0000 7f704affd640 1 -- 172.21.15.3:0/2697995143 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f704000ccb0 con 0x7f704c06ccd0 2024-09-12T12:23:51.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.116+0000 7f704affd640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f70380678f0 0x7f7038069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:51.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.116+0000 7f704affd640 1 -- 172.21.15.3:0/2697995143 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7f7040085450 con 0x7f704c06ccd0 2024-09-12T12:23:51.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.116+0000 7f70522f2640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f70380678f0 0x7f7038069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:51.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.116+0000 7f70522f2640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f70380678f0 0x7f7038069db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f703c0ac000 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:51.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.120+0000 7f704affd640 1 -- 172.21.15.3:0/2697995143 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7040056280 con 0x7f704c06ccd0 2024-09-12T12:23:51.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:51.344+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7f704c0ad260 con 0x7f704c06ccd0 2024-09-12T12:23:52.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f704affd640 1 -- 172.21.15.3:0/2697995143 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v218) ==== 148+0+0 (secure 0 0 0) 0x7f7040055b80 con 0x7f704c06ccd0 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f70380678f0 msgr2=0x7f7038069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f70380678f0 0x7f7038069db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f703c0ac000 tx=0x7f703c009040 comp rx=0 tx=0).stop 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f704c06ccd0 msgr2=0x7f704c0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f704c06ccd0 0x7f704c0afec0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f704000e7b0 tx=0x7f704000ec70 comp rx=0 tx=0).stop 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 shutdown_connections 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f70380678f0 0x7f7038069db0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f704c0ce720 0x7f704c0c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f704c06ccd0 0x7f704c0afec0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 --2- 172.21.15.3:0/2697995143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034003b50 0x7f704c1569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.308+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 >> 172.21.15.3:0/2697995143 conn(0x7f704c067e00 msgr2=0x7f704c15fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.312+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 shutdown_connections 2024-09-12T12:23:52.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.312+0000 7f7054556640 1 -- 172.21.15.3:0/2697995143 wait complete. 2024-09-12T12:23:52.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T12:23:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd99fa640 1 Processor -- start 2024-09-12T12:23:52.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd99fa640 1 -- start start 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd99fa640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3fd415e750 0x7f3fd40d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd99fa640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3fd40cf880 0x7f3fd4168400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd99fa640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fd40d10d0 con 0x7f3fd40d40c0 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd99fa640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd40d13b0 con 0x7f3fd415e750 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd99fa640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd40d1690 con 0x7f3fd40cf880 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3fd415e750 0x7f3fd40d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd2ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3fd415e750 0x7f3fd40d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46648/0 (socket says 172.21.15.3:46648) 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd27fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3fd40d40c0 0x7f3fd40cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56686/0 (socket says 172.21.15.3:56686) 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd2ffd640 1 -- 172.21.15.3:0/4133075349 learned_addr learned my addr 172.21.15.3:0/4133075349 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd2ffd640 1 -- 172.21.15.3:0/4133075349 >> v2:172.21.15.142:3300/0 conn(0x7f3fd40cf880 msgr2=0x7f3fd4168400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd37fe640 1 --2- 172.21.15.3:0/4133075349 >> v2:172.21.15.142:3300/0 conn(0x7f3fd40cf880 0x7f3fd4168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd2ffd640 1 --2- 172.21.15.3:0/4133075349 >> v2:172.21.15.142:3300/0 conn(0x7f3fd40cf880 0x7f3fd4168400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd2ffd640 1 -- 172.21.15.3:0/4133075349 >> v1:172.21.15.3:6789/0 conn(0x7f3fd40d40c0 legacy=0x7f3fd40cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd2ffd640 1 -- 172.21.15.3:0/4133075349 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd40d1970 con 0x7f3fd415e750 2024-09-12T12:23:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.464+0000 7f3fd1ffb640 1 -- 172.21.15.3:0/4133075349 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145212959 0 0) 0x7f3fd40d10d0 con 0x7f3fd40d40c0 2024-09-12T12:23:52.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd2ffd640 1 --2- 172.21.15.3:0/4133075349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd415e750 0x7f3fd40d3b80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40d0570 tx=0x7f3fc401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e84d6dd4c5adc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd1ffb640 1 -- 172.21.15.3:0/4133075349 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40b4070 con 0x7f3fd415e750 2024-09-12T12:23:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd1ffb640 1 -- 172.21.15.3:0/4133075349 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc40af400 con 0x7f3fd415e750 2024-09-12T12:23:52.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd1ffb640 1 -- 172.21.15.3:0/4133075349 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40af7d0 con 0x7f3fd415e750 2024-09-12T12:23:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- 172.21.15.3:0/4133075349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd415e750 msgr2=0x7f3fd40d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/4133075349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd415e750 0x7f3fd40d3b80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40d0570 tx=0x7f3fc401f1e0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- 172.21.15.3:0/4133075349 shutdown_connections 2024-09-12T12:23:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/4133075349 >> v2:172.21.15.142:3300/0 conn(0x7f3fd40cf880 0x7f3fd4168400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/4133075349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd415e750 0x7f3fd40d3b80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- 172.21.15.3:0/4133075349 >> 172.21.15.3:0/4133075349 conn(0x7f3fd4067a00 msgr2=0x7f3fd4067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:52.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- 172.21.15.3:0/4133075349 shutdown_connections 2024-09-12T12:23:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- 172.21.15.3:0/4133075349 wait complete. 2024-09-12T12:23:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 Processor -- start 2024-09-12T12:23:52.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- start start 2024-09-12T12:23:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fd40cf880 0x7f3fd415ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd40d40c0 0x7f3fd415fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd415e750 0x7f3fd4160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd415eb30 con 0x7f3fd40d40c0 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd40b6e10 con 0x7f3fd415e750 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.468+0000 7f3fd99fa640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd415ecb0 con 0x7f3fd40cf880 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd40d40c0 0x7f3fd415fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd40d40c0 0x7f3fd415fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46656/0 (socket says 172.21.15.3:46656) 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 -- 172.21.15.3:0/3110108552 learned_addr learned my addr 172.21.15.3:0/3110108552 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd2ffd640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fd40cf880 0x7f3fd415ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd37fe640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd415e750 0x7f3fd4160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 -- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fd40cf880 msgr2=0x7f3fd415ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fd40cf880 0x7f3fd415ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 -- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd415e750 msgr2=0x7f3fd4160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd415e750 0x7f3fd4160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 -- 172.21.15.3:0/3110108552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd41f4370 con 0x7f3fd40d40c0 2024-09-12T12:23:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd37fe640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd415e750 0x7f3fd4160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd27fc640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd40d40c0 0x7f3fd415fcd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80011f0 tx=0x7f3fc800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fbbfff640 1 -- 172.21.15.3:0/3110108552 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8018070 con 0x7f3fd40d40c0 2024-09-12T12:23:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fbbfff640 1 -- 172.21.15.3:0/3110108552 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc80074e0 con 0x7f3fd40d40c0 2024-09-12T12:23:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd41f4630 con 0x7f3fd40d40c0 2024-09-12T12:23:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fbbfff640 1 -- 172.21.15.3:0/3110108552 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8004a10 con 0x7f3fd40d40c0 2024-09-12T12:23:52.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd41f4bf0 con 0x7f3fd40d40c0 2024-09-12T12:23:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd40d3b80 con 0x7f3fd40d40c0 2024-09-12T12:23:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fbbfff640 1 -- 172.21.15.3:0/3110108552 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3fc801b070 con 0x7f3fd40d40c0 2024-09-12T12:23:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fbbfff640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fc00678c0 0x7f3fc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fbbfff640 1 -- 172.21.15.3:0/3110108552 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7f3fc8084b90 con 0x7f3fd40d40c0 2024-09-12T12:23:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.472+0000 7f3fd2ffd640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fc00678c0 0x7f3fc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.476+0000 7f3fd2ffd640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fc00678c0 0x7f3fc0069d80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b1000 tx=0x7f3fc40aa000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:52.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.476+0000 7f3fbbfff640 1 -- 172.21.15.3:0/3110108552 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3fc8055410 con 0x7f3fd40d40c0 2024-09-12T12:23:52.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.640+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3fd40d2700 con 0x7f3fd40d40c0 2024-09-12T12:23:52.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.648+0000 7f3fbbfff640 1 -- 172.21.15.3:0/3110108552 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f3fc8059710 con 0x7f3fd40d40c0 2024-09-12T12:23:52.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fc00678c0 msgr2=0x7f3fc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fc00678c0 0x7f3fc0069d80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b1000 tx=0x7f3fc40aa000 comp rx=0 tx=0).stop 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd40d40c0 msgr2=0x7f3fd415fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd40d40c0 0x7f3fd415fcd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80011f0 tx=0x7f3fc800c9b0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 shutdown_connections 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3fc00678c0 0x7f3fc0069d80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd415e750 0x7f3fd4160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fd40d40c0 0x7f3fd415fcd0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 --2- 172.21.15.3:0/3110108552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fd40cf880 0x7f3fd415ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 >> 172.21.15.3:0/3110108552 conn(0x7f3fd4067a00 msgr2=0x7f3fd40b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 shutdown_connections 2024-09-12T12:23:52.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.652+0000 7f3fd99fa640 1 -- 172.21.15.3:0/3110108552 wait complete. 2024-09-12T12:23:52.671 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:23:52.671 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:52.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:23:52.773 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:23:52.773 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:52.773 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:52.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:52.779 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:52.779 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:52.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:52.786 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:23:52.786 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:52.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:23:52.826 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:23:52.826 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:23:52.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:23:52.832 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T12:23:52.832 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:23:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa2aa4640 1 Processor -- start 2024-09-12T12:23:52.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa2aa4640 1 -- start start 2024-09-12T12:23:52.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa2aa4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a9c15c710 0x7f0a9c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa2aa4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c0ae880 0x7f0a9c0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:52.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa2aa4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a9c0b5000 con 0x7f0a9c073cb0 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa2aa4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0a9c074090 con 0x7f0a9c15c710 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa2aa4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9c0b37c0 con 0x7f0a9c0ae880 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a9c15c710 0x7f0a9c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0a9c15c710 0x7f0a9c0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46662/0 (socket says 172.21.15.3:46662) 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa0840640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0a9c073cb0 0x7f0a9c0adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56694/0 (socket says 172.21.15.3:56694) 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9bfff640 1 -- 172.21.15.3:0/886762397 learned_addr learned my addr 172.21.15.3:0/886762397 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0aa1041640 1 --2- 172.21.15.3:0/886762397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c0ae880 0x7f0a9c0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9bfff640 1 -- 172.21.15.3:0/886762397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c0ae880 msgr2=0x7f0a9c0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9bfff640 1 --2- 172.21.15.3:0/886762397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c0ae880 0x7f0a9c0aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9bfff640 1 -- 172.21.15.3:0/886762397 >> v1:172.21.15.3:6789/0 conn(0x7f0a9c073cb0 legacy=0x7f0a9c0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9b7fe640 1 -- 172.21.15.3:0/886762397 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502659386 0 0) 0x7f0a9c0b5000 con 0x7f0a9c073cb0 2024-09-12T12:23:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.992+0000 7f0a9bfff640 1 -- 172.21.15.3:0/886762397 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c0b3aa0 con 0x7f0a9c15c710 2024-09-12T12:23:52.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0a9bfff640 1 --2- 172.21.15.3:0/886762397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c15c710 0x7f0a9c0b30e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009e50 tx=0x7f0a9001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26874518db762d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:52.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0a9b7fe640 1 -- 172.21.15.3:0/886762397 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900a57f0 con 0x7f0a9c15c710 2024-09-12T12:23:52.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0a9b7fe640 1 -- 172.21.15.3:0/886762397 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900a5990 con 0x7f0a9c15c710 2024-09-12T12:23:52.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0a9b7fe640 1 -- 172.21.15.3:0/886762397 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900a5d60 con 0x7f0a9c15c710 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/886762397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c15c710 msgr2=0x7f0a9c0b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/886762397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c15c710 0x7f0a9c0b30e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009e50 tx=0x7f0a9001ca30 comp rx=0 tx=0).stop 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/886762397 shutdown_connections 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/886762397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c0ae880 0x7f0a9c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/886762397 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c15c710 0x7f0a9c0b30e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/886762397 >> 172.21.15.3:0/886762397 conn(0x7f0a9c067e00 msgr2=0x7f0a9c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/886762397 shutdown_connections 2024-09-12T12:23:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/886762397 wait complete. 2024-09-12T12:23:53.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 Processor -- start 2024-09-12T12:23:53.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- start start 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a9c073cb0 0x7f0a9c148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c0ae880 0x7f0a9c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c15c710 0x7f0a9c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c168f40 con 0x7f0a9c0ae880 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c168dc0 con 0x7f0a9c073cb0 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0aa2aa4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9c1690c0 con 0x7f0a9c15c710 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0a9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c0ae880 0x7f0a9c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0a9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c0ae880 0x7f0a9c159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46664/0 (socket says 172.21.15.3:46664) 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:52.996+0000 7f0a9bfff640 1 -- 172.21.15.3:0/299728044 learned_addr learned my addr 172.21.15.3:0/299728044 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa1041640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c15c710 0x7f0a9c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa0840640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a9c073cb0 0x7f0a9c148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a9bfff640 1 -- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c15c710 msgr2=0x7f0a9c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a9bfff640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c15c710 0x7f0a9c15e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a9bfff640 1 -- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a9c073cb0 msgr2=0x7f0a9c148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a9bfff640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a9c073cb0 0x7f0a9c148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a9bfff640 1 -- 172.21.15.3:0/299728044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c15b230 con 0x7f0a9c0ae880 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa1041640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c15c710 0x7f0a9c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a9bfff640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c0ae880 0x7f0a9c159710 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001b0e0 tx=0x7f0a900197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a997fa640 1 -- 172.21.15.3:0/299728044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900b8670 con 0x7f0a9c0ae880 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a997fa640 1 -- 172.21.15.3:0/299728044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900be070 con 0x7f0a9c0ae880 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c159c50 con 0x7f0a9c0ae880 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a997fa640 1 -- 172.21.15.3:0/299728044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900b8a50 con 0x7f0a9c0ae880 2024-09-12T12:23:53.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c0b2260 con 0x7f0a9c0ae880 2024-09-12T12:23:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9c0aecb0 con 0x7f0a9c0ae880 2024-09-12T12:23:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a997fa640 1 -- 172.21.15.3:0/299728044 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a900b8bf0 con 0x7f0a9c0ae880 2024-09-12T12:23:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a997fa640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a880678c0 0x7f0a88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0a997fa640 1 -- 172.21.15.3:0/299728044 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7f0a901215c0 con 0x7f0a9c0ae880 2024-09-12T12:23:53.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa0840640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a880678c0 0x7f0a88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.000+0000 7f0aa0840640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a880678c0 0x7f0a88069d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c000f70 tx=0x7f0a8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:53.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.004+0000 7f0a997fa640 1 -- 172.21.15.3:0/299728044 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0a90016620 con 0x7f0a9c0ae880 2024-09-12T12:23:53.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a9c0b24a0 con 0x7f0a9c0ae880 2024-09-12T12:23:53.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0a997fa640 1 -- 172.21.15.3:0/299728044 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+3911 (secure 0 0 0) 0x7f0a900f24e0 con 0x7f0a9c0ae880 2024-09-12T12:23:53.207 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:53.207 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":47,"btime":"2024-09-12T12:23:46:265706+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5084,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/3840622871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":3840622871},{"type":"v1","addr":"172.21.15.142:6837","nonce":3840622871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1160186856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1160186856},{"type":"v1","addr":"172.21.15.142:6835","nonce":1160186856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6009,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1791272808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1791272808},{"type":"v1","addr":"172.21.15.3:6834","nonce":1791272808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2116726147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2116726147},{"type":"v1","addr":"172.21.15.3:6835","nonce":2116726147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[]} 2024-09-12T12:23:53.207 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 47 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a880678c0 msgr2=0x7f0a88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a880678c0 0x7f0a88069d80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c000f70 tx=0x7f0a8c009040 comp rx=0 tx=0).stop 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c0ae880 msgr2=0x7f0a9c159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c0ae880 0x7f0a9c159710 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001b0e0 tx=0x7f0a900197d0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 shutdown_connections 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0a880678c0 0x7f0a88069d80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0a9c15c710 0x7f0a9c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a9c0ae880 0x7f0a9c159710 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 --2- 172.21.15.3:0/299728044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a9c073cb0 0x7f0a9c148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 >> 172.21.15.3:0/299728044 conn(0x7f0a9c067e00 msgr2=0x7f0a9c15ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:53.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 shutdown_connections 2024-09-12T12:23:53.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.204+0000 7f0aa2aa4640 1 -- 172.21.15.3:0/299728044 wait complete. 2024-09-12T12:23:53.226 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:23:53.226 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T12:23:53.226 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:23:53.227 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:53.227 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:23:53.227 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T12:23:53.227 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:23:53.227 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:53.227 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:23:53.227 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T12:23:53.228 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:23:53.228 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:53.228 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:23:53.228 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T12:23:53.228 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:23:53.228 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:23:53.239 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:23:53.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:23:53.239 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:23:53.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:23:53.242 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:23:53.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:23:53.245 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:23:53.246 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:23:53.283 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:23:53.283 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:53.283 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:23:53.284 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:53.288 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:23:53.288 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:53.290 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:23:53.290 DEBUG:teuthology.parallel:result is None 2024-09-12T12:23:53.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T12:23:53.347 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:23:53.353 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:23:53.361 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:23:53.362 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:23:53.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dee25640 1 Processor -- start 2024-09-12T12:23:53.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dee25640 1 -- start start 2024-09-12T12:23:53.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dee25640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26d80d1190 0x7f26d80d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dee25640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b7740 0x7f26d80b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dee25640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26d80d30b0 con 0x7f26d80d1570 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dee25640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f26d80d3cc0 con 0x7f26d80d1190 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dee25640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26d80d3390 con 0x7f26d80b7740 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dcbc1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26d80d1190 0x7f26d80d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dcbc1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26d80d1190 0x7f26d80d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46710/0 (socket says 172.21.15.3:46710) 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dcbc1640 1 -- 172.21.15.3:0/2517451411 learned_addr learned my addr 172.21.15.3:0/2517451411 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dd3c2640 1 --2- 172.21.15.3:0/2517451411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b7740 0x7f26d80b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dcbc1640 1 -- 172.21.15.3:0/2517451411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b7740 msgr2=0x7f26d80b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dcbc1640 1 --2- 172.21.15.3:0/2517451411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b7740 0x7f26d80b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dcbc1640 1 -- 172.21.15.3:0/2517451411 >> v1:172.21.15.142:6789/0 conn(0x7f26d80d1570 legacy=0x7f26d80cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.412+0000 7f26dcbc1640 1 -- 172.21.15.3:0/2517451411 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d80d3670 con 0x7f26d80d1190 2024-09-12T12:23:53.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dcbc1640 1 --2- 172.21.15.3:0/2517451411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1190 0x7f26d80d4150 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009870 tx=0x7f26c801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723fe6efb9aa210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:53.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26d77fe640 1 -- 172.21.15.3:0/2517451411 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80b5070 con 0x7f26d80d1190 2024-09-12T12:23:53.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26d77fe640 1 -- 172.21.15.3:0/2517451411 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c80a8a50 con 0x7f26d80d1190 2024-09-12T12:23:53.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26d77fe640 1 -- 172.21.15.3:0/2517451411 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80a8e20 con 0x7f26d80d1190 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 -- 172.21.15.3:0/2517451411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1190 msgr2=0x7f26d80d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 --2- 172.21.15.3:0/2517451411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1190 0x7f26d80d4150 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009870 tx=0x7f26c801ed50 comp rx=0 tx=0).stop 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 -- 172.21.15.3:0/2517451411 shutdown_connections 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 --2- 172.21.15.3:0/2517451411 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80b7740 0x7f26d80b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 --2- 172.21.15.3:0/2517451411 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1190 0x7f26d80d4150 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 -- 172.21.15.3:0/2517451411 >> 172.21.15.3:0/2517451411 conn(0x7f26d8067a00 msgr2=0x7f26d8067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 -- 172.21.15.3:0/2517451411 shutdown_connections 2024-09-12T12:23:53.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 -- 172.21.15.3:0/2517451411 wait complete. 2024-09-12T12:23:53.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 Processor -- start 2024-09-12T12:23:53.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.416+0000 7f26dee25640 1 -- start start 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80b7740 0x7f26d8154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80d1190 0x7f26d80ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1570 0x7f26d80ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26d80d2cd0 con 0x7f26d80d1570 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f26d80d1a00 con 0x7f26d80b7740 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f26d80d2e50 con 0x7f26d80d1190 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1570 0x7f26d80ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1570 0x7f26d80ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46720/0 (socket says 172.21.15.3:46720) 2024-09-12T12:23:53.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 -- 172.21.15.3:0/397104486 learned_addr learned my addr 172.21.15.3:0/397104486 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26d7fff640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80d1190 0x7f26d80ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dcbc1640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80b7740 0x7f26d8154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 -- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80d1190 msgr2=0x7f26d80ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80d1190 0x7f26d80ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 -- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80b7740 msgr2=0x7f26d8154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80b7740 0x7f26d8154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 -- 172.21.15.3:0/397104486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d81f8040 con 0x7f26d80d1570 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dd3c2640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1570 0x7f26d80ca810 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009dd0 tx=0x7f26d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26d57fa640 1 -- 172.21.15.3:0/397104486 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0018020 con 0x7f26d80d1570 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26d57fa640 1 -- 172.21.15.3:0/397104486 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d0014480 con 0x7f26d80d1570 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26d57fa640 1 -- 172.21.15.3:0/397104486 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0014870 con 0x7f26d80d1570 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d81f8320 con 0x7f26d80d1570 2024-09-12T12:23:53.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d81f88e0 con 0x7f26d80d1570 2024-09-12T12:23:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d815e860 con 0x7f26d80d1570 2024-09-12T12:23:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26d57fa640 1 -- 172.21.15.3:0/397104486 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f26d001b020 con 0x7f26d80d1570 2024-09-12T12:23:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26d57fa640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26c40678c0 0x7f26c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:53.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.420+0000 7f26d57fa640 1 -- 172.21.15.3:0/397104486 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 4943+0+0 (secure 0 0 0) 0x7f26d0086290 con 0x7f26d80d1570 2024-09-12T12:23:53.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.424+0000 7f26dcbc1640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26c40678c0 0x7f26c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:53.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.424+0000 7f26dcbc1640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26c40678c0 0x7f26c4069d80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f26c80b2000 tx=0x7f26c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:53.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.428+0000 7f26d57fa640 1 -- 172.21.15.3:0/397104486 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f26d0022070 con 0x7f26d80d1570 2024-09-12T12:23:53.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:53.608+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f26d80ba4c0 con 0x7f26d80d1570 2024-09-12T12:23:54.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.324+0000 7f26d57fa640 1 -- 172.21.15.3:0/397104486 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) ==== 97+0+0 (secure 0 0 0) 0x7f26d0010040 con 0x7f26d80d1570 2024-09-12T12:23:54.329 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-12T12:23:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26c40678c0 msgr2=0x7f26c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26c40678c0 0x7f26c4069d80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f26c80b2000 tx=0x7f26c8009040 comp rx=0 tx=0).stop 2024-09-12T12:23:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1570 msgr2=0x7f26d80ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1570 0x7f26d80ca810 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009dd0 tx=0x7f26d000c490 comp rx=0 tx=0).stop 2024-09-12T12:23:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 shutdown_connections 2024-09-12T12:23:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f26c40678c0 0x7f26c4069d80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26d80d1570 0x7f26d80ca810 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f26d80d1190 0x7f26d80ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 --2- 172.21.15.3:0/397104486 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26d80b7740 0x7f26d8154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 >> 172.21.15.3:0/397104486 conn(0x7f26d8067a00 msgr2=0x7f26d80cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:54.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 shutdown_connections 2024-09-12T12:23:54.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.328+0000 7f26dee25640 1 -- 172.21.15.3:0/397104486 wait complete. 2024-09-12T12:23:54.353 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T12:23:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.472+0000 7f60556ed640 1 Processor -- start 2024-09-12T12:23:54.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f60556ed640 1 -- start start 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f60556ed640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6050073cb0 0x7f605015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f60556ed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60500b30e0 0x7f60500adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f60556ed640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60500b5000 con 0x7f60500ae150 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f60556ed640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6050074090 con 0x7f60500b30e0 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f60556ed640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f60500b3500 con 0x7f6050073cb0 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f60500ae150 0x7f60500ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56714/0 (socket says 172.21.15.3:56714) 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604e7fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f60500b30e0 0x7f60500adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604f7fe640 1 -- 172.21.15.3:0/2459950936 learned_addr learned my addr 172.21.15.3:0/2459950936 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:54.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604effd640 1 --2- 172.21.15.3:0/2459950936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6050073cb0 0x7f605015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604e7fc640 1 -- 172.21.15.3:0/2459950936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6050073cb0 msgr2=0x7f605015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:54.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604e7fc640 1 --2- 172.21.15.3:0/2459950936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6050073cb0 0x7f605015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604e7fc640 1 -- 172.21.15.3:0/2459950936 >> v1:172.21.15.3:6789/0 conn(0x7f60500ae150 legacy=0x7f60500ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604e7fc640 1 -- 172.21.15.3:0/2459950936 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60500b37c0 con 0x7f60500b30e0 2024-09-12T12:23:54.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604e7fc640 1 --2- 172.21.15.3:0/2459950936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f60500adc10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f60440097c0 tx=0x7f604401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762a8468ea0dc2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:54.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604dffb640 1 -- 172.21.15.3:0/2459950936 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440b0070 con 0x7f60500b30e0 2024-09-12T12:23:54.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604dffb640 1 -- 172.21.15.3:0/2459950936 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60440ab600 con 0x7f60500b30e0 2024-09-12T12:23:54.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.476+0000 7f604dffb640 1 -- 172.21.15.3:0/2459950936 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440ab9d0 con 0x7f60500b30e0 2024-09-12T12:23:54.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- 172.21.15.3:0/2459950936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 msgr2=0x7f60500adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 --2- 172.21.15.3:0/2459950936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f60500adc10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f60440097c0 tx=0x7f604401c4a0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- 172.21.15.3:0/2459950936 shutdown_connections 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 --2- 172.21.15.3:0/2459950936 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f60500adc10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 --2- 172.21.15.3:0/2459950936 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6050073cb0 0x7f605015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- 172.21.15.3:0/2459950936 >> 172.21.15.3:0/2459950936 conn(0x7f6050067e00 msgr2=0x7f6050068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- 172.21.15.3:0/2459950936 shutdown_connections 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- 172.21.15.3:0/2459950936 wait complete. 2024-09-12T12:23:54.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 Processor -- start 2024-09-12T12:23:54.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- start start 2024-09-12T12:23:54.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6050073cb0 0x7f605006e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60500ae150 0x7f6050152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f605014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f60500af110 con 0x7f60500b30e0 2024-09-12T12:23:54.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f60500aef90 con 0x7f6050073cb0 2024-09-12T12:23:54.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6050168dc0 con 0x7f60500ae150 2024-09-12T12:23:54.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f605014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f605014cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46736/0 (socket says 172.21.15.3:46736) 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 -- 172.21.15.3:0/2199453484 learned_addr learned my addr 172.21.15.3:0/2199453484 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 -- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60500ae150 msgr2=0x7f6050152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60500ae150 0x7f6050152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 -- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6050073cb0 msgr2=0x7f605006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6050073cb0 0x7f605006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 -- 172.21.15.3:0/2199453484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60500cb1f0 con 0x7f60500b30e0 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f604f7fe640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f605014cf60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f604800cbf0 tx=0x7f604800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60377fe640 1 -- 172.21.15.3:0/2199453484 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048017070 con 0x7f60500b30e0 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60377fe640 1 -- 172.21.15.3:0/2199453484 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604800ad40 con 0x7f60500b30e0 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60500cb490 con 0x7f60500b30e0 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60377fe640 1 -- 172.21.15.3:0/2199453484 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048012600 con 0x7f60500b30e0 2024-09-12T12:23:54.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.480+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60500cb730 con 0x7f60500b30e0 2024-09-12T12:23:54.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.484+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60500adc10 con 0x7f60500b30e0 2024-09-12T12:23:54.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.484+0000 7f60377fe640 1 -- 172.21.15.3:0/2199453484 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f604801a070 con 0x7f60500b30e0 2024-09-12T12:23:54.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.484+0000 7f60377fe640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f603c0678c0 0x7f603c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.484+0000 7f60377fe640 1 -- 172.21.15.3:0/2199453484 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7f60480854f0 con 0x7f60500b30e0 2024-09-12T12:23:54.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.484+0000 7f604effd640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f603c0678c0 0x7f603c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.484+0000 7f604effd640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f603c0678c0 0x7f603c069d80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f604000a8b0 tx=0x7f6040008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:54.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.488+0000 7f60377fe640 1 -- 172.21.15.3:0/2199453484 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6048055dc0 con 0x7f60500b30e0 2024-09-12T12:23:54.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.716+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f60500b2710 con 0x7f60500b30e0 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.716+0000 7f60377fe640 1 -- 172.21.15.3:0/2199453484 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f604805a0c0 con 0x7f60500b30e0 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg==", 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.720 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA==", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.721 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg==", 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-12T12:23:54.722 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw==", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w==", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.723 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA==", 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.724 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw==", 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.725 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg==", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA==", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.726 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ==", 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.727 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ==", 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.728 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA==", 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA==", 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.729 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw==", 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:23:54.730 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw==", 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.731 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA==", 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.732 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKNVOAxAAMmejHiBys/ZYsGdB0oiroQ==", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA==", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:23:54.733 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg==", 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:23:54.734 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:23:54.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f603c0678c0 msgr2=0x7f603c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f603c0678c0 0x7f603c069d80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f604000a8b0 tx=0x7f6040008040 comp rx=0 tx=0).stop 2024-09-12T12:23:54.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 msgr2=0x7f605014cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f605014cf60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f604800cbf0 tx=0x7f604800a430 comp rx=0 tx=0).stop 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 shutdown_connections 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f603c0678c0 0x7f603c069d80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60500b30e0 0x7f605014cf60 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60500ae150 0x7f6050152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 --2- 172.21.15.3:0/2199453484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6050073cb0 0x7f605006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 >> 172.21.15.3:0/2199453484 conn(0x7f6050067e00 msgr2=0x7f60501538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 shutdown_connections 2024-09-12T12:23:54.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.720+0000 7f60556ed640 1 -- 172.21.15.3:0/2199453484 wait complete. 2024-09-12T12:23:54.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T12:23:54.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10d5f22640 1 Processor -- start 2024-09-12T12:23:54.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10d5f22640 1 -- start start 2024-09-12T12:23:54.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10d5f22640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d00ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10d5f22640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10d015b620 0x7f10d015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10d5f22640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d00ad0c0 con 0x7f10d00d05b0 2024-09-12T12:23:54.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10d5f22640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f10d00ad3a0 con 0x7f10d015b620 2024-09-12T12:23:54.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10d5f22640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f10d00ad680 con 0x7f10d00ae250 2024-09-12T12:23:54.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10cffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10d015b620 0x7f10d015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10ceffd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d00ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10cffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10d015b620 0x7f10d015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:48982/0 (socket says 172.21.15.3:48982) 2024-09-12T12:23:54.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10cf7fe640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f10d00d05b0 0x7f10d00d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:49090/0 (socket says 172.21.15.3:49090) 2024-09-12T12:23:54.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10cffff640 1 -- 172.21.15.3:0/3015458863 learned_addr learned my addr 172.21.15.3:0/3015458863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:54.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10ceffd640 1 -- 172.21.15.3:0/3015458863 >> v2:172.21.15.142:3300/0 conn(0x7f10d015b620 msgr2=0x7f10d015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:54.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10ceffd640 1 --2- 172.21.15.3:0/3015458863 >> v2:172.21.15.142:3300/0 conn(0x7f10d015b620 0x7f10d015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10ceffd640 1 -- 172.21.15.3:0/3015458863 >> v1:172.21.15.142:6789/0 conn(0x7f10d00d05b0 legacy=0x7f10d00d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10ceffd640 1 -- 172.21.15.3:0/3015458863 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d00ad960 con 0x7f10d00ae250 2024-09-12T12:23:54.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.848+0000 7f10cffff640 1 --2- 172.21.15.3:0/3015458863 >> v2:172.21.15.142:3300/0 conn(0x7f10d015b620 0x7f10d015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:54.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10ceffd640 1 --2- 172.21.15.3:0/3015458863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d00ae6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097c0 tx=0x7f10c401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a44837f4719b4f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:54.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10ce7fc640 1 -- 172.21.15.3:0/3015458863 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c40aa5f0 con 0x7f10d00ae250 2024-09-12T12:23:54.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10ce7fc640 1 -- 172.21.15.3:0/3015458863 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c40af070 con 0x7f10d00ae250 2024-09-12T12:23:54.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10ce7fc640 1 -- 172.21.15.3:0/3015458863 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c40aa9b0 con 0x7f10d00ae250 2024-09-12T12:23:54.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 -- 172.21.15.3:0/3015458863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 msgr2=0x7f10d00ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 --2- 172.21.15.3:0/3015458863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d00ae6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097c0 tx=0x7f10c401c470 comp rx=0 tx=0).stop 2024-09-12T12:23:54.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 -- 172.21.15.3:0/3015458863 shutdown_connections 2024-09-12T12:23:54.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 --2- 172.21.15.3:0/3015458863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d00ae6b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 --2- 172.21.15.3:0/3015458863 >> v2:172.21.15.142:3300/0 conn(0x7f10d015b620 0x7f10d015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 -- 172.21.15.3:0/3015458863 >> 172.21.15.3:0/3015458863 conn(0x7f10d0067e00 msgr2=0x7f10d0068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:54.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 -- 172.21.15.3:0/3015458863 shutdown_connections 2024-09-12T12:23:54.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 -- 172.21.15.3:0/3015458863 wait complete. 2024-09-12T12:23:54.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 Processor -- start 2024-09-12T12:23:54.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.852+0000 7f10d5f22640 1 -- start start 2024-09-12T12:23:54.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d0145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10d00d05b0 0x7f10d0072a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d015b620 0x7f10d0072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10d015c9b0 con 0x7f10d00d05b0 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10d015c830 con 0x7f10d015b620 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f10d00b31e0 con 0x7f10d00ae250 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10d00d05b0 0x7f10d0072a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10d00d05b0 0x7f10d0072a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46744/0 (socket says 172.21.15.3:46744) 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 -- 172.21.15.3:0/2500764154 learned_addr learned my addr 172.21.15.3:0/2500764154 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10cf7fe640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d0145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10cffff640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d015b620 0x7f10d0072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 -- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 msgr2=0x7f10d0145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d0145600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 -- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d015b620 msgr2=0x7f10d0072f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d015b620 0x7f10d0072f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 -- 172.21.15.3:0/2500764154 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d0152c50 con 0x7f10d00d05b0 2024-09-12T12:23:54.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10ceffd640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10d00d05b0 0x7f10d0072a40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f10c401cd10 tx=0x7f10c401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10b7fff640 1 -- 172.21.15.3:0/2500764154 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4006e20 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10b7fff640 1 -- 172.21.15.3:0/2500764154 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c40be610 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10b7fff640 1 -- 172.21.15.3:0/2500764154 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c40aae60 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d0151bb0 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d0152170 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d00ae6b0 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10b7fff640 1 -- 172.21.15.3:0/2500764154 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f10c40bc020 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10b7fff640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10bc0678c0 0x7f10bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10b7fff640 1 -- 172.21.15.3:0/2500764154 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7f10c4120000 con 0x7f10d00d05b0 2024-09-12T12:23:54.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.856+0000 7f10cf7fe640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10bc0678c0 0x7f10bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:54.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.860+0000 7f10cf7fe640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10bc0678c0 0x7f10bc069d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10c0000f70 tx=0x7f10c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:54.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:54.860+0000 7f10b7fff640 1 -- 172.21.15.3:0/2500764154 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f10c4016090 con 0x7f10d00d05b0 2024-09-12T12:23:55.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.020+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f10d006dd60 con 0x7f10d00d05b0 2024-09-12T12:23:55.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.020+0000 7f10b7fff640 1 -- 172.21.15.3:0/2500764154 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f10c40f0f20 con 0x7f10d00d05b0 2024-09-12T12:23:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:55.025 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T12:23:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:23:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T12:23:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T12:23:55.026 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:23:55.026 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T12:23:55.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10bc0678c0 msgr2=0x7f10bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:55.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10bc0678c0 0x7f10bc069d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10c0000f70 tx=0x7f10c0009040 comp rx=0 tx=0).stop 2024-09-12T12:23:55.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10d00d05b0 msgr2=0x7f10d0072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:55.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10d00d05b0 0x7f10d0072a40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f10c401cd10 tx=0x7f10c401acb0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 shutdown_connections 2024-09-12T12:23:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10bc0678c0 0x7f10bc069d80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d015b620 0x7f10d0072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10d00d05b0 0x7f10d0072a40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 --2- 172.21.15.3:0/2500764154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d00ae250 0x7f10d0145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 >> 172.21.15.3:0/2500764154 conn(0x7f10d0067e00 msgr2=0x7f10d00b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 shutdown_connections 2024-09-12T12:23:55.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.024+0000 7f10d5f22640 1 -- 172.21.15.3:0/2500764154 wait complete. 2024-09-12T12:23:55.043 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:23:55.043 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:23:55.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1358ae1640 1 Processor -- start 2024-09-12T12:23:55.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1358ae1640 1 -- start start 2024-09-12T12:23:55.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1358ae1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f13540d03a0 0x7f13540d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1358ae1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540724c0 0x7f1354072940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1358ae1640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13540d0cc0 con 0x7f13540b31c0 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1358ae1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f135415d720 con 0x7f13540d03a0 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1358ae1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f135415ce90 con 0x7f13540724c0 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f135259c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f13540d03a0 0x7f13540d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1351d9b640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f13540b31c0 0x7f135415c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:49094/0 (socket says 172.21.15.3:49094) 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1351d9b640 1 -- 172.21.15.3:0/2579680798 learned_addr learned my addr 172.21.15.3:0/2579680798 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.156+0000 7f1352d9d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540724c0 0x7f1354072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:55.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135259c640 1 -- 172.21.15.3:0/2579680798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540724c0 msgr2=0x7f1354072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:55.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135259c640 1 --2- 172.21.15.3:0/2579680798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540724c0 0x7f1354072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135259c640 1 -- 172.21.15.3:0/2579680798 >> v1:172.21.15.142:6789/0 conn(0x7f13540b31c0 legacy=0x7f135415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:55.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135259c640 1 -- 172.21.15.3:0/2579680798 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135415d160 con 0x7f13540d03a0 2024-09-12T12:23:55.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135259c640 1 --2- 172.21.15.3:0/2579680798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540d03a0 0x7f13540d0780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f134c009d90 tx=0x7f134c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fd7d2c70544fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:55.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135159a640 1 -- 172.21.15.3:0/2579680798 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0b2070 con 0x7f13540d03a0 2024-09-12T12:23:55.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135159a640 1 -- 172.21.15.3:0/2579680798 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134c0a5970 con 0x7f13540d03a0 2024-09-12T12:23:55.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f135159a640 1 -- 172.21.15.3:0/2579680798 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0a5d40 con 0x7f13540d03a0 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 -- 172.21.15.3:0/2579680798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540d03a0 msgr2=0x7f13540d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 --2- 172.21.15.3:0/2579680798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540d03a0 0x7f13540d0780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f134c009d90 tx=0x7f134c01c8b0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 -- 172.21.15.3:0/2579680798 shutdown_connections 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 --2- 172.21.15.3:0/2579680798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540724c0 0x7f1354072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 --2- 172.21.15.3:0/2579680798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540d03a0 0x7f13540d0780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 -- 172.21.15.3:0/2579680798 >> 172.21.15.3:0/2579680798 conn(0x7f1354067bd0 msgr2=0x7f1354067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 -- 172.21.15.3:0/2579680798 shutdown_connections 2024-09-12T12:23:55.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.160+0000 7f1358ae1640 1 -- 172.21.15.3:0/2579680798 wait complete. 2024-09-12T12:23:55.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 Processor -- start 2024-09-12T12:23:55.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 -- start start 2024-09-12T12:23:55.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540724c0 0x7f135414f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:55.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540b31c0 0x7f135406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:55.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13540d03a0 0x7f135406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f135415a180 con 0x7f13540d03a0 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f135415a000 con 0x7f13540724c0 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1354168dc0 con 0x7f13540b31c0 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13540d03a0 0x7f135406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13540d03a0 0x7f135406e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46758/0 (socket says 172.21.15.3:46758) 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 -- 172.21.15.3:0/3644148391 learned_addr learned my addr 172.21.15.3:0/3644148391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f135259c640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540724c0 0x7f135414f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1351d9b640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540b31c0 0x7f135406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 -- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540b31c0 msgr2=0x7f135406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540b31c0 0x7f135406e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 -- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540724c0 msgr2=0x7f135414f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:55.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540724c0 0x7f135414f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 -- 172.21.15.3:0/3644148391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1354153f60 con 0x7f13540d03a0 2024-09-12T12:23:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1352d9d640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13540d03a0 0x7f135406e6a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f134400e820 tx=0x7f134400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f134affd640 1 -- 172.21.15.3:0/3644148391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344017070 con 0x7f13540d03a0 2024-09-12T12:23:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f134affd640 1 -- 172.21.15.3:0/3644148391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134400c940 con 0x7f13540d03a0 2024-09-12T12:23:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1354154240 con 0x7f13540d03a0 2024-09-12T12:23:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f134affd640 1 -- 172.21.15.3:0/3644148391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134400cd10 con 0x7f13540d03a0 2024-09-12T12:23:55.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13540cdb10 con 0x7f13540d03a0 2024-09-12T12:23:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13540cdd10 con 0x7f13540d03a0 2024-09-12T12:23:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.164+0000 7f134affd640 1 -- 172.21.15.3:0/3644148391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f134401a070 con 0x7f13540d03a0 2024-09-12T12:23:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.168+0000 7f134affd640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13380678c0 0x7f1338069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.168+0000 7f134affd640 1 -- 172.21.15.3:0/3644148391 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 4857+0+0 (secure 0 0 0) 0x7f1344085bb0 con 0x7f13540d03a0 2024-09-12T12:23:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.168+0000 7f135259c640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13380678c0 0x7f1338069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:55.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.168+0000 7f135259c640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13380678c0 0x7f1338069d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f134c0af000 tx=0x7f134c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:55.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.172+0000 7f134affd640 1 -- 172.21.15.3:0/3644148391 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f13540cdd10 con 0x7f13540d03a0 2024-09-12T12:23:55.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:55.332+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1354151d20 con 0x7f13540d03a0 2024-09-12T12:23:56.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.332+0000 7f134affd640 1 -- 172.21.15.3:0/3644148391 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f1344056360 con 0x7f13540d03a0 2024-09-12T12:23:56.335 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T12:23:56.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13380678c0 msgr2=0x7f1338069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:56.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13380678c0 0x7f1338069d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f134c0af000 tx=0x7f134c0a7070 comp rx=0 tx=0).stop 2024-09-12T12:23:56.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13540d03a0 msgr2=0x7f135406e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:56.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13540d03a0 0x7f135406e6a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f134400e820 tx=0x7f134400ece0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 shutdown_connections 2024-09-12T12:23:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13380678c0 0x7f1338069d80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13540d03a0 0x7f135406e6a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13540b31c0 0x7f135406e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 --2- 172.21.15.3:0/3644148391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f13540724c0 0x7f135414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 >> 172.21.15.3:0/3644148391 conn(0x7f1354067bd0 msgr2=0x7f13540b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 shutdown_connections 2024-09-12T12:23:56.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.336+0000 7f1358ae1640 1 -- 172.21.15.3:0/3644148391 wait complete. 2024-09-12T12:23:56.366 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:23:56.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.496+0000 7feb75ffc640 1 Processor -- start 2024-09-12T12:23:56.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.496+0000 7feb75ffc640 1 -- start start 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb75ffc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb7006e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb75ffc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb700adf60 0x7feb7015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb75ffc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb70160600 con 0x7feb7015c210 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb75ffc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7feb701608e0 con 0x7feb700adf60 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb75ffc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb70160bc0 con 0x7feb700d0cd0 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb700adf60 0x7feb7015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feb7015c210 0x7feb700d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56718/0 (socket says 172.21.15.3:56718) 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7feb700adf60 0x7feb7015feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46766/0 (socket says 172.21.15.3:46766) 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6f7fe640 1 -- 172.21.15.3:0/3211359765 learned_addr learned my addr 172.21.15.3:0/3211359765 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:56.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6effd640 1 --2- 172.21.15.3:0/3211359765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb7006e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6ffff640 1 -- 172.21.15.3:0/3211359765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 msgr2=0x7feb7006e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6ffff640 1 --2- 172.21.15.3:0/3211359765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb7006e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6ffff640 1 -- 172.21.15.3:0/3211359765 >> v1:172.21.15.3:6789/0 conn(0x7feb7015c210 legacy=0x7feb700d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6e7fc640 1 -- 172.21.15.3:0/3211359765 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 831276637 0 0) 0x7feb70160600 con 0x7feb7015c210 2024-09-12T12:23:56.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6ffff640 1 -- 172.21.15.3:0/3211359765 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb700cec00 con 0x7feb700adf60 2024-09-12T12:23:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6ffff640 1 --2- 172.21.15.3:0/3211359765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb700adf60 0x7feb7015feb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7feb680097c0 tx=0x7feb6801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bf91761cd1c0cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.500+0000 7feb6e7fc640 1 -- 172.21.15.3:0/3211359765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680b0070 con 0x7feb700adf60 2024-09-12T12:23:56.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6e7fc640 1 -- 172.21.15.3:0/3211359765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb680ab600 con 0x7feb700adf60 2024-09-12T12:23:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6e7fc640 1 -- 172.21.15.3:0/3211359765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680ab9d0 con 0x7feb700adf60 2024-09-12T12:23:56.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- 172.21.15.3:0/3211359765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb700adf60 msgr2=0x7feb7015feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 --2- 172.21.15.3:0/3211359765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb700adf60 0x7feb7015feb0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7feb680097c0 tx=0x7feb6801c380 comp rx=0 tx=0).stop 2024-09-12T12:23:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- 172.21.15.3:0/3211359765 shutdown_connections 2024-09-12T12:23:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 --2- 172.21.15.3:0/3211359765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb700adf60 0x7feb7015feb0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 --2- 172.21.15.3:0/3211359765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb7006e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- 172.21.15.3:0/3211359765 >> 172.21.15.3:0/3211359765 conn(0x7feb70067e00 msgr2=0x7feb70068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- 172.21.15.3:0/3211359765 shutdown_connections 2024-09-12T12:23:56.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- 172.21.15.3:0/3211359765 wait complete. 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 Processor -- start 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- start start 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb700adf60 0x7feb701585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb70149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb7015c210 0x7feb70149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb700cfcc0 con 0x7feb7015c210 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb700cfb40 con 0x7feb700adf60 2024-09-12T12:23:56.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb75ffc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7feb700cfe40 con 0x7feb700d0cd0 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb7015c210 0x7feb70149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb7015c210 0x7feb70149790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46774/0 (socket says 172.21.15.3:46774) 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 -- 172.21.15.3:0/1396240659 learned_addr learned my addr 172.21.15.3:0/1396240659 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6effd640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb70149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6f7fe640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb700adf60 0x7feb701585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 -- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 msgr2=0x7feb70149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb70149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 -- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb700adf60 msgr2=0x7feb701585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb700adf60 0x7feb701585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 -- 172.21.15.3:0/1396240659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb701557e0 con 0x7feb7015c210 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6f7fe640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb700adf60 0x7feb701585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.504+0000 7feb6ffff640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb7015c210 0x7feb70149790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7feb680ae040 tx=0x7feb68019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb67fff640 1 -- 172.21.15.3:0/1396240659 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680b8020 con 0x7feb7015c210 2024-09-12T12:23:56.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb67fff640 1 -- 172.21.15.3:0/1396240659 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb680b4610 con 0x7feb7015c210 2024-09-12T12:23:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb70155ac0 con 0x7feb7015c210 2024-09-12T12:23:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb67fff640 1 -- 172.21.15.3:0/1396240659 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680b0040 con 0x7feb7015c210 2024-09-12T12:23:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7014d730 con 0x7feb7015c210 2024-09-12T12:23:56.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7014d520 con 0x7feb7015c210 2024-09-12T12:23:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb67fff640 1 -- 172.21.15.3:0/1396240659 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feb680bb020 con 0x7feb7015c210 2024-09-12T12:23:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb67fff640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb500678c0 0x7feb50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb67fff640 1 -- 172.21.15.3:0/1396240659 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5227+0+0 (secure 0 0 0) 0x7feb6811fa90 con 0x7feb7015c210 2024-09-12T12:23:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb6f7fe640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb500678c0 0x7feb50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:56.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.508+0000 7feb6f7fe640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb500678c0 0x7feb50069d80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feb60000f70 tx=0x7feb60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:56.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.512+0000 7feb67fff640 1 -- 172.21.15.3:0/1396240659 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7feb680c2030 con 0x7feb7015c210 2024-09-12T12:23:56.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:56.676+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7feb7006e450 con 0x7feb7015c210 2024-09-12T12:23:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb67fff640 1 -- 172.21.15.3:0/1396240659 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7feb68016090 con 0x7feb7015c210 2024-09-12T12:23:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb500678c0 msgr2=0x7feb50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb500678c0 0x7feb50069d80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feb60000f70 tx=0x7feb60009040 comp rx=0 tx=0).stop 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb7015c210 msgr2=0x7feb70149790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb7015c210 0x7feb70149790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7feb680ae040 tx=0x7feb68019120 comp rx=0 tx=0).stop 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 shutdown_connections 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7feb500678c0 0x7feb50069d80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb7015c210 0x7feb70149790 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feb700d0cd0 0x7feb70149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 --2- 172.21.15.3:0/1396240659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb700adf60 0x7feb701585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 >> 172.21.15.3:0/1396240659 conn(0x7feb70067e00 msgr2=0x7feb70069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.344+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 shutdown_connections 2024-09-12T12:23:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.348+0000 7feb75ffc640 1 -- 172.21.15.3:0/1396240659 wait complete. 2024-09-12T12:23:57.386 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:23:57.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.508+0000 7f658a0d2640 1 Processor -- start 2024-09-12T12:23:57.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.508+0000 7f658a0d2640 1 -- start start 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f658a0d2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f658415c210 0x7f658406e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f658a0d2640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6584160600 con 0x7f65840d0cd0 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f658a0d2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65841608e0 con 0x7f65840adf60 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f658a0d2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6584160bc0 con 0x7f658415c210 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f6583fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f65840d0cd0 0x7f65840cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:56726/0 (socket says 172.21.15.3:56726) 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f6583fff640 1 -- 172.21.15.3:0/416909350 learned_addr learned my addr 172.21.15.3:0/416909350 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65837fe640 1 --2- 172.21.15.3:0/416909350 >> v2:172.21.15.3:3300/0 conn(0x7f658415c210 0x7f658406e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:57.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65827fc640 1 -- 172.21.15.3:0/416909350 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024027526 0 0) 0x7f6584160600 con 0x7f65840d0cd0 2024-09-12T12:23:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65827fc640 1 -- 172.21.15.3:0/416909350 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6570003530 con 0x7f65840d0cd0 2024-09-12T12:23:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65837fe640 1 -- 172.21.15.3:0/416909350 >> v1:172.21.15.142:6789/0 conn(0x7f65840adf60 legacy=0x7f65840d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65837fe640 1 -- 172.21.15.3:0/416909350 >> v1:172.21.15.3:6789/0 conn(0x7f65840d0cd0 legacy=0x7f65840cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:57.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65837fe640 1 -- 172.21.15.3:0/416909350 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65840cec00 con 0x7f658415c210 2024-09-12T12:23:57.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65837fe640 1 --2- 172.21.15.3:0/416909350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658415c210 0x7f658406e450 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097c0 tx=0x7f656c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4ee1ed606a9a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:57.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.512+0000 7f65827fc640 1 -- 172.21.15.3:0/416909350 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c0b0070 con 0x7f658415c210 2024-09-12T12:23:57.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f65827fc640 1 -- 172.21.15.3:0/416909350 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c0ab600 con 0x7f658415c210 2024-09-12T12:23:57.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f65827fc640 1 -- 172.21.15.3:0/416909350 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c0ab9d0 con 0x7f658415c210 2024-09-12T12:23:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- 172.21.15.3:0/416909350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658415c210 msgr2=0x7f658406e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 --2- 172.21.15.3:0/416909350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658415c210 0x7f658406e450 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f656c0097c0 tx=0x7f656c01c3d0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- 172.21.15.3:0/416909350 shutdown_connections 2024-09-12T12:23:57.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 --2- 172.21.15.3:0/416909350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f658415c210 0x7f658406e450 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- 172.21.15.3:0/416909350 >> 172.21.15.3:0/416909350 conn(0x7f6584067e00 msgr2=0x7f6584068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- 172.21.15.3:0/416909350 shutdown_connections 2024-09-12T12:23:57.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- 172.21.15.3:0/416909350 wait complete. 2024-09-12T12:23:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 Processor -- start 2024-09-12T12:23:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- start start 2024-09-12T12:23:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65840adf60 0x7f658414ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:57.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65840d0cd0 0x7f65841505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658415c210 0x7f65840b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f658416a5f0 con 0x7f65840d0cd0 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f658416a470 con 0x7f65840adf60 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f658416a770 con 0x7f658415c210 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65840d0cd0 0x7f65841505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65840d0cd0 0x7f65841505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46784/0 (socket says 172.21.15.3:46784) 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 -- 172.21.15.3:0/3052032392 learned_addr learned my addr 172.21.15.3:0/3052032392 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:57.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f65837fe640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65840adf60 0x7f658414ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6583fff640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658415c210 0x7f65840b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 -- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658415c210 msgr2=0x7f65840b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658415c210 0x7f65840b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 -- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65840adf60 msgr2=0x7f658414ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65840adf60 0x7f658414ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 -- 172.21.15.3:0/3052032392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6584153f60 con 0x7f65840d0cd0 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f65837fe640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65840adf60 0x7f658414ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f6582ffd640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65840d0cd0 0x7f65841505c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f657400c7f0 tx=0x7f657400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f657bfff640 1 -- 172.21.15.3:0/3052032392 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574016020 con 0x7f65840d0cd0 2024-09-12T12:23:57.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f657bfff640 1 -- 172.21.15.3:0/3052032392 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657400aa50 con 0x7f65840d0cd0 2024-09-12T12:23:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65841531a0 con 0x7f65840d0cd0 2024-09-12T12:23:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f657bfff640 1 -- 172.21.15.3:0/3052032392 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657400ae40 con 0x7f65840d0cd0 2024-09-12T12:23:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.516+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6584153760 con 0x7f65840d0cd0 2024-09-12T12:23:57.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.520+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658406e450 con 0x7f65840d0cd0 2024-09-12T12:23:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.520+0000 7f657bfff640 1 -- 172.21.15.3:0/3052032392 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6574019020 con 0x7f65840d0cd0 2024-09-12T12:23:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.520+0000 7f657bfff640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65640678f0 0x7f6564069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.520+0000 7f657bfff640 1 -- 172.21.15.3:0/3052032392 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5609+0+0 (secure 0 0 0) 0x7f6574086160 con 0x7f65840d0cd0 2024-09-12T12:23:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.520+0000 7f65837fe640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65640678f0 0x7f6564069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:57.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.520+0000 7f65837fe640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65640678f0 0x7f6564069db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f656c0ad000 tx=0x7f656c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:57.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.524+0000 7f657bfff640 1 -- 172.21.15.3:0/3052032392 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6574020030 con 0x7f65840d0cd0 2024-09-12T12:23:57.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:57.724+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f65840ce840 con 0x7f65840d0cd0 2024-09-12T12:23:58.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.376+0000 7f657bfff640 1 -- 172.21.15.3:0/3052032392 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v50) ==== 174+0+0 (secure 0 0 0) 0x7f6574056ce0 con 0x7f65840d0cd0 2024-09-12T12:23:58.383 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 12 and data pool 13 2024-09-12T12:23:58.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65640678f0 msgr2=0x7f6564069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65640678f0 0x7f6564069db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f656c0ad000 tx=0x7f656c0a6000 comp rx=0 tx=0).stop 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65840d0cd0 msgr2=0x7f65841505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65840d0cd0 0x7f65841505c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f657400c7f0 tx=0x7f657400ccb0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 shutdown_connections 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65640678f0 0x7f6564069db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f658415c210 0x7f65840b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65840d0cd0 0x7f65841505c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 --2- 172.21.15.3:0/3052032392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65840adf60 0x7f658414ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 >> 172.21.15.3:0/3052032392 conn(0x7f6584067e00 msgr2=0x7f658406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 shutdown_connections 2024-09-12T12:23:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.388+0000 7f658a0d2640 1 -- 172.21.15.3:0/3052032392 wait complete. 2024-09-12T12:23:58.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:58.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.536+0000 7fefcb785640 1 Processor -- start 2024-09-12T12:23:58.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefcb785640 1 -- start start 2024-09-12T12:23:58.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefcb785640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefc40af410 0x7fefc415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefcb785640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fefc415ef90 0x7fefc40d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefcb785640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc415f430 con 0x7fefc415c000 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefcb785640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fefc415f9f0 con 0x7fefc40af410 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefcb785640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fefc415f710 con 0x7fefc415ef90 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefc40af410 0x7fefc415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fefc40af410 0x7fefc415ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60470/0 (socket says 172.21.15.3:60470) 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 -- 172.21.15.3:0/2724564893 learned_addr learned my addr 172.21.15.3:0/2724564893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc9d22640 1 --2- 172.21.15.3:0/2724564893 >> v2:172.21.15.142:3300/0 conn(0x7fefc415ef90 0x7fefc40d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 -- 172.21.15.3:0/2724564893 >> v2:172.21.15.142:3300/0 conn(0x7fefc415ef90 msgr2=0x7fefc40d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 --2- 172.21.15.3:0/2724564893 >> v2:172.21.15.142:3300/0 conn(0x7fefc415ef90 0x7fefc40d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 -- 172.21.15.3:0/2724564893 >> v1:172.21.15.142:6789/0 conn(0x7fefc415c000 legacy=0x7fefc40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:58.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 -- 172.21.15.3:0/2724564893 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40b3210 con 0x7fefc40af410 2024-09-12T12:23:58.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefc8d20640 1 --2- 172.21.15.3:0/2724564893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc415ea50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009870 tx=0x7fefb401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c052465931cb367d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:58.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.540+0000 7fefbbfff640 1 -- 172.21.15.3:0/2724564893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb40b0070 con 0x7fefc40af410 2024-09-12T12:23:58.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefbbfff640 1 -- 172.21.15.3:0/2724564893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb40ab600 con 0x7fefc40af410 2024-09-12T12:23:58.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefbbfff640 1 -- 172.21.15.3:0/2724564893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb40ab9d0 con 0x7fefc40af410 2024-09-12T12:23:58.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- 172.21.15.3:0/2724564893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 msgr2=0x7fefc415ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 --2- 172.21.15.3:0/2724564893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc415ea50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009870 tx=0x7fefb401c420 comp rx=0 tx=0).stop 2024-09-12T12:23:58.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- 172.21.15.3:0/2724564893 shutdown_connections 2024-09-12T12:23:58.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 --2- 172.21.15.3:0/2724564893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc415ea50 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 --2- 172.21.15.3:0/2724564893 >> v2:172.21.15.142:3300/0 conn(0x7fefc415ef90 0x7fefc40d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- 172.21.15.3:0/2724564893 >> 172.21.15.3:0/2724564893 conn(0x7fefc4067e00 msgr2=0x7fefc4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- 172.21.15.3:0/2724564893 shutdown_connections 2024-09-12T12:23:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- 172.21.15.3:0/2724564893 wait complete. 2024-09-12T12:23:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 Processor -- start 2024-09-12T12:23:58.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- start start 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc40ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefc415c000 0x7fefc40af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc415ef90 0x7fefc40afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fefc406dc00 con 0x7fefc40af410 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fefc416b090 con 0x7fefc415ef90 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fefc41569b0 con 0x7fefc415c000 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc40ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc40ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60480/0 (socket says 172.21.15.3:60480) 2024-09-12T12:23:58.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 -- 172.21.15.3:0/1278806586 learned_addr learned my addr 172.21.15.3:0/1278806586 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9d22640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc415ef90 0x7fefc40afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc8d20640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefc415c000 0x7fefc40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 -- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefc415c000 msgr2=0x7fefc40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefc415c000 0x7fefc40af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 -- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc415ef90 msgr2=0x7fefc40afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc415ef90 0x7fefc40afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 -- 172.21.15.3:0/1278806586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc40b19a0 con 0x7fefc40af410 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefc9521640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc40ce9b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c490 tx=0x7fefc0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefb9ffb640 1 -- 172.21.15.3:0/1278806586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0017070 con 0x7fefc40af410 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefb9ffb640 1 -- 172.21.15.3:0/1278806586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc0003600 con 0x7fefc40af410 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc40b1c80 con 0x7fefc40af410 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefb9ffb640 1 -- 172.21.15.3:0/1278806586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0003a30 con 0x7fefc40af410 2024-09-12T12:23:58.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.544+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc40b2240 con 0x7fefc40af410 2024-09-12T12:23:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.548+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc415ea50 con 0x7fefc40af410 2024-09-12T12:23:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.548+0000 7fefb9ffb640 1 -- 172.21.15.3:0/1278806586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fefc001a070 con 0x7fefc40af410 2024-09-12T12:23:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.548+0000 7fefb9ffb640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fefa80678f0 0x7fefa8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.548+0000 7fefb9ffb640 1 -- 172.21.15.3:0/1278806586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fefc00854d0 con 0x7fefc40af410 2024-09-12T12:23:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.548+0000 7fefc8d20640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fefa80678f0 0x7fefa8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.548+0000 7fefc8d20640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fefa80678f0 0x7fefa8069db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fefb40ad000 tx=0x7fefb40a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:58.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.552+0000 7fefb9ffb640 1 -- 172.21.15.3:0/1278806586 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fefc0021020 con 0x7fefc40af410 2024-09-12T12:23:58.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.716+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefc4068280 con 0x7fefc40af410 2024-09-12T12:23:58.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.720+0000 7fefb9ffb640 1 -- 172.21.15.3:0/1278806586 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16673 (secure 0 0 0) 0x7fefc0056060 con 0x7fefc40af410 2024-09-12T12:23:58.724 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:58.724 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":78,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:57.729490+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:55.370943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:56.721996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:58.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fefa80678f0 msgr2=0x7fefa8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fefa80678f0 0x7fefa8069db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fefb40ad000 tx=0x7fefb40a6000 comp rx=0 tx=0).stop 2024-09-12T12:23:58.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 msgr2=0x7fefc40ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc40ce9b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fefc000c490 tx=0x7fefc0001e80 comp rx=0 tx=0).stop 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 shutdown_connections 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fefa80678f0 0x7fefa8069db0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc415ef90 0x7fefc40afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fefc415c000 0x7fefc40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 --2- 172.21.15.3:0/1278806586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefc40af410 0x7fefc40ce9b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 >> 172.21.15.3:0/1278806586 conn(0x7fefc4067e00 msgr2=0x7fefc415b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 shutdown_connections 2024-09-12T12:23:58.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.724+0000 7fefcb785640 1 -- 172.21.15.3:0/1278806586 wait complete. 2024-09-12T12:23:58.744 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:23:58.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.856+0000 7f00aad80640 1 Processor -- start 2024-09-12T12:23:58.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.856+0000 7f00aad80640 1 -- start start 2024-09-12T12:23:58.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00aad80640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00a40b30e0 0x7f00a40d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00aad80640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a40b5000 con 0x7f00a4073cb0 2024-09-12T12:23:58.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00aad80640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a40b3510 con 0x7f00a415c710 2024-09-12T12:23:58.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00aad80640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f00a4074090 con 0x7f00a40b30e0 2024-09-12T12:23:58.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a8b1c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f00a4073cb0 0x7f00a40adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38764/0 (socket says 172.21.15.3:38764) 2024-09-12T12:23:58.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a931d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00a40b30e0 0x7f00a40d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a8b1c640 1 -- 172.21.15.3:0/3784028266 learned_addr learned my addr 172.21.15.3:0/3784028266 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:23:58.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a931d640 1 -- 172.21.15.3:0/3784028266 >> v1:172.21.15.142:6789/0 conn(0x7f00a415c710 legacy=0x7f00a40ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:58.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a931d640 1 -- 172.21.15.3:0/3784028266 >> v1:172.21.15.3:6789/0 conn(0x7f00a4073cb0 legacy=0x7f00a40adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a37fe640 1 -- 172.21.15.3:0/3784028266 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560201928 0 0) 0x7f00a40b5000 con 0x7f00a4073cb0 2024-09-12T12:23:58.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a931d640 1 -- 172.21.15.3:0/3784028266 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a40b37c0 con 0x7f00a40b30e0 2024-09-12T12:23:58.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.860+0000 7f00a931d640 1 --2- 172.21.15.3:0/3784028266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a40b30e0 0x7f00a40d05b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f00940097c0 tx=0x7f009401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d92f1095dd23b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:58.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a37fe640 1 -- 172.21.15.3:0/3784028266 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940b0070 con 0x7f00a40b30e0 2024-09-12T12:23:58.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a37fe640 1 -- 172.21.15.3:0/3784028266 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00940ab600 con 0x7f00a40b30e0 2024-09-12T12:23:58.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a37fe640 1 -- 172.21.15.3:0/3784028266 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00940ab9d0 con 0x7f00a40b30e0 2024-09-12T12:23:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- 172.21.15.3:0/3784028266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a40b30e0 msgr2=0x7f00a40d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 --2- 172.21.15.3:0/3784028266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a40b30e0 0x7f00a40d05b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f00940097c0 tx=0x7f009401c4a0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- 172.21.15.3:0/3784028266 shutdown_connections 2024-09-12T12:23:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 --2- 172.21.15.3:0/3784028266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a40b30e0 0x7f00a40d05b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- 172.21.15.3:0/3784028266 >> 172.21.15.3:0/3784028266 conn(0x7f00a4067e00 msgr2=0x7f00a4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- 172.21.15.3:0/3784028266 shutdown_connections 2024-09-12T12:23:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- 172.21.15.3:0/3784028266 wait complete. 2024-09-12T12:23:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 Processor -- start 2024-09-12T12:23:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- start start 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a4073cb0 0x7f00a41572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00a40b30e0 0x7f00a40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00a415c710 0x7f00a4147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40ae320 con 0x7f00a4073cb0 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f00a40d0f00 con 0x7f00a40b30e0 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f00a40ae4a0 con 0x7f00a415c710 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a4073cb0 0x7f00a41572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a4073cb0 0x7f00a41572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60504/0 (socket says 172.21.15.3:60504) 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 -- 172.21.15.3:0/3382012862 learned_addr learned my addr 172.21.15.3:0/3382012862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 -- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00a415c710 msgr2=0x7f00a4147f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00a415c710 0x7f00a4147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 -- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00a40b30e0 msgr2=0x7f00a40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a3fff640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00a40b30e0 0x7f00a40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00a40b30e0 0x7f00a40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 -- 172.21.15.3:0/3382012862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a40cfb10 con 0x7f00a4073cb0 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a3fff640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00a40b30e0 0x7f00a40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a8b1c640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a4073cb0 0x7f00a41572e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c8c0 tx=0x7f009c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a17fa640 1 -- 172.21.15.3:0/3382012862 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c018070 con 0x7f00a4073cb0 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a17fa640 1 -- 172.21.15.3:0/3382012862 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f009c00a640 con 0x7f00a4073cb0 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a40cdb10 con 0x7f00a4073cb0 2024-09-12T12:23:58.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00a17fa640 1 -- 172.21.15.3:0/3382012862 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c00aa70 con 0x7f00a4073cb0 2024-09-12T12:23:58.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.864+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a40ce0d0 con 0x7f00a4073cb0 2024-09-12T12:23:58.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.868+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a40d05b0 con 0x7f00a4073cb0 2024-09-12T12:23:58.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.868+0000 7f00a17fa640 1 -- 172.21.15.3:0/3382012862 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f009c01b070 con 0x7f00a4073cb0 2024-09-12T12:23:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.868+0000 7f00a17fa640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00840678c0 0x7f0084069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.868+0000 7f00a17fa640 1 -- 172.21.15.3:0/3382012862 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7f009c0853d0 con 0x7f00a4073cb0 2024-09-12T12:23:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.868+0000 7f00a3fff640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00840678c0 0x7f0084069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:58.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.868+0000 7f00a3fff640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00840678c0 0x7f0084069d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f00a4148af0 tx=0x7f008c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:58.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:58.872+0000 7f00a17fa640 1 -- 172.21.15.3:0/3382012862 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f00a40cfb10 con 0x7f00a4073cb0 2024-09-12T12:23:59.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.036+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00a4152b50 con 0x7f00a4073cb0 2024-09-12T12:23:59.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00a17fa640 1 -- 172.21.15.3:0/3382012862 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16673 (secure 0 0 0) 0x7f009c055e70 con 0x7f00a4073cb0 2024-09-12T12:23:59.043 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:23:59.043 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":78,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:57.729490+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:55.370943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:56.721996+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00840678c0 msgr2=0x7f0084069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00840678c0 0x7f0084069d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f00a4148af0 tx=0x7f008c009040 comp rx=0 tx=0).stop 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a4073cb0 msgr2=0x7f00a41572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a4073cb0 0x7f00a41572e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c8c0 tx=0x7f009c00cd80 comp rx=0 tx=0).stop 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 shutdown_connections 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00840678c0 0x7f0084069d80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00a415c710 0x7f00a4147f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00a40b30e0 0x7f00a40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 --2- 172.21.15.3:0/3382012862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00a4073cb0 0x7f00a41572e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 >> 172.21.15.3:0/3382012862 conn(0x7f00a4067e00 msgr2=0x7f00a415a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 shutdown_connections 2024-09-12T12:23:59.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.040+0000 7f00aad80640 1 -- 172.21.15.3:0/3382012862 wait complete. 2024-09-12T12:23:59.064 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:23:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd4a058640 1 Processor -- start 2024-09-12T12:23:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd4a058640 1 -- start start 2024-09-12T12:23:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd4a058640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd4415e750 0x7fcd440d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd4a058640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd440d40c0 0x7fcd440cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd4a058640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd440d10d0 con 0x7fcd440cf6b0 2024-09-12T12:23:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd4a058640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcd440d13b0 con 0x7fcd440d40c0 2024-09-12T12:23:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd4a058640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcd440d1690 con 0x7fcd4415e750 2024-09-12T12:23:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd3affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd440d40c0 0x7fcd440cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd3affd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcd440d40c0 0x7fcd440cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60508/0 (socket says 172.21.15.3:60508) 2024-09-12T12:23:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd3affd640 1 -- 172.21.15.3:0/1591372020 learned_addr learned my addr 172.21.15.3:0/1591372020 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd437fe640 1 --2- 172.21.15.3:0/1591372020 >> v2:172.21.15.142:3300/0 conn(0x7fcd4415e750 0x7fcd440d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd3affd640 1 -- 172.21.15.3:0/1591372020 >> v2:172.21.15.142:3300/0 conn(0x7fcd4415e750 msgr2=0x7fcd440d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd3affd640 1 --2- 172.21.15.3:0/1591372020 >> v2:172.21.15.142:3300/0 conn(0x7fcd4415e750 0x7fcd440d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd3affd640 1 -- 172.21.15.3:0/1591372020 >> v1:172.21.15.142:6789/0 conn(0x7fcd440cf6b0 legacy=0x7fcd440d2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.184+0000 7fcd3affd640 1 -- 172.21.15.3:0/1591372020 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd440d1970 con 0x7fcd440d40c0 2024-09-12T12:23:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd3affd640 1 --2- 172.21.15.3:0/1591372020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440d40c0 0x7fcd440cf170 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcd4408d1c0 tx=0x7fcd2c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210c2e1f491fd728 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:59.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd42ffd640 1 -- 172.21.15.3:0/1591372020 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0b5070 con 0x7fcd440d40c0 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd42ffd640 1 -- 172.21.15.3:0/1591372020 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0a89d0 con 0x7fcd440d40c0 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd42ffd640 1 -- 172.21.15.3:0/1591372020 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0a8da0 con 0x7fcd440d40c0 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- 172.21.15.3:0/1591372020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440d40c0 msgr2=0x7fcd440cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 --2- 172.21.15.3:0/1591372020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440d40c0 0x7fcd440cf170 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fcd4408d1c0 tx=0x7fcd2c01ecd0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- 172.21.15.3:0/1591372020 shutdown_connections 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 --2- 172.21.15.3:0/1591372020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440d40c0 0x7fcd440cf170 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 --2- 172.21.15.3:0/1591372020 >> v2:172.21.15.142:3300/0 conn(0x7fcd4415e750 0x7fcd440d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- 172.21.15.3:0/1591372020 >> 172.21.15.3:0/1591372020 conn(0x7fcd44067a00 msgr2=0x7fcd44067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- 172.21.15.3:0/1591372020 shutdown_connections 2024-09-12T12:23:59.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- 172.21.15.3:0/1591372020 wait complete. 2024-09-12T12:23:59.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 Processor -- start 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- start start 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440cf6b0 0x7fcd4415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd440d40c0 0x7fcd440ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd4415e750 0x7fcd440c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4415eb30 con 0x7fcd440cf6b0 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440b6e10 con 0x7fcd4415e750 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd4a058640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4415ecb0 con 0x7fcd440d40c0 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440cf6b0 0x7fcd4415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440cf6b0 0x7fcd4415d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60520/0 (socket says 172.21.15.3:60520) 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 -- 172.21.15.3:0/3420087434 learned_addr learned my addr 172.21.15.3:0/3420087434 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 -- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd440d40c0 msgr2=0x7fcd440ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:59.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd440d40c0 0x7fcd440ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 -- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd4415e750 msgr2=0x7fcd440c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:59.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd4415e750 0x7fcd440c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:59.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.188+0000 7fcd437fe640 1 -- 172.21.15.3:0/3420087434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4415d930 con 0x7fcd440cf6b0 2024-09-12T12:23:59.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd437fe640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440cf6b0 0x7fcd4415d3f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c820 tx=0x7fcd3400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:59.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd40ff9640 1 -- 172.21.15.3:0/3420087434 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34015070 con 0x7fcd440cf6b0 2024-09-12T12:23:59.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd40ff9640 1 -- 172.21.15.3:0/3420087434 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd34007a30 con 0x7fcd440cf6b0 2024-09-12T12:23:59.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd440cd810 con 0x7fcd440cf6b0 2024-09-12T12:23:59.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd40ff9640 1 -- 172.21.15.3:0/3420087434 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34007e20 con 0x7fcd440cf6b0 2024-09-12T12:23:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd440cddd0 con 0x7fcd440cf6b0 2024-09-12T12:23:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd440cf170 con 0x7fcd440cf6b0 2024-09-12T12:23:59.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd40ff9640 1 -- 172.21.15.3:0/3420087434 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd34018070 con 0x7fcd440cf6b0 2024-09-12T12:23:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd40ff9640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd240678c0 0x7fcd24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd40ff9640 1 -- 172.21.15.3:0/3420087434 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5709+0+0 (secure 0 0 0) 0x7fcd34085de0 con 0x7fcd440cf6b0 2024-09-12T12:23:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd3affd640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd240678c0 0x7fcd24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.192+0000 7fcd3affd640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd240678c0 0x7fcd24069d80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0b2000 tx=0x7fcd2c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:59.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.196+0000 7fcd40ff9640 1 -- 172.21.15.3:0/3420087434 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcd34056a90 con 0x7fcd440cf6b0 2024-09-12T12:23:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:23:59.392+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcd440b7f90 con 0x7fcd440cf6b0 2024-09-12T12:24:00.333 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-12T12:24:00.332+0000 7fceeb5c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:24:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd40ff9640 1 -- 172.21.15.3:0/3420087434 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) ==== 118+0+0 (secure 0 0 0) 0x7fcd34056390 con 0x7fcd440cf6b0 2024-09-12T12:24:00.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd240678c0 msgr2=0x7fcd24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:00.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd240678c0 0x7fcd24069d80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0b2000 tx=0x7fcd2c004740 comp rx=0 tx=0).stop 2024-09-12T12:24:00.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440cf6b0 msgr2=0x7fcd4415d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:00.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440cf6b0 0x7fcd4415d3f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c820 tx=0x7fcd3400cce0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 shutdown_connections 2024-09-12T12:24:00.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcd240678c0 0x7fcd24069d80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd4415e750 0x7fcd440c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd440d40c0 0x7fcd440ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 --2- 172.21.15.3:0/3420087434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd440cf6b0 0x7fcd4415d3f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 >> 172.21.15.3:0/3420087434 conn(0x7fcd44067a00 msgr2=0x7fcd440d44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 shutdown_connections 2024-09-12T12:24:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.384+0000 7fcd4a058640 1 -- 172.21.15.3:0/3420087434 wait complete. 2024-09-12T12:24:00.411 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T12:24:00.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T12:24:00.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a306a640 1 Processor -- start 2024-09-12T12:24:00.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a306a640 1 -- start start 2024-09-12T12:24:00.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a306a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f859c0d0cd0 0x7f859c06e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:00.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a306a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f859c160600 con 0x7f859c15c210 2024-09-12T12:24:00.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a306a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f859c1608e0 con 0x7f859c0adf60 2024-09-12T12:24:00.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a306a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f859c160bc0 con 0x7f859c0d0cd0 2024-09-12T12:24:00.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f859bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f859c0d0cd0 0x7f859c06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:00.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a0e06640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f859c15c210 0x7f859c0d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38778/0 (socket says 172.21.15.3:38778) 2024-09-12T12:24:00.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f85a0e06640 1 -- 172.21.15.3:0/2043405509 learned_addr learned my addr 172.21.15.3:0/2043405509 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:00.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f859bfff640 1 -- 172.21.15.3:0/2043405509 >> v1:172.21.15.142:6789/0 conn(0x7f859c0adf60 legacy=0x7f859c0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:00.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f859bfff640 1 -- 172.21.15.3:0/2043405509 >> v1:172.21.15.3:6789/0 conn(0x7f859c15c210 legacy=0x7f859c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:00.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.540+0000 7f859bfff640 1 -- 172.21.15.3:0/2043405509 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c0cec00 con 0x7f859c0d0cd0 2024-09-12T12:24:00.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f859bfff640 1 --2- 172.21.15.3:0/2043405509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0d0cd0 0x7f859c06e450 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f85880097c0 tx=0x7f858801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8761c477a9ce4774 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:00.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f859b7fe640 1 -- 172.21.15.3:0/2043405509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85880b0070 con 0x7f859c0d0cd0 2024-09-12T12:24:00.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f859b7fe640 1 -- 172.21.15.3:0/2043405509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85880ab600 con 0x7f859c0d0cd0 2024-09-12T12:24:00.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f859b7fe640 1 -- 172.21.15.3:0/2043405509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85880ab9d0 con 0x7f859c0d0cd0 2024-09-12T12:24:00.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 -- 172.21.15.3:0/2043405509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0d0cd0 msgr2=0x7f859c06e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:00.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 --2- 172.21.15.3:0/2043405509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0d0cd0 0x7f859c06e450 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f85880097c0 tx=0x7f858801c3d0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 -- 172.21.15.3:0/2043405509 shutdown_connections 2024-09-12T12:24:00.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 --2- 172.21.15.3:0/2043405509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0d0cd0 0x7f859c06e450 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 -- 172.21.15.3:0/2043405509 >> 172.21.15.3:0/2043405509 conn(0x7f859c067e00 msgr2=0x7f859c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:00.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 -- 172.21.15.3:0/2043405509 shutdown_connections 2024-09-12T12:24:00.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 -- 172.21.15.3:0/2043405509 wait complete. 2024-09-12T12:24:00.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 Processor -- start 2024-09-12T12:24:00.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.544+0000 7f85a306a640 1 -- start start 2024-09-12T12:24:00.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0adf60 0x7f859c14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:00.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f859c0d0cd0 0x7f859c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:00.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f859c15c210 0x7f859c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:00.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f859c16a5f0 con 0x7f859c0adf60 2024-09-12T12:24:00.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f859c16a470 con 0x7f859c0d0cd0 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f859c16a770 con 0x7f859c15c210 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0adf60 0x7f859c14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0adf60 0x7f859c14e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60540/0 (socket says 172.21.15.3:60540) 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 -- 172.21.15.3:0/3749794365 learned_addr learned my addr 172.21.15.3:0/3749794365 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f859bfff640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f859c0d0cd0 0x7f859c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 -- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f859c15c210 msgr2=0x7f859c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a1607640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f859c15c210 0x7f859c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f859c15c210 0x7f859c157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 -- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f859c0d0cd0 msgr2=0x7f859c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f859c0d0cd0 0x7f859c1572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 -- 172.21.15.3:0/3749794365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c167b50 con 0x7f859c0adf60 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f859bfff640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f859c0d0cd0 0x7f859c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a1607640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f859c15c210 0x7f859c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a0e06640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0adf60 0x7f859c14e270 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f859000c370 tx=0x7f859000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85997fa640 1 -- 172.21.15.3:0/3749794365 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590017070 con 0x7f859c0adf60 2024-09-12T12:24:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85997fa640 1 -- 172.21.15.3:0/3749794365 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8590005cf0 con 0x7f859c0adf60 2024-09-12T12:24:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c153450 con 0x7f859c0adf60 2024-09-12T12:24:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85997fa640 1 -- 172.21.15.3:0/3749794365 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85900072f0 con 0x7f859c0adf60 2024-09-12T12:24:00.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c14e7b0 con 0x7f859c0adf60 2024-09-12T12:24:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c0ce840 con 0x7f859c0adf60 2024-09-12T12:24:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85997fa640 1 -- 172.21.15.3:0/3749794365 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f859001a070 con 0x7f859c0adf60 2024-09-12T12:24:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85997fa640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f85800678c0 0x7f8580069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.548+0000 7f85997fa640 1 -- 172.21.15.3:0/3749794365 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f8590085320 con 0x7f859c0adf60 2024-09-12T12:24:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.552+0000 7f859bfff640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f85800678c0 0x7f8580069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.552+0000 7f859bfff640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f85800678c0 0x7f8580069d80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f85880ad000 tx=0x7f8588008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:00.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.552+0000 7f85997fa640 1 -- 172.21.15.3:0/3749794365 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8590021020 con 0x7f859c0adf60 2024-09-12T12:24:00.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:00.760+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f859c14ef10 con 0x7f859c0adf60 2024-09-12T12:24:01.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.392+0000 7f85997fa640 1 -- 172.21.15.3:0/3749794365 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7f8590055fd0 con 0x7f859c0adf60 2024-09-12T12:24:01.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f85800678c0 msgr2=0x7f8580069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f85800678c0 0x7f8580069d80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f85880ad000 tx=0x7f8588008040 comp rx=0 tx=0).stop 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0adf60 msgr2=0x7f859c14e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0adf60 0x7f859c14e270 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f859000c370 tx=0x7f859000c830 comp rx=0 tx=0).stop 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 shutdown_connections 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f85800678c0 0x7f8580069d80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f859c15c210 0x7f859c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f859c0d0cd0 0x7f859c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 --2- 172.21.15.3:0/3749794365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c0adf60 0x7f859c14e270 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 >> 172.21.15.3:0/3749794365 conn(0x7f859c067e00 msgr2=0x7f859c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 shutdown_connections 2024-09-12T12:24:01.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.396+0000 7f85a306a640 1 -- 172.21.15.3:0/3749794365 wait complete. 2024-09-12T12:24:01.421 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T12:24:01.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:01.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabfa916640 1 Processor -- start 2024-09-12T12:24:01.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabfa916640 1 -- start start 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabfa916640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fabf40d05b0 0x7fabf40d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabfa916640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf40d0ed0 con 0x7fabf40ae250 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabfa916640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf40ad0c0 con 0x7fabf415b620 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabfa916640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fabf40ad3a0 con 0x7fabf40d05b0 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabf37fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fabf40ae250 0x7fabf40ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38794/0 (socket says 172.21.15.3:38794) 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.548+0000 7fabf37fe640 1 -- 172.21.15.3:0/4045376061 learned_addr learned my addr 172.21.15.3:0/4045376061 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001564656 0 0) 0x7fabf40d0ed0 con 0x7fabf40ae250 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 --2- 172.21.15.3:0/4045376061 >> v2:172.21.15.142:3300/0 conn(0x7fabf40d05b0 0x7fabf40d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabe0003530 con 0x7fabf40ae250 2024-09-12T12:24:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3522416008 0 0) 0x7fabe0003530 con 0x7fabf40ae250 2024-09-12T12:24:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 >> v2:172.21.15.142:3300/0 conn(0x7fabf40d05b0 msgr2=0x7fabf40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 --2- 172.21.15.3:0/4045376061 >> v2:172.21.15.142:3300/0 conn(0x7fabf40d05b0 0x7fabf40d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 >> v1:172.21.15.142:6789/0 conn(0x7fabf415b620 legacy=0x7fabf415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf40ad680 con 0x7fabf40ae250 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fabe4001680 con 0x7fabf40ae250 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 >> v1:172.21.15.3:6789/0 conn(0x7fabf40ae250 legacy=0x7fabf40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabe0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabe0004810 0x7fabe0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabe00051d0 0x7fabe000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fabf40ad3a0 con 0x7fabe00039c0 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fabe000c120 con 0x7fabe0004810 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fabe000c320 con 0x7fabe00051d0 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 --2- 172.21.15.3:0/4045376061 >> v2:172.21.15.142:3300/0 conn(0x7fabf40d05b0 0x7fabf40d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabe0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf37fe640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabe0004810 0x7fabe0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf8eb3640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabe00051d0 0x7fabe000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 -- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabe00051d0 msgr2=0x7fabe000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabe00051d0 0x7fabe000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 -- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabe0004810 msgr2=0x7fabe0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabe0004810 0x7fabe0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 -- 172.21.15.3:0/4045376061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe0003530 con 0x7fabe00039c0 2024-09-12T12:24:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf8eb3640 1 -- 172.21.15.3:0/4045376061 reap_dead start 2024-09-12T12:24:01.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf37fe640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabe0004810 0x7fabe0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf3fff640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabe0003de0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0098b0 tx=0x7fabdc01c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6bfb0eb8b31465 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00f040 con 0x7fabe00039c0 2024-09-12T12:24:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabdc004020 con 0x7fabe00039c0 2024-09-12T12:24:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabf2ffd640 1 -- 172.21.15.3:0/4045376061 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0043f0 con 0x7fabe00039c0 2024-09-12T12:24:01.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabfa916640 1 -- 172.21.15.3:0/4045376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 msgr2=0x7fabe0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabfa916640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabe0003de0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0098b0 tx=0x7fabdc01c580 comp rx=0 tx=0).stop 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabfa916640 1 -- 172.21.15.3:0/4045376061 shutdown_connections 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabfa916640 1 --2- 172.21.15.3:0/4045376061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabe0003de0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.552+0000 7fabfa916640 1 -- 172.21.15.3:0/4045376061 >> 172.21.15.3:0/4045376061 conn(0x7fabf4067e00 msgr2=0x7fabf4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- 172.21.15.3:0/4045376061 shutdown_connections 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- 172.21.15.3:0/4045376061 wait complete. 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 Processor -- start 2024-09-12T12:24:01.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- start start 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabf41e9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabf414a770 0x7fabf40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabf4150740 0x7fabf4150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fabf415c9b0 con 0x7fabe00039c0 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fabf415c830 con 0x7fabf414a770 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fabf415ba30 con 0x7fabf4150740 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabf41e9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabf41e9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60564/0 (socket says 172.21.15.3:60564) 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 -- 172.21.15.3:0/3517362518 learned_addr learned my addr 172.21.15.3:0/3517362518 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf37fe640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabf414a770 0x7fabf40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf8eb3640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabf4150740 0x7fabf4150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 -- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabf4150740 msgr2=0x7fabf4150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabf4150740 0x7fabf4150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 -- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabf414a770 msgr2=0x7fabf40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabf414a770 0x7fabf40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 -- 172.21.15.3:0/3517362518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41e7b20 con 0x7fabe00039c0 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf8eb3640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabf4150740 0x7fabf4150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf37fe640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabf414a770 0x7fabf40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf3fff640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabf41e9960 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0ac040 tx=0x7fabe0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf0ff9640 1 -- 172.21.15.3:0/3517362518 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0ba030 con 0x7fabe00039c0 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf0ff9640 1 -- 172.21.15.3:0/3517362518 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabdc004410 con 0x7fabe00039c0 2024-09-12T12:24:01.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41e7e00 con 0x7fabe00039c0 2024-09-12T12:24:01.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabf0ff9640 1 -- 172.21.15.3:0/3517362518 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc003620 con 0x7fabe00039c0 2024-09-12T12:24:01.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.556+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41e83c0 con 0x7fabe00039c0 2024-09-12T12:24:01.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.560+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf415bdc0 con 0x7fabe00039c0 2024-09-12T12:24:01.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.560+0000 7fabf0ff9640 1 -- 172.21.15.3:0/3517362518 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fabdc0bd030 con 0x7fabe00039c0 2024-09-12T12:24:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.560+0000 7fabf0ff9640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fabd40678f0 0x7fabd4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.560+0000 7fabf0ff9640 1 -- 172.21.15.3:0/3517362518 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fabdc11fab0 con 0x7fabe00039c0 2024-09-12T12:24:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.560+0000 7fabf37fe640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fabd40678f0 0x7fabd4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.560+0000 7fabf37fe640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fabd40678f0 0x7fabd4069db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fabe4001210 tx=0x7fabe4001c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.564+0000 7fabf0ff9640 1 -- 172.21.15.3:0/3517362518 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fabdc0162c0 con 0x7fabe00039c0 2024-09-12T12:24:01.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.756+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabf4071810 con 0x7fabe00039c0 2024-09-12T12:24:01.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.760+0000 7fabf0ff9640 1 -- 172.21.15.3:0/3517362518 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5222 (secure 0 0 0) 0x7fabdc0f0560 con 0x7fabe00039c0 2024-09-12T12:24:01.762 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:01.762 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:57.730276+0000","modified":"2024-09-12T12:24:00.762240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2940169617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2940169617},{"type":"v1","addr":"172.21.15.3:6835","nonce":2940169617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-09-12T12:24:01.763 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 54 2024-09-12T12:24:01.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fabd40678f0 msgr2=0x7fabd4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fabd40678f0 0x7fabd4069db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fabe4001210 tx=0x7fabe4001c20 comp rx=0 tx=0).stop 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 msgr2=0x7fabf41e9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabf41e9960 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0ac040 tx=0x7fabe0005840 comp rx=0 tx=0).stop 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 shutdown_connections 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fabd40678f0 0x7fabd4069db0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fabf4150740 0x7fabf4150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fabf414a770 0x7fabf40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 --2- 172.21.15.3:0/3517362518 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabe00039c0 0x7fabf41e9960 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 >> 172.21.15.3:0/3517362518 conn(0x7fabf4067e00 msgr2=0x7fabf4072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 shutdown_connections 2024-09-12T12:24:01.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.764+0000 7fabfa916640 1 -- 172.21.15.3:0/3517362518 wait complete. 2024-09-12T12:24:01.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:24:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.896+0000 7f626cc7d640 1 Processor -- start 2024-09-12T12:24:01.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.896+0000 7f626cc7d640 1 -- start start 2024-09-12T12:24:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f626cc7d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f626815c000 0x7f62680b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f626cc7d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f626815ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f626cc7d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626815f430 con 0x7f626815ef90 2024-09-12T12:24:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f626cc7d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f626815f9f0 con 0x7f626815c000 2024-09-12T12:24:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f626cc7d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f626815f710 con 0x7f62680af410 2024-09-12T12:24:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f626815ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f626659c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f626815c000 0x7f62680b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f626815ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:57648/0 (socket says 172.21.15.3:57648) 2024-09-12T12:24:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 -- 172.21.15.3:0/59012412 learned_addr learned my addr 172.21.15.3:0/59012412 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 -- 172.21.15.3:0/59012412 >> v2:172.21.15.142:3300/0 conn(0x7f626815c000 msgr2=0x7f62680b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 --2- 172.21.15.3:0/59012412 >> v2:172.21.15.142:3300/0 conn(0x7f626815c000 0x7f62680b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 -- 172.21.15.3:0/59012412 >> v1:172.21.15.142:6789/0 conn(0x7f626815ef90 legacy=0x7f62680d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 -- 172.21.15.3:0/59012412 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62680b3210 con 0x7f62680af410 2024-09-12T12:24:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.900+0000 7f6265d9b640 1 --2- 172.21.15.3:0/59012412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f626815ea50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6250009870 tx=0x7f625001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a41ebd202bacc0c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626559a640 1 -- 172.21.15.3:0/59012412 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500ab640 con 0x7f62680af410 2024-09-12T12:24:01.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626559a640 1 -- 172.21.15.3:0/59012412 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62500b0070 con 0x7f62680af410 2024-09-12T12:24:01.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626559a640 1 -- 172.21.15.3:0/59012412 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500aba00 con 0x7f62680af410 2024-09-12T12:24:01.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- 172.21.15.3:0/59012412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 msgr2=0x7f626815ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 --2- 172.21.15.3:0/59012412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f626815ea50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6250009870 tx=0x7f625001c4a0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- 172.21.15.3:0/59012412 shutdown_connections 2024-09-12T12:24:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 --2- 172.21.15.3:0/59012412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f626815ea50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 --2- 172.21.15.3:0/59012412 >> v2:172.21.15.142:3300/0 conn(0x7f626815c000 0x7f62680b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- 172.21.15.3:0/59012412 >> 172.21.15.3:0/59012412 conn(0x7f6268067e00 msgr2=0x7f6268068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- 172.21.15.3:0/59012412 shutdown_connections 2024-09-12T12:24:01.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- 172.21.15.3:0/59012412 wait complete. 2024-09-12T12:24:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 Processor -- start 2024-09-12T12:24:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- start start 2024-09-12T12:24:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f62681588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626815c000 0x7f6268141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626815ef90 0x7f6268071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f62680d0e40 con 0x7f626815ef90 2024-09-12T12:24:01.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f62680d0cc0 con 0x7f626815c000 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626cc7d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f626806d9f0 con 0x7f62680af410 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626815ef90 0x7f6268071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626815ef90 0x7f6268071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60566/0 (socket says 172.21.15.3:60566) 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 -- 172.21.15.3:0/2696224615 learned_addr learned my addr 172.21.15.3:0/2696224615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626659c640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f62681588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6265d9b640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626815c000 0x7f6268141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 -- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 msgr2=0x7f62681588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f62681588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 -- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626815c000 msgr2=0x7f6268141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626815c000 0x7f6268141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 -- 172.21.15.3:0/2696224615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6268141610 con 0x7f626815ef90 2024-09-12T12:24:01.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f626659c640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f62681588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.904+0000 7f6266d9d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626815ef90 0x7f6268071630 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f626000c370 tx=0x7f626000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f625effd640 1 -- 172.21.15.3:0/2696224615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260005220 con 0x7f626815ef90 2024-09-12T12:24:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f625effd640 1 -- 172.21.15.3:0/2696224615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62600053c0 con 0x7f626815ef90 2024-09-12T12:24:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6268071fa0 con 0x7f626815ef90 2024-09-12T12:24:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f625effd640 1 -- 172.21.15.3:0/2696224615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260007270 con 0x7f626815ef90 2024-09-12T12:24:01.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6268072560 con 0x7f626815ef90 2024-09-12T12:24:01.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626815ea50 con 0x7f626815ef90 2024-09-12T12:24:01.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f625effd640 1 -- 172.21.15.3:0/2696224615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6260019020 con 0x7f626815ef90 2024-09-12T12:24:01.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f625effd640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f62480678c0 0x7f6248069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f625effd640 1 -- 172.21.15.3:0/2696224615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f6260084f80 con 0x7f626815ef90 2024-09-12T12:24:01.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f626659c640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f62480678c0 0x7f6248069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.908+0000 7f626659c640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f62480678c0 0x7f6248069d80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6268146010 tx=0x7f625800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:01.912+0000 7f625effd640 1 -- 172.21.15.3:0/2696224615 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6260055a40 con 0x7f626815ef90 2024-09-12T12:24:02.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.076+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62681592d0 con 0x7f626815ef90 2024-09-12T12:24:02.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.076+0000 7f625effd640 1 -- 172.21.15.3:0/2696224615 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16660 (secure 0 0 0) 0x7f6260055340 con 0x7f626815ef90 2024-09-12T12:24:02.081 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:02.081 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:59.375425+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:55.370943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:56.721996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:24:02.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f62480678c0 msgr2=0x7f6248069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f62480678c0 0x7f6248069d80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6268146010 tx=0x7f625800b040 comp rx=0 tx=0).stop 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626815ef90 msgr2=0x7f6268071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626815ef90 0x7f6268071630 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f626000c370 tx=0x7f626000c830 comp rx=0 tx=0).stop 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 shutdown_connections 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f62480678c0 0x7f6248069d80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f626815ef90 0x7f6268071630 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f626815c000 0x7f6268141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 --2- 172.21.15.3:0/2696224615 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f62680af410 0x7f62681588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 >> 172.21.15.3:0/2696224615 conn(0x7f6268067e00 msgr2=0x7f62680681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 shutdown_connections 2024-09-12T12:24:02.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.080+0000 7f626cc7d640 1 -- 172.21.15.3:0/2696224615 wait complete. 2024-09-12T12:24:02.100 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:24:02.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:02.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb056611640 1 Processor -- start 2024-09-12T12:24:02.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb056611640 1 -- start start 2024-09-12T12:24:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb056611640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0500adfd0 0x7fb050067a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb056611640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb050067f40 0x7fb0500cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb056611640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0500cc6d0 con 0x7fb05015e860 2024-09-12T12:24:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb056611640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb05015ec40 con 0x7fb050067f40 2024-09-12T12:24:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb056611640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb0500d0b60 con 0x7fb0500adfd0 2024-09-12T12:24:02.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb054bae640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb050067f40 0x7fb0500cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb054bae640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb050067f40 0x7fb0500cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60572/0 (socket says 172.21.15.3:60572) 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb04ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb05015e860 0x7fb0500cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38800/0 (socket says 172.21.15.3:38800) 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb054bae640 1 -- 172.21.15.3:0/2361169052 learned_addr learned my addr 172.21.15.3:0/2361169052 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb04f7fe640 1 --2- 172.21.15.3:0/2361169052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0500adfd0 0x7fb050067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb04effd640 1 -- 172.21.15.3:0/2361169052 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340219687 0 0) 0x7fb0500cc6d0 con 0x7fb05015e860 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb054bae640 1 -- 172.21.15.3:0/2361169052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0500adfd0 msgr2=0x7fb050067a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb054bae640 1 --2- 172.21.15.3:0/2361169052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0500adfd0 0x7fb050067a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb054bae640 1 -- 172.21.15.3:0/2361169052 >> v1:172.21.15.3:6789/0 conn(0x7fb05015e860 legacy=0x7fb0500cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.220+0000 7fb054bae640 1 -- 172.21.15.3:0/2361169052 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0500ae3f0 con 0x7fb050067f40 2024-09-12T12:24:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb054bae640 1 --2- 172.21.15.3:0/2361169052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb050067f40 0x7fb0500cce50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb040009870 tx=0x7fb04001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcf49f1c4c5ac04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb04effd640 1 -- 172.21.15.3:0/2361169052 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400b5070 con 0x7fb050067f40 2024-09-12T12:24:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb04effd640 1 -- 172.21.15.3:0/2361169052 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0400a8a50 con 0x7fb050067f40 2024-09-12T12:24:02.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb04effd640 1 -- 172.21.15.3:0/2361169052 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400a8e20 con 0x7fb050067f40 2024-09-12T12:24:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 -- 172.21.15.3:0/2361169052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb050067f40 msgr2=0x7fb0500cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 --2- 172.21.15.3:0/2361169052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb050067f40 0x7fb0500cce50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb040009870 tx=0x7fb04001ed50 comp rx=0 tx=0).stop 2024-09-12T12:24:02.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 -- 172.21.15.3:0/2361169052 shutdown_connections 2024-09-12T12:24:02.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 --2- 172.21.15.3:0/2361169052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb050067f40 0x7fb0500cce50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 --2- 172.21.15.3:0/2361169052 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0500adfd0 0x7fb050067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 -- 172.21.15.3:0/2361169052 >> 172.21.15.3:0/2361169052 conn(0x7fb0500d1220 msgr2=0x7fb0500d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:02.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 -- 172.21.15.3:0/2361169052 shutdown_connections 2024-09-12T12:24:02.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.224+0000 7fb056611640 1 -- 172.21.15.3:0/2361169052 wait complete. 2024-09-12T12:24:02.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 Processor -- start 2024-09-12T12:24:02.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 -- start start 2024-09-12T12:24:02.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb050067f40 0x7fb0500af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0500adfd0 0x7fb0501f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb05015e860 0x7fb05015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb0500d0880 con 0x7fb05015e860 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb0500d05a0 con 0x7fb0500adfd0 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb0500cfee0 con 0x7fb050067f40 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb05015e860 0x7fb05015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb05015e860 0x7fb05015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60582/0 (socket says 172.21.15.3:60582) 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 -- 172.21.15.3:0/3827225001 learned_addr learned my addr 172.21.15.3:0/3827225001 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 -- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb050067f40 msgr2=0x7fb0500af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb050067f40 0x7fb0500af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 -- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0500adfd0 msgr2=0x7fb0501f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb04f7fe640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0500adfd0 0x7fb0501f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0500adfd0 0x7fb0501f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 -- 172.21.15.3:0/3827225001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0500b6660 con 0x7fb05015e860 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb054bae640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb05015e860 0x7fb05015b9d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb04001c980 tx=0x7fb04001baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb04cff9640 1 -- 172.21.15.3:0/3827225001 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400b5070 con 0x7fb05015e860 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb04cff9640 1 -- 172.21.15.3:0/3827225001 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0400b0d80 con 0x7fb05015e860 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb04cff9640 1 -- 172.21.15.3:0/3827225001 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400bb630 con 0x7fb05015e860 2024-09-12T12:24:02.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0500ae610 con 0x7fb05015e860 2024-09-12T12:24:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0500aebd0 con 0x7fb05015e860 2024-09-12T12:24:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.228+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb050067a00 con 0x7fb05015e860 2024-09-12T12:24:02.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.232+0000 7fb04cff9640 1 -- 172.21.15.3:0/3827225001 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb0400b9030 con 0x7fb05015e860 2024-09-12T12:24:02.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.232+0000 7fb04cff9640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0300678f0 0x7fb030069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.232+0000 7fb04cff9640 1 -- 172.21.15.3:0/3827225001 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fb040123140 con 0x7fb05015e860 2024-09-12T12:24:02.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.232+0000 7fb04ffff640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0300678f0 0x7fb030069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.232+0000 7fb04ffff640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0300678f0 0x7fb030069db0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb048001570 tx=0x7fb048007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.236+0000 7fb04cff9640 1 -- 172.21.15.3:0/3827225001 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb0400188b0 con 0x7fb05015e860 2024-09-12T12:24:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.392+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb050068840 con 0x7fb0300678f0 2024-09-12T12:24:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.412+0000 7fb04cff9640 1 -- 172.21.15.3:0/3827225001 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247011 (secure 0 0 0) 0x7fb050068840 con 0x7fb0300678f0 2024-09-12T12:24:02.419 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:02.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0300678f0 msgr2=0x7fb030069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0300678f0 0x7fb030069db0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb048001570 tx=0x7fb048007cb0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb05015e860 msgr2=0x7fb05015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb05015e860 0x7fb05015b9d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb04001c980 tx=0x7fb04001baf0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 shutdown_connections 2024-09-12T12:24:02.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb0300678f0 0x7fb030069db0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb05015e860 0x7fb05015b9d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0500adfd0 0x7fb0501f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 --2- 172.21.15.3:0/3827225001 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb050067f40 0x7fb0500af3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 >> 172.21.15.3:0/3827225001 conn(0x7fb0500d1220 msgr2=0x7fb05006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:02.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 shutdown_connections 2024-09-12T12:24:02.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.416+0000 7fb056611640 1 -- 172.21.15.3:0/3827225001 wait complete. 2024-09-12T12:24:02.435 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-12T12:24:01.742076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241412,"kb_used_data":8496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733308,"statfs":{"total":773094113280,"available":772846907392,"internally_reserved":0,"allocated":8699904,"data_stored":4329111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.404559"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372179+0000","last_change":"2024-09-12T12:23:57.372179+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372179+0000","last_undegraded":"2024-09-12T12:23:57.372179+0000","last_fullsized":"2024-09-12T12:23:57.372179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358274+0000","last_change":"2024-09-12T12:23:57.358274+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358274+0000","last_undegraded":"2024-09-12T12:23:57.358274+0000","last_fullsized":"2024-09-12T12:23:57.358274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367757+0000","last_change":"2024-09-12T12:23:57.367757+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367757+0000","last_undegraded":"2024-09-12T12:23:57.367757+0000","last_fullsized":"2024-09-12T12:23:57.367757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357518+0000","last_change":"2024-09-12T12:23:57.357518+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357518+0000","last_undegraded":"2024-09-12T12:23:57.357518+0000","last_fullsized":"2024-09-12T12:23:57.357518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367639+0000","last_change":"2024-09-12T12:23:57.367639+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367639+0000","last_undegraded":"2024-09-12T12:23:57.367639+0000","last_fullsized":"2024-09-12T12:23:57.367639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358945+0000","last_change":"2024-09-12T12:23:57.358945+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358945+0000","last_undegraded":"2024-09-12T12:23:57.358945+0000","last_fullsized":"2024-09-12T12:23:57.358945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358058+0000","last_change":"2024-09-12T12:23:57.358058+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358058+0000","last_undegraded":"2024-09-12T12:23:57.358058+0000","last_fullsized":"2024-09-12T12:23:57.358058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357626+0000","last_change":"2024-09-12T12:23:57.357626+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357626+0000","last_undegraded":"2024-09-12T12:23:57.357626+0000","last_fullsized":"2024-09-12T12:23:57.357626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.368766+0000","last_change":"2024-09-12T12:23:57.368766+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.368766+0000","last_undegraded":"2024-09-12T12:23:57.368766+0000","last_fullsized":"2024-09-12T12:23:57.368766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.363177+0000","last_change":"2024-09-12T12:23:57.363177+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.363177+0000","last_undegraded":"2024-09-12T12:23:57.363177+0000","last_fullsized":"2024-09-12T12:23:57.363177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358093+0000","last_change":"2024-09-12T12:23:57.358093+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358093+0000","last_undegraded":"2024-09-12T12:23:57.358093+0000","last_fullsized":"2024-09-12T12:23:57.358093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372464+0000","last_change":"2024-09-12T12:23:57.372464+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372464+0000","last_undegraded":"2024-09-12T12:23:57.372464+0000","last_fullsized":"2024-09-12T12:23:57.372464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.370220+0000","last_change":"2024-09-12T12:23:57.370220+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.370220+0000","last_undegraded":"2024-09-12T12:23:57.370220+0000","last_fullsized":"2024-09-12T12:23:57.370220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358797+0000","last_change":"2024-09-12T12:23:57.358797+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358797+0000","last_undegraded":"2024-09-12T12:23:57.358797+0000","last_fullsized":"2024-09-12T12:23:57.358797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.390665+0000","last_change":"2024-09-12T12:23:58.386557+0000","last_active":"2024-09-12T12:23:58.390665+0000","last_peered":"2024-09-12T12:23:58.390665+0000","last_clean":"2024-09-12T12:23:58.390665+0000","last_became_active":"2024-09-12T12:23:58.386226+0000","last_became_peered":"2024-09-12T12:23:58.386226+0000","last_unstale":"2024-09-12T12:23:58.390665+0000","last_undegraded":"2024-09-12T12:23:58.390665+0000","last_fullsized":"2024-09-12T12:23:58.390665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:52:12.537859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367681+0000","last_change":"2024-09-12T12:23:57.367681+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367681+0000","last_undegraded":"2024-09-12T12:23:57.367681+0000","last_fullsized":"2024-09-12T12:23:57.367681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362775+0000","last_change":"2024-09-12T12:23:57.362775+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362775+0000","last_undegraded":"2024-09-12T12:23:57.362775+0000","last_fullsized":"2024-09-12T12:23:57.362775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355804+0000","last_change":"2024-09-12T12:23:57.355804+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355804+0000","last_undegraded":"2024-09-12T12:23:57.355804+0000","last_fullsized":"2024-09-12T12:23:57.355804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362500+0000","last_change":"2024-09-12T12:23:57.362500+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362500+0000","last_undegraded":"2024-09-12T12:23:57.362500+0000","last_fullsized":"2024-09-12T12:23:57.362500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355757+0000","last_change":"2024-09-12T12:23:57.355757+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355757+0000","last_undegraded":"2024-09-12T12:23:57.355757+0000","last_fullsized":"2024-09-12T12:23:57.355757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356788+0000","last_change":"2024-09-12T12:23:57.356788+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356788+0000","last_undegraded":"2024-09-12T12:23:57.356788+0000","last_fullsized":"2024-09-12T12:23:57.356788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358620+0000","last_change":"2024-09-12T12:23:57.358620+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358620+0000","last_undegraded":"2024-09-12T12:23:57.358620+0000","last_fullsized":"2024-09-12T12:23:57.358620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.365333+0000","last_change":"2024-09-12T12:23:57.365333+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.365333+0000","last_undegraded":"2024-09-12T12:23:57.365333+0000","last_fullsized":"2024-09-12T12:23:57.365333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356474+0000","last_change":"2024-09-12T12:23:57.356474+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356474+0000","last_undegraded":"2024-09-12T12:23:57.356474+0000","last_fullsized":"2024-09-12T12:23:57.356474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357760+0000","last_change":"2024-09-12T12:23:57.357760+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357760+0000","last_undegraded":"2024-09-12T12:23:57.357760+0000","last_fullsized":"2024-09-12T12:23:57.357760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371939+0000","last_change":"2024-09-12T12:23:57.371939+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371939+0000","last_undegraded":"2024-09-12T12:23:57.371939+0000","last_fullsized":"2024-09-12T12:23:57.371939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361557+0000","last_change":"2024-09-12T12:23:57.361557+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361557+0000","last_undegraded":"2024-09-12T12:23:57.361557+0000","last_fullsized":"2024-09-12T12:23:57.361557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.369582+0000","last_change":"2024-09-12T12:23:57.369582+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.369582+0000","last_undegraded":"2024-09-12T12:23:57.369582+0000","last_fullsized":"2024-09-12T12:23:57.369582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357253+0000","last_change":"2024-09-12T12:23:57.357253+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357253+0000","last_undegraded":"2024-09-12T12:23:57.357253+0000","last_fullsized":"2024-09-12T12:23:57.357253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356976+0000","last_change":"2024-09-12T12:23:57.356976+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356976+0000","last_undegraded":"2024-09-12T12:23:57.356976+0000","last_fullsized":"2024-09-12T12:23:57.356976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360481+0000","last_change":"2024-09-12T12:23:57.360481+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360481+0000","last_undegraded":"2024-09-12T12:23:57.360481+0000","last_fullsized":"2024-09-12T12:23:57.360481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.366248+0000","last_change":"2024-09-12T12:23:57.366248+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.366248+0000","last_undegraded":"2024-09-12T12:23:57.366248+0000","last_fullsized":"2024-09-12T12:23:57.366248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360338+0000","last_change":"2024-09-12T12:23:57.360338+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360338+0000","last_undegraded":"2024-09-12T12:23:57.360338+0000","last_fullsized":"2024-09-12T12:23:57.360338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395539+0000","last_change":"2024-09-12T12:23:57.391198+0000","last_active":"2024-09-12T12:23:57.395539+0000","last_peered":"2024-09-12T12:23:57.395539+0000","last_clean":"2024-09-12T12:23:57.395539+0000","last_became_active":"2024-09-12T12:23:57.390552+0000","last_became_peered":"2024-09-12T12:23:57.390552+0000","last_unstale":"2024-09-12T12:23:57.395539+0000","last_undegraded":"2024-09-12T12:23:57.395539+0000","last_fullsized":"2024-09-12T12:23:57.395539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:37.934018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371198+0000","last_change":"2024-09-12T12:23:57.371198+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371198+0000","last_undegraded":"2024-09-12T12:23:57.371198+0000","last_fullsized":"2024-09-12T12:23:57.371198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375369+0000","last_change":"2024-09-12T12:23:57.367632+0000","last_active":"2024-09-12T12:23:57.375369+0000","last_peered":"2024-09-12T12:23:57.375369+0000","last_clean":"2024-09-12T12:23:57.375369+0000","last_became_active":"2024-09-12T12:23:57.366903+0000","last_became_peered":"2024-09-12T12:23:57.366903+0000","last_unstale":"2024-09-12T12:23:57.375369+0000","last_undegraded":"2024-09-12T12:23:57.375369+0000","last_fullsized":"2024-09-12T12:23:57.375369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:37.934018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.364066+0000","last_change":"2024-09-12T12:23:57.364066+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.364066+0000","last_undegraded":"2024-09-12T12:23:57.364066+0000","last_fullsized":"2024-09-12T12:23:57.364066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.374007+0000","last_change":"2024-09-12T12:23:57.368053+0000","last_active":"2024-09-12T12:23:57.374007+0000","last_peered":"2024-09-12T12:23:57.374007+0000","last_clean":"2024-09-12T12:23:57.374007+0000","last_became_active":"2024-09-12T12:23:57.367668+0000","last_became_peered":"2024-09-12T12:23:57.367668+0000","last_unstale":"2024-09-12T12:23:57.374007+0000","last_undegraded":"2024-09-12T12:23:57.374007+0000","last_fullsized":"2024-09-12T12:23:57.374007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:10:36.243189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362837+0000","last_change":"2024-09-12T12:23:57.362837+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362837+0000","last_undegraded":"2024-09-12T12:23:57.362837+0000","last_fullsized":"2024-09-12T12:23:57.362837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.374051+0000","last_change":"2024-09-12T12:23:57.368958+0000","last_active":"2024-09-12T12:23:57.374051+0000","last_peered":"2024-09-12T12:23:57.374051+0000","last_clean":"2024-09-12T12:23:57.374051+0000","last_became_active":"2024-09-12T12:23:57.368562+0000","last_became_peered":"2024-09-12T12:23:57.368562+0000","last_unstale":"2024-09-12T12:23:57.374051+0000","last_undegraded":"2024-09-12T12:23:57.374051+0000","last_fullsized":"2024-09-12T12:23:57.374051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:37:27.794694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356053+0000","last_change":"2024-09-12T12:23:57.356053+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356053+0000","last_undegraded":"2024-09-12T12:23:57.356053+0000","last_fullsized":"2024-09-12T12:23:57.356053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379835+0000","last_change":"2024-09-12T12:23:57.371998+0000","last_active":"2024-09-12T12:23:57.379835+0000","last_peered":"2024-09-12T12:23:57.379835+0000","last_clean":"2024-09-12T12:23:57.379835+0000","last_became_active":"2024-09-12T12:23:57.371152+0000","last_became_peered":"2024-09-12T12:23:57.371152+0000","last_unstale":"2024-09-12T12:23:57.379835+0000","last_undegraded":"2024-09-12T12:23:57.379835+0000","last_fullsized":"2024-09-12T12:23:57.379835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:50.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358969+0000","last_change":"2024-09-12T12:23:57.358969+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358969+0000","last_undegraded":"2024-09-12T12:23:57.358969+0000","last_fullsized":"2024-09-12T12:23:57.358969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375373+0000","last_change":"2024-09-12T12:23:57.366528+0000","last_active":"2024-09-12T12:23:57.375373+0000","last_peered":"2024-09-12T12:23:57.375373+0000","last_clean":"2024-09-12T12:23:57.375373+0000","last_became_active":"2024-09-12T12:23:57.365702+0000","last_became_peered":"2024-09-12T12:23:57.365702+0000","last_unstale":"2024-09-12T12:23:57.375373+0000","last_undegraded":"2024-09-12T12:23:57.375373+0000","last_fullsized":"2024-09-12T12:23:57.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:32.123568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372956+0000","last_change":"2024-09-12T12:23:57.372956+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372956+0000","last_undegraded":"2024-09-12T12:23:57.372956+0000","last_fullsized":"2024-09-12T12:23:57.372956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.377034+0000","last_change":"2024-09-12T12:23:57.375695+0000","last_active":"2024-09-12T12:23:57.377034+0000","last_peered":"2024-09-12T12:23:57.377034+0000","last_clean":"2024-09-12T12:23:57.377034+0000","last_became_active":"2024-09-12T12:23:57.375373+0000","last_became_peered":"2024-09-12T12:23:57.375373+0000","last_unstale":"2024-09-12T12:23:57.377034+0000","last_undegraded":"2024-09-12T12:23:57.377034+0000","last_fullsized":"2024-09-12T12:23:57.377034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:18.222157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.382382+0000","last_change":"2024-09-12T12:23:58.377377+0000","last_active":"2024-09-12T12:23:58.382382+0000","last_peered":"2024-09-12T12:23:58.382382+0000","last_clean":"2024-09-12T12:23:58.382382+0000","last_became_active":"2024-09-12T12:23:58.374901+0000","last_became_peered":"2024-09-12T12:23:58.374901+0000","last_unstale":"2024-09-12T12:23:58.382382+0000","last_undegraded":"2024-09-12T12:23:58.382382+0000","last_fullsized":"2024-09-12T12:23:58.382382+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:05.911408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.376659+0000","last_change":"2024-09-12T12:23:57.370326+0000","last_active":"2024-09-12T12:23:57.376659+0000","last_peered":"2024-09-12T12:23:57.376659+0000","last_clean":"2024-09-12T12:23:57.376659+0000","last_became_active":"2024-09-12T12:23:57.369836+0000","last_became_peered":"2024-09-12T12:23:57.369836+0000","last_unstale":"2024-09-12T12:23:57.376659+0000","last_undegraded":"2024-09-12T12:23:57.376659+0000","last_fullsized":"2024-09-12T12:23:57.376659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:34:14.111064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.400544+0000","last_change":"2024-09-12T12:23:58.395462+0000","last_active":"2024-09-12T12:23:58.400544+0000","last_peered":"2024-09-12T12:23:58.400544+0000","last_clean":"2024-09-12T12:23:58.400544+0000","last_became_active":"2024-09-12T12:23:58.395205+0000","last_became_peered":"2024-09-12T12:23:58.395205+0000","last_unstale":"2024-09-12T12:23:58.400544+0000","last_undegraded":"2024-09-12T12:23:58.400544+0000","last_fullsized":"2024-09-12T12:23:58.400544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:22.945497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395479+0000","last_change":"2024-09-12T12:23:57.390954+0000","last_active":"2024-09-12T12:23:57.395479+0000","last_peered":"2024-09-12T12:23:57.395479+0000","last_clean":"2024-09-12T12:23:57.395479+0000","last_became_active":"2024-09-12T12:23:57.389910+0000","last_became_peered":"2024-09-12T12:23:57.389910+0000","last_unstale":"2024-09-12T12:23:57.395479+0000","last_undegraded":"2024-09-12T12:23:57.395479+0000","last_fullsized":"2024-09-12T12:23:57.395479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:32.123568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360621+0000","last_change":"2024-09-12T12:23:57.360621+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360621+0000","last_undegraded":"2024-09-12T12:23:57.360621+0000","last_fullsized":"2024-09-12T12:23:57.360621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.389643+0000","last_change":"2024-09-12T12:23:57.387882+0000","last_active":"2024-09-12T12:23:57.389643+0000","last_peered":"2024-09-12T12:23:57.389643+0000","last_clean":"2024-09-12T12:23:57.389643+0000","last_became_active":"2024-09-12T12:23:57.387183+0000","last_became_peered":"2024-09-12T12:23:57.387183+0000","last_unstale":"2024-09-12T12:23:57.389643+0000","last_undegraded":"2024-09-12T12:23:57.389643+0000","last_fullsized":"2024-09-12T12:23:57.389643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:38.887300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361801+0000","last_change":"2024-09-12T12:23:57.361801+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361801+0000","last_undegraded":"2024-09-12T12:23:57.361801+0000","last_fullsized":"2024-09-12T12:23:57.361801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388590+0000","last_change":"2024-09-12T12:23:57.381940+0000","last_active":"2024-09-12T12:23:57.388590+0000","last_peered":"2024-09-12T12:23:57.388590+0000","last_clean":"2024-09-12T12:23:57.388590+0000","last_became_active":"2024-09-12T12:23:57.380630+0000","last_became_peered":"2024-09-12T12:23:57.380630+0000","last_unstale":"2024-09-12T12:23:57.388590+0000","last_undegraded":"2024-09-12T12:23:57.388590+0000","last_fullsized":"2024-09-12T12:23:57.388590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:25:32.135659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357304+0000","last_change":"2024-09-12T12:23:57.357304+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357304+0000","last_undegraded":"2024-09-12T12:23:57.357304+0000","last_fullsized":"2024-09-12T12:23:57.357304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.384198+0000","last_change":"2024-09-12T12:23:57.377798+0000","last_active":"2024-09-12T12:23:57.384198+0000","last_peered":"2024-09-12T12:23:57.384198+0000","last_clean":"2024-09-12T12:23:57.384198+0000","last_became_active":"2024-09-12T12:23:57.376811+0000","last_became_peered":"2024-09-12T12:23:57.376811+0000","last_unstale":"2024-09-12T12:23:57.384198+0000","last_undegraded":"2024-09-12T12:23:57.384198+0000","last_fullsized":"2024-09-12T12:23:57.384198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:05.313097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361216+0000","last_change":"2024-09-12T12:23:57.361216+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361216+0000","last_undegraded":"2024-09-12T12:23:57.361216+0000","last_fullsized":"2024-09-12T12:23:57.361216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.377291+0000","last_change":"2024-09-12T12:23:57.372742+0000","last_active":"2024-09-12T12:23:57.377291+0000","last_peered":"2024-09-12T12:23:57.377291+0000","last_clean":"2024-09-12T12:23:57.377291+0000","last_became_active":"2024-09-12T12:23:57.372354+0000","last_became_peered":"2024-09-12T12:23:57.372354+0000","last_unstale":"2024-09-12T12:23:57.377291+0000","last_undegraded":"2024-09-12T12:23:57.377291+0000","last_fullsized":"2024-09-12T12:23:57.377291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:04:30.827600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.368232+0000","last_change":"2024-09-12T12:23:57.368232+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.368232+0000","last_undegraded":"2024-09-12T12:23:57.368232+0000","last_fullsized":"2024-09-12T12:23:57.368232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.364221+0000","last_change":"2024-09-12T12:23:57.367013+0000","last_active":"2024-09-12T12:23:58.364221+0000","last_peered":"2024-09-12T12:23:58.364221+0000","last_clean":"2024-09-12T12:23:58.364221+0000","last_became_active":"2024-09-12T12:23:57.366614+0000","last_became_peered":"2024-09-12T12:23:57.366614+0000","last_unstale":"2024-09-12T12:23:58.364221+0000","last_undegraded":"2024-09-12T12:23:58.364221+0000","last_fullsized":"2024-09-12T12:23:58.364221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:16:37.971791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354943+0000","last_change":"2024-09-12T12:23:57.354943+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354943+0000","last_undegraded":"2024-09-12T12:23:57.354943+0000","last_fullsized":"2024-09-12T12:23:57.354943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387273+0000","last_change":"2024-09-12T12:23:57.385811+0000","last_active":"2024-09-12T12:23:57.387273+0000","last_peered":"2024-09-12T12:23:57.387273+0000","last_clean":"2024-09-12T12:23:57.387273+0000","last_became_active":"2024-09-12T12:23:57.384813+0000","last_became_peered":"2024-09-12T12:23:57.384813+0000","last_unstale":"2024-09-12T12:23:57.387273+0000","last_undegraded":"2024-09-12T12:23:57.387273+0000","last_fullsized":"2024-09-12T12:23:57.387273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:39.937210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355561+0000","last_change":"2024-09-12T12:23:57.355561+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355561+0000","last_undegraded":"2024-09-12T12:23:57.355561+0000","last_fullsized":"2024-09-12T12:23:57.355561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387439+0000","last_change":"2024-09-12T12:23:57.385905+0000","last_active":"2024-09-12T12:23:57.387439+0000","last_peered":"2024-09-12T12:23:57.387439+0000","last_clean":"2024-09-12T12:23:57.387439+0000","last_became_active":"2024-09-12T12:23:57.385555+0000","last_became_peered":"2024-09-12T12:23:57.385555+0000","last_unstale":"2024-09-12T12:23:57.387439+0000","last_undegraded":"2024-09-12T12:23:57.387439+0000","last_fullsized":"2024-09-12T12:23:57.387439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:53.460264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362662+0000","last_change":"2024-09-12T12:23:57.362662+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362662+0000","last_undegraded":"2024-09-12T12:23:57.362662+0000","last_fullsized":"2024-09-12T12:23:57.362662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395428+0000","last_change":"2024-09-12T12:23:57.390472+0000","last_active":"2024-09-12T12:23:57.395428+0000","last_peered":"2024-09-12T12:23:57.395428+0000","last_clean":"2024-09-12T12:23:57.395428+0000","last_became_active":"2024-09-12T12:23:57.389725+0000","last_became_peered":"2024-09-12T12:23:57.389725+0000","last_unstale":"2024-09-12T12:23:57.395428+0000","last_undegraded":"2024-09-12T12:23:57.395428+0000","last_fullsized":"2024-09-12T12:23:57.395428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:16.093565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.370155+0000","last_change":"2024-09-12T12:23:57.370155+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.370155+0000","last_undegraded":"2024-09-12T12:23:57.370155+0000","last_fullsized":"2024-09-12T12:23:57.370155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379930+0000","last_change":"2024-09-12T12:23:57.378862+0000","last_active":"2024-09-12T12:23:57.379930+0000","last_peered":"2024-09-12T12:23:57.379930+0000","last_clean":"2024-09-12T12:23:57.379930+0000","last_became_active":"2024-09-12T12:23:57.378463+0000","last_became_peered":"2024-09-12T12:23:57.378463+0000","last_unstale":"2024-09-12T12:23:57.379930+0000","last_undegraded":"2024-09-12T12:23:57.379930+0000","last_fullsized":"2024-09-12T12:23:57.379930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:07.040284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356236+0000","last_change":"2024-09-12T12:23:57.356236+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356236+0000","last_undegraded":"2024-09-12T12:23:57.356236+0000","last_fullsized":"2024-09-12T12:23:57.356236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379895+0000","last_change":"2024-09-12T12:23:57.377815+0000","last_active":"2024-09-12T12:23:57.379895+0000","last_peered":"2024-09-12T12:23:57.379895+0000","last_clean":"2024-09-12T12:23:57.379895+0000","last_became_active":"2024-09-12T12:23:57.377401+0000","last_became_peered":"2024-09-12T12:23:57.377401+0000","last_unstale":"2024-09-12T12:23:57.379895+0000","last_undegraded":"2024-09-12T12:23:57.379895+0000","last_fullsized":"2024-09-12T12:23:57.379895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:56.378083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355613+0000","last_change":"2024-09-12T12:23:57.355613+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355613+0000","last_undegraded":"2024-09-12T12:23:57.355613+0000","last_fullsized":"2024-09-12T12:23:57.355613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.403505+0000","last_change":"2024-09-12T12:23:57.367701+0000","last_active":"2024-09-12T12:23:58.403505+0000","last_peered":"2024-09-12T12:23:58.403505+0000","last_clean":"2024-09-12T12:23:58.403505+0000","last_became_active":"2024-09-12T12:23:57.367397+0000","last_became_peered":"2024-09-12T12:23:57.367397+0000","last_unstale":"2024-09-12T12:23:58.403505+0000","last_undegraded":"2024-09-12T12:23:58.403505+0000","last_fullsized":"2024-09-12T12:23:58.403505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:26:48.411535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358286+0000","last_change":"2024-09-12T12:23:57.358286+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358286+0000","last_undegraded":"2024-09-12T12:23:57.358286+0000","last_fullsized":"2024-09-12T12:23:57.358286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.368927+0000","last_change":"2024-09-12T12:23:57.365147+0000","last_active":"2024-09-12T12:23:57.368927+0000","last_peered":"2024-09-12T12:23:57.368927+0000","last_clean":"2024-09-12T12:23:57.368927+0000","last_became_active":"2024-09-12T12:23:57.364796+0000","last_became_peered":"2024-09-12T12:23:57.364796+0000","last_unstale":"2024-09-12T12:23:57.368927+0000","last_undegraded":"2024-09-12T12:23:57.368927+0000","last_fullsized":"2024-09-12T12:23:57.368927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:46:18.962036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354804+0000","last_change":"2024-09-12T12:23:57.354804+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354804+0000","last_undegraded":"2024-09-12T12:23:57.354804+0000","last_fullsized":"2024-09-12T12:23:57.354804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387960+0000","last_change":"2024-09-12T12:23:57.386864+0000","last_active":"2024-09-12T12:23:57.387960+0000","last_peered":"2024-09-12T12:23:57.387960+0000","last_clean":"2024-09-12T12:23:57.387960+0000","last_became_active":"2024-09-12T12:23:57.386550+0000","last_became_peered":"2024-09-12T12:23:57.386550+0000","last_unstale":"2024-09-12T12:23:57.387960+0000","last_undegraded":"2024-09-12T12:23:57.387960+0000","last_fullsized":"2024-09-12T12:23:57.387960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:03.518217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.364654+0000","last_change":"2024-09-12T12:23:57.364654+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.364654+0000","last_undegraded":"2024-09-12T12:23:57.364654+0000","last_fullsized":"2024-09-12T12:23:57.364654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.360414+0000","last_change":"2024-09-12T12:23:57.368319+0000","last_active":"2024-09-12T12:23:58.360414+0000","last_peered":"2024-09-12T12:23:58.360414+0000","last_clean":"2024-09-12T12:23:58.360414+0000","last_became_active":"2024-09-12T12:23:57.367912+0000","last_became_peered":"2024-09-12T12:23:57.367912+0000","last_unstale":"2024-09-12T12:23:58.360414+0000","last_undegraded":"2024-09-12T12:23:58.360414+0000","last_fullsized":"2024-09-12T12:23:58.360414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:35.720409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354991+0000","last_change":"2024-09-12T12:23:57.354991+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354991+0000","last_undegraded":"2024-09-12T12:23:57.354991+0000","last_fullsized":"2024-09-12T12:23:57.354991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387956+0000","last_change":"2024-09-12T12:23:57.386214+0000","last_active":"2024-09-12T12:23:57.387956+0000","last_peered":"2024-09-12T12:23:57.387956+0000","last_clean":"2024-09-12T12:23:57.387956+0000","last_became_active":"2024-09-12T12:23:57.385756+0000","last_became_peered":"2024-09-12T12:23:57.385756+0000","last_unstale":"2024-09-12T12:23:57.387956+0000","last_undegraded":"2024-09-12T12:23:57.387956+0000","last_fullsized":"2024-09-12T12:23:57.387956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:58.282124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.376205+0000","last_change":"2024-09-12T12:23:57.376205+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.376205+0000","last_undegraded":"2024-09-12T12:23:57.376205+0000","last_fullsized":"2024-09-12T12:23:57.376205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.376824+0000","last_change":"2024-09-12T12:23:57.369090+0000","last_active":"2024-09-12T12:23:57.376824+0000","last_peered":"2024-09-12T12:23:57.376824+0000","last_clean":"2024-09-12T12:23:57.376824+0000","last_became_active":"2024-09-12T12:23:57.368802+0000","last_became_peered":"2024-09-12T12:23:57.368802+0000","last_unstale":"2024-09-12T12:23:57.376824+0000","last_undegraded":"2024-09-12T12:23:57.376824+0000","last_fullsized":"2024-09-12T12:23:57.376824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:37:27.347542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362894+0000","last_change":"2024-09-12T12:23:57.362894+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362894+0000","last_undegraded":"2024-09-12T12:23:57.362894+0000","last_fullsized":"2024-09-12T12:23:57.362894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388670+0000","last_change":"2024-09-12T12:23:57.382433+0000","last_active":"2024-09-12T12:23:57.388670+0000","last_peered":"2024-09-12T12:23:57.388670+0000","last_clean":"2024-09-12T12:23:57.388670+0000","last_became_active":"2024-09-12T12:23:57.381303+0000","last_became_peered":"2024-09-12T12:23:57.381303+0000","last_unstale":"2024-09-12T12:23:57.388670+0000","last_undegraded":"2024-09-12T12:23:57.388670+0000","last_fullsized":"2024-09-12T12:23:57.388670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:38:15.474579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355099+0000","last_change":"2024-09-12T12:23:57.355099+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355099+0000","last_undegraded":"2024-09-12T12:23:57.355099+0000","last_fullsized":"2024-09-12T12:23:57.355099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388602+0000","last_change":"2024-09-12T12:23:57.382205+0000","last_active":"2024-09-12T12:23:57.388602+0000","last_peered":"2024-09-12T12:23:57.388602+0000","last_clean":"2024-09-12T12:23:57.388602+0000","last_became_active":"2024-09-12T12:23:57.381644+0000","last_became_peered":"2024-09-12T12:23:57.381644+0000","last_unstale":"2024-09-12T12:23:57.388602+0000","last_undegraded":"2024-09-12T12:23:57.388602+0000","last_fullsized":"2024-09-12T12:23:57.388602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:41.425596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357647+0000","last_change":"2024-09-12T12:23:57.357647+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357647+0000","last_undegraded":"2024-09-12T12:23:57.357647+0000","last_fullsized":"2024-09-12T12:23:57.357647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.382580+0000","last_change":"2024-09-12T12:23:57.381829+0000","last_active":"2024-09-12T12:23:57.382580+0000","last_peered":"2024-09-12T12:23:57.382580+0000","last_clean":"2024-09-12T12:23:57.382580+0000","last_became_active":"2024-09-12T12:23:57.381422+0000","last_became_peered":"2024-09-12T12:23:57.381422+0000","last_unstale":"2024-09-12T12:23:57.382580+0000","last_undegraded":"2024-09-12T12:23:57.382580+0000","last_fullsized":"2024-09-12T12:23:57.382580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:12.617164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361245+0000","last_change":"2024-09-12T12:23:57.361245+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361245+0000","last_undegraded":"2024-09-12T12:23:57.361245+0000","last_fullsized":"2024-09-12T12:23:57.361245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375315+0000","last_change":"2024-09-12T12:23:57.366499+0000","last_active":"2024-09-12T12:23:57.375315+0000","last_peered":"2024-09-12T12:23:57.375315+0000","last_clean":"2024-09-12T12:23:57.375315+0000","last_became_active":"2024-09-12T12:23:57.364717+0000","last_became_peered":"2024-09-12T12:23:57.364717+0000","last_unstale":"2024-09-12T12:23:57.375315+0000","last_undegraded":"2024-09-12T12:23:57.375315+0000","last_fullsized":"2024-09-12T12:23:57.375315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:16.093565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371930+0000","last_change":"2024-09-12T12:23:57.371930+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371930+0000","last_undegraded":"2024-09-12T12:23:57.371930+0000","last_fullsized":"2024-09-12T12:23:57.371930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.392519+0000","last_change":"2024-09-12T12:23:57.388887+0000","last_active":"2024-09-12T12:23:57.392519+0000","last_peered":"2024-09-12T12:23:57.392519+0000","last_clean":"2024-09-12T12:23:57.392519+0000","last_became_active":"2024-09-12T12:23:57.388290+0000","last_became_peered":"2024-09-12T12:23:57.388290+0000","last_unstale":"2024-09-12T12:23:57.392519+0000","last_undegraded":"2024-09-12T12:23:57.392519+0000","last_fullsized":"2024-09-12T12:23:57.392519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:19:31.787885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.363040+0000","last_change":"2024-09-12T12:23:57.363040+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.363040+0000","last_undegraded":"2024-09-12T12:23:57.363040+0000","last_fullsized":"2024-09-12T12:23:57.363040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.394938+0000","last_change":"2024-09-12T12:23:57.390177+0000","last_active":"2024-09-12T12:23:57.394938+0000","last_peered":"2024-09-12T12:23:57.394938+0000","last_clean":"2024-09-12T12:23:57.394938+0000","last_became_active":"2024-09-12T12:23:57.389410+0000","last_became_peered":"2024-09-12T12:23:57.389410+0000","last_unstale":"2024-09-12T12:23:57.394938+0000","last_undegraded":"2024-09-12T12:23:57.394938+0000","last_fullsized":"2024-09-12T12:23:57.394938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:21.483924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354931+0000","last_change":"2024-09-12T12:23:57.354931+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354931+0000","last_undegraded":"2024-09-12T12:23:57.354931+0000","last_fullsized":"2024-09-12T12:23:57.354931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.390536+0000","last_change":"2024-09-12T12:23:57.386525+0000","last_active":"2024-09-12T12:23:57.390536+0000","last_peered":"2024-09-12T12:23:57.390536+0000","last_clean":"2024-09-12T12:23:57.390536+0000","last_became_active":"2024-09-12T12:23:57.385710+0000","last_became_peered":"2024-09-12T12:23:57.385710+0000","last_unstale":"2024-09-12T12:23:57.390536+0000","last_undegraded":"2024-09-12T12:23:57.390536+0000","last_fullsized":"2024-09-12T12:23:57.390536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:11.831899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":273,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.385148+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:57.385148+0000","last_peered":"2024-09-12T12:23:57.385148+0000","last_clean":"2024-09-12T12:23:57.385148+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:57.385148+0000","last_undegraded":"2024-09-12T12:23:57.385148+0000","last_fullsized":"2024-09-12T12:23:57.385148+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":966656,"data_stored":404023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542200,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1523712,"data_stored":979729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":395636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":395636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1536000,"data_stored":986004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:02.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.588+0000 7f2192b60640 1 Processor -- start 2024-09-12T12:24:02.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.588+0000 7f2192b60640 1 -- start start 2024-09-12T12:24:02.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f2192b60640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f218c0b30e0 0x7f218c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f2192b60640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c0b5000 con 0x7f218c073cb0 2024-09-12T12:24:02.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f2192b60640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c0b3510 con 0x7f218c15c710 2024-09-12T12:24:02.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f2192b60640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f218c074090 con 0x7f218c0b30e0 2024-09-12T12:24:02.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f218c073cb0 0x7f218c0adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38802/0 (socket says 172.21.15.3:38802) 2024-09-12T12:24:02.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 -- 172.21.15.3:0/1680099629 learned_addr learned my addr 172.21.15.3:0/1680099629 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 438192734 0 0) 0x7f218c0b5000 con 0x7f218c073cb0 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2174003530 con 0x7f218c073cb0 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21910fd640 1 --2- 172.21.15.3:0/1680099629 >> v2:172.21.15.142:3300/0 conn(0x7f218c0b30e0 0x7f218c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3078149177 0 0) 0x7f2174003530 con 0x7f218c073cb0 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 >> v2:172.21.15.142:3300/0 conn(0x7f218c0b30e0 msgr2=0x7f218c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 --2- 172.21.15.3:0/1680099629 >> v2:172.21.15.142:3300/0 conn(0x7f218c0b30e0 0x7f218c0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 >> v1:172.21.15.142:6789/0 conn(0x7f218c15c710 legacy=0x7f218c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c0b37c0 con 0x7f218c073cb0 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f217c001680 con 0x7f218c073cb0 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 >> v1:172.21.15.3:6789/0 conn(0x7f218c073cb0 legacy=0x7f218c0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f2174003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174004810 0x7f2174004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21740051d0 0x7f217400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f217400c0a0 con 0x7f2174003a10 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f217400c250 con 0x7f2174004810 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f217400c450 con 0x7f21740051d0 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f2174003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218bfff640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174004810 0x7f2174004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 -- 172.21.15.3:0/1680099629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21740051d0 msgr2=0x7f217400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21740051d0 0x7f217400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 -- 172.21.15.3:0/1680099629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174004810 msgr2=0x7f2174004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2174004810 0x7f2174004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 -- 172.21.15.3:0/1680099629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2174003530 con 0x7f2174003a10 2024-09-12T12:24:02.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21910fd640 1 -- 172.21.15.3:0/1680099629 reap_dead start 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f21908fc640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f2174003e30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c110 tx=0x7f217c01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f941a02821f0d705 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c007c50 con 0x7f2174003a10 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217c0a9070 con 0x7f2174003a10 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.592+0000 7f218b7fe640 1 -- 172.21.15.3:0/1680099629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c005d80 con 0x7f2174003a10 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- 172.21.15.3:0/1680099629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 msgr2=0x7f2174003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f2174003e30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c110 tx=0x7f217c01cb80 comp rx=0 tx=0).stop 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- 172.21.15.3:0/1680099629 shutdown_connections 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 --2- 172.21.15.3:0/1680099629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f2174003e30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- 172.21.15.3:0/1680099629 >> 172.21.15.3:0/1680099629 conn(0x7f218c067e00 msgr2=0x7f218c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:02.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- 172.21.15.3:0/1680099629 shutdown_connections 2024-09-12T12:24:02.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- 172.21.15.3:0/1680099629 wait complete. 2024-09-12T12:24:02.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 Processor -- start 2024-09-12T12:24:02.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- start start 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f218c151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f218c152030 0x7f218c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f218c0cd610 0x7f218c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f218c0aea00 con 0x7f2174003a10 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f218c0ae880 con 0x7f218c0cd610 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f218c0aeb80 con 0x7f218c152030 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f218c151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f218c151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60608/0 (socket says 172.21.15.3:60608) 2024-09-12T12:24:02.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 -- 172.21.15.3:0/335481953 learned_addr learned my addr 172.21.15.3:0/335481953 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21910fd640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f218c0cd610 0x7f218c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 -- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f218c152030 msgr2=0x7f218c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f218bfff640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f218c152030 0x7f218c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f218c152030 0x7f218c152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 -- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f218c0cd610 msgr2=0x7f218c153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f218c0cd610 0x7f218c153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 -- 172.21.15.3:0/335481953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c0cda90 con 0x7f2174003a10 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21910fd640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f218c0cd610 0x7f218c153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f218bfff640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f218c152030 0x7f218c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21908fc640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f218c151a50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c110 tx=0x7f2174005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21897fa640 1 -- 172.21.15.3:0/335481953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c007140 con 0x7f2174003a10 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21897fa640 1 -- 172.21.15.3:0/335481953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217c0072e0 con 0x7f2174003a10 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218c0b0e90 con 0x7f2174003a10 2024-09-12T12:24:02.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f21897fa640 1 -- 172.21.15.3:0/335481953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c007690 con 0x7f2174003a10 2024-09-12T12:24:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.596+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218c0b1450 con 0x7f2174003a10 2024-09-12T12:24:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.600+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218c0af6d0 con 0x7f2174003a10 2024-09-12T12:24:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.600+0000 7f21897fa640 1 -- 172.21.15.3:0/335481953 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f217c0047f0 con 0x7f2174003a10 2024-09-12T12:24:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.600+0000 7f21897fa640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21780678f0 0x7f2178069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.600+0000 7f21897fa640 1 -- 172.21.15.3:0/335481953 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f217c11f430 con 0x7f2174003a10 2024-09-12T12:24:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.600+0000 7f218bfff640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21780678f0 0x7f2178069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.600+0000 7f218bfff640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21780678f0 0x7f2178069db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c10 tx=0x7f2180009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.604+0000 7f21897fa640 1 -- 172.21.15.3:0/335481953 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f218c0cda90 con 0x7f2174003a10 2024-09-12T12:24:02.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.764+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f218c15e4c0 con 0x7f21780678f0 2024-09-12T12:24:02.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.784+0000 7f21897fa640 1 -- 172.21.15.3:0/335481953 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247011 (secure 0 0 0) 0x7f218c15e4c0 con 0x7f21780678f0 2024-09-12T12:24:02.787 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:02.789 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21780678f0 msgr2=0x7f2178069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21780678f0 0x7f2178069db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c10 tx=0x7f2180009040 comp rx=0 tx=0).stop 2024-09-12T12:24:02.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 msgr2=0x7f218c151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f218c151a50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f217c00c110 tx=0x7f2174005840 comp rx=0 tx=0).stop 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 shutdown_connections 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21780678f0 0x7f2178069db0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f218c0cd610 0x7f218c153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f218c152030 0x7f218c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 --2- 172.21.15.3:0/335481953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2174003a10 0x7f218c151a50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 >> 172.21.15.3:0/335481953 conn(0x7f218c067e00 msgr2=0x7f218c15d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 shutdown_connections 2024-09-12T12:24:02.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.788+0000 7f2192b60640 1 -- 172.21.15.3:0/335481953 wait complete. 2024-09-12T12:24:02.808 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-12T12:24:01.742076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241412,"kb_used_data":8496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733308,"statfs":{"total":773094113280,"available":772846907392,"internally_reserved":0,"allocated":8699904,"data_stored":4329111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.404559"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372179+0000","last_change":"2024-09-12T12:23:57.372179+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372179+0000","last_undegraded":"2024-09-12T12:23:57.372179+0000","last_fullsized":"2024-09-12T12:23:57.372179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358274+0000","last_change":"2024-09-12T12:23:57.358274+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358274+0000","last_undegraded":"2024-09-12T12:23:57.358274+0000","last_fullsized":"2024-09-12T12:23:57.358274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367757+0000","last_change":"2024-09-12T12:23:57.367757+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367757+0000","last_undegraded":"2024-09-12T12:23:57.367757+0000","last_fullsized":"2024-09-12T12:23:57.367757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357518+0000","last_change":"2024-09-12T12:23:57.357518+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357518+0000","last_undegraded":"2024-09-12T12:23:57.357518+0000","last_fullsized":"2024-09-12T12:23:57.357518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367639+0000","last_change":"2024-09-12T12:23:57.367639+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367639+0000","last_undegraded":"2024-09-12T12:23:57.367639+0000","last_fullsized":"2024-09-12T12:23:57.367639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358945+0000","last_change":"2024-09-12T12:23:57.358945+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358945+0000","last_undegraded":"2024-09-12T12:23:57.358945+0000","last_fullsized":"2024-09-12T12:23:57.358945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358058+0000","last_change":"2024-09-12T12:23:57.358058+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358058+0000","last_undegraded":"2024-09-12T12:23:57.358058+0000","last_fullsized":"2024-09-12T12:23:57.358058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357626+0000","last_change":"2024-09-12T12:23:57.357626+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357626+0000","last_undegraded":"2024-09-12T12:23:57.357626+0000","last_fullsized":"2024-09-12T12:23:57.357626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.368766+0000","last_change":"2024-09-12T12:23:57.368766+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.368766+0000","last_undegraded":"2024-09-12T12:23:57.368766+0000","last_fullsized":"2024-09-12T12:23:57.368766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.363177+0000","last_change":"2024-09-12T12:23:57.363177+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.363177+0000","last_undegraded":"2024-09-12T12:23:57.363177+0000","last_fullsized":"2024-09-12T12:23:57.363177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358093+0000","last_change":"2024-09-12T12:23:57.358093+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358093+0000","last_undegraded":"2024-09-12T12:23:57.358093+0000","last_fullsized":"2024-09-12T12:23:57.358093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372464+0000","last_change":"2024-09-12T12:23:57.372464+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372464+0000","last_undegraded":"2024-09-12T12:23:57.372464+0000","last_fullsized":"2024-09-12T12:23:57.372464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.370220+0000","last_change":"2024-09-12T12:23:57.370220+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.370220+0000","last_undegraded":"2024-09-12T12:23:57.370220+0000","last_fullsized":"2024-09-12T12:23:57.370220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358797+0000","last_change":"2024-09-12T12:23:57.358797+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358797+0000","last_undegraded":"2024-09-12T12:23:57.358797+0000","last_fullsized":"2024-09-12T12:23:57.358797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.390665+0000","last_change":"2024-09-12T12:23:58.386557+0000","last_active":"2024-09-12T12:23:58.390665+0000","last_peered":"2024-09-12T12:23:58.390665+0000","last_clean":"2024-09-12T12:23:58.390665+0000","last_became_active":"2024-09-12T12:23:58.386226+0000","last_became_peered":"2024-09-12T12:23:58.386226+0000","last_unstale":"2024-09-12T12:23:58.390665+0000","last_undegraded":"2024-09-12T12:23:58.390665+0000","last_fullsized":"2024-09-12T12:23:58.390665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:52:12.537859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367681+0000","last_change":"2024-09-12T12:23:57.367681+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367681+0000","last_undegraded":"2024-09-12T12:23:57.367681+0000","last_fullsized":"2024-09-12T12:23:57.367681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362775+0000","last_change":"2024-09-12T12:23:57.362775+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362775+0000","last_undegraded":"2024-09-12T12:23:57.362775+0000","last_fullsized":"2024-09-12T12:23:57.362775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355804+0000","last_change":"2024-09-12T12:23:57.355804+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355804+0000","last_undegraded":"2024-09-12T12:23:57.355804+0000","last_fullsized":"2024-09-12T12:23:57.355804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362500+0000","last_change":"2024-09-12T12:23:57.362500+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362500+0000","last_undegraded":"2024-09-12T12:23:57.362500+0000","last_fullsized":"2024-09-12T12:23:57.362500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355757+0000","last_change":"2024-09-12T12:23:57.355757+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355757+0000","last_undegraded":"2024-09-12T12:23:57.355757+0000","last_fullsized":"2024-09-12T12:23:57.355757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356788+0000","last_change":"2024-09-12T12:23:57.356788+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356788+0000","last_undegraded":"2024-09-12T12:23:57.356788+0000","last_fullsized":"2024-09-12T12:23:57.356788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358620+0000","last_change":"2024-09-12T12:23:57.358620+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358620+0000","last_undegraded":"2024-09-12T12:23:57.358620+0000","last_fullsized":"2024-09-12T12:23:57.358620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.365333+0000","last_change":"2024-09-12T12:23:57.365333+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.365333+0000","last_undegraded":"2024-09-12T12:23:57.365333+0000","last_fullsized":"2024-09-12T12:23:57.365333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356474+0000","last_change":"2024-09-12T12:23:57.356474+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356474+0000","last_undegraded":"2024-09-12T12:23:57.356474+0000","last_fullsized":"2024-09-12T12:23:57.356474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357760+0000","last_change":"2024-09-12T12:23:57.357760+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357760+0000","last_undegraded":"2024-09-12T12:23:57.357760+0000","last_fullsized":"2024-09-12T12:23:57.357760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371939+0000","last_change":"2024-09-12T12:23:57.371939+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371939+0000","last_undegraded":"2024-09-12T12:23:57.371939+0000","last_fullsized":"2024-09-12T12:23:57.371939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361557+0000","last_change":"2024-09-12T12:23:57.361557+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361557+0000","last_undegraded":"2024-09-12T12:23:57.361557+0000","last_fullsized":"2024-09-12T12:23:57.361557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.369582+0000","last_change":"2024-09-12T12:23:57.369582+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.369582+0000","last_undegraded":"2024-09-12T12:23:57.369582+0000","last_fullsized":"2024-09-12T12:23:57.369582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357253+0000","last_change":"2024-09-12T12:23:57.357253+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357253+0000","last_undegraded":"2024-09-12T12:23:57.357253+0000","last_fullsized":"2024-09-12T12:23:57.357253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356976+0000","last_change":"2024-09-12T12:23:57.356976+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356976+0000","last_undegraded":"2024-09-12T12:23:57.356976+0000","last_fullsized":"2024-09-12T12:23:57.356976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360481+0000","last_change":"2024-09-12T12:23:57.360481+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360481+0000","last_undegraded":"2024-09-12T12:23:57.360481+0000","last_fullsized":"2024-09-12T12:23:57.360481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.366248+0000","last_change":"2024-09-12T12:23:57.366248+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.366248+0000","last_undegraded":"2024-09-12T12:23:57.366248+0000","last_fullsized":"2024-09-12T12:23:57.366248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360338+0000","last_change":"2024-09-12T12:23:57.360338+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360338+0000","last_undegraded":"2024-09-12T12:23:57.360338+0000","last_fullsized":"2024-09-12T12:23:57.360338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395539+0000","last_change":"2024-09-12T12:23:57.391198+0000","last_active":"2024-09-12T12:23:57.395539+0000","last_peered":"2024-09-12T12:23:57.395539+0000","last_clean":"2024-09-12T12:23:57.395539+0000","last_became_active":"2024-09-12T12:23:57.390552+0000","last_became_peered":"2024-09-12T12:23:57.390552+0000","last_unstale":"2024-09-12T12:23:57.395539+0000","last_undegraded":"2024-09-12T12:23:57.395539+0000","last_fullsized":"2024-09-12T12:23:57.395539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:37.934018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371198+0000","last_change":"2024-09-12T12:23:57.371198+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371198+0000","last_undegraded":"2024-09-12T12:23:57.371198+0000","last_fullsized":"2024-09-12T12:23:57.371198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375369+0000","last_change":"2024-09-12T12:23:57.367632+0000","last_active":"2024-09-12T12:23:57.375369+0000","last_peered":"2024-09-12T12:23:57.375369+0000","last_clean":"2024-09-12T12:23:57.375369+0000","last_became_active":"2024-09-12T12:23:57.366903+0000","last_became_peered":"2024-09-12T12:23:57.366903+0000","last_unstale":"2024-09-12T12:23:57.375369+0000","last_undegraded":"2024-09-12T12:23:57.375369+0000","last_fullsized":"2024-09-12T12:23:57.375369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:37.934018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.364066+0000","last_change":"2024-09-12T12:23:57.364066+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.364066+0000","last_undegraded":"2024-09-12T12:23:57.364066+0000","last_fullsized":"2024-09-12T12:23:57.364066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.374007+0000","last_change":"2024-09-12T12:23:57.368053+0000","last_active":"2024-09-12T12:23:57.374007+0000","last_peered":"2024-09-12T12:23:57.374007+0000","last_clean":"2024-09-12T12:23:57.374007+0000","last_became_active":"2024-09-12T12:23:57.367668+0000","last_became_peered":"2024-09-12T12:23:57.367668+0000","last_unstale":"2024-09-12T12:23:57.374007+0000","last_undegraded":"2024-09-12T12:23:57.374007+0000","last_fullsized":"2024-09-12T12:23:57.374007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:10:36.243189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362837+0000","last_change":"2024-09-12T12:23:57.362837+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362837+0000","last_undegraded":"2024-09-12T12:23:57.362837+0000","last_fullsized":"2024-09-12T12:23:57.362837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.374051+0000","last_change":"2024-09-12T12:23:57.368958+0000","last_active":"2024-09-12T12:23:57.374051+0000","last_peered":"2024-09-12T12:23:57.374051+0000","last_clean":"2024-09-12T12:23:57.374051+0000","last_became_active":"2024-09-12T12:23:57.368562+0000","last_became_peered":"2024-09-12T12:23:57.368562+0000","last_unstale":"2024-09-12T12:23:57.374051+0000","last_undegraded":"2024-09-12T12:23:57.374051+0000","last_fullsized":"2024-09-12T12:23:57.374051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:37:27.794694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356053+0000","last_change":"2024-09-12T12:23:57.356053+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356053+0000","last_undegraded":"2024-09-12T12:23:57.356053+0000","last_fullsized":"2024-09-12T12:23:57.356053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379835+0000","last_change":"2024-09-12T12:23:57.371998+0000","last_active":"2024-09-12T12:23:57.379835+0000","last_peered":"2024-09-12T12:23:57.379835+0000","last_clean":"2024-09-12T12:23:57.379835+0000","last_became_active":"2024-09-12T12:23:57.371152+0000","last_became_peered":"2024-09-12T12:23:57.371152+0000","last_unstale":"2024-09-12T12:23:57.379835+0000","last_undegraded":"2024-09-12T12:23:57.379835+0000","last_fullsized":"2024-09-12T12:23:57.379835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:50.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358969+0000","last_change":"2024-09-12T12:23:57.358969+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358969+0000","last_undegraded":"2024-09-12T12:23:57.358969+0000","last_fullsized":"2024-09-12T12:23:57.358969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375373+0000","last_change":"2024-09-12T12:23:57.366528+0000","last_active":"2024-09-12T12:23:57.375373+0000","last_peered":"2024-09-12T12:23:57.375373+0000","last_clean":"2024-09-12T12:23:57.375373+0000","last_became_active":"2024-09-12T12:23:57.365702+0000","last_became_peered":"2024-09-12T12:23:57.365702+0000","last_unstale":"2024-09-12T12:23:57.375373+0000","last_undegraded":"2024-09-12T12:23:57.375373+0000","last_fullsized":"2024-09-12T12:23:57.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:32.123568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372956+0000","last_change":"2024-09-12T12:23:57.372956+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372956+0000","last_undegraded":"2024-09-12T12:23:57.372956+0000","last_fullsized":"2024-09-12T12:23:57.372956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.377034+0000","last_change":"2024-09-12T12:23:57.375695+0000","last_active":"2024-09-12T12:23:57.377034+0000","last_peered":"2024-09-12T12:23:57.377034+0000","last_clean":"2024-09-12T12:23:57.377034+0000","last_became_active":"2024-09-12T12:23:57.375373+0000","last_became_peered":"2024-09-12T12:23:57.375373+0000","last_unstale":"2024-09-12T12:23:57.377034+0000","last_undegraded":"2024-09-12T12:23:57.377034+0000","last_fullsized":"2024-09-12T12:23:57.377034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:18.222157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.382382+0000","last_change":"2024-09-12T12:23:58.377377+0000","last_active":"2024-09-12T12:23:58.382382+0000","last_peered":"2024-09-12T12:23:58.382382+0000","last_clean":"2024-09-12T12:23:58.382382+0000","last_became_active":"2024-09-12T12:23:58.374901+0000","last_became_peered":"2024-09-12T12:23:58.374901+0000","last_unstale":"2024-09-12T12:23:58.382382+0000","last_undegraded":"2024-09-12T12:23:58.382382+0000","last_fullsized":"2024-09-12T12:23:58.382382+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:05.911408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.376659+0000","last_change":"2024-09-12T12:23:57.370326+0000","last_active":"2024-09-12T12:23:57.376659+0000","last_peered":"2024-09-12T12:23:57.376659+0000","last_clean":"2024-09-12T12:23:57.376659+0000","last_became_active":"2024-09-12T12:23:57.369836+0000","last_became_peered":"2024-09-12T12:23:57.369836+0000","last_unstale":"2024-09-12T12:23:57.376659+0000","last_undegraded":"2024-09-12T12:23:57.376659+0000","last_fullsized":"2024-09-12T12:23:57.376659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:34:14.111064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.400544+0000","last_change":"2024-09-12T12:23:58.395462+0000","last_active":"2024-09-12T12:23:58.400544+0000","last_peered":"2024-09-12T12:23:58.400544+0000","last_clean":"2024-09-12T12:23:58.400544+0000","last_became_active":"2024-09-12T12:23:58.395205+0000","last_became_peered":"2024-09-12T12:23:58.395205+0000","last_unstale":"2024-09-12T12:23:58.400544+0000","last_undegraded":"2024-09-12T12:23:58.400544+0000","last_fullsized":"2024-09-12T12:23:58.400544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:22.945497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395479+0000","last_change":"2024-09-12T12:23:57.390954+0000","last_active":"2024-09-12T12:23:57.395479+0000","last_peered":"2024-09-12T12:23:57.395479+0000","last_clean":"2024-09-12T12:23:57.395479+0000","last_became_active":"2024-09-12T12:23:57.389910+0000","last_became_peered":"2024-09-12T12:23:57.389910+0000","last_unstale":"2024-09-12T12:23:57.395479+0000","last_undegraded":"2024-09-12T12:23:57.395479+0000","last_fullsized":"2024-09-12T12:23:57.395479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:32.123568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360621+0000","last_change":"2024-09-12T12:23:57.360621+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360621+0000","last_undegraded":"2024-09-12T12:23:57.360621+0000","last_fullsized":"2024-09-12T12:23:57.360621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.389643+0000","last_change":"2024-09-12T12:23:57.387882+0000","last_active":"2024-09-12T12:23:57.389643+0000","last_peered":"2024-09-12T12:23:57.389643+0000","last_clean":"2024-09-12T12:23:57.389643+0000","last_became_active":"2024-09-12T12:23:57.387183+0000","last_became_peered":"2024-09-12T12:23:57.387183+0000","last_unstale":"2024-09-12T12:23:57.389643+0000","last_undegraded":"2024-09-12T12:23:57.389643+0000","last_fullsized":"2024-09-12T12:23:57.389643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:38.887300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361801+0000","last_change":"2024-09-12T12:23:57.361801+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361801+0000","last_undegraded":"2024-09-12T12:23:57.361801+0000","last_fullsized":"2024-09-12T12:23:57.361801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388590+0000","last_change":"2024-09-12T12:23:57.381940+0000","last_active":"2024-09-12T12:23:57.388590+0000","last_peered":"2024-09-12T12:23:57.388590+0000","last_clean":"2024-09-12T12:23:57.388590+0000","last_became_active":"2024-09-12T12:23:57.380630+0000","last_became_peered":"2024-09-12T12:23:57.380630+0000","last_unstale":"2024-09-12T12:23:57.388590+0000","last_undegraded":"2024-09-12T12:23:57.388590+0000","last_fullsized":"2024-09-12T12:23:57.388590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:25:32.135659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357304+0000","last_change":"2024-09-12T12:23:57.357304+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357304+0000","last_undegraded":"2024-09-12T12:23:57.357304+0000","last_fullsized":"2024-09-12T12:23:57.357304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.384198+0000","last_change":"2024-09-12T12:23:57.377798+0000","last_active":"2024-09-12T12:23:57.384198+0000","last_peered":"2024-09-12T12:23:57.384198+0000","last_clean":"2024-09-12T12:23:57.384198+0000","last_became_active":"2024-09-12T12:23:57.376811+0000","last_became_peered":"2024-09-12T12:23:57.376811+0000","last_unstale":"2024-09-12T12:23:57.384198+0000","last_undegraded":"2024-09-12T12:23:57.384198+0000","last_fullsized":"2024-09-12T12:23:57.384198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:05.313097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361216+0000","last_change":"2024-09-12T12:23:57.361216+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361216+0000","last_undegraded":"2024-09-12T12:23:57.361216+0000","last_fullsized":"2024-09-12T12:23:57.361216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.377291+0000","last_change":"2024-09-12T12:23:57.372742+0000","last_active":"2024-09-12T12:23:57.377291+0000","last_peered":"2024-09-12T12:23:57.377291+0000","last_clean":"2024-09-12T12:23:57.377291+0000","last_became_active":"2024-09-12T12:23:57.372354+0000","last_became_peered":"2024-09-12T12:23:57.372354+0000","last_unstale":"2024-09-12T12:23:57.377291+0000","last_undegraded":"2024-09-12T12:23:57.377291+0000","last_fullsized":"2024-09-12T12:23:57.377291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:04:30.827600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.368232+0000","last_change":"2024-09-12T12:23:57.368232+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.368232+0000","last_undegraded":"2024-09-12T12:23:57.368232+0000","last_fullsized":"2024-09-12T12:23:57.368232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.364221+0000","last_change":"2024-09-12T12:23:57.367013+0000","last_active":"2024-09-12T12:23:58.364221+0000","last_peered":"2024-09-12T12:23:58.364221+0000","last_clean":"2024-09-12T12:23:58.364221+0000","last_became_active":"2024-09-12T12:23:57.366614+0000","last_became_peered":"2024-09-12T12:23:57.366614+0000","last_unstale":"2024-09-12T12:23:58.364221+0000","last_undegraded":"2024-09-12T12:23:58.364221+0000","last_fullsized":"2024-09-12T12:23:58.364221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:16:37.971791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354943+0000","last_change":"2024-09-12T12:23:57.354943+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354943+0000","last_undegraded":"2024-09-12T12:23:57.354943+0000","last_fullsized":"2024-09-12T12:23:57.354943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387273+0000","last_change":"2024-09-12T12:23:57.385811+0000","last_active":"2024-09-12T12:23:57.387273+0000","last_peered":"2024-09-12T12:23:57.387273+0000","last_clean":"2024-09-12T12:23:57.387273+0000","last_became_active":"2024-09-12T12:23:57.384813+0000","last_became_peered":"2024-09-12T12:23:57.384813+0000","last_unstale":"2024-09-12T12:23:57.387273+0000","last_undegraded":"2024-09-12T12:23:57.387273+0000","last_fullsized":"2024-09-12T12:23:57.387273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:39.937210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355561+0000","last_change":"2024-09-12T12:23:57.355561+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355561+0000","last_undegraded":"2024-09-12T12:23:57.355561+0000","last_fullsized":"2024-09-12T12:23:57.355561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387439+0000","last_change":"2024-09-12T12:23:57.385905+0000","last_active":"2024-09-12T12:23:57.387439+0000","last_peered":"2024-09-12T12:23:57.387439+0000","last_clean":"2024-09-12T12:23:57.387439+0000","last_became_active":"2024-09-12T12:23:57.385555+0000","last_became_peered":"2024-09-12T12:23:57.385555+0000","last_unstale":"2024-09-12T12:23:57.387439+0000","last_undegraded":"2024-09-12T12:23:57.387439+0000","last_fullsized":"2024-09-12T12:23:57.387439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:53.460264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362662+0000","last_change":"2024-09-12T12:23:57.362662+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362662+0000","last_undegraded":"2024-09-12T12:23:57.362662+0000","last_fullsized":"2024-09-12T12:23:57.362662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395428+0000","last_change":"2024-09-12T12:23:57.390472+0000","last_active":"2024-09-12T12:23:57.395428+0000","last_peered":"2024-09-12T12:23:57.395428+0000","last_clean":"2024-09-12T12:23:57.395428+0000","last_became_active":"2024-09-12T12:23:57.389725+0000","last_became_peered":"2024-09-12T12:23:57.389725+0000","last_unstale":"2024-09-12T12:23:57.395428+0000","last_undegraded":"2024-09-12T12:23:57.395428+0000","last_fullsized":"2024-09-12T12:23:57.395428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:16.093565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.370155+0000","last_change":"2024-09-12T12:23:57.370155+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.370155+0000","last_undegraded":"2024-09-12T12:23:57.370155+0000","last_fullsized":"2024-09-12T12:23:57.370155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379930+0000","last_change":"2024-09-12T12:23:57.378862+0000","last_active":"2024-09-12T12:23:57.379930+0000","last_peered":"2024-09-12T12:23:57.379930+0000","last_clean":"2024-09-12T12:23:57.379930+0000","last_became_active":"2024-09-12T12:23:57.378463+0000","last_became_peered":"2024-09-12T12:23:57.378463+0000","last_unstale":"2024-09-12T12:23:57.379930+0000","last_undegraded":"2024-09-12T12:23:57.379930+0000","last_fullsized":"2024-09-12T12:23:57.379930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:07.040284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356236+0000","last_change":"2024-09-12T12:23:57.356236+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356236+0000","last_undegraded":"2024-09-12T12:23:57.356236+0000","last_fullsized":"2024-09-12T12:23:57.356236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379895+0000","last_change":"2024-09-12T12:23:57.377815+0000","last_active":"2024-09-12T12:23:57.379895+0000","last_peered":"2024-09-12T12:23:57.379895+0000","last_clean":"2024-09-12T12:23:57.379895+0000","last_became_active":"2024-09-12T12:23:57.377401+0000","last_became_peered":"2024-09-12T12:23:57.377401+0000","last_unstale":"2024-09-12T12:23:57.379895+0000","last_undegraded":"2024-09-12T12:23:57.379895+0000","last_fullsized":"2024-09-12T12:23:57.379895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:56.378083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355613+0000","last_change":"2024-09-12T12:23:57.355613+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355613+0000","last_undegraded":"2024-09-12T12:23:57.355613+0000","last_fullsized":"2024-09-12T12:23:57.355613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.403505+0000","last_change":"2024-09-12T12:23:57.367701+0000","last_active":"2024-09-12T12:23:58.403505+0000","last_peered":"2024-09-12T12:23:58.403505+0000","last_clean":"2024-09-12T12:23:58.403505+0000","last_became_active":"2024-09-12T12:23:57.367397+0000","last_became_peered":"2024-09-12T12:23:57.367397+0000","last_unstale":"2024-09-12T12:23:58.403505+0000","last_undegraded":"2024-09-12T12:23:58.403505+0000","last_fullsized":"2024-09-12T12:23:58.403505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:26:48.411535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358286+0000","last_change":"2024-09-12T12:23:57.358286+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358286+0000","last_undegraded":"2024-09-12T12:23:57.358286+0000","last_fullsized":"2024-09-12T12:23:57.358286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.368927+0000","last_change":"2024-09-12T12:23:57.365147+0000","last_active":"2024-09-12T12:23:57.368927+0000","last_peered":"2024-09-12T12:23:57.368927+0000","last_clean":"2024-09-12T12:23:57.368927+0000","last_became_active":"2024-09-12T12:23:57.364796+0000","last_became_peered":"2024-09-12T12:23:57.364796+0000","last_unstale":"2024-09-12T12:23:57.368927+0000","last_undegraded":"2024-09-12T12:23:57.368927+0000","last_fullsized":"2024-09-12T12:23:57.368927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:46:18.962036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354804+0000","last_change":"2024-09-12T12:23:57.354804+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354804+0000","last_undegraded":"2024-09-12T12:23:57.354804+0000","last_fullsized":"2024-09-12T12:23:57.354804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387960+0000","last_change":"2024-09-12T12:23:57.386864+0000","last_active":"2024-09-12T12:23:57.387960+0000","last_peered":"2024-09-12T12:23:57.387960+0000","last_clean":"2024-09-12T12:23:57.387960+0000","last_became_active":"2024-09-12T12:23:57.386550+0000","last_became_peered":"2024-09-12T12:23:57.386550+0000","last_unstale":"2024-09-12T12:23:57.387960+0000","last_undegraded":"2024-09-12T12:23:57.387960+0000","last_fullsized":"2024-09-12T12:23:57.387960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:03.518217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.364654+0000","last_change":"2024-09-12T12:23:57.364654+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.364654+0000","last_undegraded":"2024-09-12T12:23:57.364654+0000","last_fullsized":"2024-09-12T12:23:57.364654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.360414+0000","last_change":"2024-09-12T12:23:57.368319+0000","last_active":"2024-09-12T12:23:58.360414+0000","last_peered":"2024-09-12T12:23:58.360414+0000","last_clean":"2024-09-12T12:23:58.360414+0000","last_became_active":"2024-09-12T12:23:57.367912+0000","last_became_peered":"2024-09-12T12:23:57.367912+0000","last_unstale":"2024-09-12T12:23:58.360414+0000","last_undegraded":"2024-09-12T12:23:58.360414+0000","last_fullsized":"2024-09-12T12:23:58.360414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:35.720409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354991+0000","last_change":"2024-09-12T12:23:57.354991+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354991+0000","last_undegraded":"2024-09-12T12:23:57.354991+0000","last_fullsized":"2024-09-12T12:23:57.354991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387956+0000","last_change":"2024-09-12T12:23:57.386214+0000","last_active":"2024-09-12T12:23:57.387956+0000","last_peered":"2024-09-12T12:23:57.387956+0000","last_clean":"2024-09-12T12:23:57.387956+0000","last_became_active":"2024-09-12T12:23:57.385756+0000","last_became_peered":"2024-09-12T12:23:57.385756+0000","last_unstale":"2024-09-12T12:23:57.387956+0000","last_undegraded":"2024-09-12T12:23:57.387956+0000","last_fullsized":"2024-09-12T12:23:57.387956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:58.282124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.376205+0000","last_change":"2024-09-12T12:23:57.376205+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.376205+0000","last_undegraded":"2024-09-12T12:23:57.376205+0000","last_fullsized":"2024-09-12T12:23:57.376205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.376824+0000","last_change":"2024-09-12T12:23:57.369090+0000","last_active":"2024-09-12T12:23:57.376824+0000","last_peered":"2024-09-12T12:23:57.376824+0000","last_clean":"2024-09-12T12:23:57.376824+0000","last_became_active":"2024-09-12T12:23:57.368802+0000","last_became_peered":"2024-09-12T12:23:57.368802+0000","last_unstale":"2024-09-12T12:23:57.376824+0000","last_undegraded":"2024-09-12T12:23:57.376824+0000","last_fullsized":"2024-09-12T12:23:57.376824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:37:27.347542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362894+0000","last_change":"2024-09-12T12:23:57.362894+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362894+0000","last_undegraded":"2024-09-12T12:23:57.362894+0000","last_fullsized":"2024-09-12T12:23:57.362894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388670+0000","last_change":"2024-09-12T12:23:57.382433+0000","last_active":"2024-09-12T12:23:57.388670+0000","last_peered":"2024-09-12T12:23:57.388670+0000","last_clean":"2024-09-12T12:23:57.388670+0000","last_became_active":"2024-09-12T12:23:57.381303+0000","last_became_peered":"2024-09-12T12:23:57.381303+0000","last_unstale":"2024-09-12T12:23:57.388670+0000","last_undegraded":"2024-09-12T12:23:57.388670+0000","last_fullsized":"2024-09-12T12:23:57.388670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:38:15.474579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355099+0000","last_change":"2024-09-12T12:23:57.355099+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355099+0000","last_undegraded":"2024-09-12T12:23:57.355099+0000","last_fullsized":"2024-09-12T12:23:57.355099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388602+0000","last_change":"2024-09-12T12:23:57.382205+0000","last_active":"2024-09-12T12:23:57.388602+0000","last_peered":"2024-09-12T12:23:57.388602+0000","last_clean":"2024-09-12T12:23:57.388602+0000","last_became_active":"2024-09-12T12:23:57.381644+0000","last_became_peered":"2024-09-12T12:23:57.381644+0000","last_unstale":"2024-09-12T12:23:57.388602+0000","last_undegraded":"2024-09-12T12:23:57.388602+0000","last_fullsized":"2024-09-12T12:23:57.388602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:41.425596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357647+0000","last_change":"2024-09-12T12:23:57.357647+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357647+0000","last_undegraded":"2024-09-12T12:23:57.357647+0000","last_fullsized":"2024-09-12T12:23:57.357647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.382580+0000","last_change":"2024-09-12T12:23:57.381829+0000","last_active":"2024-09-12T12:23:57.382580+0000","last_peered":"2024-09-12T12:23:57.382580+0000","last_clean":"2024-09-12T12:23:57.382580+0000","last_became_active":"2024-09-12T12:23:57.381422+0000","last_became_peered":"2024-09-12T12:23:57.381422+0000","last_unstale":"2024-09-12T12:23:57.382580+0000","last_undegraded":"2024-09-12T12:23:57.382580+0000","last_fullsized":"2024-09-12T12:23:57.382580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:12.617164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361245+0000","last_change":"2024-09-12T12:23:57.361245+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361245+0000","last_undegraded":"2024-09-12T12:23:57.361245+0000","last_fullsized":"2024-09-12T12:23:57.361245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375315+0000","last_change":"2024-09-12T12:23:57.366499+0000","last_active":"2024-09-12T12:23:57.375315+0000","last_peered":"2024-09-12T12:23:57.375315+0000","last_clean":"2024-09-12T12:23:57.375315+0000","last_became_active":"2024-09-12T12:23:57.364717+0000","last_became_peered":"2024-09-12T12:23:57.364717+0000","last_unstale":"2024-09-12T12:23:57.375315+0000","last_undegraded":"2024-09-12T12:23:57.375315+0000","last_fullsized":"2024-09-12T12:23:57.375315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:16.093565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371930+0000","last_change":"2024-09-12T12:23:57.371930+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371930+0000","last_undegraded":"2024-09-12T12:23:57.371930+0000","last_fullsized":"2024-09-12T12:23:57.371930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.392519+0000","last_change":"2024-09-12T12:23:57.388887+0000","last_active":"2024-09-12T12:23:57.392519+0000","last_peered":"2024-09-12T12:23:57.392519+0000","last_clean":"2024-09-12T12:23:57.392519+0000","last_became_active":"2024-09-12T12:23:57.388290+0000","last_became_peered":"2024-09-12T12:23:57.388290+0000","last_unstale":"2024-09-12T12:23:57.392519+0000","last_undegraded":"2024-09-12T12:23:57.392519+0000","last_fullsized":"2024-09-12T12:23:57.392519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:19:31.787885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.363040+0000","last_change":"2024-09-12T12:23:57.363040+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.363040+0000","last_undegraded":"2024-09-12T12:23:57.363040+0000","last_fullsized":"2024-09-12T12:23:57.363040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.394938+0000","last_change":"2024-09-12T12:23:57.390177+0000","last_active":"2024-09-12T12:23:57.394938+0000","last_peered":"2024-09-12T12:23:57.394938+0000","last_clean":"2024-09-12T12:23:57.394938+0000","last_became_active":"2024-09-12T12:23:57.389410+0000","last_became_peered":"2024-09-12T12:23:57.389410+0000","last_unstale":"2024-09-12T12:23:57.394938+0000","last_undegraded":"2024-09-12T12:23:57.394938+0000","last_fullsized":"2024-09-12T12:23:57.394938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:21.483924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354931+0000","last_change":"2024-09-12T12:23:57.354931+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354931+0000","last_undegraded":"2024-09-12T12:23:57.354931+0000","last_fullsized":"2024-09-12T12:23:57.354931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.390536+0000","last_change":"2024-09-12T12:23:57.386525+0000","last_active":"2024-09-12T12:23:57.390536+0000","last_peered":"2024-09-12T12:23:57.390536+0000","last_clean":"2024-09-12T12:23:57.390536+0000","last_became_active":"2024-09-12T12:23:57.385710+0000","last_became_peered":"2024-09-12T12:23:57.385710+0000","last_unstale":"2024-09-12T12:23:57.390536+0000","last_undegraded":"2024-09-12T12:23:57.390536+0000","last_fullsized":"2024-09-12T12:23:57.390536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:11.831899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":273,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.385148+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:57.385148+0000","last_peered":"2024-09-12T12:23:57.385148+0000","last_clean":"2024-09-12T12:23:57.385148+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:57.385148+0000","last_undegraded":"2024-09-12T12:23:57.385148+0000","last_fullsized":"2024-09-12T12:23:57.385148+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":966656,"data_stored":404023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542200,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1523712,"data_stored":979729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":395636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":395636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1536000,"data_stored":986004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:02.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:02.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f063d9c0640 1 Processor -- start 2024-09-12T12:24:02.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f063d9c0640 1 -- start start 2024-09-12T12:24:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f063d9c0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 0x7f063806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f063d9c0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06380b3280 con 0x7f06380add30 2024-09-12T12:24:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f063d9c0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06380b3560 con 0x7f063806e870 2024-09-12T12:24:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f063d9c0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06380b3840 con 0x7f06380ae110 2024-09-12T12:24:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f06380add30 0x7f0638070d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38812/0 (socket says 172.21.15.3:38812) 2024-09-12T12:24:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 -- 172.21.15.3:0/199889937 learned_addr learned my addr 172.21.15.3:0/199889937 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:02.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f06367fc640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 0x7f063806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865528210 0 0) 0x7f06380b3280 con 0x7f06380add30 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0620003450 con 0x7f06380add30 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 94122505 0 0) 0x7f0620003450 con 0x7f06380add30 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 msgr2=0x7f063806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 0x7f063806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 >> v1:172.21.15.142:6789/0 conn(0x7f063806e870 legacy=0x7f063815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06380b3b20 con 0x7f06380add30 2024-09-12T12:24:02.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f0628001a00 con 0x7f06380add30 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 >> v1:172.21.15.3:6789/0 conn(0x7f06380add30 legacy=0x7f0638070d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f0620003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 0x7f062000a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062000aff0 0x7f062000d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06380b3840 con 0x7f0620003ae0 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f062000dc90 con 0x7f062000aff0 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f062000de90 con 0x7f06380ae110 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f0620003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f06367fc640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 0x7f062000a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f06377fe640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062000aff0 0x7f062000d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 -- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 msgr2=0x7f062000a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06380ae110 0x7f062000a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 -- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062000aff0 msgr2=0x7f062000d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062000aff0 0x7f062000d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.968+0000 7f0636ffd640 1 -- 172.21.15.3:0/199889937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620003450 con 0x7f0620003ae0 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f0636ffd640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f0620003f00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f062800a860 tx=0x7f062801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e849cf92ebd2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062800a480 con 0x7f0620003ae0 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063815aef0 con 0x7f0620003ae0 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f0635ffb640 1 -- 172.21.15.3:0/199889937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06280ad890 con 0x7f0620003ae0 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f063d9c0640 1 -- 172.21.15.3:0/199889937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 msgr2=0x7f0620003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f063d9c0640 1 --2- 172.21.15.3:0/199889937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f0620003f00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f062800a860 tx=0x7f062801d380 comp rx=0 tx=0).stop 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f06377fe640 1 -- 172.21.15.3:0/199889937 reap_dead start 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f063d9c0640 1 -- 172.21.15.3:0/199889937 shutdown_connections 2024-09-12T12:24:02.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f063d9c0640 1 -- 172.21.15.3:0/199889937 >> 172.21.15.3:0/199889937 conn(0x7f0638067e00 msgr2=0x7f0638068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f063d9c0640 1 -- 172.21.15.3:0/199889937 shutdown_connections 2024-09-12T12:24:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.972+0000 7f063d9c0640 1 -- 172.21.15.3:0/199889937 wait complete. 2024-09-12T12:24:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 Processor -- start 2024-09-12T12:24:02.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 -- start start 2024-09-12T12:24:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f063814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f063815f4b0 0x7f0638160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638071d80 0x7f0638072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06380b4470 con 0x7f0620003ae0 2024-09-12T12:24:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06380b42f0 con 0x7f0638071d80 2024-09-12T12:24:02.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06380b45f0 con 0x7f063815f4b0 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f063814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f063814b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60628/0 (socket says 172.21.15.3:60628) 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 -- 172.21.15.3:0/4115330428 learned_addr learned my addr 172.21.15.3:0/4115330428 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f06377fe640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638071d80 0x7f0638072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f06367fc640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f063815f4b0 0x7f0638160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 -- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f063815f4b0 msgr2=0x7f0638160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f063815f4b0 0x7f0638160ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 -- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638071d80 msgr2=0x7f0638072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638071d80 0x7f0638072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:02.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 -- 172.21.15.3:0/4115330428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063815fa20 con 0x7f0620003ae0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f0636ffd640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f063814b560 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f062801d1b0 tx=0x7f0620003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f061ffff640 1 -- 172.21.15.3:0/4115330428 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06280bd070 con 0x7f0620003ae0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f061ffff640 1 -- 172.21.15.3:0/4115330428 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06280019c0 con 0x7f0620003ae0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063815fd00 con 0x7f0620003ae0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f061ffff640 1 -- 172.21.15.3:0/4115330428 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06280b2040 con 0x7f0620003ae0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381602c0 con 0x7f0620003ae0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0638072740 con 0x7f0620003ae0 2024-09-12T12:24:02.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f061ffff640 1 -- 172.21.15.3:0/4115330428 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06280c0070 con 0x7f0620003ae0 2024-09-12T12:24:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f061ffff640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06240678f0 0x7f0624069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.976+0000 7f061ffff640 1 -- 172.21.15.3:0/4115330428 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f0628120320 con 0x7f0620003ae0 2024-09-12T12:24:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.980+0000 7f06367fc640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06240678f0 0x7f0624069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:02.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.980+0000 7f06367fc640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06240678f0 0x7f0624069db0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06380c5390 tx=0x7f062c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:02.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:02.984+0000 7f061ffff640 1 -- 172.21.15.3:0/4115330428 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0628017090 con 0x7f0620003ae0 2024-09-12T12:24:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.140+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f063815aef0 con 0x7f06240678f0 2024-09-12T12:24:03.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.160+0000 7f061ffff640 1 -- 172.21.15.3:0/4115330428 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247011 (secure 0 0 0) 0x7f063815aef0 con 0x7f06240678f0 2024-09-12T12:24:03.165 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:03.166 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06240678f0 msgr2=0x7f0624069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06240678f0 0x7f0624069db0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06380c5390 tx=0x7f062c008040 comp rx=0 tx=0).stop 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 msgr2=0x7f063814b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f063814b560 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f062801d1b0 tx=0x7f0620003740 comp rx=0 tx=0).stop 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 shutdown_connections 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06240678f0 0x7f0624069db0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0638071d80 0x7f0638072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f063815f4b0 0x7f0638160ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 --2- 172.21.15.3:0/4115330428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0620003ae0 0x7f063814b560 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 >> 172.21.15.3:0/4115330428 conn(0x7f0638067e00 msgr2=0x7f0638159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:03.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 shutdown_connections 2024-09-12T12:24:03.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.164+0000 7f063d9c0640 1 -- 172.21.15.3:0/4115330428 wait complete. 2024-09-12T12:24:03.183 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-12T12:24:01.742076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241412,"kb_used_data":8496,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733308,"statfs":{"total":773094113280,"available":772846907392,"internally_reserved":0,"allocated":8699904,"data_stored":4329111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.404559"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372179+0000","last_change":"2024-09-12T12:23:57.372179+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372179+0000","last_undegraded":"2024-09-12T12:23:57.372179+0000","last_fullsized":"2024-09-12T12:23:57.372179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358274+0000","last_change":"2024-09-12T12:23:57.358274+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358274+0000","last_undegraded":"2024-09-12T12:23:57.358274+0000","last_fullsized":"2024-09-12T12:23:57.358274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367757+0000","last_change":"2024-09-12T12:23:57.367757+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367757+0000","last_undegraded":"2024-09-12T12:23:57.367757+0000","last_fullsized":"2024-09-12T12:23:57.367757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357518+0000","last_change":"2024-09-12T12:23:57.357518+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357518+0000","last_undegraded":"2024-09-12T12:23:57.357518+0000","last_fullsized":"2024-09-12T12:23:57.357518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367639+0000","last_change":"2024-09-12T12:23:57.367639+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367639+0000","last_undegraded":"2024-09-12T12:23:57.367639+0000","last_fullsized":"2024-09-12T12:23:57.367639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358945+0000","last_change":"2024-09-12T12:23:57.358945+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358945+0000","last_undegraded":"2024-09-12T12:23:57.358945+0000","last_fullsized":"2024-09-12T12:23:57.358945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358058+0000","last_change":"2024-09-12T12:23:57.358058+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358058+0000","last_undegraded":"2024-09-12T12:23:57.358058+0000","last_fullsized":"2024-09-12T12:23:57.358058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357626+0000","last_change":"2024-09-12T12:23:57.357626+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357626+0000","last_undegraded":"2024-09-12T12:23:57.357626+0000","last_fullsized":"2024-09-12T12:23:57.357626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.368766+0000","last_change":"2024-09-12T12:23:57.368766+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.368766+0000","last_undegraded":"2024-09-12T12:23:57.368766+0000","last_fullsized":"2024-09-12T12:23:57.368766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.363177+0000","last_change":"2024-09-12T12:23:57.363177+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.363177+0000","last_undegraded":"2024-09-12T12:23:57.363177+0000","last_fullsized":"2024-09-12T12:23:57.363177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358093+0000","last_change":"2024-09-12T12:23:57.358093+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358093+0000","last_undegraded":"2024-09-12T12:23:57.358093+0000","last_fullsized":"2024-09-12T12:23:57.358093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372464+0000","last_change":"2024-09-12T12:23:57.372464+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372464+0000","last_undegraded":"2024-09-12T12:23:57.372464+0000","last_fullsized":"2024-09-12T12:23:57.372464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.370220+0000","last_change":"2024-09-12T12:23:57.370220+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.370220+0000","last_undegraded":"2024-09-12T12:23:57.370220+0000","last_fullsized":"2024-09-12T12:23:57.370220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358797+0000","last_change":"2024-09-12T12:23:57.358797+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358797+0000","last_undegraded":"2024-09-12T12:23:57.358797+0000","last_fullsized":"2024-09-12T12:23:57.358797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.390665+0000","last_change":"2024-09-12T12:23:58.386557+0000","last_active":"2024-09-12T12:23:58.390665+0000","last_peered":"2024-09-12T12:23:58.390665+0000","last_clean":"2024-09-12T12:23:58.390665+0000","last_became_active":"2024-09-12T12:23:58.386226+0000","last_became_peered":"2024-09-12T12:23:58.386226+0000","last_unstale":"2024-09-12T12:23:58.390665+0000","last_undegraded":"2024-09-12T12:23:58.390665+0000","last_fullsized":"2024-09-12T12:23:58.390665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:52:12.537859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.367681+0000","last_change":"2024-09-12T12:23:57.367681+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.367681+0000","last_undegraded":"2024-09-12T12:23:57.367681+0000","last_fullsized":"2024-09-12T12:23:57.367681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362775+0000","last_change":"2024-09-12T12:23:57.362775+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362775+0000","last_undegraded":"2024-09-12T12:23:57.362775+0000","last_fullsized":"2024-09-12T12:23:57.362775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355804+0000","last_change":"2024-09-12T12:23:57.355804+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355804+0000","last_undegraded":"2024-09-12T12:23:57.355804+0000","last_fullsized":"2024-09-12T12:23:57.355804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362500+0000","last_change":"2024-09-12T12:23:57.362500+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362500+0000","last_undegraded":"2024-09-12T12:23:57.362500+0000","last_fullsized":"2024-09-12T12:23:57.362500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355757+0000","last_change":"2024-09-12T12:23:57.355757+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355757+0000","last_undegraded":"2024-09-12T12:23:57.355757+0000","last_fullsized":"2024-09-12T12:23:57.355757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356788+0000","last_change":"2024-09-12T12:23:57.356788+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356788+0000","last_undegraded":"2024-09-12T12:23:57.356788+0000","last_fullsized":"2024-09-12T12:23:57.356788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358620+0000","last_change":"2024-09-12T12:23:57.358620+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358620+0000","last_undegraded":"2024-09-12T12:23:57.358620+0000","last_fullsized":"2024-09-12T12:23:57.358620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.365333+0000","last_change":"2024-09-12T12:23:57.365333+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.365333+0000","last_undegraded":"2024-09-12T12:23:57.365333+0000","last_fullsized":"2024-09-12T12:23:57.365333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356474+0000","last_change":"2024-09-12T12:23:57.356474+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356474+0000","last_undegraded":"2024-09-12T12:23:57.356474+0000","last_fullsized":"2024-09-12T12:23:57.356474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357760+0000","last_change":"2024-09-12T12:23:57.357760+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357760+0000","last_undegraded":"2024-09-12T12:23:57.357760+0000","last_fullsized":"2024-09-12T12:23:57.357760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371939+0000","last_change":"2024-09-12T12:23:57.371939+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371939+0000","last_undegraded":"2024-09-12T12:23:57.371939+0000","last_fullsized":"2024-09-12T12:23:57.371939+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361557+0000","last_change":"2024-09-12T12:23:57.361557+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361557+0000","last_undegraded":"2024-09-12T12:23:57.361557+0000","last_fullsized":"2024-09-12T12:23:57.361557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.369582+0000","last_change":"2024-09-12T12:23:57.369582+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.369582+0000","last_undegraded":"2024-09-12T12:23:57.369582+0000","last_fullsized":"2024-09-12T12:23:57.369582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357253+0000","last_change":"2024-09-12T12:23:57.357253+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357253+0000","last_undegraded":"2024-09-12T12:23:57.357253+0000","last_fullsized":"2024-09-12T12:23:57.357253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356976+0000","last_change":"2024-09-12T12:23:57.356976+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356976+0000","last_undegraded":"2024-09-12T12:23:57.356976+0000","last_fullsized":"2024-09-12T12:23:57.356976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360481+0000","last_change":"2024-09-12T12:23:57.360481+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360481+0000","last_undegraded":"2024-09-12T12:23:57.360481+0000","last_fullsized":"2024-09-12T12:23:57.360481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.366248+0000","last_change":"2024-09-12T12:23:57.366248+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.366248+0000","last_undegraded":"2024-09-12T12:23:57.366248+0000","last_fullsized":"2024-09-12T12:23:57.366248+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360338+0000","last_change":"2024-09-12T12:23:57.360338+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360338+0000","last_undegraded":"2024-09-12T12:23:57.360338+0000","last_fullsized":"2024-09-12T12:23:57.360338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395539+0000","last_change":"2024-09-12T12:23:57.391198+0000","last_active":"2024-09-12T12:23:57.395539+0000","last_peered":"2024-09-12T12:23:57.395539+0000","last_clean":"2024-09-12T12:23:57.395539+0000","last_became_active":"2024-09-12T12:23:57.390552+0000","last_became_peered":"2024-09-12T12:23:57.390552+0000","last_unstale":"2024-09-12T12:23:57.395539+0000","last_undegraded":"2024-09-12T12:23:57.395539+0000","last_fullsized":"2024-09-12T12:23:57.395539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:37.934018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371198+0000","last_change":"2024-09-12T12:23:57.371198+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371198+0000","last_undegraded":"2024-09-12T12:23:57.371198+0000","last_fullsized":"2024-09-12T12:23:57.371198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375369+0000","last_change":"2024-09-12T12:23:57.367632+0000","last_active":"2024-09-12T12:23:57.375369+0000","last_peered":"2024-09-12T12:23:57.375369+0000","last_clean":"2024-09-12T12:23:57.375369+0000","last_became_active":"2024-09-12T12:23:57.366903+0000","last_became_peered":"2024-09-12T12:23:57.366903+0000","last_unstale":"2024-09-12T12:23:57.375369+0000","last_undegraded":"2024-09-12T12:23:57.375369+0000","last_fullsized":"2024-09-12T12:23:57.375369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:37.934018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.364066+0000","last_change":"2024-09-12T12:23:57.364066+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.364066+0000","last_undegraded":"2024-09-12T12:23:57.364066+0000","last_fullsized":"2024-09-12T12:23:57.364066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.374007+0000","last_change":"2024-09-12T12:23:57.368053+0000","last_active":"2024-09-12T12:23:57.374007+0000","last_peered":"2024-09-12T12:23:57.374007+0000","last_clean":"2024-09-12T12:23:57.374007+0000","last_became_active":"2024-09-12T12:23:57.367668+0000","last_became_peered":"2024-09-12T12:23:57.367668+0000","last_unstale":"2024-09-12T12:23:57.374007+0000","last_undegraded":"2024-09-12T12:23:57.374007+0000","last_fullsized":"2024-09-12T12:23:57.374007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:10:36.243189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362837+0000","last_change":"2024-09-12T12:23:57.362837+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362837+0000","last_undegraded":"2024-09-12T12:23:57.362837+0000","last_fullsized":"2024-09-12T12:23:57.362837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.374051+0000","last_change":"2024-09-12T12:23:57.368958+0000","last_active":"2024-09-12T12:23:57.374051+0000","last_peered":"2024-09-12T12:23:57.374051+0000","last_clean":"2024-09-12T12:23:57.374051+0000","last_became_active":"2024-09-12T12:23:57.368562+0000","last_became_peered":"2024-09-12T12:23:57.368562+0000","last_unstale":"2024-09-12T12:23:57.374051+0000","last_undegraded":"2024-09-12T12:23:57.374051+0000","last_fullsized":"2024-09-12T12:23:57.374051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:37:27.794694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356053+0000","last_change":"2024-09-12T12:23:57.356053+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356053+0000","last_undegraded":"2024-09-12T12:23:57.356053+0000","last_fullsized":"2024-09-12T12:23:57.356053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379835+0000","last_change":"2024-09-12T12:23:57.371998+0000","last_active":"2024-09-12T12:23:57.379835+0000","last_peered":"2024-09-12T12:23:57.379835+0000","last_clean":"2024-09-12T12:23:57.379835+0000","last_became_active":"2024-09-12T12:23:57.371152+0000","last_became_peered":"2024-09-12T12:23:57.371152+0000","last_unstale":"2024-09-12T12:23:57.379835+0000","last_undegraded":"2024-09-12T12:23:57.379835+0000","last_fullsized":"2024-09-12T12:23:57.379835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:50.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358969+0000","last_change":"2024-09-12T12:23:57.358969+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358969+0000","last_undegraded":"2024-09-12T12:23:57.358969+0000","last_fullsized":"2024-09-12T12:23:57.358969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375373+0000","last_change":"2024-09-12T12:23:57.366528+0000","last_active":"2024-09-12T12:23:57.375373+0000","last_peered":"2024-09-12T12:23:57.375373+0000","last_clean":"2024-09-12T12:23:57.375373+0000","last_became_active":"2024-09-12T12:23:57.365702+0000","last_became_peered":"2024-09-12T12:23:57.365702+0000","last_unstale":"2024-09-12T12:23:57.375373+0000","last_undegraded":"2024-09-12T12:23:57.375373+0000","last_fullsized":"2024-09-12T12:23:57.375373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:32.123568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.372956+0000","last_change":"2024-09-12T12:23:57.372956+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.372956+0000","last_undegraded":"2024-09-12T12:23:57.372956+0000","last_fullsized":"2024-09-12T12:23:57.372956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.377034+0000","last_change":"2024-09-12T12:23:57.375695+0000","last_active":"2024-09-12T12:23:57.377034+0000","last_peered":"2024-09-12T12:23:57.377034+0000","last_clean":"2024-09-12T12:23:57.377034+0000","last_became_active":"2024-09-12T12:23:57.375373+0000","last_became_peered":"2024-09-12T12:23:57.375373+0000","last_unstale":"2024-09-12T12:23:57.377034+0000","last_undegraded":"2024-09-12T12:23:57.377034+0000","last_fullsized":"2024-09-12T12:23:57.377034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:17:18.222157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.382382+0000","last_change":"2024-09-12T12:23:58.377377+0000","last_active":"2024-09-12T12:23:58.382382+0000","last_peered":"2024-09-12T12:23:58.382382+0000","last_clean":"2024-09-12T12:23:58.382382+0000","last_became_active":"2024-09-12T12:23:58.374901+0000","last_became_peered":"2024-09-12T12:23:58.374901+0000","last_unstale":"2024-09-12T12:23:58.382382+0000","last_undegraded":"2024-09-12T12:23:58.382382+0000","last_fullsized":"2024-09-12T12:23:58.382382+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:05.911408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.376659+0000","last_change":"2024-09-12T12:23:57.370326+0000","last_active":"2024-09-12T12:23:57.376659+0000","last_peered":"2024-09-12T12:23:57.376659+0000","last_clean":"2024-09-12T12:23:57.376659+0000","last_became_active":"2024-09-12T12:23:57.369836+0000","last_became_peered":"2024-09-12T12:23:57.369836+0000","last_unstale":"2024-09-12T12:23:57.376659+0000","last_undegraded":"2024-09-12T12:23:57.376659+0000","last_fullsized":"2024-09-12T12:23:57.376659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:34:14.111064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.400544+0000","last_change":"2024-09-12T12:23:58.395462+0000","last_active":"2024-09-12T12:23:58.400544+0000","last_peered":"2024-09-12T12:23:58.400544+0000","last_clean":"2024-09-12T12:23:58.400544+0000","last_became_active":"2024-09-12T12:23:58.395205+0000","last_became_peered":"2024-09-12T12:23:58.395205+0000","last_unstale":"2024-09-12T12:23:58.400544+0000","last_undegraded":"2024-09-12T12:23:58.400544+0000","last_fullsized":"2024-09-12T12:23:58.400544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:22.945497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395479+0000","last_change":"2024-09-12T12:23:57.390954+0000","last_active":"2024-09-12T12:23:57.395479+0000","last_peered":"2024-09-12T12:23:57.395479+0000","last_clean":"2024-09-12T12:23:57.395479+0000","last_became_active":"2024-09-12T12:23:57.389910+0000","last_became_peered":"2024-09-12T12:23:57.389910+0000","last_unstale":"2024-09-12T12:23:57.395479+0000","last_undegraded":"2024-09-12T12:23:57.395479+0000","last_fullsized":"2024-09-12T12:23:57.395479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:32.123568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.360621+0000","last_change":"2024-09-12T12:23:57.360621+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.360621+0000","last_undegraded":"2024-09-12T12:23:57.360621+0000","last_fullsized":"2024-09-12T12:23:57.360621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.389643+0000","last_change":"2024-09-12T12:23:57.387882+0000","last_active":"2024-09-12T12:23:57.389643+0000","last_peered":"2024-09-12T12:23:57.389643+0000","last_clean":"2024-09-12T12:23:57.389643+0000","last_became_active":"2024-09-12T12:23:57.387183+0000","last_became_peered":"2024-09-12T12:23:57.387183+0000","last_unstale":"2024-09-12T12:23:57.389643+0000","last_undegraded":"2024-09-12T12:23:57.389643+0000","last_fullsized":"2024-09-12T12:23:57.389643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:21:38.887300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361801+0000","last_change":"2024-09-12T12:23:57.361801+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361801+0000","last_undegraded":"2024-09-12T12:23:57.361801+0000","last_fullsized":"2024-09-12T12:23:57.361801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388590+0000","last_change":"2024-09-12T12:23:57.381940+0000","last_active":"2024-09-12T12:23:57.388590+0000","last_peered":"2024-09-12T12:23:57.388590+0000","last_clean":"2024-09-12T12:23:57.388590+0000","last_became_active":"2024-09-12T12:23:57.380630+0000","last_became_peered":"2024-09-12T12:23:57.380630+0000","last_unstale":"2024-09-12T12:23:57.388590+0000","last_undegraded":"2024-09-12T12:23:57.388590+0000","last_fullsized":"2024-09-12T12:23:57.388590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:25:32.135659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357304+0000","last_change":"2024-09-12T12:23:57.357304+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357304+0000","last_undegraded":"2024-09-12T12:23:57.357304+0000","last_fullsized":"2024-09-12T12:23:57.357304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.384198+0000","last_change":"2024-09-12T12:23:57.377798+0000","last_active":"2024-09-12T12:23:57.384198+0000","last_peered":"2024-09-12T12:23:57.384198+0000","last_clean":"2024-09-12T12:23:57.384198+0000","last_became_active":"2024-09-12T12:23:57.376811+0000","last_became_peered":"2024-09-12T12:23:57.376811+0000","last_unstale":"2024-09-12T12:23:57.384198+0000","last_undegraded":"2024-09-12T12:23:57.384198+0000","last_fullsized":"2024-09-12T12:23:57.384198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:05.313097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361216+0000","last_change":"2024-09-12T12:23:57.361216+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361216+0000","last_undegraded":"2024-09-12T12:23:57.361216+0000","last_fullsized":"2024-09-12T12:23:57.361216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.377291+0000","last_change":"2024-09-12T12:23:57.372742+0000","last_active":"2024-09-12T12:23:57.377291+0000","last_peered":"2024-09-12T12:23:57.377291+0000","last_clean":"2024-09-12T12:23:57.377291+0000","last_became_active":"2024-09-12T12:23:57.372354+0000","last_became_peered":"2024-09-12T12:23:57.372354+0000","last_unstale":"2024-09-12T12:23:57.377291+0000","last_undegraded":"2024-09-12T12:23:57.377291+0000","last_fullsized":"2024-09-12T12:23:57.377291+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:04:30.827600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.368232+0000","last_change":"2024-09-12T12:23:57.368232+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.368232+0000","last_undegraded":"2024-09-12T12:23:57.368232+0000","last_fullsized":"2024-09-12T12:23:57.368232+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.364221+0000","last_change":"2024-09-12T12:23:57.367013+0000","last_active":"2024-09-12T12:23:58.364221+0000","last_peered":"2024-09-12T12:23:58.364221+0000","last_clean":"2024-09-12T12:23:58.364221+0000","last_became_active":"2024-09-12T12:23:57.366614+0000","last_became_peered":"2024-09-12T12:23:57.366614+0000","last_unstale":"2024-09-12T12:23:58.364221+0000","last_undegraded":"2024-09-12T12:23:58.364221+0000","last_fullsized":"2024-09-12T12:23:58.364221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:16:37.971791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354943+0000","last_change":"2024-09-12T12:23:57.354943+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354943+0000","last_undegraded":"2024-09-12T12:23:57.354943+0000","last_fullsized":"2024-09-12T12:23:57.354943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387273+0000","last_change":"2024-09-12T12:23:57.385811+0000","last_active":"2024-09-12T12:23:57.387273+0000","last_peered":"2024-09-12T12:23:57.387273+0000","last_clean":"2024-09-12T12:23:57.387273+0000","last_became_active":"2024-09-12T12:23:57.384813+0000","last_became_peered":"2024-09-12T12:23:57.384813+0000","last_unstale":"2024-09-12T12:23:57.387273+0000","last_undegraded":"2024-09-12T12:23:57.387273+0000","last_fullsized":"2024-09-12T12:23:57.387273+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:39.937210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355561+0000","last_change":"2024-09-12T12:23:57.355561+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355561+0000","last_undegraded":"2024-09-12T12:23:57.355561+0000","last_fullsized":"2024-09-12T12:23:57.355561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387439+0000","last_change":"2024-09-12T12:23:57.385905+0000","last_active":"2024-09-12T12:23:57.387439+0000","last_peered":"2024-09-12T12:23:57.387439+0000","last_clean":"2024-09-12T12:23:57.387439+0000","last_became_active":"2024-09-12T12:23:57.385555+0000","last_became_peered":"2024-09-12T12:23:57.385555+0000","last_unstale":"2024-09-12T12:23:57.387439+0000","last_undegraded":"2024-09-12T12:23:57.387439+0000","last_fullsized":"2024-09-12T12:23:57.387439+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:53.460264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362662+0000","last_change":"2024-09-12T12:23:57.362662+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362662+0000","last_undegraded":"2024-09-12T12:23:57.362662+0000","last_fullsized":"2024-09-12T12:23:57.362662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.395428+0000","last_change":"2024-09-12T12:23:57.390472+0000","last_active":"2024-09-12T12:23:57.395428+0000","last_peered":"2024-09-12T12:23:57.395428+0000","last_clean":"2024-09-12T12:23:57.395428+0000","last_became_active":"2024-09-12T12:23:57.389725+0000","last_became_peered":"2024-09-12T12:23:57.389725+0000","last_unstale":"2024-09-12T12:23:57.395428+0000","last_undegraded":"2024-09-12T12:23:57.395428+0000","last_fullsized":"2024-09-12T12:23:57.395428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:16.093565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.370155+0000","last_change":"2024-09-12T12:23:57.370155+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.370155+0000","last_undegraded":"2024-09-12T12:23:57.370155+0000","last_fullsized":"2024-09-12T12:23:57.370155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379930+0000","last_change":"2024-09-12T12:23:57.378862+0000","last_active":"2024-09-12T12:23:57.379930+0000","last_peered":"2024-09-12T12:23:57.379930+0000","last_clean":"2024-09-12T12:23:57.379930+0000","last_became_active":"2024-09-12T12:23:57.378463+0000","last_became_peered":"2024-09-12T12:23:57.378463+0000","last_unstale":"2024-09-12T12:23:57.379930+0000","last_undegraded":"2024-09-12T12:23:57.379930+0000","last_fullsized":"2024-09-12T12:23:57.379930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:45:07.040284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.356236+0000","last_change":"2024-09-12T12:23:57.356236+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.356236+0000","last_undegraded":"2024-09-12T12:23:57.356236+0000","last_fullsized":"2024-09-12T12:23:57.356236+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.379895+0000","last_change":"2024-09-12T12:23:57.377815+0000","last_active":"2024-09-12T12:23:57.379895+0000","last_peered":"2024-09-12T12:23:57.379895+0000","last_clean":"2024-09-12T12:23:57.379895+0000","last_became_active":"2024-09-12T12:23:57.377401+0000","last_became_peered":"2024-09-12T12:23:57.377401+0000","last_unstale":"2024-09-12T12:23:57.379895+0000","last_undegraded":"2024-09-12T12:23:57.379895+0000","last_fullsized":"2024-09-12T12:23:57.379895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:56.378083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355613+0000","last_change":"2024-09-12T12:23:57.355613+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355613+0000","last_undegraded":"2024-09-12T12:23:57.355613+0000","last_fullsized":"2024-09-12T12:23:57.355613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.403505+0000","last_change":"2024-09-12T12:23:57.367701+0000","last_active":"2024-09-12T12:23:58.403505+0000","last_peered":"2024-09-12T12:23:58.403505+0000","last_clean":"2024-09-12T12:23:58.403505+0000","last_became_active":"2024-09-12T12:23:57.367397+0000","last_became_peered":"2024-09-12T12:23:57.367397+0000","last_unstale":"2024-09-12T12:23:58.403505+0000","last_undegraded":"2024-09-12T12:23:58.403505+0000","last_fullsized":"2024-09-12T12:23:58.403505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:26:48.411535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.358286+0000","last_change":"2024-09-12T12:23:57.358286+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.358286+0000","last_undegraded":"2024-09-12T12:23:57.358286+0000","last_fullsized":"2024-09-12T12:23:57.358286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.368927+0000","last_change":"2024-09-12T12:23:57.365147+0000","last_active":"2024-09-12T12:23:57.368927+0000","last_peered":"2024-09-12T12:23:57.368927+0000","last_clean":"2024-09-12T12:23:57.368927+0000","last_became_active":"2024-09-12T12:23:57.364796+0000","last_became_peered":"2024-09-12T12:23:57.364796+0000","last_unstale":"2024-09-12T12:23:57.368927+0000","last_undegraded":"2024-09-12T12:23:57.368927+0000","last_fullsized":"2024-09-12T12:23:57.368927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:46:18.962036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354804+0000","last_change":"2024-09-12T12:23:57.354804+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354804+0000","last_undegraded":"2024-09-12T12:23:57.354804+0000","last_fullsized":"2024-09-12T12:23:57.354804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387960+0000","last_change":"2024-09-12T12:23:57.386864+0000","last_active":"2024-09-12T12:23:57.387960+0000","last_peered":"2024-09-12T12:23:57.387960+0000","last_clean":"2024-09-12T12:23:57.387960+0000","last_became_active":"2024-09-12T12:23:57.386550+0000","last_became_peered":"2024-09-12T12:23:57.386550+0000","last_unstale":"2024-09-12T12:23:57.387960+0000","last_undegraded":"2024-09-12T12:23:57.387960+0000","last_fullsized":"2024-09-12T12:23:57.387960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:03.518217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.364654+0000","last_change":"2024-09-12T12:23:57.364654+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.364654+0000","last_undegraded":"2024-09-12T12:23:57.364654+0000","last_fullsized":"2024-09-12T12:23:57.364654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.360414+0000","last_change":"2024-09-12T12:23:57.368319+0000","last_active":"2024-09-12T12:23:58.360414+0000","last_peered":"2024-09-12T12:23:58.360414+0000","last_clean":"2024-09-12T12:23:58.360414+0000","last_became_active":"2024-09-12T12:23:57.367912+0000","last_became_peered":"2024-09-12T12:23:57.367912+0000","last_unstale":"2024-09-12T12:23:58.360414+0000","last_undegraded":"2024-09-12T12:23:58.360414+0000","last_fullsized":"2024-09-12T12:23:58.360414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:35.720409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354991+0000","last_change":"2024-09-12T12:23:57.354991+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354991+0000","last_undegraded":"2024-09-12T12:23:57.354991+0000","last_fullsized":"2024-09-12T12:23:57.354991+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.387956+0000","last_change":"2024-09-12T12:23:57.386214+0000","last_active":"2024-09-12T12:23:57.387956+0000","last_peered":"2024-09-12T12:23:57.387956+0000","last_clean":"2024-09-12T12:23:57.387956+0000","last_became_active":"2024-09-12T12:23:57.385756+0000","last_became_peered":"2024-09-12T12:23:57.385756+0000","last_unstale":"2024-09-12T12:23:57.387956+0000","last_undegraded":"2024-09-12T12:23:57.387956+0000","last_fullsized":"2024-09-12T12:23:57.387956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:09:58.282124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.376205+0000","last_change":"2024-09-12T12:23:57.376205+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.376205+0000","last_undegraded":"2024-09-12T12:23:57.376205+0000","last_fullsized":"2024-09-12T12:23:57.376205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.376824+0000","last_change":"2024-09-12T12:23:57.369090+0000","last_active":"2024-09-12T12:23:57.376824+0000","last_peered":"2024-09-12T12:23:57.376824+0000","last_clean":"2024-09-12T12:23:57.376824+0000","last_became_active":"2024-09-12T12:23:57.368802+0000","last_became_peered":"2024-09-12T12:23:57.368802+0000","last_unstale":"2024-09-12T12:23:57.376824+0000","last_undegraded":"2024-09-12T12:23:57.376824+0000","last_fullsized":"2024-09-12T12:23:57.376824+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:37:27.347542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.362894+0000","last_change":"2024-09-12T12:23:57.362894+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.362894+0000","last_undegraded":"2024-09-12T12:23:57.362894+0000","last_fullsized":"2024-09-12T12:23:57.362894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388670+0000","last_change":"2024-09-12T12:23:57.382433+0000","last_active":"2024-09-12T12:23:57.388670+0000","last_peered":"2024-09-12T12:23:57.388670+0000","last_clean":"2024-09-12T12:23:57.388670+0000","last_became_active":"2024-09-12T12:23:57.381303+0000","last_became_peered":"2024-09-12T12:23:57.381303+0000","last_unstale":"2024-09-12T12:23:57.388670+0000","last_undegraded":"2024-09-12T12:23:57.388670+0000","last_fullsized":"2024-09-12T12:23:57.388670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:38:15.474579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.355099+0000","last_change":"2024-09-12T12:23:57.355099+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.355099+0000","last_undegraded":"2024-09-12T12:23:57.355099+0000","last_fullsized":"2024-09-12T12:23:57.355099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.388602+0000","last_change":"2024-09-12T12:23:57.382205+0000","last_active":"2024-09-12T12:23:57.388602+0000","last_peered":"2024-09-12T12:23:57.388602+0000","last_clean":"2024-09-12T12:23:57.388602+0000","last_became_active":"2024-09-12T12:23:57.381644+0000","last_became_peered":"2024-09-12T12:23:57.381644+0000","last_unstale":"2024-09-12T12:23:57.388602+0000","last_undegraded":"2024-09-12T12:23:57.388602+0000","last_fullsized":"2024-09-12T12:23:57.388602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:41.425596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.357647+0000","last_change":"2024-09-12T12:23:57.357647+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.357647+0000","last_undegraded":"2024-09-12T12:23:57.357647+0000","last_fullsized":"2024-09-12T12:23:57.357647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.382580+0000","last_change":"2024-09-12T12:23:57.381829+0000","last_active":"2024-09-12T12:23:57.382580+0000","last_peered":"2024-09-12T12:23:57.382580+0000","last_clean":"2024-09-12T12:23:57.382580+0000","last_became_active":"2024-09-12T12:23:57.381422+0000","last_became_peered":"2024-09-12T12:23:57.381422+0000","last_unstale":"2024-09-12T12:23:57.382580+0000","last_undegraded":"2024-09-12T12:23:57.382580+0000","last_fullsized":"2024-09-12T12:23:57.382580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:12.617164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.361245+0000","last_change":"2024-09-12T12:23:57.361245+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.361245+0000","last_undegraded":"2024-09-12T12:23:57.361245+0000","last_fullsized":"2024-09-12T12:23:57.361245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.375315+0000","last_change":"2024-09-12T12:23:57.366499+0000","last_active":"2024-09-12T12:23:57.375315+0000","last_peered":"2024-09-12T12:23:57.375315+0000","last_clean":"2024-09-12T12:23:57.375315+0000","last_became_active":"2024-09-12T12:23:57.364717+0000","last_became_peered":"2024-09-12T12:23:57.364717+0000","last_unstale":"2024-09-12T12:23:57.375315+0000","last_undegraded":"2024-09-12T12:23:57.375315+0000","last_fullsized":"2024-09-12T12:23:57.375315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:25:16.093565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.371930+0000","last_change":"2024-09-12T12:23:57.371930+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.371930+0000","last_undegraded":"2024-09-12T12:23:57.371930+0000","last_fullsized":"2024-09-12T12:23:57.371930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.392519+0000","last_change":"2024-09-12T12:23:57.388887+0000","last_active":"2024-09-12T12:23:57.392519+0000","last_peered":"2024-09-12T12:23:57.392519+0000","last_clean":"2024-09-12T12:23:57.392519+0000","last_became_active":"2024-09-12T12:23:57.388290+0000","last_became_peered":"2024-09-12T12:23:57.388290+0000","last_unstale":"2024-09-12T12:23:57.392519+0000","last_undegraded":"2024-09-12T12:23:57.392519+0000","last_fullsized":"2024-09-12T12:23:57.392519+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:19:31.787885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.363040+0000","last_change":"2024-09-12T12:23:57.363040+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.363040+0000","last_undegraded":"2024-09-12T12:23:57.363040+0000","last_fullsized":"2024-09-12T12:23:57.363040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.394938+0000","last_change":"2024-09-12T12:23:57.390177+0000","last_active":"2024-09-12T12:23:57.394938+0000","last_peered":"2024-09-12T12:23:57.394938+0000","last_clean":"2024-09-12T12:23:57.394938+0000","last_became_active":"2024-09-12T12:23:57.389410+0000","last_became_peered":"2024-09-12T12:23:57.389410+0000","last_unstale":"2024-09-12T12:23:57.394938+0000","last_undegraded":"2024-09-12T12:23:57.394938+0000","last_fullsized":"2024-09-12T12:23:57.394938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:21.483924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-12T12:23:57.354931+0000","last_change":"2024-09-12T12:23:57.354931+0000","last_active":"2024-09-12T12:23:57.335036+0000","last_peered":"2024-09-12T12:23:57.335036+0000","last_clean":"2024-09-12T12:23:57.335036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:23:57.354931+0000","last_undegraded":"2024-09-12T12:23:57.354931+0000","last_fullsized":"2024-09-12T12:23:57.354931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.390536+0000","last_change":"2024-09-12T12:23:57.386525+0000","last_active":"2024-09-12T12:23:57.390536+0000","last_peered":"2024-09-12T12:23:57.390536+0000","last_clean":"2024-09-12T12:23:57.390536+0000","last_became_active":"2024-09-12T12:23:57.385710+0000","last_became_peered":"2024-09-12T12:23:57.385710+0000","last_unstale":"2024-09-12T12:23:57.390536+0000","last_undegraded":"2024-09-12T12:23:57.390536+0000","last_fullsized":"2024-09-12T12:23:57.390536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:11.831899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":273,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-12T12:23:57.385148+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:23:57.385148+0000","last_peered":"2024-09-12T12:23:57.385148+0000","last_clean":"2024-09-12T12:23:57.385148+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:23:57.385148+0000","last_undegraded":"2024-09-12T12:23:57.385148+0000","last_fullsized":"2024-09-12T12:23:57.385148+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":966656,"data_stored":404023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542200,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1523712,"data_stored":979729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":942080,"data_stored":395636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":395636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":929792,"data_stored":389361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1536000,"data_stored":986004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:03.187 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3f is not active+clean 2024-09-12T12:24:03.187 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.372179+0000', 'last_change': '2024-09-12T12:23:57.372179+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.372179+0000', 'last_undegraded': '2024-09-12T12:23:57.372179+0000', 'last_fullsized': '2024-09-12T12:23:57.372179+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.187 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3e is not active+clean 2024-09-12T12:24:03.187 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358274+0000', 'last_change': '2024-09-12T12:23:57.358274+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358274+0000', 'last_undegraded': '2024-09-12T12:23:57.358274+0000', 'last_fullsized': '2024-09-12T12:23:57.358274+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3d is not active+clean 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.367757+0000', 'last_change': '2024-09-12T12:23:57.367757+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.367757+0000', 'last_undegraded': '2024-09-12T12:23:57.367757+0000', 'last_fullsized': '2024-09-12T12:23:57.367757+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3c is not active+clean 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.357518+0000', 'last_change': '2024-09-12T12:23:57.357518+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.357518+0000', 'last_undegraded': '2024-09-12T12:23:57.357518+0000', 'last_fullsized': '2024-09-12T12:23:57.357518+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3b is not active+clean 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.367639+0000', 'last_change': '2024-09-12T12:23:57.367639+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.367639+0000', 'last_undegraded': '2024-09-12T12:23:57.367639+0000', 'last_fullsized': '2024-09-12T12:23:57.367639+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3a is not active+clean 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358945+0000', 'last_change': '2024-09-12T12:23:57.358945+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358945+0000', 'last_undegraded': '2024-09-12T12:23:57.358945+0000', 'last_fullsized': '2024-09-12T12:23:57.358945+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:PG 13.39 is not active+clean 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358058+0000', 'last_change': '2024-09-12T12:23:57.358058+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358058+0000', 'last_undegraded': '2024-09-12T12:23:57.358058+0000', 'last_fullsized': '2024-09-12T12:23:57.358058+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.188 INFO:tasks.ceph.ceph_manager.ceph:PG 13.38 is not active+clean 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.357626+0000', 'last_change': '2024-09-12T12:23:57.357626+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.357626+0000', 'last_undegraded': '2024-09-12T12:23:57.357626+0000', 'last_fullsized': '2024-09-12T12:23:57.357626+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:PG 13.37 is not active+clean 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.368766+0000', 'last_change': '2024-09-12T12:23:57.368766+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.368766+0000', 'last_undegraded': '2024-09-12T12:23:57.368766+0000', 'last_fullsized': '2024-09-12T12:23:57.368766+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:PG 13.36 is not active+clean 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.363177+0000', 'last_change': '2024-09-12T12:23:57.363177+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.363177+0000', 'last_undegraded': '2024-09-12T12:23:57.363177+0000', 'last_fullsized': '2024-09-12T12:23:57.363177+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:PG 13.35 is not active+clean 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358093+0000', 'last_change': '2024-09-12T12:23:57.358093+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358093+0000', 'last_undegraded': '2024-09-12T12:23:57.358093+0000', 'last_fullsized': '2024-09-12T12:23:57.358093+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:PG 13.34 is not active+clean 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.372464+0000', 'last_change': '2024-09-12T12:23:57.372464+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.372464+0000', 'last_undegraded': '2024-09-12T12:23:57.372464+0000', 'last_fullsized': '2024-09-12T12:23:57.372464+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.189 INFO:tasks.ceph.ceph_manager.ceph:PG 13.33 is not active+clean 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.370220+0000', 'last_change': '2024-09-12T12:23:57.370220+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.370220+0000', 'last_undegraded': '2024-09-12T12:23:57.370220+0000', 'last_fullsized': '2024-09-12T12:23:57.370220+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:PG 13.32 is not active+clean 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358797+0000', 'last_change': '2024-09-12T12:23:57.358797+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358797+0000', 'last_undegraded': '2024-09-12T12:23:57.358797+0000', 'last_fullsized': '2024-09-12T12:23:57.358797+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:PG 13.30 is not active+clean 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.367681+0000', 'last_change': '2024-09-12T12:23:57.367681+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.367681+0000', 'last_undegraded': '2024-09-12T12:23:57.367681+0000', 'last_fullsized': '2024-09-12T12:23:57.367681+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2f is not active+clean 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.362775+0000', 'last_change': '2024-09-12T12:23:57.362775+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.362775+0000', 'last_undegraded': '2024-09-12T12:23:57.362775+0000', 'last_fullsized': '2024-09-12T12:23:57.362775+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2e is not active+clean 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.355804+0000', 'last_change': '2024-09-12T12:23:57.355804+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.355804+0000', 'last_undegraded': '2024-09-12T12:23:57.355804+0000', 'last_fullsized': '2024-09-12T12:23:57.355804+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.190 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2d is not active+clean 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.362500+0000', 'last_change': '2024-09-12T12:23:57.362500+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.362500+0000', 'last_undegraded': '2024-09-12T12:23:57.362500+0000', 'last_fullsized': '2024-09-12T12:23:57.362500+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2c is not active+clean 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.355757+0000', 'last_change': '2024-09-12T12:23:57.355757+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.355757+0000', 'last_undegraded': '2024-09-12T12:23:57.355757+0000', 'last_fullsized': '2024-09-12T12:23:57.355757+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2b is not active+clean 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.356788+0000', 'last_change': '2024-09-12T12:23:57.356788+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.356788+0000', 'last_undegraded': '2024-09-12T12:23:57.356788+0000', 'last_fullsized': '2024-09-12T12:23:57.356788+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2a is not active+clean 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358620+0000', 'last_change': '2024-09-12T12:23:57.358620+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358620+0000', 'last_undegraded': '2024-09-12T12:23:57.358620+0000', 'last_fullsized': '2024-09-12T12:23:57.358620+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:PG 13.29 is not active+clean 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.365333+0000', 'last_change': '2024-09-12T12:23:57.365333+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.365333+0000', 'last_undegraded': '2024-09-12T12:23:57.365333+0000', 'last_fullsized': '2024-09-12T12:23:57.365333+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:PG 13.28 is not active+clean 2024-09-12T12:24:03.191 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.356474+0000', 'last_change': '2024-09-12T12:23:57.356474+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.356474+0000', 'last_undegraded': '2024-09-12T12:23:57.356474+0000', 'last_fullsized': '2024-09-12T12:23:57.356474+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:PG 13.27 is not active+clean 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.357760+0000', 'last_change': '2024-09-12T12:23:57.357760+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.357760+0000', 'last_undegraded': '2024-09-12T12:23:57.357760+0000', 'last_fullsized': '2024-09-12T12:23:57.357760+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:PG 13.26 is not active+clean 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.371939+0000', 'last_change': '2024-09-12T12:23:57.371939+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.371939+0000', 'last_undegraded': '2024-09-12T12:23:57.371939+0000', 'last_fullsized': '2024-09-12T12:23:57.371939+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:PG 13.25 is not active+clean 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.361557+0000', 'last_change': '2024-09-12T12:23:57.361557+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.361557+0000', 'last_undegraded': '2024-09-12T12:23:57.361557+0000', 'last_fullsized': '2024-09-12T12:23:57.361557+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:PG 13.24 is not active+clean 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.369582+0000', 'last_change': '2024-09-12T12:23:57.369582+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.369582+0000', 'last_undegraded': '2024-09-12T12:23:57.369582+0000', 'last_fullsized': '2024-09-12T12:23:57.369582+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:PG 13.23 is not active+clean 2024-09-12T12:24:03.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.357253+0000', 'last_change': '2024-09-12T12:23:57.357253+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.357253+0000', 'last_undegraded': '2024-09-12T12:23:57.357253+0000', 'last_fullsized': '2024-09-12T12:23:57.357253+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:PG 13.22 is not active+clean 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.356976+0000', 'last_change': '2024-09-12T12:23:57.356976+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.356976+0000', 'last_undegraded': '2024-09-12T12:23:57.356976+0000', 'last_fullsized': '2024-09-12T12:23:57.356976+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:PG 13.21 is not active+clean 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.360481+0000', 'last_change': '2024-09-12T12:23:57.360481+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.360481+0000', 'last_undegraded': '2024-09-12T12:23:57.360481+0000', 'last_fullsized': '2024-09-12T12:23:57.360481+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:PG 13.20 is not active+clean 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.366248+0000', 'last_change': '2024-09-12T12:23:57.366248+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.366248+0000', 'last_undegraded': '2024-09-12T12:23:57.366248+0000', 'last_fullsized': '2024-09-12T12:23:57.366248+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1e is not active+clean 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.360338+0000', 'last_change': '2024-09-12T12:23:57.360338+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.360338+0000', 'last_undegraded': '2024-09-12T12:23:57.360338+0000', 'last_fullsized': '2024-09-12T12:23:57.360338+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1f is not active+clean 2024-09-12T12:24:03.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.371198+0000', 'last_change': '2024-09-12T12:23:57.371198+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.371198+0000', 'last_undegraded': '2024-09-12T12:23:57.371198+0000', 'last_fullsized': '2024-09-12T12:23:57.371198+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1c is not active+clean 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.364066+0000', 'last_change': '2024-09-12T12:23:57.364066+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.364066+0000', 'last_undegraded': '2024-09-12T12:23:57.364066+0000', 'last_fullsized': '2024-09-12T12:23:57.364066+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1d is not active+clean 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.362837+0000', 'last_change': '2024-09-12T12:23:57.362837+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.362837+0000', 'last_undegraded': '2024-09-12T12:23:57.362837+0000', 'last_fullsized': '2024-09-12T12:23:57.362837+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1a is not active+clean 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.356053+0000', 'last_change': '2024-09-12T12:23:57.356053+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.356053+0000', 'last_undegraded': '2024-09-12T12:23:57.356053+0000', 'last_fullsized': '2024-09-12T12:23:57.356053+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1b is not active+clean 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358969+0000', 'last_change': '2024-09-12T12:23:57.358969+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358969+0000', 'last_undegraded': '2024-09-12T12:23:57.358969+0000', 'last_fullsized': '2024-09-12T12:23:57.358969+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:PG 13.18 is not active+clean 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.372956+0000', 'last_change': '2024-09-12T12:23:57.372956+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.372956+0000', 'last_undegraded': '2024-09-12T12:23:57.372956+0000', 'last_fullsized': '2024-09-12T12:23:57.372956+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.194 INFO:tasks.ceph.ceph_manager.ceph:PG 13.17 is not active+clean 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.360621+0000', 'last_change': '2024-09-12T12:23:57.360621+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.360621+0000', 'last_undegraded': '2024-09-12T12:23:57.360621+0000', 'last_fullsized': '2024-09-12T12:23:57.360621+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:PG 13.14 is not active+clean 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.361801+0000', 'last_change': '2024-09-12T12:23:57.361801+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.361801+0000', 'last_undegraded': '2024-09-12T12:23:57.361801+0000', 'last_fullsized': '2024-09-12T12:23:57.361801+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:PG 13.15 is not active+clean 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.357304+0000', 'last_change': '2024-09-12T12:23:57.357304+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.357304+0000', 'last_undegraded': '2024-09-12T12:23:57.357304+0000', 'last_fullsized': '2024-09-12T12:23:57.357304+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:PG 13.12 is not active+clean 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.361216+0000', 'last_change': '2024-09-12T12:23:57.361216+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.361216+0000', 'last_undegraded': '2024-09-12T12:23:57.361216+0000', 'last_fullsized': '2024-09-12T12:23:57.361216+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:PG 13.13 is not active+clean 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.368232+0000', 'last_change': '2024-09-12T12:23:57.368232+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.368232+0000', 'last_undegraded': '2024-09-12T12:23:57.368232+0000', 'last_fullsized': '2024-09-12T12:23:57.368232+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.195 INFO:tasks.ceph.ceph_manager.ceph:PG 13.10 is not active+clean 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.354943+0000', 'last_change': '2024-09-12T12:23:57.354943+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.354943+0000', 'last_undegraded': '2024-09-12T12:23:57.354943+0000', 'last_fullsized': '2024-09-12T12:23:57.354943+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:PG 13.11 is not active+clean 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.355561+0000', 'last_change': '2024-09-12T12:23:57.355561+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.355561+0000', 'last_undegraded': '2024-09-12T12:23:57.355561+0000', 'last_fullsized': '2024-09-12T12:23:57.355561+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:PG 13.e is not active+clean 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.362662+0000', 'last_change': '2024-09-12T12:23:57.362662+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.362662+0000', 'last_undegraded': '2024-09-12T12:23:57.362662+0000', 'last_fullsized': '2024-09-12T12:23:57.362662+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:PG 13.f is not active+clean 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.370155+0000', 'last_change': '2024-09-12T12:23:57.370155+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.370155+0000', 'last_undegraded': '2024-09-12T12:23:57.370155+0000', 'last_fullsized': '2024-09-12T12:23:57.370155+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:PG 13.d is not active+clean 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.356236+0000', 'last_change': '2024-09-12T12:23:57.356236+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.356236+0000', 'last_undegraded': '2024-09-12T12:23:57.356236+0000', 'last_fullsized': '2024-09-12T12:23:57.356236+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:PG 13.a is not active+clean 2024-09-12T12:24:03.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.355613+0000', 'last_change': '2024-09-12T12:23:57.355613+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.355613+0000', 'last_undegraded': '2024-09-12T12:23:57.355613+0000', 'last_fullsized': '2024-09-12T12:23:57.355613+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:PG 13.b is not active+clean 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.358286+0000', 'last_change': '2024-09-12T12:23:57.358286+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.358286+0000', 'last_undegraded': '2024-09-12T12:23:57.358286+0000', 'last_fullsized': '2024-09-12T12:23:57.358286+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:PG 13.8 is not active+clean 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.354804+0000', 'last_change': '2024-09-12T12:23:57.354804+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.354804+0000', 'last_undegraded': '2024-09-12T12:23:57.354804+0000', 'last_fullsized': '2024-09-12T12:23:57.354804+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:PG 13.9 is not active+clean 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.364654+0000', 'last_change': '2024-09-12T12:23:57.364654+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.364654+0000', 'last_undegraded': '2024-09-12T12:23:57.364654+0000', 'last_fullsized': '2024-09-12T12:23:57.364654+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:PG 13.6 is not active+clean 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.354991+0000', 'last_change': '2024-09-12T12:23:57.354991+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.354991+0000', 'last_undegraded': '2024-09-12T12:23:57.354991+0000', 'last_fullsized': '2024-09-12T12:23:57.354991+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:PG 13.7 is not active+clean 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.376205+0000', 'last_change': '2024-09-12T12:23:57.376205+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.376205+0000', 'last_undegraded': '2024-09-12T12:23:57.376205+0000', 'last_fullsized': '2024-09-12T12:23:57.376205+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:PG 13.4 is not active+clean 2024-09-12T12:24:03.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.362894+0000', 'last_change': '2024-09-12T12:23:57.362894+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.362894+0000', 'last_undegraded': '2024-09-12T12:23:57.362894+0000', 'last_fullsized': '2024-09-12T12:23:57.362894+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:PG 13.5 is not active+clean 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.355099+0000', 'last_change': '2024-09-12T12:23:57.355099+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.355099+0000', 'last_undegraded': '2024-09-12T12:23:57.355099+0000', 'last_fullsized': '2024-09-12T12:23:57.355099+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2 is not active+clean 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.357647+0000', 'last_change': '2024-09-12T12:23:57.357647+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.357647+0000', 'last_undegraded': '2024-09-12T12:23:57.357647+0000', 'last_fullsized': '2024-09-12T12:23:57.357647+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3 is not active+clean 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.361245+0000', 'last_change': '2024-09-12T12:23:57.361245+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.361245+0000', 'last_undegraded': '2024-09-12T12:23:57.361245+0000', 'last_fullsized': '2024-09-12T12:23:57.361245+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:PG 13.0 is not active+clean 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.371930+0000', 'last_change': '2024-09-12T12:23:57.371930+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.371930+0000', 'last_undegraded': '2024-09-12T12:23:57.371930+0000', 'last_fullsized': '2024-09-12T12:23:57.371930+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1 is not active+clean 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.363040+0000', 'last_change': '2024-09-12T12:23:57.363040+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.363040+0000', 'last_undegraded': '2024-09-12T12:23:57.363040+0000', 'last_fullsized': '2024-09-12T12:23:57.363040+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-12T12:24:03.198 INFO:tasks.ceph.ceph_manager.ceph:PG 13.c is not active+clean 2024-09-12T12:24:03.199 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:23:57.354931+0000', 'last_change': '2024-09-12T12:23:57.354931+0000', 'last_active': '2024-09-12T12:23:57.335036+0000', 'last_peered': '2024-09-12T12:23:57.335036+0000', 'last_clean': '2024-09-12T12:23:57.335036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:23:57.354931+0000', 'last_undegraded': '2024-09-12T12:23:57.354931+0000', 'last_fullsized': '2024-09-12T12:23:57.354931+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'last_clean_scrub_stamp': '2024-09-12T12:23:57.335036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-12T12:24:03.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T12:24:03.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c42f2640 1 Processor -- start 2024-09-12T12:24:03.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c42f2640 1 -- start start 2024-09-12T12:24:03.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c42f2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72bc0d1190 0x7f72bc0d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c42f2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0b7740 0x7f72bc0b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c42f2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72bc0d30b0 con 0x7f72bc0d1570 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c42f2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f72bc0d3cc0 con 0x7f72bc0d1190 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c42f2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72bc0d3390 con 0x7f72bc0b7740 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c288f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0b7740 0x7f72bc0b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c288f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0b7740 0x7f72bc0b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:57748/0 (socket says 172.21.15.3:57748) 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c208e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72bc0d1190 0x7f72bc0d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c288f640 1 -- 172.21.15.3:0/1950946717 learned_addr learned my addr 172.21.15.3:0/1950946717 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c288f640 1 -- 172.21.15.3:0/1950946717 >> v2:172.21.15.142:3300/0 conn(0x7f72bc0d1190 msgr2=0x7f72bc0d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c288f640 1 --2- 172.21.15.3:0/1950946717 >> v2:172.21.15.142:3300/0 conn(0x7f72bc0d1190 0x7f72bc0d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c288f640 1 -- 172.21.15.3:0/1950946717 >> v1:172.21.15.142:6789/0 conn(0x7f72bc0d1570 legacy=0x7f72bc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.348+0000 7f72c288f640 1 -- 172.21.15.3:0/1950946717 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc0d3670 con 0x7f72bc0b7740 2024-09-12T12:24:03.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c288f640 1 --2- 172.21.15.3:0/1950946717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0b7740 0x7f72bc0b7bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=361d0ecb02f8e27c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c108c640 1 -- 172.21.15.3:0/1950946717 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b40b5070 con 0x7f72bc0b7740 2024-09-12T12:24:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c108c640 1 -- 172.21.15.3:0/1950946717 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b40a8a50 con 0x7f72bc0b7740 2024-09-12T12:24:03.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c108c640 1 -- 172.21.15.3:0/1950946717 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b40a8e20 con 0x7f72bc0b7740 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- 172.21.15.3:0/1950946717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0b7740 msgr2=0x7f72bc0b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 --2- 172.21.15.3:0/1950946717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0b7740 0x7f72bc0b7bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b401ed50 comp rx=0 tx=0).stop 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- 172.21.15.3:0/1950946717 shutdown_connections 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 --2- 172.21.15.3:0/1950946717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0b7740 0x7f72bc0b7bd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 --2- 172.21.15.3:0/1950946717 >> v2:172.21.15.142:3300/0 conn(0x7f72bc0d1190 0x7f72bc0d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- 172.21.15.3:0/1950946717 >> 172.21.15.3:0/1950946717 conn(0x7f72bc067a00 msgr2=0x7f72bc067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- 172.21.15.3:0/1950946717 shutdown_connections 2024-09-12T12:24:03.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- 172.21.15.3:0/1950946717 wait complete. 2024-09-12T12:24:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 Processor -- start 2024-09-12T12:24:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- start start 2024-09-12T12:24:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72bc0b7740 0x7f72bc0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0d1190 0x7f72bc1e9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72bc0d1570 0x7f72bc155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc0d2cd0 con 0x7f72bc0d1570 2024-09-12T12:24:03.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc0d1a00 con 0x7f72bc0b7740 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.352+0000 7f72c42f2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f72bc0d2e50 con 0x7f72bc0d1190 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72bc0d1570 0x7f72bc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72bc0d1570 0x7f72bc155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60638/0 (socket says 172.21.15.3:60638) 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c208e640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72bc0b7740 0x7f72bc0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 -- 172.21.15.3:0/323186366 learned_addr learned my addr 172.21.15.3:0/323186366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c188d640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0d1190 0x7f72bc1e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 -- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0d1190 msgr2=0x7f72bc1e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0d1190 0x7f72bc1e9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 -- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72bc0b7740 msgr2=0x7f72bc0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72bc0b7740 0x7f72bc0ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 -- 172.21.15.3:0/323186366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc0ce590 con 0x7f72bc0d1570 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c288f640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72bc0d1570 0x7f72bc155870 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b3040 tx=0x7f72b401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72b2ffd640 1 -- 172.21.15.3:0/323186366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b40b5070 con 0x7f72bc0d1570 2024-09-12T12:24:03.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72b2ffd640 1 -- 172.21.15.3:0/323186366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b40b0de0 con 0x7f72bc0d1570 2024-09-12T12:24:03.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc160200 con 0x7f72bc0d1570 2024-09-12T12:24:03.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72b2ffd640 1 -- 172.21.15.3:0/323186366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b40bb670 con 0x7f72bc0d1570 2024-09-12T12:24:03.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc1607c0 con 0x7f72bc0d1570 2024-09-12T12:24:03.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72bc161b30 con 0x7f72bc0d1570 2024-09-12T12:24:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72b2ffd640 1 -- 172.21.15.3:0/323186366 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f72b40c2060 con 0x7f72bc0d1570 2024-09-12T12:24:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72b2ffd640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72a00678c0 0x7f72a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72b2ffd640 1 -- 172.21.15.3:0/323186366 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f72b4123010 con 0x7f72bc0d1570 2024-09-12T12:24:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.356+0000 7f72c208e640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72a00678c0 0x7f72a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.360+0000 7f72c208e640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72a00678c0 0x7f72a0069d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72b80113e0 tx=0x7f72b800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:03.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.360+0000 7f72b2ffd640 1 -- 172.21.15.3:0/323186366 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f72b40188b0 con 0x7f72bc0d1570 2024-09-12T12:24:03.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.584+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f72bc0ba4c0 con 0x7f72bc0d1570 2024-09-12T12:24:03.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72b2ffd640 1 -- 172.21.15.3:0/323186366 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1556 (secure 0 0 0) 0x7f72b40f3500 con 0x7f72bc0d1570 2024-09-12T12:24:03.590 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:03.590 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":278,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":79,"num_osds":8,"num_up_osds":8,"osd_up_since":1726143578,"num_in_osds":8,"osd_in_since":1726143567,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":61},{"state_name":"active+clean","count":36}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":247205888,"bytes_avail":772846907392,"bytes_total":773094113280,"inactive_pgs_ratio":0.62886595726013184,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":6375}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-12T12:23:45.737426+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T12:24:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72a00678c0 msgr2=0x7f72a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72a00678c0 0x7f72a0069d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f72b80113e0 tx=0x7f72b800b040 comp rx=0 tx=0).stop 2024-09-12T12:24:03.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72bc0d1570 msgr2=0x7f72bc155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72bc0d1570 0x7f72bc155870 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f72b40b3040 tx=0x7f72b401baf0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 shutdown_connections 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f72a00678c0 0x7f72a0069d80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72bc0d1570 0x7f72bc155870 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72bc0d1190 0x7f72bc1e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 --2- 172.21.15.3:0/323186366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72bc0b7740 0x7f72bc0ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 >> 172.21.15.3:0/323186366 conn(0x7f72bc067a00 msgr2=0x7f72bc0b98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 shutdown_connections 2024-09-12T12:24:03.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.588+0000 7f72c42f2640 1 -- 172.21.15.3:0/323186366 wait complete. 2024-09-12T12:24:03.608 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T12:24:03.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:03.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c07a3e640 1 Processor -- start 2024-09-12T12:24:03.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c07a3e640 1 -- start start 2024-09-12T12:24:03.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c07a3e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6c000d05b0 0x7f6c000d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c07a3e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c0015b620 0x7f6c0015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c07a3e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c000d0ed0 con 0x7f6c000ae250 2024-09-12T12:24:03.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c07a3e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6c000ad0c0 con 0x7f6c0015b620 2024-09-12T12:24:03.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c07a3e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6c000ad3a0 con 0x7f6c000d05b0 2024-09-12T12:24:03.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c05fdb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c0015b620 0x7f6c0015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c05fdb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6c0015b620 0x7f6c0015ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60642/0 (socket says 172.21.15.3:60642) 2024-09-12T12:24:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c05fdb640 1 -- 172.21.15.3:0/557955039 learned_addr learned my addr 172.21.15.3:0/557955039 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c057da640 1 --2- 172.21.15.3:0/557955039 >> v2:172.21.15.142:3300/0 conn(0x7f6c000d05b0 0x7f6c000d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c05fdb640 1 -- 172.21.15.3:0/557955039 >> v2:172.21.15.142:3300/0 conn(0x7f6c000d05b0 msgr2=0x7f6c000d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c05fdb640 1 --2- 172.21.15.3:0/557955039 >> v2:172.21.15.142:3300/0 conn(0x7f6c000d05b0 0x7f6c000d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c05fdb640 1 -- 172.21.15.3:0/557955039 >> v1:172.21.15.142:6789/0 conn(0x7f6c000ae250 legacy=0x7f6c000ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:03.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.724+0000 7f6c05fdb640 1 -- 172.21.15.3:0/557955039 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c000ad680 con 0x7f6c0015b620 2024-09-12T12:24:03.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c05fdb640 1 --2- 172.21.15.3:0/557955039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c0015b620 0x7f6c0015ba50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c0006d8b0 tx=0x7f6bf801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899c2c1231b44be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:03.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6bf7fff640 1 -- 172.21.15.3:0/557955039 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80a5760 con 0x7f6c0015b620 2024-09-12T12:24:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6bf7fff640 1 -- 172.21.15.3:0/557955039 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf80a5900 con 0x7f6c0015b620 2024-09-12T12:24:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6bf7fff640 1 -- 172.21.15.3:0/557955039 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf80a5cd0 con 0x7f6c0015b620 2024-09-12T12:24:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- 172.21.15.3:0/557955039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c0015b620 msgr2=0x7f6c0015ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/557955039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c0015b620 0x7f6c0015ba50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c0006d8b0 tx=0x7f6bf801c9b0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- 172.21.15.3:0/557955039 shutdown_connections 2024-09-12T12:24:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/557955039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c0015b620 0x7f6c0015ba50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/557955039 >> v2:172.21.15.142:3300/0 conn(0x7f6c000d05b0 0x7f6c000d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- 172.21.15.3:0/557955039 >> 172.21.15.3:0/557955039 conn(0x7f6c00067e00 msgr2=0x7f6c00068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- 172.21.15.3:0/557955039 shutdown_connections 2024-09-12T12:24:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- 172.21.15.3:0/557955039 wait complete. 2024-09-12T12:24:03.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 Processor -- start 2024-09-12T12:24:03.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- start start 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c000ae250 0x7f6c00143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c000d05b0 0x7f6c00144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c0015b620 0x7f6c000cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0015c9b0 con 0x7f6c000ae250 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0015c830 con 0x7f6c000d05b0 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c07a3e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6c00165ca0 con 0x7f6c0015b620 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c000ae250 0x7f6c00143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c000ae250 0x7f6c00143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60646/0 (socket says 172.21.15.3:60646) 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 -- 172.21.15.3:0/1079597338 learned_addr learned my addr 172.21.15.3:0/1079597338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:03.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c05fdb640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c0015b620 0x7f6c000cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c04fd9640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c000d05b0 0x7f6c00144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 -- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c0015b620 msgr2=0x7f6c000cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c0015b620 0x7f6c000cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 -- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c000d05b0 msgr2=0x7f6c00144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c000d05b0 0x7f6c00144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 -- 172.21.15.3:0/1079597338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c00152c20 con 0x7f6c000ae250 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c04fd9640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c000d05b0 0x7f6c00144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.728+0000 7f6c057da640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c000ae250 0x7f6c00143120 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d800 tx=0x7f6bfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6bf5ffb640 1 -- 172.21.15.3:0/1079597338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00b6c0 con 0x7f6c000ae250 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6bf5ffb640 1 -- 172.21.15.3:0/1079597338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bfc019070 con 0x7f6c000ae250 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c00143860 con 0x7f6c000ae250 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6bf5ffb640 1 -- 172.21.15.3:0/1079597338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00ba60 con 0x7f6c000ae250 2024-09-12T12:24:03.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c00143e20 con 0x7f6c000ae250 2024-09-12T12:24:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c000d0990 con 0x7f6c000ae250 2024-09-12T12:24:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6bf5ffb640 1 -- 172.21.15.3:0/1079597338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6bfc00bc00 con 0x7f6c000ae250 2024-09-12T12:24:03.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6bf5ffb640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6be40678c0 0x7f6be4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6bf5ffb640 1 -- 172.21.15.3:0/1079597338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f6bfc086c30 con 0x7f6c000ae250 2024-09-12T12:24:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.732+0000 7f6c04fd9640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6be40678c0 0x7f6be4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:03.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.736+0000 7f6c04fd9640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6be40678c0 0x7f6be4069d80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6c000b5140 tx=0x7f6bf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:03.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.736+0000 7f6bf5ffb640 1 -- 172.21.15.3:0/1079597338 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6bfc0577e0 con 0x7f6c000ae250 2024-09-12T12:24:03.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.892+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c00144b60 con 0x7f6be40678c0 2024-09-12T12:24:03.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6bf5ffb640 1 -- 172.21.15.3:0/1079597338 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f6bf0005050 con 0x7f6be40678c0 2024-09-12T12:24:03.915 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:03.917 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6be40678c0 msgr2=0x7f6be4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6be40678c0 0x7f6be4069d80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6c000b5140 tx=0x7f6bf0009040 comp rx=0 tx=0).stop 2024-09-12T12:24:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c000ae250 msgr2=0x7f6c00143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c000ae250 0x7f6c00143120 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d800 tx=0x7f6bfc00dcc0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 shutdown_connections 2024-09-12T12:24:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6be40678c0 0x7f6be4069d80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6c0015b620 0x7f6c000cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c000d05b0 0x7f6c00144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 --2- 172.21.15.3:0/1079597338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c000ae250 0x7f6c00143120 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 >> 172.21.15.3:0/1079597338 conn(0x7f6c00067e00 msgr2=0x7f6c000ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.912+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 shutdown_connections 2024-09-12T12:24:03.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:03.916+0000 7f6c07a3e640 1 -- 172.21.15.3:0/1079597338 wait complete. 2024-09-12T12:24:03.932 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-12T12:24:03.742674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249984,"kb_used_data":8764,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724736,"statfs":{"total":773094113280,"available":772838129664,"internally_reserved":0,"allocated":8974336,"data_stored":4448944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":57,"apply_latency_ms":57,"commit_latency_ns":57000000,"apply_latency_ns":57000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.393685"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393467+0000","last_change":"2024-09-12T12:23:58.407134+0000","last_active":"2024-09-12T12:23:59.393467+0000","last_peered":"2024-09-12T12:23:59.393467+0000","last_clean":"2024-09-12T12:23:59.393467+0000","last_became_active":"2024-09-12T12:23:58.406341+0000","last_became_peered":"2024-09-12T12:23:58.406341+0000","last_unstale":"2024-09-12T12:23:59.393467+0000","last_undegraded":"2024-09-12T12:23:59.393467+0000","last_fullsized":"2024-09-12T12:23:59.393467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:09:55.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393394+0000","last_change":"2024-09-12T12:23:58.412147+0000","last_active":"2024-09-12T12:23:59.393394+0000","last_peered":"2024-09-12T12:23:59.393394+0000","last_clean":"2024-09-12T12:23:59.393394+0000","last_became_active":"2024-09-12T12:23:58.411838+0000","last_became_peered":"2024-09-12T12:23:58.411838+0000","last_unstale":"2024-09-12T12:23:59.393394+0000","last_undegraded":"2024-09-12T12:23:59.393394+0000","last_fullsized":"2024-09-12T12:23:59.393394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:15:25.592214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.640016+0000","last_change":"2024-09-12T12:23:58.398025+0000","last_active":"2024-09-12T12:24:00.640016+0000","last_peered":"2024-09-12T12:24:00.640016+0000","last_clean":"2024-09-12T12:24:00.640016+0000","last_became_active":"2024-09-12T12:23:58.397289+0000","last_became_peered":"2024-09-12T12:23:58.397289+0000","last_unstale":"2024-09-12T12:24:00.640016+0000","last_undegraded":"2024-09-12T12:24:00.640016+0000","last_fullsized":"2024-09-12T12:24:00.640016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:01:38.815943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.636947+0000","last_change":"2024-09-12T12:23:58.395534+0000","last_active":"2024-09-12T12:24:00.636947+0000","last_peered":"2024-09-12T12:24:00.636947+0000","last_clean":"2024-09-12T12:24:00.636947+0000","last_became_active":"2024-09-12T12:23:58.395256+0000","last_became_peered":"2024-09-12T12:23:58.395256+0000","last_unstale":"2024-09-12T12:24:00.636947+0000","last_undegraded":"2024-09-12T12:24:00.636947+0000","last_fullsized":"2024-09-12T12:24:00.636947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:58.574908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639149+0000","last_change":"2024-09-12T12:23:58.395496+0000","last_active":"2024-09-12T12:24:00.639149+0000","last_peered":"2024-09-12T12:24:00.639149+0000","last_clean":"2024-09-12T12:24:00.639149+0000","last_became_active":"2024-09-12T12:23:58.394775+0000","last_became_peered":"2024-09-12T12:23:58.394775+0000","last_unstale":"2024-09-12T12:24:00.639149+0000","last_undegraded":"2024-09-12T12:24:00.639149+0000","last_fullsized":"2024-09-12T12:24:00.639149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:52:54.437455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390369+0000","last_change":"2024-09-12T12:23:58.380493+0000","last_active":"2024-09-12T12:23:59.390369+0000","last_peered":"2024-09-12T12:23:59.390369+0000","last_clean":"2024-09-12T12:23:59.390369+0000","last_became_active":"2024-09-12T12:23:58.379752+0000","last_became_peered":"2024-09-12T12:23:58.379752+0000","last_unstale":"2024-09-12T12:23:59.390369+0000","last_undegraded":"2024-09-12T12:23:59.390369+0000","last_fullsized":"2024-09-12T12:23:59.390369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:56.787085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.634755+0000","last_change":"2024-09-12T12:23:58.377300+0000","last_active":"2024-09-12T12:24:00.634755+0000","last_peered":"2024-09-12T12:24:00.634755+0000","last_clean":"2024-09-12T12:24:00.634755+0000","last_became_active":"2024-09-12T12:23:58.376305+0000","last_became_peered":"2024-09-12T12:23:58.376305+0000","last_unstale":"2024-09-12T12:24:00.634755+0000","last_undegraded":"2024-09-12T12:24:00.634755+0000","last_fullsized":"2024-09-12T12:24:00.634755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:34:56.335568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390257+0000","last_change":"2024-09-12T12:23:58.395247+0000","last_active":"2024-09-12T12:23:59.390257+0000","last_peered":"2024-09-12T12:23:59.390257+0000","last_clean":"2024-09-12T12:23:59.390257+0000","last_became_active":"2024-09-12T12:23:58.394534+0000","last_became_peered":"2024-09-12T12:23:58.394534+0000","last_unstale":"2024-09-12T12:23:59.390257+0000","last_undegraded":"2024-09-12T12:23:59.390257+0000","last_fullsized":"2024-09-12T12:23:59.390257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:31.089085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390933+0000","last_change":"2024-09-12T12:23:58.371385+0000","last_active":"2024-09-12T12:23:59.390933+0000","last_peered":"2024-09-12T12:23:59.390933+0000","last_clean":"2024-09-12T12:23:59.390933+0000","last_became_active":"2024-09-12T12:23:58.370715+0000","last_became_peered":"2024-09-12T12:23:58.370715+0000","last_unstale":"2024-09-12T12:23:59.390933+0000","last_undegraded":"2024-09-12T12:23:59.390933+0000","last_fullsized":"2024-09-12T12:23:59.390933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:18.770724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393154+0000","last_change":"2024-09-12T12:23:58.395206+0000","last_active":"2024-09-12T12:23:59.393154+0000","last_peered":"2024-09-12T12:23:59.393154+0000","last_clean":"2024-09-12T12:23:59.393154+0000","last_became_active":"2024-09-12T12:23:58.394053+0000","last_became_peered":"2024-09-12T12:23:58.394053+0000","last_unstale":"2024-09-12T12:23:59.393154+0000","last_undegraded":"2024-09-12T12:23:59.393154+0000","last_fullsized":"2024-09-12T12:23:59.393154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:08:35.556909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393209+0000","last_change":"2024-09-12T12:23:58.392493+0000","last_active":"2024-09-12T12:23:59.393209+0000","last_peered":"2024-09-12T12:23:59.393209+0000","last_clean":"2024-09-12T12:23:59.393209+0000","last_became_active":"2024-09-12T12:23:58.391962+0000","last_became_peered":"2024-09-12T12:23:58.391962+0000","last_unstale":"2024-09-12T12:23:59.393209+0000","last_undegraded":"2024-09-12T12:23:59.393209+0000","last_fullsized":"2024-09-12T12:23:59.393209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:28.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392428+0000","last_change":"2024-09-12T12:23:58.401621+0000","last_active":"2024-09-12T12:23:59.392428+0000","last_peered":"2024-09-12T12:23:59.392428+0000","last_clean":"2024-09-12T12:23:59.392428+0000","last_became_active":"2024-09-12T12:23:58.394576+0000","last_became_peered":"2024-09-12T12:23:58.394576+0000","last_unstale":"2024-09-12T12:23:59.392428+0000","last_undegraded":"2024-09-12T12:23:59.392428+0000","last_fullsized":"2024-09-12T12:23:59.392428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:54.581603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638976+0000","last_change":"2024-09-12T12:23:58.374604+0000","last_active":"2024-09-12T12:24:00.638976+0000","last_peered":"2024-09-12T12:24:00.638976+0000","last_clean":"2024-09-12T12:24:00.638976+0000","last_became_active":"2024-09-12T12:23:58.373831+0000","last_became_peered":"2024-09-12T12:23:58.373831+0000","last_unstale":"2024-09-12T12:24:00.638976+0000","last_undegraded":"2024-09-12T12:24:00.638976+0000","last_fullsized":"2024-09-12T12:24:00.638976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:10.558765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635353+0000","last_change":"2024-09-12T12:23:58.394035+0000","last_active":"2024-09-12T12:24:00.635353+0000","last_peered":"2024-09-12T12:24:00.635353+0000","last_clean":"2024-09-12T12:24:00.635353+0000","last_became_active":"2024-09-12T12:23:58.393238+0000","last_became_peered":"2024-09-12T12:23:58.393238+0000","last_unstale":"2024-09-12T12:24:00.635353+0000","last_undegraded":"2024-09-12T12:24:00.635353+0000","last_fullsized":"2024-09-12T12:24:00.635353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:47:30.744183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.390665+0000","last_change":"2024-09-12T12:23:58.386557+0000","last_active":"2024-09-12T12:23:58.390665+0000","last_peered":"2024-09-12T12:23:58.390665+0000","last_clean":"2024-09-12T12:23:58.390665+0000","last_became_active":"2024-09-12T12:23:58.386226+0000","last_became_peered":"2024-09-12T12:23:58.386226+0000","last_unstale":"2024-09-12T12:23:58.390665+0000","last_undegraded":"2024-09-12T12:23:58.390665+0000","last_fullsized":"2024-09-12T12:23:58.390665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:52:12.537859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638959+0000","last_change":"2024-09-12T12:23:58.412247+0000","last_active":"2024-09-12T12:24:00.638959+0000","last_peered":"2024-09-12T12:24:00.638959+0000","last_clean":"2024-09-12T12:24:00.638959+0000","last_became_active":"2024-09-12T12:23:58.411587+0000","last_became_peered":"2024-09-12T12:23:58.411587+0000","last_unstale":"2024-09-12T12:24:00.638959+0000","last_undegraded":"2024-09-12T12:24:00.638959+0000","last_fullsized":"2024-09-12T12:24:00.638959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:18.770724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639605+0000","last_change":"2024-09-12T12:23:58.377530+0000","last_active":"2024-09-12T12:24:00.639605+0000","last_peered":"2024-09-12T12:24:00.639605+0000","last_clean":"2024-09-12T12:24:00.639605+0000","last_became_active":"2024-09-12T12:23:58.376736+0000","last_became_peered":"2024-09-12T12:23:58.376736+0000","last_unstale":"2024-09-12T12:24:00.639605+0000","last_undegraded":"2024-09-12T12:24:00.639605+0000","last_fullsized":"2024-09-12T12:24:00.639605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:08:35.556909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637437+0000","last_change":"2024-09-12T12:23:58.394232+0000","last_active":"2024-09-12T12:24:00.637437+0000","last_peered":"2024-09-12T12:24:00.637437+0000","last_clean":"2024-09-12T12:24:00.637437+0000","last_became_active":"2024-09-12T12:23:58.393925+0000","last_became_peered":"2024-09-12T12:23:58.393925+0000","last_unstale":"2024-09-12T12:24:00.637437+0000","last_undegraded":"2024-09-12T12:24:00.637437+0000","last_fullsized":"2024-09-12T12:24:00.637437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:42:05.083901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395624+0000","last_change":"2024-09-12T12:23:58.384760+0000","last_active":"2024-09-12T12:23:59.395624+0000","last_peered":"2024-09-12T12:23:59.395624+0000","last_clean":"2024-09-12T12:23:59.395624+0000","last_became_active":"2024-09-12T12:23:58.382109+0000","last_became_peered":"2024-09-12T12:23:58.382109+0000","last_unstale":"2024-09-12T12:23:59.395624+0000","last_undegraded":"2024-09-12T12:23:59.395624+0000","last_fullsized":"2024-09-12T12:23:59.395624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:49.144078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637268+0000","last_change":"2024-09-12T12:23:58.383770+0000","last_active":"2024-09-12T12:24:00.637268+0000","last_peered":"2024-09-12T12:24:00.637268+0000","last_clean":"2024-09-12T12:24:00.637268+0000","last_became_active":"2024-09-12T12:23:58.381588+0000","last_became_peered":"2024-09-12T12:23:58.381588+0000","last_unstale":"2024-09-12T12:24:00.637268+0000","last_undegraded":"2024-09-12T12:24:00.637268+0000","last_fullsized":"2024-09-12T12:24:00.637268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:27:07.541371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.636665+0000","last_change":"2024-09-12T12:23:58.373789+0000","last_active":"2024-09-12T12:24:00.636665+0000","last_peered":"2024-09-12T12:24:00.636665+0000","last_clean":"2024-09-12T12:24:00.636665+0000","last_became_active":"2024-09-12T12:23:58.372969+0000","last_became_peered":"2024-09-12T12:23:58.372969+0000","last_unstale":"2024-09-12T12:24:00.636665+0000","last_undegraded":"2024-09-12T12:24:00.636665+0000","last_fullsized":"2024-09-12T12:24:00.636665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:14:31.576588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392280+0000","last_change":"2024-09-12T12:23:58.392638+0000","last_active":"2024-09-12T12:23:59.392280+0000","last_peered":"2024-09-12T12:23:59.392280+0000","last_clean":"2024-09-12T12:23:59.392280+0000","last_became_active":"2024-09-12T12:23:58.392083+0000","last_became_peered":"2024-09-12T12:23:58.392083+0000","last_unstale":"2024-09-12T12:23:59.392280+0000","last_undegraded":"2024-09-12T12:23:59.392280+0000","last_fullsized":"2024-09-12T12:23:59.392280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:10:20.142471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392234+0000","last_change":"2024-09-12T12:23:58.412797+0000","last_active":"2024-09-12T12:23:59.392234+0000","last_peered":"2024-09-12T12:23:59.392234+0000","last_clean":"2024-09-12T12:23:59.392234+0000","last_became_active":"2024-09-12T12:23:58.411231+0000","last_became_peered":"2024-09-12T12:23:58.411231+0000","last_unstale":"2024-09-12T12:23:59.392234+0000","last_undegraded":"2024-09-12T12:23:59.392234+0000","last_fullsized":"2024-09-12T12:23:59.392234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:52:54.437455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.634853+0000","last_change":"2024-09-12T12:23:58.378166+0000","last_active":"2024-09-12T12:24:00.634853+0000","last_peered":"2024-09-12T12:24:00.634853+0000","last_clean":"2024-09-12T12:24:00.634853+0000","last_became_active":"2024-09-12T12:23:58.377393+0000","last_became_peered":"2024-09-12T12:23:58.377393+0000","last_unstale":"2024-09-12T12:24:00.634853+0000","last_undegraded":"2024-09-12T12:24:00.634853+0000","last_fullsized":"2024-09-12T12:24:00.634853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:41.164230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393348+0000","last_change":"2024-09-12T12:23:58.416477+0000","last_active":"2024-09-12T12:23:59.393348+0000","last_peered":"2024-09-12T12:23:59.393348+0000","last_clean":"2024-09-12T12:23:59.393348+0000","last_became_active":"2024-09-12T12:23:58.416249+0000","last_became_peered":"2024-09-12T12:23:58.416249+0000","last_unstale":"2024-09-12T12:23:59.393348+0000","last_undegraded":"2024-09-12T12:23:59.393348+0000","last_fullsized":"2024-09-12T12:23:59.393348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:15.188997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639055+0000","last_change":"2024-09-12T12:23:58.395622+0000","last_active":"2024-09-12T12:24:00.639055+0000","last_peered":"2024-09-12T12:24:00.639055+0000","last_clean":"2024-09-12T12:24:00.639055+0000","last_became_active":"2024-09-12T12:23:58.394982+0000","last_became_peered":"2024-09-12T12:23:58.394982+0000","last_unstale":"2024-09-12T12:24:00.639055+0000","last_undegraded":"2024-09-12T12:24:00.639055+0000","last_fullsized":"2024-09-12T12:24:00.639055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:18.727703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639493+0000","last_change":"2024-09-12T12:23:58.384692+0000","last_active":"2024-09-12T12:24:00.639493+0000","last_peered":"2024-09-12T12:24:00.639493+0000","last_clean":"2024-09-12T12:24:00.639493+0000","last_became_active":"2024-09-12T12:23:58.384001+0000","last_became_peered":"2024-09-12T12:23:58.384001+0000","last_unstale":"2024-09-12T12:24:00.639493+0000","last_undegraded":"2024-09-12T12:24:00.639493+0000","last_fullsized":"2024-09-12T12:24:00.639493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:16.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393246+0000","last_change":"2024-09-12T12:23:58.407174+0000","last_active":"2024-09-12T12:23:59.393246+0000","last_peered":"2024-09-12T12:23:59.393246+0000","last_clean":"2024-09-12T12:23:59.393246+0000","last_became_active":"2024-09-12T12:23:58.406429+0000","last_became_peered":"2024-09-12T12:23:58.406429+0000","last_unstale":"2024-09-12T12:23:59.393246+0000","last_undegraded":"2024-09-12T12:23:59.393246+0000","last_fullsized":"2024-09-12T12:23:59.393246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:32:37.144121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392309+0000","last_change":"2024-09-12T12:23:58.393242+0000","last_active":"2024-09-12T12:23:59.392309+0000","last_peered":"2024-09-12T12:23:59.392309+0000","last_clean":"2024-09-12T12:23:59.392309+0000","last_became_active":"2024-09-12T12:23:58.391479+0000","last_became_peered":"2024-09-12T12:23:58.391479+0000","last_unstale":"2024-09-12T12:23:59.392309+0000","last_undegraded":"2024-09-12T12:23:59.392309+0000","last_fullsized":"2024-09-12T12:23:59.392309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:50.943216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392431+0000","last_change":"2024-09-12T12:23:58.391885+0000","last_active":"2024-09-12T12:23:59.392431+0000","last_peered":"2024-09-12T12:23:59.392431+0000","last_clean":"2024-09-12T12:23:59.392431+0000","last_became_active":"2024-09-12T12:23:58.391554+0000","last_became_peered":"2024-09-12T12:23:58.391554+0000","last_unstale":"2024-09-12T12:23:59.392431+0000","last_undegraded":"2024-09-12T12:23:59.392431+0000","last_fullsized":"2024-09-12T12:23:59.392431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:06.323352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638373+0000","last_change":"2024-09-12T12:23:58.374951+0000","last_active":"2024-09-12T12:24:00.638373+0000","last_peered":"2024-09-12T12:24:00.638373+0000","last_clean":"2024-09-12T12:24:00.638373+0000","last_became_active":"2024-09-12T12:23:58.374134+0000","last_became_peered":"2024-09-12T12:23:58.374134+0000","last_unstale":"2024-09-12T12:24:00.638373+0000","last_undegraded":"2024-09-12T12:24:00.638373+0000","last_fullsized":"2024-09-12T12:24:00.638373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:12.842154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392079+0000","last_change":"2024-09-12T12:23:58.415959+0000","last_active":"2024-09-12T12:23:59.392079+0000","last_peered":"2024-09-12T12:23:59.392079+0000","last_clean":"2024-09-12T12:23:59.392079+0000","last_became_active":"2024-09-12T12:23:58.412091+0000","last_became_peered":"2024-09-12T12:23:58.412091+0000","last_unstale":"2024-09-12T12:23:59.392079+0000","last_undegraded":"2024-09-12T12:23:59.392079+0000","last_fullsized":"2024-09-12T12:23:59.392079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:18.770724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639576+0000","last_change":"2024-09-12T12:23:58.399145+0000","last_active":"2024-09-12T12:24:00.639576+0000","last_peered":"2024-09-12T12:24:00.639576+0000","last_clean":"2024-09-12T12:24:00.639576+0000","last_became_active":"2024-09-12T12:23:58.398277+0000","last_became_peered":"2024-09-12T12:23:58.398277+0000","last_unstale":"2024-09-12T12:24:00.639576+0000","last_undegraded":"2024-09-12T12:24:00.639576+0000","last_fullsized":"2024-09-12T12:24:00.639576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:54.581603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393769+0000","last_change":"2024-09-12T12:23:57.391198+0000","last_active":"2024-09-12T12:23:59.393769+0000","last_peered":"2024-09-12T12:23:59.393769+0000","last_clean":"2024-09-12T12:23:59.393769+0000","last_became_active":"2024-09-12T12:23:57.390552+0000","last_became_peered":"2024-09-12T12:23:57.390552+0000","last_unstale":"2024-09-12T12:23:59.393769+0000","last_undegraded":"2024-09-12T12:23:59.393769+0000","last_fullsized":"2024-09-12T12:23:59.393769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:49.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393493+0000","last_change":"2024-09-12T12:23:58.398834+0000","last_active":"2024-09-12T12:23:59.393493+0000","last_peered":"2024-09-12T12:23:59.393493+0000","last_clean":"2024-09-12T12:23:59.393493+0000","last_became_active":"2024-09-12T12:23:58.398012+0000","last_became_peered":"2024-09-12T12:23:58.398012+0000","last_unstale":"2024-09-12T12:23:59.393493+0000","last_undegraded":"2024-09-12T12:23:59.393493+0000","last_fullsized":"2024-09-12T12:23:59.393493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:14:12.942908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639904+0000","last_change":"2024-09-12T12:23:57.367632+0000","last_active":"2024-09-12T12:24:00.639904+0000","last_peered":"2024-09-12T12:24:00.639904+0000","last_clean":"2024-09-12T12:24:00.639904+0000","last_became_active":"2024-09-12T12:23:57.366903+0000","last_became_peered":"2024-09-12T12:23:57.366903+0000","last_unstale":"2024-09-12T12:24:00.639904+0000","last_undegraded":"2024-09-12T12:24:00.639904+0000","last_fullsized":"2024-09-12T12:24:00.639904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:27.962118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392829+0000","last_change":"2024-09-12T12:23:58.401731+0000","last_active":"2024-09-12T12:23:59.392829+0000","last_peered":"2024-09-12T12:23:59.392829+0000","last_clean":"2024-09-12T12:23:59.392829+0000","last_became_active":"2024-09-12T12:23:58.395518+0000","last_became_peered":"2024-09-12T12:23:58.395518+0000","last_unstale":"2024-09-12T12:23:59.392829+0000","last_undegraded":"2024-09-12T12:23:59.392829+0000","last_fullsized":"2024-09-12T12:23:59.392829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:39.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395705+0000","last_change":"2024-09-12T12:23:57.368053+0000","last_active":"2024-09-12T12:23:59.395705+0000","last_peered":"2024-09-12T12:23:59.395705+0000","last_clean":"2024-09-12T12:23:59.395705+0000","last_became_active":"2024-09-12T12:23:57.367668+0000","last_became_peered":"2024-09-12T12:23:57.367668+0000","last_unstale":"2024-09-12T12:23:59.395705+0000","last_undegraded":"2024-09-12T12:23:59.395705+0000","last_fullsized":"2024-09-12T12:23:59.395705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:51:00.133402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638622+0000","last_change":"2024-09-12T12:23:58.381382+0000","last_active":"2024-09-12T12:24:00.638622+0000","last_peered":"2024-09-12T12:24:00.638622+0000","last_clean":"2024-09-12T12:24:00.638622+0000","last_became_active":"2024-09-12T12:23:58.380618+0000","last_became_peered":"2024-09-12T12:23:58.380618+0000","last_unstale":"2024-09-12T12:24:00.638622+0000","last_undegraded":"2024-09-12T12:24:00.638622+0000","last_fullsized":"2024-09-12T12:24:00.638622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:06:03.030474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396288+0000","last_change":"2024-09-12T12:23:57.368958+0000","last_active":"2024-09-12T12:23:59.396288+0000","last_peered":"2024-09-12T12:23:59.396288+0000","last_clean":"2024-09-12T12:23:59.396288+0000","last_became_active":"2024-09-12T12:23:57.368562+0000","last_became_peered":"2024-09-12T12:23:57.368562+0000","last_unstale":"2024-09-12T12:23:59.396288+0000","last_undegraded":"2024-09-12T12:23:59.396288+0000","last_fullsized":"2024-09-12T12:23:59.396288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:58.364890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392575+0000","last_change":"2024-09-12T12:23:58.391783+0000","last_active":"2024-09-12T12:23:59.392575+0000","last_peered":"2024-09-12T12:23:59.392575+0000","last_clean":"2024-09-12T12:23:59.392575+0000","last_became_active":"2024-09-12T12:23:58.391447+0000","last_became_peered":"2024-09-12T12:23:58.391447+0000","last_unstale":"2024-09-12T12:23:59.392575+0000","last_undegraded":"2024-09-12T12:23:59.392575+0000","last_fullsized":"2024-09-12T12:23:59.392575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:40.947465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.640107+0000","last_change":"2024-09-12T12:23:57.371998+0000","last_active":"2024-09-12T12:24:00.640107+0000","last_peered":"2024-09-12T12:24:00.640107+0000","last_clean":"2024-09-12T12:24:00.640107+0000","last_became_active":"2024-09-12T12:23:57.371152+0000","last_became_peered":"2024-09-12T12:23:57.371152+0000","last_unstale":"2024-09-12T12:24:00.640107+0000","last_undegraded":"2024-09-12T12:24:00.640107+0000","last_fullsized":"2024-09-12T12:24:00.640107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:23.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639558+0000","last_change":"2024-09-12T12:23:58.377812+0000","last_active":"2024-09-12T12:24:00.639558+0000","last_peered":"2024-09-12T12:24:00.639558+0000","last_clean":"2024-09-12T12:24:00.639558+0000","last_became_active":"2024-09-12T12:23:58.377054+0000","last_became_peered":"2024-09-12T12:23:58.377054+0000","last_unstale":"2024-09-12T12:24:00.639558+0000","last_undegraded":"2024-09-12T12:24:00.639558+0000","last_fullsized":"2024-09-12T12:24:00.639558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:39.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638568+0000","last_change":"2024-09-12T12:23:57.366528+0000","last_active":"2024-09-12T12:24:00.638568+0000","last_peered":"2024-09-12T12:24:00.638568+0000","last_clean":"2024-09-12T12:24:00.638568+0000","last_became_active":"2024-09-12T12:23:57.365702+0000","last_became_peered":"2024-09-12T12:23:57.365702+0000","last_unstale":"2024-09-12T12:24:00.638568+0000","last_undegraded":"2024-09-12T12:24:00.638568+0000","last_fullsized":"2024-09-12T12:24:00.638568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:49.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637778+0000","last_change":"2024-09-12T12:23:58.392087+0000","last_active":"2024-09-12T12:24:00.637778+0000","last_peered":"2024-09-12T12:24:00.637778+0000","last_clean":"2024-09-12T12:24:00.637778+0000","last_became_active":"2024-09-12T12:23:58.391215+0000","last_became_peered":"2024-09-12T12:23:58.391215+0000","last_unstale":"2024-09-12T12:24:00.637778+0000","last_undegraded":"2024-09-12T12:24:00.637778+0000","last_fullsized":"2024-09-12T12:24:00.637778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:38.102031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395360+0000","last_change":"2024-09-12T12:23:57.375695+0000","last_active":"2024-09-12T12:23:59.395360+0000","last_peered":"2024-09-12T12:23:59.395360+0000","last_clean":"2024-09-12T12:23:59.395360+0000","last_became_active":"2024-09-12T12:23:57.375373+0000","last_became_peered":"2024-09-12T12:23:57.375373+0000","last_unstale":"2024-09-12T12:23:59.395360+0000","last_undegraded":"2024-09-12T12:23:59.395360+0000","last_fullsized":"2024-09-12T12:23:59.395360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:09.585468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.382382+0000","last_change":"2024-09-12T12:23:58.377377+0000","last_active":"2024-09-12T12:23:58.382382+0000","last_peered":"2024-09-12T12:23:58.382382+0000","last_clean":"2024-09-12T12:23:58.382382+0000","last_became_active":"2024-09-12T12:23:58.374901+0000","last_became_peered":"2024-09-12T12:23:58.374901+0000","last_unstale":"2024-09-12T12:23:58.382382+0000","last_undegraded":"2024-09-12T12:23:58.382382+0000","last_fullsized":"2024-09-12T12:23:58.382382+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:31:05.911408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392198+0000","last_change":"2024-09-12T12:23:57.370326+0000","last_active":"2024-09-12T12:23:59.392198+0000","last_peered":"2024-09-12T12:23:59.392198+0000","last_clean":"2024-09-12T12:23:59.392198+0000","last_became_active":"2024-09-12T12:23:57.369836+0000","last_became_peered":"2024-09-12T12:23:57.369836+0000","last_unstale":"2024-09-12T12:23:59.392198+0000","last_undegraded":"2024-09-12T12:23:59.392198+0000","last_fullsized":"2024-09-12T12:23:59.392198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:30.448876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.400544+0000","last_change":"2024-09-12T12:23:58.395462+0000","last_active":"2024-09-12T12:23:58.400544+0000","last_peered":"2024-09-12T12:23:58.400544+0000","last_clean":"2024-09-12T12:23:58.400544+0000","last_became_active":"2024-09-12T12:23:58.395205+0000","last_became_peered":"2024-09-12T12:23:58.395205+0000","last_unstale":"2024-09-12T12:23:58.400544+0000","last_undegraded":"2024-09-12T12:23:58.400544+0000","last_fullsized":"2024-09-12T12:23:58.400544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:11:22.945497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.394044+0000","last_change":"2024-09-12T12:23:57.390954+0000","last_active":"2024-09-12T12:23:59.394044+0000","last_peered":"2024-09-12T12:23:59.394044+0000","last_clean":"2024-09-12T12:23:59.394044+0000","last_became_active":"2024-09-12T12:23:57.389910+0000","last_became_peered":"2024-09-12T12:23:57.389910+0000","last_unstale":"2024-09-12T12:23:59.394044+0000","last_undegraded":"2024-09-12T12:23:59.394044+0000","last_fullsized":"2024-09-12T12:23:59.394044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:05.313097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.391219+0000","last_change":"2024-09-12T12:23:58.380267+0000","last_active":"2024-09-12T12:23:59.391219+0000","last_peered":"2024-09-12T12:23:59.391219+0000","last_clean":"2024-09-12T12:23:59.391219+0000","last_became_active":"2024-09-12T12:23:58.379518+0000","last_became_peered":"2024-09-12T12:23:58.379518+0000","last_unstale":"2024-09-12T12:23:59.391219+0000","last_undegraded":"2024-09-12T12:23:59.391219+0000","last_fullsized":"2024-09-12T12:23:59.391219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:16.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393100+0000","last_change":"2024-09-12T12:23:57.387882+0000","last_active":"2024-09-12T12:23:59.393100+0000","last_peered":"2024-09-12T12:23:59.393100+0000","last_clean":"2024-09-12T12:23:59.393100+0000","last_became_active":"2024-09-12T12:23:57.387183+0000","last_became_peered":"2024-09-12T12:23:57.387183+0000","last_unstale":"2024-09-12T12:23:59.393100+0000","last_undegraded":"2024-09-12T12:23:59.393100+0000","last_fullsized":"2024-09-12T12:23:59.393100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:10:19.132216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396219+0000","last_change":"2024-09-12T12:23:58.411509+0000","last_active":"2024-09-12T12:23:59.396219+0000","last_peered":"2024-09-12T12:23:59.396219+0000","last_clean":"2024-09-12T12:23:59.396219+0000","last_became_active":"2024-09-12T12:23:58.411275+0000","last_became_peered":"2024-09-12T12:23:58.411275+0000","last_unstale":"2024-09-12T12:23:59.396219+0000","last_undegraded":"2024-09-12T12:23:59.396219+0000","last_fullsized":"2024-09-12T12:23:59.396219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:22.858881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392617+0000","last_change":"2024-09-12T12:23:57.381940+0000","last_active":"2024-09-12T12:23:59.392617+0000","last_peered":"2024-09-12T12:23:59.392617+0000","last_clean":"2024-09-12T12:23:59.392617+0000","last_became_active":"2024-09-12T12:23:57.380630+0000","last_became_peered":"2024-09-12T12:23:57.380630+0000","last_unstale":"2024-09-12T12:23:59.392617+0000","last_undegraded":"2024-09-12T12:23:59.392617+0000","last_fullsized":"2024-09-12T12:23:59.392617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:50.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639986+0000","last_change":"2024-09-12T12:23:58.393498+0000","last_active":"2024-09-12T12:24:00.639986+0000","last_peered":"2024-09-12T12:24:00.639986+0000","last_clean":"2024-09-12T12:24:00.639986+0000","last_became_active":"2024-09-12T12:23:58.392463+0000","last_became_peered":"2024-09-12T12:23:58.392463+0000","last_unstale":"2024-09-12T12:24:00.639986+0000","last_undegraded":"2024-09-12T12:24:00.639986+0000","last_fullsized":"2024-09-12T12:24:00.639986+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:15.001374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390791+0000","last_change":"2024-09-12T12:23:57.377798+0000","last_active":"2024-09-12T12:23:59.390791+0000","last_peered":"2024-09-12T12:23:59.390791+0000","last_clean":"2024-09-12T12:23:59.390791+0000","last_became_active":"2024-09-12T12:23:57.376811+0000","last_became_peered":"2024-09-12T12:23:57.376811+0000","last_unstale":"2024-09-12T12:23:59.390791+0000","last_undegraded":"2024-09-12T12:23:59.390791+0000","last_fullsized":"2024-09-12T12:23:59.390791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:27.962118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395505+0000","last_change":"2024-09-12T12:23:58.374150+0000","last_active":"2024-09-12T12:23:59.395505+0000","last_peered":"2024-09-12T12:23:59.395505+0000","last_clean":"2024-09-12T12:23:59.395505+0000","last_became_active":"2024-09-12T12:23:58.373423+0000","last_became_peered":"2024-09-12T12:23:58.373423+0000","last_unstale":"2024-09-12T12:23:59.395505+0000","last_undegraded":"2024-09-12T12:23:59.395505+0000","last_fullsized":"2024-09-12T12:23:59.395505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:10.956156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392286+0000","last_change":"2024-09-12T12:23:57.372742+0000","last_active":"2024-09-12T12:23:59.392286+0000","last_peered":"2024-09-12T12:23:59.392286+0000","last_clean":"2024-09-12T12:23:59.392286+0000","last_became_active":"2024-09-12T12:23:57.372354+0000","last_became_peered":"2024-09-12T12:23:57.372354+0000","last_unstale":"2024-09-12T12:23:59.392286+0000","last_undegraded":"2024-09-12T12:23:59.392286+0000","last_fullsized":"2024-09-12T12:23:59.392286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:14:55.671206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638433+0000","last_change":"2024-09-12T12:23:58.391541+0000","last_active":"2024-09-12T12:24:00.638433+0000","last_peered":"2024-09-12T12:24:00.638433+0000","last_clean":"2024-09-12T12:24:00.638433+0000","last_became_active":"2024-09-12T12:23:58.390938+0000","last_became_peered":"2024-09-12T12:23:58.390938+0000","last_unstale":"2024-09-12T12:24:00.638433+0000","last_undegraded":"2024-09-12T12:24:00.638433+0000","last_fullsized":"2024-09-12T12:24:00.638433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:43.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.364221+0000","last_change":"2024-09-12T12:23:57.367013+0000","last_active":"2024-09-12T12:23:58.364221+0000","last_peered":"2024-09-12T12:23:58.364221+0000","last_clean":"2024-09-12T12:23:58.364221+0000","last_became_active":"2024-09-12T12:23:57.366614+0000","last_became_peered":"2024-09-12T12:23:57.366614+0000","last_unstale":"2024-09-12T12:23:58.364221+0000","last_undegraded":"2024-09-12T12:23:58.364221+0000","last_fullsized":"2024-09-12T12:23:58.364221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:16:37.971791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392098+0000","last_change":"2024-09-12T12:23:58.391761+0000","last_active":"2024-09-12T12:23:59.392098+0000","last_peered":"2024-09-12T12:23:59.392098+0000","last_clean":"2024-09-12T12:23:59.392098+0000","last_became_active":"2024-09-12T12:23:58.391272+0000","last_became_peered":"2024-09-12T12:23:58.391272+0000","last_unstale":"2024-09-12T12:23:59.392098+0000","last_undegraded":"2024-09-12T12:23:59.392098+0000","last_fullsized":"2024-09-12T12:23:59.392098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:22.494179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390380+0000","last_change":"2024-09-12T12:23:57.385811+0000","last_active":"2024-09-12T12:23:59.390380+0000","last_peered":"2024-09-12T12:23:59.390380+0000","last_clean":"2024-09-12T12:23:59.390380+0000","last_became_active":"2024-09-12T12:23:57.384813+0000","last_became_peered":"2024-09-12T12:23:57.384813+0000","last_unstale":"2024-09-12T12:23:59.390380+0000","last_undegraded":"2024-09-12T12:23:59.390380+0000","last_fullsized":"2024-09-12T12:23:59.390380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:14.178742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392042+0000","last_change":"2024-09-12T12:23:58.415030+0000","last_active":"2024-09-12T12:23:59.392042+0000","last_peered":"2024-09-12T12:23:59.392042+0000","last_clean":"2024-09-12T12:23:59.392042+0000","last_became_active":"2024-09-12T12:23:58.414677+0000","last_became_peered":"2024-09-12T12:23:58.414677+0000","last_unstale":"2024-09-12T12:23:59.392042+0000","last_undegraded":"2024-09-12T12:23:59.392042+0000","last_fullsized":"2024-09-12T12:23:59.392042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:51.942684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395215+0000","last_change":"2024-09-12T12:23:57.385905+0000","last_active":"2024-09-12T12:23:59.395215+0000","last_peered":"2024-09-12T12:23:59.395215+0000","last_clean":"2024-09-12T12:23:59.395215+0000","last_became_active":"2024-09-12T12:23:57.385555+0000","last_became_peered":"2024-09-12T12:23:57.385555+0000","last_unstale":"2024-09-12T12:23:59.395215+0000","last_undegraded":"2024-09-12T12:23:59.395215+0000","last_fullsized":"2024-09-12T12:23:59.395215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:22.740585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639455+0000","last_change":"2024-09-12T12:23:58.380280+0000","last_active":"2024-09-12T12:24:00.639455+0000","last_peered":"2024-09-12T12:24:00.639455+0000","last_clean":"2024-09-12T12:24:00.639455+0000","last_became_active":"2024-09-12T12:23:58.379576+0000","last_became_peered":"2024-09-12T12:23:58.379576+0000","last_unstale":"2024-09-12T12:24:00.639455+0000","last_undegraded":"2024-09-12T12:24:00.639455+0000","last_fullsized":"2024-09-12T12:24:00.639455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:12:20.884415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.394285+0000","last_change":"2024-09-12T12:23:57.390472+0000","last_active":"2024-09-12T12:23:59.394285+0000","last_peered":"2024-09-12T12:23:59.394285+0000","last_clean":"2024-09-12T12:23:59.394285+0000","last_became_active":"2024-09-12T12:23:57.389725+0000","last_became_peered":"2024-09-12T12:23:57.389725+0000","last_unstale":"2024-09-12T12:23:59.394285+0000","last_undegraded":"2024-09-12T12:23:59.394285+0000","last_fullsized":"2024-09-12T12:23:59.394285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:39.937210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393846+0000","last_change":"2024-09-12T12:23:58.398413+0000","last_active":"2024-09-12T12:23:59.393846+0000","last_peered":"2024-09-12T12:23:59.393846+0000","last_clean":"2024-09-12T12:23:59.393846+0000","last_became_active":"2024-09-12T12:23:58.397352+0000","last_became_peered":"2024-09-12T12:23:58.397352+0000","last_unstale":"2024-09-12T12:23:59.393846+0000","last_undegraded":"2024-09-12T12:23:59.393846+0000","last_fullsized":"2024-09-12T12:23:59.393846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:03:56.610591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395396+0000","last_change":"2024-09-12T12:23:57.378862+0000","last_active":"2024-09-12T12:23:59.395396+0000","last_peered":"2024-09-12T12:23:59.395396+0000","last_clean":"2024-09-12T12:23:59.395396+0000","last_became_active":"2024-09-12T12:23:57.378463+0000","last_became_peered":"2024-09-12T12:23:57.378463+0000","last_unstale":"2024-09-12T12:23:59.395396+0000","last_undegraded":"2024-09-12T12:23:59.395396+0000","last_fullsized":"2024-09-12T12:23:59.395396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:05.881468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639984+0000","last_change":"2024-09-12T12:23:58.398669+0000","last_active":"2024-09-12T12:24:00.639984+0000","last_peered":"2024-09-12T12:24:00.639984+0000","last_clean":"2024-09-12T12:24:00.639984+0000","last_became_active":"2024-09-12T12:23:58.397659+0000","last_became_peered":"2024-09-12T12:23:58.397659+0000","last_unstale":"2024-09-12T12:24:00.639984+0000","last_undegraded":"2024-09-12T12:24:00.639984+0000","last_fullsized":"2024-09-12T12:24:00.639984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:09:55.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396340+0000","last_change":"2024-09-12T12:23:57.377815+0000","last_active":"2024-09-12T12:23:59.396340+0000","last_peered":"2024-09-12T12:23:59.396340+0000","last_clean":"2024-09-12T12:23:59.396340+0000","last_became_active":"2024-09-12T12:23:57.377401+0000","last_became_peered":"2024-09-12T12:23:57.377401+0000","last_unstale":"2024-09-12T12:23:59.396340+0000","last_undegraded":"2024-09-12T12:23:59.396340+0000","last_fullsized":"2024-09-12T12:23:59.396340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:06.342439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635769+0000","last_change":"2024-09-12T12:23:58.393822+0000","last_active":"2024-09-12T12:24:00.635769+0000","last_peered":"2024-09-12T12:24:00.635769+0000","last_clean":"2024-09-12T12:24:00.635769+0000","last_became_active":"2024-09-12T12:23:58.393144+0000","last_became_peered":"2024-09-12T12:23:58.393144+0000","last_unstale":"2024-09-12T12:24:00.635769+0000","last_undegraded":"2024-09-12T12:24:00.635769+0000","last_fullsized":"2024-09-12T12:24:00.635769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:19.984058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"78'2","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.403505+0000","last_change":"2024-09-12T12:23:57.367701+0000","last_active":"2024-09-12T12:23:58.403505+0000","last_peered":"2024-09-12T12:23:58.403505+0000","last_clean":"2024-09-12T12:23:58.403505+0000","last_became_active":"2024-09-12T12:23:57.367397+0000","last_became_peered":"2024-09-12T12:23:57.367397+0000","last_unstale":"2024-09-12T12:23:58.403505+0000","last_undegraded":"2024-09-12T12:23:58.403505+0000","last_fullsized":"2024-09-12T12:23:58.403505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:26:48.411535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.391045+0000","last_change":"2024-09-12T12:23:58.388342+0000","last_active":"2024-09-12T12:23:59.391045+0000","last_peered":"2024-09-12T12:23:59.391045+0000","last_clean":"2024-09-12T12:23:59.391045+0000","last_became_active":"2024-09-12T12:23:58.387470+0000","last_became_peered":"2024-09-12T12:23:58.387470+0000","last_unstale":"2024-09-12T12:23:59.391045+0000","last_undegraded":"2024-09-12T12:23:59.391045+0000","last_fullsized":"2024-09-12T12:23:59.391045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:52:54.437455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.636539+0000","last_change":"2024-09-12T12:23:57.365147+0000","last_active":"2024-09-12T12:24:00.636539+0000","last_peered":"2024-09-12T12:24:00.636539+0000","last_clean":"2024-09-12T12:24:00.636539+0000","last_became_active":"2024-09-12T12:23:57.364796+0000","last_became_peered":"2024-09-12T12:23:57.364796+0000","last_unstale":"2024-09-12T12:24:00.636539+0000","last_undegraded":"2024-09-12T12:24:00.636539+0000","last_fullsized":"2024-09-12T12:24:00.636539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:10:39.896309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635247+0000","last_change":"2024-09-12T12:23:58.386311+0000","last_active":"2024-09-12T12:24:00.635247+0000","last_peered":"2024-09-12T12:24:00.635247+0000","last_clean":"2024-09-12T12:24:00.635247+0000","last_became_active":"2024-09-12T12:23:58.385942+0000","last_became_peered":"2024-09-12T12:23:58.385942+0000","last_unstale":"2024-09-12T12:24:00.635247+0000","last_undegraded":"2024-09-12T12:24:00.635247+0000","last_fullsized":"2024-09-12T12:24:00.635247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:53:18.140011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.634866+0000","last_change":"2024-09-12T12:23:57.386864+0000","last_active":"2024-09-12T12:24:00.634866+0000","last_peered":"2024-09-12T12:24:00.634866+0000","last_clean":"2024-09-12T12:24:00.634866+0000","last_became_active":"2024-09-12T12:23:57.386550+0000","last_became_peered":"2024-09-12T12:23:57.386550+0000","last_unstale":"2024-09-12T12:24:00.634866+0000","last_undegraded":"2024-09-12T12:24:00.634866+0000","last_fullsized":"2024-09-12T12:24:00.634866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:12:10.152000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390713+0000","last_change":"2024-09-12T12:23:58.374811+0000","last_active":"2024-09-12T12:23:59.390713+0000","last_peered":"2024-09-12T12:23:59.390713+0000","last_clean":"2024-09-12T12:23:59.390713+0000","last_became_active":"2024-09-12T12:23:58.373952+0000","last_became_peered":"2024-09-12T12:23:58.373952+0000","last_unstale":"2024-09-12T12:23:59.390713+0000","last_undegraded":"2024-09-12T12:23:59.390713+0000","last_fullsized":"2024-09-12T12:23:59.390713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:52.908148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-12T12:23:58.360414+0000","last_change":"2024-09-12T12:23:57.368319+0000","last_active":"2024-09-12T12:23:58.360414+0000","last_peered":"2024-09-12T12:23:58.360414+0000","last_clean":"2024-09-12T12:23:58.360414+0000","last_became_active":"2024-09-12T12:23:57.367912+0000","last_became_peered":"2024-09-12T12:23:57.367912+0000","last_unstale":"2024-09-12T12:23:58.360414+0000","last_undegraded":"2024-09-12T12:23:58.360414+0000","last_fullsized":"2024-09-12T12:23:58.360414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:35.720409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639813+0000","last_change":"2024-09-12T12:23:58.379545+0000","last_active":"2024-09-12T12:24:00.639813+0000","last_peered":"2024-09-12T12:24:00.639813+0000","last_clean":"2024-09-12T12:24:00.639813+0000","last_became_active":"2024-09-12T12:23:58.378816+0000","last_became_peered":"2024-09-12T12:23:58.378816+0000","last_unstale":"2024-09-12T12:24:00.639813+0000","last_undegraded":"2024-09-12T12:24:00.639813+0000","last_fullsized":"2024-09-12T12:24:00.639813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:32:37.144121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393386+0000","last_change":"2024-09-12T12:23:57.386214+0000","last_active":"2024-09-12T12:23:59.393386+0000","last_peered":"2024-09-12T12:23:59.393386+0000","last_clean":"2024-09-12T12:23:59.393386+0000","last_became_active":"2024-09-12T12:23:57.385756+0000","last_became_peered":"2024-09-12T12:23:57.385756+0000","last_unstale":"2024-09-12T12:23:59.393386+0000","last_undegraded":"2024-09-12T12:23:59.393386+0000","last_fullsized":"2024-09-12T12:23:59.393386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:50.660652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638459+0000","last_change":"2024-09-12T12:23:58.400966+0000","last_active":"2024-09-12T12:24:00.638459+0000","last_peered":"2024-09-12T12:24:00.638459+0000","last_clean":"2024-09-12T12:24:00.638459+0000","last_became_active":"2024-09-12T12:23:58.400073+0000","last_became_peered":"2024-09-12T12:23:58.400073+0000","last_unstale":"2024-09-12T12:24:00.638459+0000","last_undegraded":"2024-09-12T12:24:00.638459+0000","last_fullsized":"2024-09-12T12:24:00.638459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:47.090366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393431+0000","last_change":"2024-09-12T12:23:57.369090+0000","last_active":"2024-09-12T12:23:59.393431+0000","last_peered":"2024-09-12T12:23:59.393431+0000","last_clean":"2024-09-12T12:23:59.393431+0000","last_became_active":"2024-09-12T12:23:57.368802+0000","last_became_peered":"2024-09-12T12:23:57.368802+0000","last_unstale":"2024-09-12T12:23:59.393431+0000","last_undegraded":"2024-09-12T12:23:59.393431+0000","last_fullsized":"2024-09-12T12:23:59.393431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:38.077870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393429+0000","last_change":"2024-09-12T12:23:58.402393+0000","last_active":"2024-09-12T12:23:59.393429+0000","last_peered":"2024-09-12T12:23:59.393429+0000","last_clean":"2024-09-12T12:23:59.393429+0000","last_became_active":"2024-09-12T12:23:58.399806+0000","last_became_peered":"2024-09-12T12:23:58.399806+0000","last_unstale":"2024-09-12T12:23:59.393429+0000","last_undegraded":"2024-09-12T12:23:59.393429+0000","last_fullsized":"2024-09-12T12:23:59.393429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:15.001374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393093+0000","last_change":"2024-09-12T12:23:57.382433+0000","last_active":"2024-09-12T12:23:59.393093+0000","last_peered":"2024-09-12T12:23:59.393093+0000","last_clean":"2024-09-12T12:23:59.393093+0000","last_became_active":"2024-09-12T12:23:57.381303+0000","last_became_peered":"2024-09-12T12:23:57.381303+0000","last_unstale":"2024-09-12T12:23:59.393093+0000","last_undegraded":"2024-09-12T12:23:59.393093+0000","last_fullsized":"2024-09-12T12:23:59.393093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:21.483924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.640380+0000","last_change":"2024-09-12T12:23:58.393099+0000","last_active":"2024-09-12T12:24:00.640380+0000","last_peered":"2024-09-12T12:24:00.640380+0000","last_clean":"2024-09-12T12:24:00.640380+0000","last_became_active":"2024-09-12T12:23:58.392406+0000","last_became_peered":"2024-09-12T12:23:58.392406+0000","last_unstale":"2024-09-12T12:24:00.640380+0000","last_undegraded":"2024-09-12T12:24:00.640380+0000","last_fullsized":"2024-09-12T12:24:00.640380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:14:12.942908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393467+0000","last_change":"2024-09-12T12:23:57.382205+0000","last_active":"2024-09-12T12:23:59.393467+0000","last_peered":"2024-09-12T12:23:59.393467+0000","last_clean":"2024-09-12T12:23:59.393467+0000","last_became_active":"2024-09-12T12:23:57.381644+0000","last_became_peered":"2024-09-12T12:23:57.381644+0000","last_unstale":"2024-09-12T12:23:59.393467+0000","last_undegraded":"2024-09-12T12:23:59.393467+0000","last_fullsized":"2024-09-12T12:23:59.393467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:23.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390832+0000","last_change":"2024-09-12T12:23:58.384197+0000","last_active":"2024-09-12T12:23:59.390832+0000","last_peered":"2024-09-12T12:23:59.390832+0000","last_clean":"2024-09-12T12:23:59.390832+0000","last_became_active":"2024-09-12T12:23:58.383235+0000","last_became_peered":"2024-09-12T12:23:58.383235+0000","last_unstale":"2024-09-12T12:23:59.390832+0000","last_undegraded":"2024-09-12T12:23:59.390832+0000","last_fullsized":"2024-09-12T12:23:59.390832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:12.842154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396013+0000","last_change":"2024-09-12T12:23:57.381829+0000","last_active":"2024-09-12T12:23:59.396013+0000","last_peered":"2024-09-12T12:23:59.396013+0000","last_clean":"2024-09-12T12:23:59.396013+0000","last_became_active":"2024-09-12T12:23:57.381422+0000","last_became_peered":"2024-09-12T12:23:57.381422+0000","last_unstale":"2024-09-12T12:23:59.396013+0000","last_undegraded":"2024-09-12T12:23:59.396013+0000","last_fullsized":"2024-09-12T12:23:59.396013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:10.834460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395928+0000","last_change":"2024-09-12T12:23:58.378283+0000","last_active":"2024-09-12T12:23:59.395928+0000","last_peered":"2024-09-12T12:23:59.395928+0000","last_clean":"2024-09-12T12:23:59.395928+0000","last_became_active":"2024-09-12T12:23:58.377451+0000","last_became_peered":"2024-09-12T12:23:58.377451+0000","last_unstale":"2024-09-12T12:23:59.395928+0000","last_undegraded":"2024-09-12T12:23:59.395928+0000","last_fullsized":"2024-09-12T12:23:59.395928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:11:22.657305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638615+0000","last_change":"2024-09-12T12:23:57.366499+0000","last_active":"2024-09-12T12:24:00.638615+0000","last_peered":"2024-09-12T12:24:00.638615+0000","last_clean":"2024-09-12T12:24:00.638615+0000","last_became_active":"2024-09-12T12:23:57.364717+0000","last_became_peered":"2024-09-12T12:23:57.364717+0000","last_unstale":"2024-09-12T12:24:00.638615+0000","last_undegraded":"2024-09-12T12:24:00.638615+0000","last_fullsized":"2024-09-12T12:24:00.638615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:39.937210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637851+0000","last_change":"2024-09-12T12:23:58.396186+0000","last_active":"2024-09-12T12:24:00.637851+0000","last_peered":"2024-09-12T12:24:00.637851+0000","last_clean":"2024-09-12T12:24:00.637851+0000","last_became_active":"2024-09-12T12:23:58.393902+0000","last_became_peered":"2024-09-12T12:23:58.393902+0000","last_unstale":"2024-09-12T12:24:00.637851+0000","last_undegraded":"2024-09-12T12:24:00.637851+0000","last_fullsized":"2024-09-12T12:24:00.637851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:50:19.418045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392449+0000","last_change":"2024-09-12T12:23:57.388887+0000","last_active":"2024-09-12T12:23:59.392449+0000","last_peered":"2024-09-12T12:23:59.392449+0000","last_clean":"2024-09-12T12:23:59.392449+0000","last_became_active":"2024-09-12T12:23:57.388290+0000","last_became_peered":"2024-09-12T12:23:57.388290+0000","last_unstale":"2024-09-12T12:23:59.392449+0000","last_undegraded":"2024-09-12T12:23:59.392449+0000","last_fullsized":"2024-09-12T12:23:59.392449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:27.962118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392267+0000","last_change":"2024-09-12T12:23:58.397164+0000","last_active":"2024-09-12T12:23:59.392267+0000","last_peered":"2024-09-12T12:23:59.392267+0000","last_clean":"2024-09-12T12:23:59.392267+0000","last_became_active":"2024-09-12T12:23:58.396442+0000","last_became_peered":"2024-09-12T12:23:58.396442+0000","last_unstale":"2024-09-12T12:23:59.392267+0000","last_undegraded":"2024-09-12T12:23:59.392267+0000","last_fullsized":"2024-09-12T12:23:59.392267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:16.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637962+0000","last_change":"2024-09-12T12:23:57.390177+0000","last_active":"2024-09-12T12:24:00.637962+0000","last_peered":"2024-09-12T12:24:00.637962+0000","last_clean":"2024-09-12T12:24:00.637962+0000","last_became_active":"2024-09-12T12:23:57.389410+0000","last_became_peered":"2024-09-12T12:23:57.389410+0000","last_unstale":"2024-09-12T12:24:00.637962+0000","last_undegraded":"2024-09-12T12:24:00.637962+0000","last_fullsized":"2024-09-12T12:24:00.637962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:05.313097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392877+0000","last_change":"2024-09-12T12:23:58.390793+0000","last_active":"2024-09-12T12:23:59.392877+0000","last_peered":"2024-09-12T12:23:59.392877+0000","last_clean":"2024-09-12T12:23:59.392877+0000","last_became_active":"2024-09-12T12:23:58.390529+0000","last_became_peered":"2024-09-12T12:23:58.390529+0000","last_unstale":"2024-09-12T12:23:59.392877+0000","last_undegraded":"2024-09-12T12:23:59.392877+0000","last_fullsized":"2024-09-12T12:23:59.392877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:24.352342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638879+0000","last_change":"2024-09-12T12:23:57.386525+0000","last_active":"2024-09-12T12:24:00.638879+0000","last_peered":"2024-09-12T12:24:00.638879+0000","last_clean":"2024-09-12T12:24:00.638879+0000","last_became_active":"2024-09-12T12:23:57.385710+0000","last_became_peered":"2024-09-12T12:23:57.385710+0000","last_unstale":"2024-09-12T12:24:00.638879+0000","last_undegraded":"2024-09-12T12:24:00.638879+0000","last_fullsized":"2024-09-12T12:24:00.638879+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:15.396855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638136+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:24:00.638136+0000","last_peered":"2024-09-12T12:24:00.638136+0000","last_clean":"2024-09-12T12:24:00.638136+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:24:00.638136+0000","last_undegraded":"2024-09-12T12:24:00.638136+0000","last_fullsized":"2024-09-12T12:24:00.638136+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":966656,"data_stored":404023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542201,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":999149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":978944,"data_stored":409405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542200,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":983040,"data_stored":410345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":408711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542199,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":409293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":978944,"data_stored":408905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":999113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:06.938 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f12178bc640 1 Processor -- start 2024-09-12T12:24:07.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f12178bc640 1 -- start start 2024-09-12T12:24:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f12178bc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f121015c710 0x7f12100b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f12178bc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12100b5000 con 0x7f12100ae880 2024-09-12T12:24:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f12178bc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12100b37c0 con 0x7f1210073cb0 2024-09-12T12:24:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f12178bc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1210074090 con 0x7f121015c710 2024-09-12T12:24:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f1214e57640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f121015c710 0x7f12100b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f1215e59640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f12100ae880 0x7f12100d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38820/0 (socket says 172.21.15.3:38820) 2024-09-12T12:24:07.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f1215e59640 1 -- 172.21.15.3:0/4186562308 learned_addr learned my addr 172.21.15.3:0/4186562308 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f1214e57640 1 -- 172.21.15.3:0/4186562308 >> v1:172.21.15.142:6789/0 conn(0x7f1210073cb0 legacy=0x7f12100adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f1214e57640 1 -- 172.21.15.3:0/4186562308 >> v1:172.21.15.3:6789/0 conn(0x7f12100ae880 legacy=0x7f12100d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.072+0000 7f1214e57640 1 -- 172.21.15.3:0/4186562308 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12100b3aa0 con 0x7f121015c710 2024-09-12T12:24:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f1214e57640 1 --2- 172.21.15.3:0/4186562308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121015c710 0x7f12100b30e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f12040097c0 tx=0x7f120401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88236751f81d2ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f120ffff640 1 -- 172.21.15.3:0/4186562308 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040b0070 con 0x7f121015c710 2024-09-12T12:24:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f120ffff640 1 -- 172.21.15.3:0/4186562308 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204004230 con 0x7f121015c710 2024-09-12T12:24:07.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f120ffff640 1 -- 172.21.15.3:0/4186562308 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204003620 con 0x7f121015c710 2024-09-12T12:24:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- 172.21.15.3:0/4186562308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121015c710 msgr2=0x7f12100b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 --2- 172.21.15.3:0/4186562308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121015c710 0x7f12100b30e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f12040097c0 tx=0x7f120401c460 comp rx=0 tx=0).stop 2024-09-12T12:24:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- 172.21.15.3:0/4186562308 shutdown_connections 2024-09-12T12:24:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 --2- 172.21.15.3:0/4186562308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f121015c710 0x7f12100b30e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- 172.21.15.3:0/4186562308 >> 172.21.15.3:0/4186562308 conn(0x7f1210067e00 msgr2=0x7f1210068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:07.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- 172.21.15.3:0/4186562308 shutdown_connections 2024-09-12T12:24:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- 172.21.15.3:0/4186562308 wait complete. 2024-09-12T12:24:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 Processor -- start 2024-09-12T12:24:07.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- start start 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1210073cb0 0x7f121006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12100ae880 0x7f1210155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f121015c710 0x7f12101557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f12100d0e40 con 0x7f12100ae880 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12100d0cc0 con 0x7f1210073cb0 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.076+0000 7f12178bc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f12100aec90 con 0x7f121015c710 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12100ae880 0x7f1210155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12100ae880 0x7f1210155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60664/0 (socket says 172.21.15.3:60664) 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 -- 172.21.15.3:0/626726140 learned_addr learned my addr 172.21.15.3:0/626726140 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:07.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1215e59640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f121015c710 0x7f12101557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 -- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f121015c710 msgr2=0x7f12101557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1215658640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1210073cb0 0x7f121006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f121015c710 0x7f12101557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 -- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1210073cb0 msgr2=0x7f121006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1210073cb0 0x7f121006e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 -- 172.21.15.3:0/626726140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1210151740 con 0x7f12100ae880 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1215e59640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f121015c710 0x7f12101557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1215658640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1210073cb0 0x7f121006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1214e57640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12100ae880 0x7f1210155270 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f12040ae040 tx=0x7f1204019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f120dffb640 1 -- 172.21.15.3:0/626726140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040b9020 con 0x7f12100ae880 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f120dffb640 1 -- 172.21.15.3:0/626726140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12040043e0 con 0x7f12100ae880 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f120dffb640 1 -- 172.21.15.3:0/626726140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040b0040 con 0x7f12100ae880 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12100c9500 con 0x7f12100ae880 2024-09-12T12:24:07.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12100c9d00 con 0x7f12100ae880 2024-09-12T12:24:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12100c9ff0 con 0x7f12100ae880 2024-09-12T12:24:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f120dffb640 1 -- 172.21.15.3:0/626726140 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f12040bc020 con 0x7f12100ae880 2024-09-12T12:24:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f120dffb640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11fc0678c0 0x7f11fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f120dffb640 1 -- 172.21.15.3:0/626726140 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f120411fce0 con 0x7f12100ae880 2024-09-12T12:24:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.080+0000 7f1215658640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11fc0678c0 0x7f11fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.084+0000 7f1215658640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11fc0678c0 0x7f11fc069d80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1200007fd0 tx=0x7f1200009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.084+0000 7f120dffb640 1 -- 172.21.15.3:0/626726140 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f12100c9ff0 con 0x7f12100ae880 2024-09-12T12:24:07.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.248+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12101514f0 con 0x7f11fc0678c0 2024-09-12T12:24:07.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.264+0000 7f120dffb640 1 -- 172.21.15.3:0/626726140 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252076 (secure 0 0 0) 0x7f12101514f0 con 0x7f11fc0678c0 2024-09-12T12:24:07.270 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:07.271 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11fc0678c0 msgr2=0x7f11fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11fc0678c0 0x7f11fc069d80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1200007fd0 tx=0x7f1200009040 comp rx=0 tx=0).stop 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12100ae880 msgr2=0x7f1210155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12100ae880 0x7f1210155270 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f12040ae040 tx=0x7f1204019200 comp rx=0 tx=0).stop 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 shutdown_connections 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f11fc0678c0 0x7f11fc069d80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f121015c710 0x7f12101557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f12100ae880 0x7f1210155270 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 --2- 172.21.15.3:0/626726140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1210073cb0 0x7f121006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 >> 172.21.15.3:0/626726140 conn(0x7f1210067e00 msgr2=0x7f12100681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 shutdown_connections 2024-09-12T12:24:07.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.268+0000 7f12178bc640 1 -- 172.21.15.3:0/626726140 wait complete. 2024-09-12T12:24:07.287 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-12T12:24:05.743633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249988,"kb_used_data":8776,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724732,"statfs":{"total":773094113280,"available":772838125568,"internally_reserved":0,"allocated":8986624,"data_stored":4455244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002939"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393467+0000","last_change":"2024-09-12T12:23:58.407134+0000","last_active":"2024-09-12T12:23:59.393467+0000","last_peered":"2024-09-12T12:23:59.393467+0000","last_clean":"2024-09-12T12:23:59.393467+0000","last_became_active":"2024-09-12T12:23:58.406341+0000","last_became_peered":"2024-09-12T12:23:58.406341+0000","last_unstale":"2024-09-12T12:23:59.393467+0000","last_undegraded":"2024-09-12T12:23:59.393467+0000","last_fullsized":"2024-09-12T12:23:59.393467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:09:55.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393394+0000","last_change":"2024-09-12T12:23:58.412147+0000","last_active":"2024-09-12T12:23:59.393394+0000","last_peered":"2024-09-12T12:23:59.393394+0000","last_clean":"2024-09-12T12:23:59.393394+0000","last_became_active":"2024-09-12T12:23:58.411838+0000","last_became_peered":"2024-09-12T12:23:58.411838+0000","last_unstale":"2024-09-12T12:23:59.393394+0000","last_undegraded":"2024-09-12T12:23:59.393394+0000","last_fullsized":"2024-09-12T12:23:59.393394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:15:25.592214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.640016+0000","last_change":"2024-09-12T12:23:58.398025+0000","last_active":"2024-09-12T12:24:00.640016+0000","last_peered":"2024-09-12T12:24:00.640016+0000","last_clean":"2024-09-12T12:24:00.640016+0000","last_became_active":"2024-09-12T12:23:58.397289+0000","last_became_peered":"2024-09-12T12:23:58.397289+0000","last_unstale":"2024-09-12T12:24:00.640016+0000","last_undegraded":"2024-09-12T12:24:00.640016+0000","last_fullsized":"2024-09-12T12:24:00.640016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:01:38.815943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.636947+0000","last_change":"2024-09-12T12:23:58.395534+0000","last_active":"2024-09-12T12:24:00.636947+0000","last_peered":"2024-09-12T12:24:00.636947+0000","last_clean":"2024-09-12T12:24:00.636947+0000","last_became_active":"2024-09-12T12:23:58.395256+0000","last_became_peered":"2024-09-12T12:23:58.395256+0000","last_unstale":"2024-09-12T12:24:00.636947+0000","last_undegraded":"2024-09-12T12:24:00.636947+0000","last_fullsized":"2024-09-12T12:24:00.636947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:58.574908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639149+0000","last_change":"2024-09-12T12:23:58.395496+0000","last_active":"2024-09-12T12:24:00.639149+0000","last_peered":"2024-09-12T12:24:00.639149+0000","last_clean":"2024-09-12T12:24:00.639149+0000","last_became_active":"2024-09-12T12:23:58.394775+0000","last_became_peered":"2024-09-12T12:23:58.394775+0000","last_unstale":"2024-09-12T12:24:00.639149+0000","last_undegraded":"2024-09-12T12:24:00.639149+0000","last_fullsized":"2024-09-12T12:24:00.639149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:52:54.437455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390369+0000","last_change":"2024-09-12T12:23:58.380493+0000","last_active":"2024-09-12T12:23:59.390369+0000","last_peered":"2024-09-12T12:23:59.390369+0000","last_clean":"2024-09-12T12:23:59.390369+0000","last_became_active":"2024-09-12T12:23:58.379752+0000","last_became_peered":"2024-09-12T12:23:58.379752+0000","last_unstale":"2024-09-12T12:23:59.390369+0000","last_undegraded":"2024-09-12T12:23:59.390369+0000","last_fullsized":"2024-09-12T12:23:59.390369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:19:56.787085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.634755+0000","last_change":"2024-09-12T12:23:58.377300+0000","last_active":"2024-09-12T12:24:00.634755+0000","last_peered":"2024-09-12T12:24:00.634755+0000","last_clean":"2024-09-12T12:24:00.634755+0000","last_became_active":"2024-09-12T12:23:58.376305+0000","last_became_peered":"2024-09-12T12:23:58.376305+0000","last_unstale":"2024-09-12T12:24:00.634755+0000","last_undegraded":"2024-09-12T12:24:00.634755+0000","last_fullsized":"2024-09-12T12:24:00.634755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:34:56.335568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390257+0000","last_change":"2024-09-12T12:23:58.395247+0000","last_active":"2024-09-12T12:23:59.390257+0000","last_peered":"2024-09-12T12:23:59.390257+0000","last_clean":"2024-09-12T12:23:59.390257+0000","last_became_active":"2024-09-12T12:23:58.394534+0000","last_became_peered":"2024-09-12T12:23:58.394534+0000","last_unstale":"2024-09-12T12:23:59.390257+0000","last_undegraded":"2024-09-12T12:23:59.390257+0000","last_fullsized":"2024-09-12T12:23:59.390257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:31.089085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390933+0000","last_change":"2024-09-12T12:23:58.371385+0000","last_active":"2024-09-12T12:23:59.390933+0000","last_peered":"2024-09-12T12:23:59.390933+0000","last_clean":"2024-09-12T12:23:59.390933+0000","last_became_active":"2024-09-12T12:23:58.370715+0000","last_became_peered":"2024-09-12T12:23:58.370715+0000","last_unstale":"2024-09-12T12:23:59.390933+0000","last_undegraded":"2024-09-12T12:23:59.390933+0000","last_fullsized":"2024-09-12T12:23:59.390933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:18.770724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393154+0000","last_change":"2024-09-12T12:23:58.395206+0000","last_active":"2024-09-12T12:23:59.393154+0000","last_peered":"2024-09-12T12:23:59.393154+0000","last_clean":"2024-09-12T12:23:59.393154+0000","last_became_active":"2024-09-12T12:23:58.394053+0000","last_became_peered":"2024-09-12T12:23:58.394053+0000","last_unstale":"2024-09-12T12:23:59.393154+0000","last_undegraded":"2024-09-12T12:23:59.393154+0000","last_fullsized":"2024-09-12T12:23:59.393154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:08:35.556909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393209+0000","last_change":"2024-09-12T12:23:58.392493+0000","last_active":"2024-09-12T12:23:59.393209+0000","last_peered":"2024-09-12T12:23:59.393209+0000","last_clean":"2024-09-12T12:23:59.393209+0000","last_became_active":"2024-09-12T12:23:58.391962+0000","last_became_peered":"2024-09-12T12:23:58.391962+0000","last_unstale":"2024-09-12T12:23:59.393209+0000","last_undegraded":"2024-09-12T12:23:59.393209+0000","last_fullsized":"2024-09-12T12:23:59.393209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:28.972373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392428+0000","last_change":"2024-09-12T12:23:58.401621+0000","last_active":"2024-09-12T12:23:59.392428+0000","last_peered":"2024-09-12T12:23:59.392428+0000","last_clean":"2024-09-12T12:23:59.392428+0000","last_became_active":"2024-09-12T12:23:58.394576+0000","last_became_peered":"2024-09-12T12:23:58.394576+0000","last_unstale":"2024-09-12T12:23:59.392428+0000","last_undegraded":"2024-09-12T12:23:59.392428+0000","last_fullsized":"2024-09-12T12:23:59.392428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:54.581603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638976+0000","last_change":"2024-09-12T12:23:58.374604+0000","last_active":"2024-09-12T12:24:00.638976+0000","last_peered":"2024-09-12T12:24:00.638976+0000","last_clean":"2024-09-12T12:24:00.638976+0000","last_became_active":"2024-09-12T12:23:58.373831+0000","last_became_peered":"2024-09-12T12:23:58.373831+0000","last_unstale":"2024-09-12T12:24:00.638976+0000","last_undegraded":"2024-09-12T12:24:00.638976+0000","last_fullsized":"2024-09-12T12:24:00.638976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:10.558765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635353+0000","last_change":"2024-09-12T12:23:58.394035+0000","last_active":"2024-09-12T12:24:00.635353+0000","last_peered":"2024-09-12T12:24:00.635353+0000","last_clean":"2024-09-12T12:24:00.635353+0000","last_became_active":"2024-09-12T12:23:58.393238+0000","last_became_peered":"2024-09-12T12:23:58.393238+0000","last_unstale":"2024-09-12T12:24:00.635353+0000","last_undegraded":"2024-09-12T12:24:00.635353+0000","last_fullsized":"2024-09-12T12:24:00.635353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:47:30.744183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635152+0000","last_change":"2024-09-12T12:23:58.386557+0000","last_active":"2024-09-12T12:24:00.635152+0000","last_peered":"2024-09-12T12:24:00.635152+0000","last_clean":"2024-09-12T12:24:00.635152+0000","last_became_active":"2024-09-12T12:23:58.386226+0000","last_became_peered":"2024-09-12T12:23:58.386226+0000","last_unstale":"2024-09-12T12:24:00.635152+0000","last_undegraded":"2024-09-12T12:24:00.635152+0000","last_fullsized":"2024-09-12T12:24:00.635152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:52.962546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638959+0000","last_change":"2024-09-12T12:23:58.412247+0000","last_active":"2024-09-12T12:24:00.638959+0000","last_peered":"2024-09-12T12:24:00.638959+0000","last_clean":"2024-09-12T12:24:00.638959+0000","last_became_active":"2024-09-12T12:23:58.411587+0000","last_became_peered":"2024-09-12T12:23:58.411587+0000","last_unstale":"2024-09-12T12:24:00.638959+0000","last_undegraded":"2024-09-12T12:24:00.638959+0000","last_fullsized":"2024-09-12T12:24:00.638959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:18.770724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639605+0000","last_change":"2024-09-12T12:23:58.377530+0000","last_active":"2024-09-12T12:24:00.639605+0000","last_peered":"2024-09-12T12:24:00.639605+0000","last_clean":"2024-09-12T12:24:00.639605+0000","last_became_active":"2024-09-12T12:23:58.376736+0000","last_became_peered":"2024-09-12T12:23:58.376736+0000","last_unstale":"2024-09-12T12:24:00.639605+0000","last_undegraded":"2024-09-12T12:24:00.639605+0000","last_fullsized":"2024-09-12T12:24:00.639605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:08:35.556909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637437+0000","last_change":"2024-09-12T12:23:58.394232+0000","last_active":"2024-09-12T12:24:00.637437+0000","last_peered":"2024-09-12T12:24:00.637437+0000","last_clean":"2024-09-12T12:24:00.637437+0000","last_became_active":"2024-09-12T12:23:58.393925+0000","last_became_peered":"2024-09-12T12:23:58.393925+0000","last_unstale":"2024-09-12T12:24:00.637437+0000","last_undegraded":"2024-09-12T12:24:00.637437+0000","last_fullsized":"2024-09-12T12:24:00.637437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:42:05.083901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395624+0000","last_change":"2024-09-12T12:23:58.384760+0000","last_active":"2024-09-12T12:23:59.395624+0000","last_peered":"2024-09-12T12:23:59.395624+0000","last_clean":"2024-09-12T12:23:59.395624+0000","last_became_active":"2024-09-12T12:23:58.382109+0000","last_became_peered":"2024-09-12T12:23:58.382109+0000","last_unstale":"2024-09-12T12:23:59.395624+0000","last_undegraded":"2024-09-12T12:23:59.395624+0000","last_fullsized":"2024-09-12T12:23:59.395624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:49.144078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637268+0000","last_change":"2024-09-12T12:23:58.383770+0000","last_active":"2024-09-12T12:24:00.637268+0000","last_peered":"2024-09-12T12:24:00.637268+0000","last_clean":"2024-09-12T12:24:00.637268+0000","last_became_active":"2024-09-12T12:23:58.381588+0000","last_became_peered":"2024-09-12T12:23:58.381588+0000","last_unstale":"2024-09-12T12:24:00.637268+0000","last_undegraded":"2024-09-12T12:24:00.637268+0000","last_fullsized":"2024-09-12T12:24:00.637268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:27:07.541371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.636665+0000","last_change":"2024-09-12T12:23:58.373789+0000","last_active":"2024-09-12T12:24:00.636665+0000","last_peered":"2024-09-12T12:24:00.636665+0000","last_clean":"2024-09-12T12:24:00.636665+0000","last_became_active":"2024-09-12T12:23:58.372969+0000","last_became_peered":"2024-09-12T12:23:58.372969+0000","last_unstale":"2024-09-12T12:24:00.636665+0000","last_undegraded":"2024-09-12T12:24:00.636665+0000","last_fullsized":"2024-09-12T12:24:00.636665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:14:31.576588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392280+0000","last_change":"2024-09-12T12:23:58.392638+0000","last_active":"2024-09-12T12:23:59.392280+0000","last_peered":"2024-09-12T12:23:59.392280+0000","last_clean":"2024-09-12T12:23:59.392280+0000","last_became_active":"2024-09-12T12:23:58.392083+0000","last_became_peered":"2024-09-12T12:23:58.392083+0000","last_unstale":"2024-09-12T12:23:59.392280+0000","last_undegraded":"2024-09-12T12:23:59.392280+0000","last_fullsized":"2024-09-12T12:23:59.392280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:10:20.142471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392234+0000","last_change":"2024-09-12T12:23:58.412797+0000","last_active":"2024-09-12T12:23:59.392234+0000","last_peered":"2024-09-12T12:23:59.392234+0000","last_clean":"2024-09-12T12:23:59.392234+0000","last_became_active":"2024-09-12T12:23:58.411231+0000","last_became_peered":"2024-09-12T12:23:58.411231+0000","last_unstale":"2024-09-12T12:23:59.392234+0000","last_undegraded":"2024-09-12T12:23:59.392234+0000","last_fullsized":"2024-09-12T12:23:59.392234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:52:54.437455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.634853+0000","last_change":"2024-09-12T12:23:58.378166+0000","last_active":"2024-09-12T12:24:00.634853+0000","last_peered":"2024-09-12T12:24:00.634853+0000","last_clean":"2024-09-12T12:24:00.634853+0000","last_became_active":"2024-09-12T12:23:58.377393+0000","last_became_peered":"2024-09-12T12:23:58.377393+0000","last_unstale":"2024-09-12T12:24:00.634853+0000","last_undegraded":"2024-09-12T12:24:00.634853+0000","last_fullsized":"2024-09-12T12:24:00.634853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:22:41.164230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393348+0000","last_change":"2024-09-12T12:23:58.416477+0000","last_active":"2024-09-12T12:23:59.393348+0000","last_peered":"2024-09-12T12:23:59.393348+0000","last_clean":"2024-09-12T12:23:59.393348+0000","last_became_active":"2024-09-12T12:23:58.416249+0000","last_became_peered":"2024-09-12T12:23:58.416249+0000","last_unstale":"2024-09-12T12:23:59.393348+0000","last_undegraded":"2024-09-12T12:23:59.393348+0000","last_fullsized":"2024-09-12T12:23:59.393348+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:15.188997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639055+0000","last_change":"2024-09-12T12:23:58.395622+0000","last_active":"2024-09-12T12:24:00.639055+0000","last_peered":"2024-09-12T12:24:00.639055+0000","last_clean":"2024-09-12T12:24:00.639055+0000","last_became_active":"2024-09-12T12:23:58.394982+0000","last_became_peered":"2024-09-12T12:23:58.394982+0000","last_unstale":"2024-09-12T12:24:00.639055+0000","last_undegraded":"2024-09-12T12:24:00.639055+0000","last_fullsized":"2024-09-12T12:24:00.639055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:43:18.727703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639493+0000","last_change":"2024-09-12T12:23:58.384692+0000","last_active":"2024-09-12T12:24:00.639493+0000","last_peered":"2024-09-12T12:24:00.639493+0000","last_clean":"2024-09-12T12:24:00.639493+0000","last_became_active":"2024-09-12T12:23:58.384001+0000","last_became_peered":"2024-09-12T12:23:58.384001+0000","last_unstale":"2024-09-12T12:24:00.639493+0000","last_undegraded":"2024-09-12T12:24:00.639493+0000","last_fullsized":"2024-09-12T12:24:00.639493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:16.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393246+0000","last_change":"2024-09-12T12:23:58.407174+0000","last_active":"2024-09-12T12:23:59.393246+0000","last_peered":"2024-09-12T12:23:59.393246+0000","last_clean":"2024-09-12T12:23:59.393246+0000","last_became_active":"2024-09-12T12:23:58.406429+0000","last_became_peered":"2024-09-12T12:23:58.406429+0000","last_unstale":"2024-09-12T12:23:59.393246+0000","last_undegraded":"2024-09-12T12:23:59.393246+0000","last_fullsized":"2024-09-12T12:23:59.393246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:32:37.144121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392309+0000","last_change":"2024-09-12T12:23:58.393242+0000","last_active":"2024-09-12T12:23:59.392309+0000","last_peered":"2024-09-12T12:23:59.392309+0000","last_clean":"2024-09-12T12:23:59.392309+0000","last_became_active":"2024-09-12T12:23:58.391479+0000","last_became_peered":"2024-09-12T12:23:58.391479+0000","last_unstale":"2024-09-12T12:23:59.392309+0000","last_undegraded":"2024-09-12T12:23:59.392309+0000","last_fullsized":"2024-09-12T12:23:59.392309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:50.943216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392431+0000","last_change":"2024-09-12T12:23:58.391885+0000","last_active":"2024-09-12T12:23:59.392431+0000","last_peered":"2024-09-12T12:23:59.392431+0000","last_clean":"2024-09-12T12:23:59.392431+0000","last_became_active":"2024-09-12T12:23:58.391554+0000","last_became_peered":"2024-09-12T12:23:58.391554+0000","last_unstale":"2024-09-12T12:23:59.392431+0000","last_undegraded":"2024-09-12T12:23:59.392431+0000","last_fullsized":"2024-09-12T12:23:59.392431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:06.323352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638373+0000","last_change":"2024-09-12T12:23:58.374951+0000","last_active":"2024-09-12T12:24:00.638373+0000","last_peered":"2024-09-12T12:24:00.638373+0000","last_clean":"2024-09-12T12:24:00.638373+0000","last_became_active":"2024-09-12T12:23:58.374134+0000","last_became_peered":"2024-09-12T12:23:58.374134+0000","last_unstale":"2024-09-12T12:24:00.638373+0000","last_undegraded":"2024-09-12T12:24:00.638373+0000","last_fullsized":"2024-09-12T12:24:00.638373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:12.842154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392079+0000","last_change":"2024-09-12T12:23:58.415959+0000","last_active":"2024-09-12T12:23:59.392079+0000","last_peered":"2024-09-12T12:23:59.392079+0000","last_clean":"2024-09-12T12:23:59.392079+0000","last_became_active":"2024-09-12T12:23:58.412091+0000","last_became_peered":"2024-09-12T12:23:58.412091+0000","last_unstale":"2024-09-12T12:23:59.392079+0000","last_undegraded":"2024-09-12T12:23:59.392079+0000","last_fullsized":"2024-09-12T12:23:59.392079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:18.770724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639576+0000","last_change":"2024-09-12T12:23:58.399145+0000","last_active":"2024-09-12T12:24:00.639576+0000","last_peered":"2024-09-12T12:24:00.639576+0000","last_clean":"2024-09-12T12:24:00.639576+0000","last_became_active":"2024-09-12T12:23:58.398277+0000","last_became_peered":"2024-09-12T12:23:58.398277+0000","last_unstale":"2024-09-12T12:24:00.639576+0000","last_undegraded":"2024-09-12T12:24:00.639576+0000","last_fullsized":"2024-09-12T12:24:00.639576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:54.581603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393769+0000","last_change":"2024-09-12T12:23:57.391198+0000","last_active":"2024-09-12T12:23:59.393769+0000","last_peered":"2024-09-12T12:23:59.393769+0000","last_clean":"2024-09-12T12:23:59.393769+0000","last_became_active":"2024-09-12T12:23:57.390552+0000","last_became_peered":"2024-09-12T12:23:57.390552+0000","last_unstale":"2024-09-12T12:23:59.393769+0000","last_undegraded":"2024-09-12T12:23:59.393769+0000","last_fullsized":"2024-09-12T12:23:59.393769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:49.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393493+0000","last_change":"2024-09-12T12:23:58.398834+0000","last_active":"2024-09-12T12:23:59.393493+0000","last_peered":"2024-09-12T12:23:59.393493+0000","last_clean":"2024-09-12T12:23:59.393493+0000","last_became_active":"2024-09-12T12:23:58.398012+0000","last_became_peered":"2024-09-12T12:23:58.398012+0000","last_unstale":"2024-09-12T12:23:59.393493+0000","last_undegraded":"2024-09-12T12:23:59.393493+0000","last_fullsized":"2024-09-12T12:23:59.393493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:14:12.942908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639904+0000","last_change":"2024-09-12T12:23:57.367632+0000","last_active":"2024-09-12T12:24:00.639904+0000","last_peered":"2024-09-12T12:24:00.639904+0000","last_clean":"2024-09-12T12:24:00.639904+0000","last_became_active":"2024-09-12T12:23:57.366903+0000","last_became_peered":"2024-09-12T12:23:57.366903+0000","last_unstale":"2024-09-12T12:24:00.639904+0000","last_undegraded":"2024-09-12T12:24:00.639904+0000","last_fullsized":"2024-09-12T12:24:00.639904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:27.962118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392829+0000","last_change":"2024-09-12T12:23:58.401731+0000","last_active":"2024-09-12T12:23:59.392829+0000","last_peered":"2024-09-12T12:23:59.392829+0000","last_clean":"2024-09-12T12:23:59.392829+0000","last_became_active":"2024-09-12T12:23:58.395518+0000","last_became_peered":"2024-09-12T12:23:58.395518+0000","last_unstale":"2024-09-12T12:23:59.392829+0000","last_undegraded":"2024-09-12T12:23:59.392829+0000","last_fullsized":"2024-09-12T12:23:59.392829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:39.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395705+0000","last_change":"2024-09-12T12:23:57.368053+0000","last_active":"2024-09-12T12:23:59.395705+0000","last_peered":"2024-09-12T12:23:59.395705+0000","last_clean":"2024-09-12T12:23:59.395705+0000","last_became_active":"2024-09-12T12:23:57.367668+0000","last_became_peered":"2024-09-12T12:23:57.367668+0000","last_unstale":"2024-09-12T12:23:59.395705+0000","last_undegraded":"2024-09-12T12:23:59.395705+0000","last_fullsized":"2024-09-12T12:23:59.395705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:51:00.133402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638622+0000","last_change":"2024-09-12T12:23:58.381382+0000","last_active":"2024-09-12T12:24:00.638622+0000","last_peered":"2024-09-12T12:24:00.638622+0000","last_clean":"2024-09-12T12:24:00.638622+0000","last_became_active":"2024-09-12T12:23:58.380618+0000","last_became_peered":"2024-09-12T12:23:58.380618+0000","last_unstale":"2024-09-12T12:24:00.638622+0000","last_undegraded":"2024-09-12T12:24:00.638622+0000","last_fullsized":"2024-09-12T12:24:00.638622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:06:03.030474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396288+0000","last_change":"2024-09-12T12:23:57.368958+0000","last_active":"2024-09-12T12:23:59.396288+0000","last_peered":"2024-09-12T12:23:59.396288+0000","last_clean":"2024-09-12T12:23:59.396288+0000","last_became_active":"2024-09-12T12:23:57.368562+0000","last_became_peered":"2024-09-12T12:23:57.368562+0000","last_unstale":"2024-09-12T12:23:59.396288+0000","last_undegraded":"2024-09-12T12:23:59.396288+0000","last_fullsized":"2024-09-12T12:23:59.396288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:10:58.364890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392575+0000","last_change":"2024-09-12T12:23:58.391783+0000","last_active":"2024-09-12T12:23:59.392575+0000","last_peered":"2024-09-12T12:23:59.392575+0000","last_clean":"2024-09-12T12:23:59.392575+0000","last_became_active":"2024-09-12T12:23:58.391447+0000","last_became_peered":"2024-09-12T12:23:58.391447+0000","last_unstale":"2024-09-12T12:23:59.392575+0000","last_undegraded":"2024-09-12T12:23:59.392575+0000","last_fullsized":"2024-09-12T12:23:59.392575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:40.947465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.640107+0000","last_change":"2024-09-12T12:23:57.371998+0000","last_active":"2024-09-12T12:24:00.640107+0000","last_peered":"2024-09-12T12:24:00.640107+0000","last_clean":"2024-09-12T12:24:00.640107+0000","last_became_active":"2024-09-12T12:23:57.371152+0000","last_became_peered":"2024-09-12T12:23:57.371152+0000","last_unstale":"2024-09-12T12:24:00.640107+0000","last_undegraded":"2024-09-12T12:24:00.640107+0000","last_fullsized":"2024-09-12T12:24:00.640107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:23.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639558+0000","last_change":"2024-09-12T12:23:58.377812+0000","last_active":"2024-09-12T12:24:00.639558+0000","last_peered":"2024-09-12T12:24:00.639558+0000","last_clean":"2024-09-12T12:24:00.639558+0000","last_became_active":"2024-09-12T12:23:58.377054+0000","last_became_peered":"2024-09-12T12:23:58.377054+0000","last_unstale":"2024-09-12T12:24:00.639558+0000","last_undegraded":"2024-09-12T12:24:00.639558+0000","last_fullsized":"2024-09-12T12:24:00.639558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:46:39.538270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638568+0000","last_change":"2024-09-12T12:23:57.366528+0000","last_active":"2024-09-12T12:24:00.638568+0000","last_peered":"2024-09-12T12:24:00.638568+0000","last_clean":"2024-09-12T12:24:00.638568+0000","last_became_active":"2024-09-12T12:23:57.365702+0000","last_became_peered":"2024-09-12T12:23:57.365702+0000","last_unstale":"2024-09-12T12:24:00.638568+0000","last_undegraded":"2024-09-12T12:24:00.638568+0000","last_fullsized":"2024-09-12T12:24:00.638568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:49.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637778+0000","last_change":"2024-09-12T12:23:58.392087+0000","last_active":"2024-09-12T12:24:00.637778+0000","last_peered":"2024-09-12T12:24:00.637778+0000","last_clean":"2024-09-12T12:24:00.637778+0000","last_became_active":"2024-09-12T12:23:58.391215+0000","last_became_peered":"2024-09-12T12:23:58.391215+0000","last_unstale":"2024-09-12T12:24:00.637778+0000","last_undegraded":"2024-09-12T12:24:00.637778+0000","last_fullsized":"2024-09-12T12:24:00.637778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:05:38.102031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395360+0000","last_change":"2024-09-12T12:23:57.375695+0000","last_active":"2024-09-12T12:23:59.395360+0000","last_peered":"2024-09-12T12:23:59.395360+0000","last_clean":"2024-09-12T12:23:59.395360+0000","last_became_active":"2024-09-12T12:23:57.375373+0000","last_became_peered":"2024-09-12T12:23:57.375373+0000","last_unstale":"2024-09-12T12:23:59.395360+0000","last_undegraded":"2024-09-12T12:23:59.395360+0000","last_fullsized":"2024-09-12T12:23:59.395360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:09.585468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635311+0000","last_change":"2024-09-12T12:23:58.377377+0000","last_active":"2024-09-12T12:24:00.635311+0000","last_peered":"2024-09-12T12:24:00.635311+0000","last_clean":"2024-09-12T12:24:00.635311+0000","last_became_active":"2024-09-12T12:23:58.374901+0000","last_became_peered":"2024-09-12T12:23:58.374901+0000","last_unstale":"2024-09-12T12:24:00.635311+0000","last_undegraded":"2024-09-12T12:24:00.635311+0000","last_fullsized":"2024-09-12T12:24:00.635311+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:58:02.863904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392198+0000","last_change":"2024-09-12T12:23:57.370326+0000","last_active":"2024-09-12T12:23:59.392198+0000","last_peered":"2024-09-12T12:23:59.392198+0000","last_clean":"2024-09-12T12:23:59.392198+0000","last_became_active":"2024-09-12T12:23:57.369836+0000","last_became_peered":"2024-09-12T12:23:57.369836+0000","last_unstale":"2024-09-12T12:23:59.392198+0000","last_undegraded":"2024-09-12T12:23:59.392198+0000","last_fullsized":"2024-09-12T12:23:59.392198+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:30.448876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.636381+0000","last_change":"2024-09-12T12:23:58.395462+0000","last_active":"2024-09-12T12:24:00.636381+0000","last_peered":"2024-09-12T12:24:00.636381+0000","last_clean":"2024-09-12T12:24:00.636381+0000","last_became_active":"2024-09-12T12:23:58.395205+0000","last_became_peered":"2024-09-12T12:23:58.395205+0000","last_unstale":"2024-09-12T12:24:00.636381+0000","last_undegraded":"2024-09-12T12:24:00.636381+0000","last_fullsized":"2024-09-12T12:24:00.636381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:07.048644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.394044+0000","last_change":"2024-09-12T12:23:57.390954+0000","last_active":"2024-09-12T12:23:59.394044+0000","last_peered":"2024-09-12T12:23:59.394044+0000","last_clean":"2024-09-12T12:23:59.394044+0000","last_became_active":"2024-09-12T12:23:57.389910+0000","last_became_peered":"2024-09-12T12:23:57.389910+0000","last_unstale":"2024-09-12T12:23:59.394044+0000","last_undegraded":"2024-09-12T12:23:59.394044+0000","last_fullsized":"2024-09-12T12:23:59.394044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:05.313097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.391219+0000","last_change":"2024-09-12T12:23:58.380267+0000","last_active":"2024-09-12T12:23:59.391219+0000","last_peered":"2024-09-12T12:23:59.391219+0000","last_clean":"2024-09-12T12:23:59.391219+0000","last_became_active":"2024-09-12T12:23:58.379518+0000","last_became_peered":"2024-09-12T12:23:58.379518+0000","last_unstale":"2024-09-12T12:23:59.391219+0000","last_undegraded":"2024-09-12T12:23:59.391219+0000","last_fullsized":"2024-09-12T12:23:59.391219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:16.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393100+0000","last_change":"2024-09-12T12:23:57.387882+0000","last_active":"2024-09-12T12:23:59.393100+0000","last_peered":"2024-09-12T12:23:59.393100+0000","last_clean":"2024-09-12T12:23:59.393100+0000","last_became_active":"2024-09-12T12:23:57.387183+0000","last_became_peered":"2024-09-12T12:23:57.387183+0000","last_unstale":"2024-09-12T12:23:59.393100+0000","last_undegraded":"2024-09-12T12:23:59.393100+0000","last_fullsized":"2024-09-12T12:23:59.393100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:10:19.132216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396219+0000","last_change":"2024-09-12T12:23:58.411509+0000","last_active":"2024-09-12T12:23:59.396219+0000","last_peered":"2024-09-12T12:23:59.396219+0000","last_clean":"2024-09-12T12:23:59.396219+0000","last_became_active":"2024-09-12T12:23:58.411275+0000","last_became_peered":"2024-09-12T12:23:58.411275+0000","last_unstale":"2024-09-12T12:23:59.396219+0000","last_undegraded":"2024-09-12T12:23:59.396219+0000","last_fullsized":"2024-09-12T12:23:59.396219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:56:22.858881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392617+0000","last_change":"2024-09-12T12:23:57.381940+0000","last_active":"2024-09-12T12:23:59.392617+0000","last_peered":"2024-09-12T12:23:59.392617+0000","last_clean":"2024-09-12T12:23:59.392617+0000","last_became_active":"2024-09-12T12:23:57.380630+0000","last_became_peered":"2024-09-12T12:23:57.380630+0000","last_unstale":"2024-09-12T12:23:59.392617+0000","last_undegraded":"2024-09-12T12:23:59.392617+0000","last_fullsized":"2024-09-12T12:23:59.392617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:50.932429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639986+0000","last_change":"2024-09-12T12:23:58.393498+0000","last_active":"2024-09-12T12:24:00.639986+0000","last_peered":"2024-09-12T12:24:00.639986+0000","last_clean":"2024-09-12T12:24:00.639986+0000","last_became_active":"2024-09-12T12:23:58.392463+0000","last_became_peered":"2024-09-12T12:23:58.392463+0000","last_unstale":"2024-09-12T12:24:00.639986+0000","last_undegraded":"2024-09-12T12:24:00.639986+0000","last_fullsized":"2024-09-12T12:24:00.639986+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:15.001374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390791+0000","last_change":"2024-09-12T12:23:57.377798+0000","last_active":"2024-09-12T12:23:59.390791+0000","last_peered":"2024-09-12T12:23:59.390791+0000","last_clean":"2024-09-12T12:23:59.390791+0000","last_became_active":"2024-09-12T12:23:57.376811+0000","last_became_peered":"2024-09-12T12:23:57.376811+0000","last_unstale":"2024-09-12T12:23:59.390791+0000","last_undegraded":"2024-09-12T12:23:59.390791+0000","last_fullsized":"2024-09-12T12:23:59.390791+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:27.962118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395505+0000","last_change":"2024-09-12T12:23:58.374150+0000","last_active":"2024-09-12T12:23:59.395505+0000","last_peered":"2024-09-12T12:23:59.395505+0000","last_clean":"2024-09-12T12:23:59.395505+0000","last_became_active":"2024-09-12T12:23:58.373423+0000","last_became_peered":"2024-09-12T12:23:58.373423+0000","last_unstale":"2024-09-12T12:23:59.395505+0000","last_undegraded":"2024-09-12T12:23:59.395505+0000","last_fullsized":"2024-09-12T12:23:59.395505+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:10.956156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392286+0000","last_change":"2024-09-12T12:23:57.372742+0000","last_active":"2024-09-12T12:23:59.392286+0000","last_peered":"2024-09-12T12:23:59.392286+0000","last_clean":"2024-09-12T12:23:59.392286+0000","last_became_active":"2024-09-12T12:23:57.372354+0000","last_became_peered":"2024-09-12T12:23:57.372354+0000","last_unstale":"2024-09-12T12:23:59.392286+0000","last_undegraded":"2024-09-12T12:23:59.392286+0000","last_fullsized":"2024-09-12T12:23:59.392286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:14:55.671206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638433+0000","last_change":"2024-09-12T12:23:58.391541+0000","last_active":"2024-09-12T12:24:00.638433+0000","last_peered":"2024-09-12T12:24:00.638433+0000","last_clean":"2024-09-12T12:24:00.638433+0000","last_became_active":"2024-09-12T12:23:58.390938+0000","last_became_peered":"2024-09-12T12:23:58.390938+0000","last_unstale":"2024-09-12T12:24:00.638433+0000","last_undegraded":"2024-09-12T12:24:00.638433+0000","last_fullsized":"2024-09-12T12:24:00.638433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:50:43.541480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635714+0000","last_change":"2024-09-12T12:23:57.367013+0000","last_active":"2024-09-12T12:24:00.635714+0000","last_peered":"2024-09-12T12:24:00.635714+0000","last_clean":"2024-09-12T12:24:00.635714+0000","last_became_active":"2024-09-12T12:23:57.366614+0000","last_became_peered":"2024-09-12T12:23:57.366614+0000","last_unstale":"2024-09-12T12:24:00.635714+0000","last_undegraded":"2024-09-12T12:24:00.635714+0000","last_fullsized":"2024-09-12T12:24:00.635714+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:16:37.971791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392098+0000","last_change":"2024-09-12T12:23:58.391761+0000","last_active":"2024-09-12T12:23:59.392098+0000","last_peered":"2024-09-12T12:23:59.392098+0000","last_clean":"2024-09-12T12:23:59.392098+0000","last_became_active":"2024-09-12T12:23:58.391272+0000","last_became_peered":"2024-09-12T12:23:58.391272+0000","last_unstale":"2024-09-12T12:23:59.392098+0000","last_undegraded":"2024-09-12T12:23:59.392098+0000","last_fullsized":"2024-09-12T12:23:59.392098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:22.494179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390380+0000","last_change":"2024-09-12T12:23:57.385811+0000","last_active":"2024-09-12T12:23:59.390380+0000","last_peered":"2024-09-12T12:23:59.390380+0000","last_clean":"2024-09-12T12:23:59.390380+0000","last_became_active":"2024-09-12T12:23:57.384813+0000","last_became_peered":"2024-09-12T12:23:57.384813+0000","last_unstale":"2024-09-12T12:23:59.390380+0000","last_undegraded":"2024-09-12T12:23:59.390380+0000","last_fullsized":"2024-09-12T12:23:59.390380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:30:14.178742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392042+0000","last_change":"2024-09-12T12:23:58.415030+0000","last_active":"2024-09-12T12:23:59.392042+0000","last_peered":"2024-09-12T12:23:59.392042+0000","last_clean":"2024-09-12T12:23:59.392042+0000","last_became_active":"2024-09-12T12:23:58.414677+0000","last_became_peered":"2024-09-12T12:23:58.414677+0000","last_unstale":"2024-09-12T12:23:59.392042+0000","last_undegraded":"2024-09-12T12:23:59.392042+0000","last_fullsized":"2024-09-12T12:23:59.392042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:08:51.942684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395215+0000","last_change":"2024-09-12T12:23:57.385905+0000","last_active":"2024-09-12T12:23:59.395215+0000","last_peered":"2024-09-12T12:23:59.395215+0000","last_clean":"2024-09-12T12:23:59.395215+0000","last_became_active":"2024-09-12T12:23:57.385555+0000","last_became_peered":"2024-09-12T12:23:57.385555+0000","last_unstale":"2024-09-12T12:23:59.395215+0000","last_undegraded":"2024-09-12T12:23:59.395215+0000","last_fullsized":"2024-09-12T12:23:59.395215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:22.740585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639455+0000","last_change":"2024-09-12T12:23:58.380280+0000","last_active":"2024-09-12T12:24:00.639455+0000","last_peered":"2024-09-12T12:24:00.639455+0000","last_clean":"2024-09-12T12:24:00.639455+0000","last_became_active":"2024-09-12T12:23:58.379576+0000","last_became_peered":"2024-09-12T12:23:58.379576+0000","last_unstale":"2024-09-12T12:24:00.639455+0000","last_undegraded":"2024-09-12T12:24:00.639455+0000","last_fullsized":"2024-09-12T12:24:00.639455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:12:20.884415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.394285+0000","last_change":"2024-09-12T12:23:57.390472+0000","last_active":"2024-09-12T12:23:59.394285+0000","last_peered":"2024-09-12T12:23:59.394285+0000","last_clean":"2024-09-12T12:23:59.394285+0000","last_became_active":"2024-09-12T12:23:57.389725+0000","last_became_peered":"2024-09-12T12:23:57.389725+0000","last_unstale":"2024-09-12T12:23:59.394285+0000","last_undegraded":"2024-09-12T12:23:59.394285+0000","last_fullsized":"2024-09-12T12:23:59.394285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:39.937210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393846+0000","last_change":"2024-09-12T12:23:58.398413+0000","last_active":"2024-09-12T12:23:59.393846+0000","last_peered":"2024-09-12T12:23:59.393846+0000","last_clean":"2024-09-12T12:23:59.393846+0000","last_became_active":"2024-09-12T12:23:58.397352+0000","last_became_peered":"2024-09-12T12:23:58.397352+0000","last_unstale":"2024-09-12T12:23:59.393846+0000","last_undegraded":"2024-09-12T12:23:59.393846+0000","last_fullsized":"2024-09-12T12:23:59.393846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:03:56.610591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395396+0000","last_change":"2024-09-12T12:23:57.378862+0000","last_active":"2024-09-12T12:23:59.395396+0000","last_peered":"2024-09-12T12:23:59.395396+0000","last_clean":"2024-09-12T12:23:59.395396+0000","last_became_active":"2024-09-12T12:23:57.378463+0000","last_became_peered":"2024-09-12T12:23:57.378463+0000","last_unstale":"2024-09-12T12:23:59.395396+0000","last_undegraded":"2024-09-12T12:23:59.395396+0000","last_fullsized":"2024-09-12T12:23:59.395396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:05.881468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639984+0000","last_change":"2024-09-12T12:23:58.398669+0000","last_active":"2024-09-12T12:24:00.639984+0000","last_peered":"2024-09-12T12:24:00.639984+0000","last_clean":"2024-09-12T12:24:00.639984+0000","last_became_active":"2024-09-12T12:23:58.397659+0000","last_became_peered":"2024-09-12T12:23:58.397659+0000","last_unstale":"2024-09-12T12:24:00.639984+0000","last_undegraded":"2024-09-12T12:24:00.639984+0000","last_fullsized":"2024-09-12T12:24:00.639984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:09:55.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396340+0000","last_change":"2024-09-12T12:23:57.377815+0000","last_active":"2024-09-12T12:23:59.396340+0000","last_peered":"2024-09-12T12:23:59.396340+0000","last_clean":"2024-09-12T12:23:59.396340+0000","last_became_active":"2024-09-12T12:23:57.377401+0000","last_became_peered":"2024-09-12T12:23:57.377401+0000","last_unstale":"2024-09-12T12:23:59.396340+0000","last_undegraded":"2024-09-12T12:23:59.396340+0000","last_fullsized":"2024-09-12T12:23:59.396340+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:06.342439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635769+0000","last_change":"2024-09-12T12:23:58.393822+0000","last_active":"2024-09-12T12:24:00.635769+0000","last_peered":"2024-09-12T12:24:00.635769+0000","last_clean":"2024-09-12T12:24:00.635769+0000","last_became_active":"2024-09-12T12:23:58.393144+0000","last_became_peered":"2024-09-12T12:23:58.393144+0000","last_unstale":"2024-09-12T12:24:00.635769+0000","last_undegraded":"2024-09-12T12:24:00.635769+0000","last_fullsized":"2024-09-12T12:24:00.635769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:21:19.984058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635748+0000","last_change":"2024-09-12T12:23:57.367701+0000","last_active":"2024-09-12T12:24:00.635748+0000","last_peered":"2024-09-12T12:24:00.635748+0000","last_clean":"2024-09-12T12:24:00.635748+0000","last_became_active":"2024-09-12T12:23:57.367397+0000","last_became_peered":"2024-09-12T12:23:57.367397+0000","last_unstale":"2024-09-12T12:24:00.635748+0000","last_undegraded":"2024-09-12T12:24:00.635748+0000","last_fullsized":"2024-09-12T12:24:00.635748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:26:48.411535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.391045+0000","last_change":"2024-09-12T12:23:58.388342+0000","last_active":"2024-09-12T12:23:59.391045+0000","last_peered":"2024-09-12T12:23:59.391045+0000","last_clean":"2024-09-12T12:23:59.391045+0000","last_became_active":"2024-09-12T12:23:58.387470+0000","last_became_peered":"2024-09-12T12:23:58.387470+0000","last_unstale":"2024-09-12T12:23:59.391045+0000","last_undegraded":"2024-09-12T12:23:59.391045+0000","last_fullsized":"2024-09-12T12:23:59.391045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:52:54.437455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.636539+0000","last_change":"2024-09-12T12:23:57.365147+0000","last_active":"2024-09-12T12:24:00.636539+0000","last_peered":"2024-09-12T12:24:00.636539+0000","last_clean":"2024-09-12T12:24:00.636539+0000","last_became_active":"2024-09-12T12:23:57.364796+0000","last_became_peered":"2024-09-12T12:23:57.364796+0000","last_unstale":"2024-09-12T12:24:00.636539+0000","last_undegraded":"2024-09-12T12:24:00.636539+0000","last_fullsized":"2024-09-12T12:24:00.636539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:10:39.896309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635247+0000","last_change":"2024-09-12T12:23:58.386311+0000","last_active":"2024-09-12T12:24:00.635247+0000","last_peered":"2024-09-12T12:24:00.635247+0000","last_clean":"2024-09-12T12:24:00.635247+0000","last_became_active":"2024-09-12T12:23:58.385942+0000","last_became_peered":"2024-09-12T12:23:58.385942+0000","last_unstale":"2024-09-12T12:24:00.635247+0000","last_undegraded":"2024-09-12T12:24:00.635247+0000","last_fullsized":"2024-09-12T12:24:00.635247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:53:18.140011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.634866+0000","last_change":"2024-09-12T12:23:57.386864+0000","last_active":"2024-09-12T12:24:00.634866+0000","last_peered":"2024-09-12T12:24:00.634866+0000","last_clean":"2024-09-12T12:24:00.634866+0000","last_became_active":"2024-09-12T12:23:57.386550+0000","last_became_peered":"2024-09-12T12:23:57.386550+0000","last_unstale":"2024-09-12T12:24:00.634866+0000","last_undegraded":"2024-09-12T12:24:00.634866+0000","last_fullsized":"2024-09-12T12:24:00.634866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:12:10.152000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390713+0000","last_change":"2024-09-12T12:23:58.374811+0000","last_active":"2024-09-12T12:23:59.390713+0000","last_peered":"2024-09-12T12:23:59.390713+0000","last_clean":"2024-09-12T12:23:59.390713+0000","last_became_active":"2024-09-12T12:23:58.373952+0000","last_became_peered":"2024-09-12T12:23:58.373952+0000","last_unstale":"2024-09-12T12:23:59.390713+0000","last_undegraded":"2024-09-12T12:23:59.390713+0000","last_fullsized":"2024-09-12T12:23:59.390713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:45:52.908148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.635040+0000","last_change":"2024-09-12T12:23:57.368319+0000","last_active":"2024-09-12T12:24:00.635040+0000","last_peered":"2024-09-12T12:24:00.635040+0000","last_clean":"2024-09-12T12:24:00.635040+0000","last_became_active":"2024-09-12T12:23:57.367912+0000","last_became_peered":"2024-09-12T12:23:57.367912+0000","last_unstale":"2024-09-12T12:24:00.635040+0000","last_undegraded":"2024-09-12T12:24:00.635040+0000","last_fullsized":"2024-09-12T12:24:00.635040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:45:35.720409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.639813+0000","last_change":"2024-09-12T12:23:58.379545+0000","last_active":"2024-09-12T12:24:00.639813+0000","last_peered":"2024-09-12T12:24:00.639813+0000","last_clean":"2024-09-12T12:24:00.639813+0000","last_became_active":"2024-09-12T12:23:58.378816+0000","last_became_peered":"2024-09-12T12:23:58.378816+0000","last_unstale":"2024-09-12T12:24:00.639813+0000","last_undegraded":"2024-09-12T12:24:00.639813+0000","last_fullsized":"2024-09-12T12:24:00.639813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:32:37.144121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393386+0000","last_change":"2024-09-12T12:23:57.386214+0000","last_active":"2024-09-12T12:23:59.393386+0000","last_peered":"2024-09-12T12:23:59.393386+0000","last_clean":"2024-09-12T12:23:59.393386+0000","last_became_active":"2024-09-12T12:23:57.385756+0000","last_became_peered":"2024-09-12T12:23:57.385756+0000","last_unstale":"2024-09-12T12:23:59.393386+0000","last_undegraded":"2024-09-12T12:23:59.393386+0000","last_fullsized":"2024-09-12T12:23:59.393386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:54:50.660652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638459+0000","last_change":"2024-09-12T12:23:58.400966+0000","last_active":"2024-09-12T12:24:00.638459+0000","last_peered":"2024-09-12T12:24:00.638459+0000","last_clean":"2024-09-12T12:24:00.638459+0000","last_became_active":"2024-09-12T12:23:58.400073+0000","last_became_peered":"2024-09-12T12:23:58.400073+0000","last_unstale":"2024-09-12T12:24:00.638459+0000","last_undegraded":"2024-09-12T12:24:00.638459+0000","last_fullsized":"2024-09-12T12:24:00.638459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:47.090366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393431+0000","last_change":"2024-09-12T12:23:57.369090+0000","last_active":"2024-09-12T12:23:59.393431+0000","last_peered":"2024-09-12T12:23:59.393431+0000","last_clean":"2024-09-12T12:23:59.393431+0000","last_became_active":"2024-09-12T12:23:57.368802+0000","last_became_peered":"2024-09-12T12:23:57.368802+0000","last_unstale":"2024-09-12T12:23:59.393431+0000","last_undegraded":"2024-09-12T12:23:59.393431+0000","last_fullsized":"2024-09-12T12:23:59.393431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:38.077870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393429+0000","last_change":"2024-09-12T12:23:58.402393+0000","last_active":"2024-09-12T12:23:59.393429+0000","last_peered":"2024-09-12T12:23:59.393429+0000","last_clean":"2024-09-12T12:23:59.393429+0000","last_became_active":"2024-09-12T12:23:58.399806+0000","last_became_peered":"2024-09-12T12:23:58.399806+0000","last_unstale":"2024-09-12T12:23:59.393429+0000","last_undegraded":"2024-09-12T12:23:59.393429+0000","last_fullsized":"2024-09-12T12:23:59.393429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:15.001374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393093+0000","last_change":"2024-09-12T12:23:57.382433+0000","last_active":"2024-09-12T12:23:59.393093+0000","last_peered":"2024-09-12T12:23:59.393093+0000","last_clean":"2024-09-12T12:23:59.393093+0000","last_became_active":"2024-09-12T12:23:57.381303+0000","last_became_peered":"2024-09-12T12:23:57.381303+0000","last_unstale":"2024-09-12T12:23:59.393093+0000","last_undegraded":"2024-09-12T12:23:59.393093+0000","last_fullsized":"2024-09-12T12:23:59.393093+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:21.483924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.640380+0000","last_change":"2024-09-12T12:23:58.393099+0000","last_active":"2024-09-12T12:24:00.640380+0000","last_peered":"2024-09-12T12:24:00.640380+0000","last_clean":"2024-09-12T12:24:00.640380+0000","last_became_active":"2024-09-12T12:23:58.392406+0000","last_became_peered":"2024-09-12T12:23:58.392406+0000","last_unstale":"2024-09-12T12:24:00.640380+0000","last_undegraded":"2024-09-12T12:24:00.640380+0000","last_fullsized":"2024-09-12T12:24:00.640380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:14:12.942908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.393467+0000","last_change":"2024-09-12T12:23:57.382205+0000","last_active":"2024-09-12T12:23:59.393467+0000","last_peered":"2024-09-12T12:23:59.393467+0000","last_clean":"2024-09-12T12:23:59.393467+0000","last_became_active":"2024-09-12T12:23:57.381644+0000","last_became_peered":"2024-09-12T12:23:57.381644+0000","last_unstale":"2024-09-12T12:23:59.393467+0000","last_undegraded":"2024-09-12T12:23:59.393467+0000","last_fullsized":"2024-09-12T12:23:59.393467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:23.342087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.390832+0000","last_change":"2024-09-12T12:23:58.384197+0000","last_active":"2024-09-12T12:23:59.390832+0000","last_peered":"2024-09-12T12:23:59.390832+0000","last_clean":"2024-09-12T12:23:59.390832+0000","last_became_active":"2024-09-12T12:23:58.383235+0000","last_became_peered":"2024-09-12T12:23:58.383235+0000","last_unstale":"2024-09-12T12:23:59.390832+0000","last_undegraded":"2024-09-12T12:23:59.390832+0000","last_fullsized":"2024-09-12T12:23:59.390832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:12.842154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.396013+0000","last_change":"2024-09-12T12:23:57.381829+0000","last_active":"2024-09-12T12:23:59.396013+0000","last_peered":"2024-09-12T12:23:59.396013+0000","last_clean":"2024-09-12T12:23:59.396013+0000","last_became_active":"2024-09-12T12:23:57.381422+0000","last_became_peered":"2024-09-12T12:23:57.381422+0000","last_unstale":"2024-09-12T12:23:59.396013+0000","last_undegraded":"2024-09-12T12:23:59.396013+0000","last_fullsized":"2024-09-12T12:23:59.396013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:52:10.834460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.395928+0000","last_change":"2024-09-12T12:23:58.378283+0000","last_active":"2024-09-12T12:23:59.395928+0000","last_peered":"2024-09-12T12:23:59.395928+0000","last_clean":"2024-09-12T12:23:59.395928+0000","last_became_active":"2024-09-12T12:23:58.377451+0000","last_became_peered":"2024-09-12T12:23:58.377451+0000","last_unstale":"2024-09-12T12:23:59.395928+0000","last_undegraded":"2024-09-12T12:23:59.395928+0000","last_fullsized":"2024-09-12T12:23:59.395928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:11:22.657305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638615+0000","last_change":"2024-09-12T12:23:57.366499+0000","last_active":"2024-09-12T12:24:00.638615+0000","last_peered":"2024-09-12T12:24:00.638615+0000","last_clean":"2024-09-12T12:24:00.638615+0000","last_became_active":"2024-09-12T12:23:57.364717+0000","last_became_peered":"2024-09-12T12:23:57.364717+0000","last_unstale":"2024-09-12T12:24:00.638615+0000","last_undegraded":"2024-09-12T12:24:00.638615+0000","last_fullsized":"2024-09-12T12:24:00.638615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:39.937210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637851+0000","last_change":"2024-09-12T12:23:58.396186+0000","last_active":"2024-09-12T12:24:00.637851+0000","last_peered":"2024-09-12T12:24:00.637851+0000","last_clean":"2024-09-12T12:24:00.637851+0000","last_became_active":"2024-09-12T12:23:58.393902+0000","last_became_peered":"2024-09-12T12:23:58.393902+0000","last_unstale":"2024-09-12T12:24:00.637851+0000","last_undegraded":"2024-09-12T12:24:00.637851+0000","last_fullsized":"2024-09-12T12:24:00.637851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:50:19.418045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392449+0000","last_change":"2024-09-12T12:23:57.388887+0000","last_active":"2024-09-12T12:23:59.392449+0000","last_peered":"2024-09-12T12:23:59.392449+0000","last_clean":"2024-09-12T12:23:59.392449+0000","last_became_active":"2024-09-12T12:23:57.388290+0000","last_became_peered":"2024-09-12T12:23:57.388290+0000","last_unstale":"2024-09-12T12:23:59.392449+0000","last_undegraded":"2024-09-12T12:23:59.392449+0000","last_fullsized":"2024-09-12T12:23:59.392449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:27.962118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392267+0000","last_change":"2024-09-12T12:23:58.397164+0000","last_active":"2024-09-12T12:23:59.392267+0000","last_peered":"2024-09-12T12:23:59.392267+0000","last_clean":"2024-09-12T12:23:59.392267+0000","last_became_active":"2024-09-12T12:23:58.396442+0000","last_became_peered":"2024-09-12T12:23:58.396442+0000","last_unstale":"2024-09-12T12:23:59.392267+0000","last_undegraded":"2024-09-12T12:23:59.392267+0000","last_fullsized":"2024-09-12T12:23:59.392267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:16.407111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.637962+0000","last_change":"2024-09-12T12:23:57.390177+0000","last_active":"2024-09-12T12:24:00.637962+0000","last_peered":"2024-09-12T12:24:00.637962+0000","last_clean":"2024-09-12T12:24:00.637962+0000","last_became_active":"2024-09-12T12:23:57.389410+0000","last_became_peered":"2024-09-12T12:23:57.389410+0000","last_unstale":"2024-09-12T12:24:00.637962+0000","last_undegraded":"2024-09-12T12:24:00.637962+0000","last_fullsized":"2024-09-12T12:24:00.637962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:14:05.313097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:23:59.392877+0000","last_change":"2024-09-12T12:23:58.390793+0000","last_active":"2024-09-12T12:23:59.392877+0000","last_peered":"2024-09-12T12:23:59.392877+0000","last_clean":"2024-09-12T12:23:59.392877+0000","last_became_active":"2024-09-12T12:23:58.390529+0000","last_became_peered":"2024-09-12T12:23:58.390529+0000","last_unstale":"2024-09-12T12:23:59.392877+0000","last_undegraded":"2024-09-12T12:23:59.392877+0000","last_fullsized":"2024-09-12T12:23:59.392877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:57.335036+0000","last_clean_scrub_stamp":"2024-09-12T12:23:57.335036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:24.352342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638879+0000","last_change":"2024-09-12T12:23:57.386525+0000","last_active":"2024-09-12T12:24:00.638879+0000","last_peered":"2024-09-12T12:24:00.638879+0000","last_clean":"2024-09-12T12:24:00.638879+0000","last_became_active":"2024-09-12T12:23:57.385710+0000","last_became_peered":"2024-09-12T12:23:57.385710+0000","last_unstale":"2024-09-12T12:24:00.638879+0000","last_undegraded":"2024-09-12T12:24:00.638879+0000","last_fullsized":"2024-09-12T12:24:00.638879+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:23:56.324781+0000","last_clean_scrub_stamp":"2024-09-12T12:23:56.324781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:54:15.396855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":277,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-12T12:24:00.638136+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:24:00.638136+0000","last_peered":"2024-09-12T12:24:00.638136+0000","last_clean":"2024-09-12T12:24:00.638136+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:24:00.638136+0000","last_undegraded":"2024-09-12T12:24:00.638136+0000","last_fullsized":"2024-09-12T12:24:00.638136+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509497,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":410323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542201,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1564672,"data_stored":999149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":978944,"data_stored":409405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542200,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":983040,"data_stored":410345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":408711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542199,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":409293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":978944,"data_stored":408905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":999113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:07.291 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:24:07.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T12:24:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc706129640 1 Processor -- start 2024-09-12T12:24:07.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc706129640 1 -- start start 2024-09-12T12:24:07.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc706129640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc70015c210 0x7fc70006e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc706129640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7000adf60 0x7fc70015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc706129640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc700160600 con 0x7fc7000d05c0 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc706129640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc7001608e0 con 0x7fc7000adf60 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc706129640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc700160bc0 con 0x7fc70015c210 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6feffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7000adf60 0x7fc70015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6fffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc7000d05c0 0x7fc7000ccb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38832/0 (socket says 172.21.15.3:38832) 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6fffff640 1 -- 172.21.15.3:0/3378271860 learned_addr learned my addr 172.21.15.3:0/3378271860 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6ff7fe640 1 --2- 172.21.15.3:0/3378271860 >> v2:172.21.15.142:3300/0 conn(0x7fc70015c210 0x7fc70006e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6feffd640 1 -- 172.21.15.3:0/3378271860 >> v2:172.21.15.142:3300/0 conn(0x7fc70015c210 msgr2=0x7fc70006e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6feffd640 1 --2- 172.21.15.3:0/3378271860 >> v2:172.21.15.142:3300/0 conn(0x7fc70015c210 0x7fc70006e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6feffd640 1 -- 172.21.15.3:0/3378271860 >> v1:172.21.15.3:6789/0 conn(0x7fc7000d05c0 legacy=0x7fc7000ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6feffd640 1 -- 172.21.15.3:0/3378271860 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7000cec00 con 0x7fc7000adf60 2024-09-12T12:24:07.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.464+0000 7fc6fe7fc640 1 -- 172.21.15.3:0/3378271860 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934479876 0 0) 0x7fc700160600 con 0x7fc7000d05c0 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc6feffd640 1 --2- 172.21.15.3:0/3378271860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc70015feb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc7000b5410 tx=0x7fc6f401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0edcaabb3750a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc6fe7fc640 1 -- 172.21.15.3:0/3378271860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40b0070 con 0x7fc7000adf60 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc6fe7fc640 1 -- 172.21.15.3:0/3378271860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f40ab600 con 0x7fc7000adf60 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc6fe7fc640 1 -- 172.21.15.3:0/3378271860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40ab9d0 con 0x7fc7000adf60 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 -- 172.21.15.3:0/3378271860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 msgr2=0x7fc70015feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 --2- 172.21.15.3:0/3378271860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc70015feb0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc7000b5410 tx=0x7fc6f401c3f0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 -- 172.21.15.3:0/3378271860 shutdown_connections 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 --2- 172.21.15.3:0/3378271860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc70015feb0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 --2- 172.21.15.3:0/3378271860 >> v2:172.21.15.142:3300/0 conn(0x7fc70015c210 0x7fc70006e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 -- 172.21.15.3:0/3378271860 >> 172.21.15.3:0/3378271860 conn(0x7fc700067e00 msgr2=0x7fc700068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:07.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 -- 172.21.15.3:0/3378271860 shutdown_connections 2024-09-12T12:24:07.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.468+0000 7fc706129640 1 -- 172.21.15.3:0/3378271860 wait complete. 2024-09-12T12:24:07.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 Processor -- start 2024-09-12T12:24:07.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 -- start start 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc7000b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7000d05c0 0x7fc700165d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70015c210 0x7fc70014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7001695e0 con 0x7fc7000adf60 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc700169460 con 0x7fc7000d05c0 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc700169760 con 0x7fc70015c210 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc7000b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc7000b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60688/0 (socket says 172.21.15.3:60688) 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 -- 172.21.15.3:0/2513522926 learned_addr learned my addr 172.21.15.3:0/2513522926 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 -- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70015c210 msgr2=0x7fc70014e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6fffff640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70015c210 0x7fc70014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6feffd640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7000d05c0 0x7fc700165d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70015c210 0x7fc70014e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 -- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7000d05c0 msgr2=0x7fc700165d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7000d05c0 0x7fc700165d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 -- 172.21.15.3:0/2513522926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7000b3870 con 0x7fc7000adf60 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6ff7fe640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc7000b3170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000d5f0 tx=0x7fc6f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6e7fff640 1 -- 172.21.15.3:0/2513522926 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00125f0 con 0x7fc7000adf60 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6e7fff640 1 -- 172.21.15.3:0/2513522926 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f0019070 con 0x7fc7000adf60 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7000b3b50 con 0x7fc7000adf60 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6e7fff640 1 -- 172.21.15.3:0/2513522926 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00129b0 con 0x7fc7000adf60 2024-09-12T12:24:07.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7000b4110 con 0x7fc7000adf60 2024-09-12T12:24:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70006e450 con 0x7fc7000adf60 2024-09-12T12:24:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6e7fff640 1 -- 172.21.15.3:0/2513522926 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc6f0012b50 con 0x7fc7000adf60 2024-09-12T12:24:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.472+0000 7fc6e7fff640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6ec0678c0 0x7fc6ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.476+0000 7fc6e7fff640 1 -- 172.21.15.3:0/2513522926 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fc6f00860d0 con 0x7fc7000adf60 2024-09-12T12:24:07.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.476+0000 7fc6feffd640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6ec0678c0 0x7fc6ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.476+0000 7fc6feffd640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6ec0678c0 0x7fc6ec069d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40ad000 tx=0x7fc6f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.480+0000 7fc6e7fff640 1 -- 172.21.15.3:0/2513522926 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc6f0056c10 con 0x7fc7000adf60 2024-09-12T12:24:07.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.708+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc7000ce840 con 0x7fc7000adf60 2024-09-12T12:24:07.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc6e7fff640 1 -- 172.21.15.3:0/2513522926 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7fc6f0056510 con 0x7fc7000adf60 2024-09-12T12:24:07.714 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T12:24:07.714 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:24:07.714 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-12T12:24:07.715 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-12T12:24:07.715 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T12:24:07.715 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6ec0678c0 msgr2=0x7fc6ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6ec0678c0 0x7fc6ec069d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40ad000 tx=0x7fc6f4008040 comp rx=0 tx=0).stop 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 msgr2=0x7fc7000b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc7000b3170 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000d5f0 tx=0x7fc6f000dab0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 shutdown_connections 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6ec0678c0 0x7fc6ec069d80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc70015c210 0x7fc70014e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7000d05c0 0x7fc700165d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 --2- 172.21.15.3:0/2513522926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7000adf60 0x7fc7000b3170 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 >> 172.21.15.3:0/2513522926 conn(0x7fc700067e00 msgr2=0x7fc70015a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 shutdown_connections 2024-09-12T12:24:07.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.712+0000 7fc706129640 1 -- 172.21.15.3:0/2513522926 wait complete. 2024-09-12T12:24:07.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:07.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f4867c5d640 1 Processor -- start 2024-09-12T12:24:07.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f4867c5d640 1 -- start start 2024-09-12T12:24:07.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f4867c5d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f486015c710 0x7f48600b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f4867c5d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48600b5000 con 0x7f4860073cb0 2024-09-12T12:24:07.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f4867c5d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48600b37c0 con 0x7f48600ae880 2024-09-12T12:24:07.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f4867c5d640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4860074090 con 0x7f486015c710 2024-09-12T12:24:07.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48659f9640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4860073cb0 0x7f48600adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38842/0 (socket says 172.21.15.3:38842) 2024-09-12T12:24:07.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48659f9640 1 -- 172.21.15.3:0/1237850994 learned_addr learned my addr 172.21.15.3:0/1237850994 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:07.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48651f8640 1 --2- 172.21.15.3:0/1237850994 >> v2:172.21.15.142:3300/0 conn(0x7f486015c710 0x7f48600b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437166001 0 0) 0x7f48600b5000 con 0x7f4860073cb0 2024-09-12T12:24:07.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f484c003530 con 0x7f4860073cb0 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3518278985 0 0) 0x7f484c003530 con 0x7f4860073cb0 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 >> v2:172.21.15.142:3300/0 conn(0x7f486015c710 msgr2=0x7f48600b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 --2- 172.21.15.3:0/1237850994 >> v2:172.21.15.142:3300/0 conn(0x7f486015c710 0x7f48600b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 >> v1:172.21.15.142:6789/0 conn(0x7f48600ae880 legacy=0x7f48600d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48600b3aa0 con 0x7f4860073cb0 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f485c001a00 con 0x7f4860073cb0 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 >> v1:172.21.15.3:6789/0 conn(0x7f4860073cb0 legacy=0x7f48600adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c0039c0 0x7f484c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484c0043c0 0x7f484c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c005090 0x7f484c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f484c00bfc0 con 0x7f484c0043c0 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f484c00c170 con 0x7f484c0039c0 2024-09-12T12:24:07.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f484c00c370 con 0x7f484c005090 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48651f8640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484c0043c0 0x7f484c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48659f9640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c0039c0 0x7f484c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48661fa640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c005090 0x7f484c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48651f8640 1 -- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c005090 msgr2=0x7f484c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48651f8640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c005090 0x7f484c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48651f8640 1 -- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c0039c0 msgr2=0x7f484c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48651f8640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f484c0039c0 0x7f484c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48651f8640 1 -- 172.21.15.3:0/1237850994 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c003530 con 0x7f484c0043c0 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48661fa640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c005090 0x7f484c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:07.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.860+0000 7f48661fa640 1 -- 172.21.15.3:0/1237850994 reap_dead start 2024-09-12T12:24:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f48651f8640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484c0043c0 0x7f484c004890 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4850009ae0 tx=0x7f485001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70269418292b087 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500b2070 con 0x7f484c0043c0 2024-09-12T12:24:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48500a5660 con 0x7f484c0043c0 2024-09-12T12:24:07.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f48649f7640 1 -- 172.21.15.3:0/1237850994 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500096b0 con 0x7f484c0043c0 2024-09-12T12:24:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f4867c5d640 1 -- 172.21.15.3:0/1237850994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484c0043c0 msgr2=0x7f484c004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f4867c5d640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484c0043c0 0x7f484c004890 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4850009ae0 tx=0x7f485001ca70 comp rx=0 tx=0).stop 2024-09-12T12:24:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f4867c5d640 1 -- 172.21.15.3:0/1237850994 shutdown_connections 2024-09-12T12:24:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f4867c5d640 1 --2- 172.21.15.3:0/1237850994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f484c0043c0 0x7f484c004890 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f4867c5d640 1 -- 172.21.15.3:0/1237850994 >> 172.21.15.3:0/1237850994 conn(0x7f4860067e00 msgr2=0x7f4860068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f4867c5d640 1 -- 172.21.15.3:0/1237850994 shutdown_connections 2024-09-12T12:24:07.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.864+0000 7f4867c5d640 1 -- 172.21.15.3:0/1237850994 wait complete. 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 Processor -- start 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 -- start start 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c0043c0 0x7f4860152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48601560b0 0x7f48601f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48601f3f90 0x7f486015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f48600b30e0 con 0x7f48601560b0 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48600ad6f0 con 0x7f48601f3f90 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f48600b3260 con 0x7f484c0043c0 2024-09-12T12:24:07.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48601560b0 0x7f48601f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48601560b0 0x7f48601f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60712/0 (socket says 172.21.15.3:60712) 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 -- 172.21.15.3:0/3521017420 learned_addr learned my addr 172.21.15.3:0/3521017420 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48661fa640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48601f3f90 0x7f486015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 -- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c0043c0 msgr2=0x7f4860152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48659f9640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c0043c0 0x7f4860152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c0043c0 0x7f4860152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 -- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48601f3f90 msgr2=0x7f486015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48601f3f90 0x7f486015ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 -- 172.21.15.3:0/3521017420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48601f4410 con 0x7f48601560b0 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48661fa640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48601f3f90 0x7f486015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:07.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48651f8640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48601560b0 0x7f48601f3a50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f485001c8a0 tx=0x7f4850019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48567fc640 1 -- 172.21.15.3:0/3521017420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500b2040 con 0x7f48601560b0 2024-09-12T12:24:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48567fc640 1 -- 172.21.15.3:0/3521017420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4850004020 con 0x7f48601560b0 2024-09-12T12:24:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48601f46f0 con 0x7f48601560b0 2024-09-12T12:24:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f48567fc640 1 -- 172.21.15.3:0/3521017420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500043f0 con 0x7f48601560b0 2024-09-12T12:24:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48600b13d0 con 0x7f48601560b0 2024-09-12T12:24:07.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.868+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48600aecb0 con 0x7f48601560b0 2024-09-12T12:24:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.872+0000 7f48567fc640 1 -- 172.21.15.3:0/3521017420 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f48500bf030 con 0x7f48601560b0 2024-09-12T12:24:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.872+0000 7f48567fc640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48440678c0 0x7f4844069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.872+0000 7f48567fc640 1 -- 172.21.15.3:0/3521017420 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f485011fb10 con 0x7f48601560b0 2024-09-12T12:24:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.872+0000 7f48659f9640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48440678c0 0x7f4844069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:07.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.872+0000 7f48659f9640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48440678c0 0x7f4844069d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f485c005c40 tx=0x7f485c0021e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:07.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:07.876+0000 7f48567fc640 1 -- 172.21.15.3:0/3521017420 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f48500c6060 con 0x7f48601560b0 2024-09-12T12:24:08.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.072+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48600af940 con 0x7f48601560b0 2024-09-12T12:24:08.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.072+0000 7f48567fc640 1 -- 172.21.15.3:0/3521017420 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5222 (secure 0 0 0) 0x7f4850016780 con 0x7f48601560b0 2024-09-12T12:24:08.077 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:08.077 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:57.730276+0000","modified":"2024-09-12T12:24:00.762240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2940169617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2940169617},{"type":"v1","addr":"172.21.15.3:6835","nonce":2940169617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-09-12T12:24:08.077 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 54 2024-09-12T12:24:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48440678c0 msgr2=0x7f4844069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48440678c0 0x7f4844069d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f485c005c40 tx=0x7f485c0021e0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48601560b0 msgr2=0x7f48601f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48601560b0 0x7f48601f3a50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f485001c8a0 tx=0x7f4850019810 comp rx=0 tx=0).stop 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 shutdown_connections 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f48440678c0 0x7f4844069d80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48601f3f90 0x7f486015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48601560b0 0x7f48601f3a50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 --2- 172.21.15.3:0/3521017420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f484c0043c0 0x7f4860152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 >> 172.21.15.3:0/3521017420 conn(0x7f4860067e00 msgr2=0x7f48600aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 shutdown_connections 2024-09-12T12:24:08.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.076+0000 7f4867c5d640 1 -- 172.21.15.3:0/3521017420 wait complete. 2024-09-12T12:24:08.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:23:57.730276+0000', 'modified': '2024-09-12T12:24:00.762240+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6375}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6375': {'gid': 6375, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/2940169617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 2940169617}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 2940169617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6375, 'qdb_cluster': [6375]} 2024-09-12T12:24:08.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:24:08.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T12:24:08.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3447f2a640 1 Processor -- start 2024-09-12T12:24:08.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3447f2a640 1 -- start start 2024-09-12T12:24:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3447f2a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f34400ae250 0x7f34400ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3447f2a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34400ad0c0 con 0x7f34400d05b0 2024-09-12T12:24:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3447f2a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34400ad3a0 con 0x7f344015b620 2024-09-12T12:24:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3447f2a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f34400ad680 con 0x7f34400ae250 2024-09-12T12:24:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3445cc6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f34400d05b0 0x7f34400d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45056/0 (socket says 172.21.15.3:45056) 2024-09-12T12:24:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3445cc6640 1 -- 172.21.15.3:0/2482068066 learned_addr learned my addr 172.21.15.3:0/2482068066 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:08.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f34454c5640 1 --2- 172.21.15.3:0/2482068066 >> v2:172.21.15.142:3300/0 conn(0x7f34400ae250 0x7f34400ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2126352217 0 0) 0x7f34400ad0c0 con 0x7f34400d05b0 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f342c003530 con 0x7f34400d05b0 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3227641680 0 0) 0x7f342c003530 con 0x7f34400d05b0 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 >> v2:172.21.15.142:3300/0 conn(0x7f34400ae250 msgr2=0x7f34400ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 --2- 172.21.15.3:0/2482068066 >> v2:172.21.15.142:3300/0 conn(0x7f34400ae250 0x7f34400ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 >> v1:172.21.15.142:6789/0 conn(0x7f344015b620 legacy=0x7f344015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34400ad960 con 0x7f34400d05b0 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f343c001680 con 0x7f34400d05b0 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 >> v1:172.21.15.3:6789/0 conn(0x7f34400d05b0 legacy=0x7f34400d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0039f0 0x7f342c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c004810 0x7f342c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c0051d0 0x7f342c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34400ad680 con 0x7f342c0039f0 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f342c00c0a0 con 0x7f342c0051d0 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.632+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f342c00c2a0 con 0x7f342c004810 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0039f0 0x7f342c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f34454c5640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c004810 0x7f342c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3445cc6640 1 -- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c004810 msgr2=0x7f342c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c004810 0x7f342c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3445cc6640 1 -- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c0051d0 msgr2=0x7f342c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f34464c7640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c0051d0 0x7f342c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f342c0051d0 0x7f342c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f34464c7640 1 -- 172.21.15.3:0/2482068066 reap_dead start 2024-09-12T12:24:08.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3445cc6640 1 -- 172.21.15.3:0/2482068066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c003530 con 0x7f342c0039f0 2024-09-12T12:24:08.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f34454c5640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c004810 0x7f342c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0039f0 0x7f342c003e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c240 tx=0x7f343c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6d55b5b477aaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c007d20 con 0x7f342c0039f0 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c0a9070 con 0x7f342c0039f0 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3444cc4640 1 -- 172.21.15.3:0/2482068066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c0a5e10 con 0x7f342c0039f0 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3447f2a640 1 -- 172.21.15.3:0/2482068066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0039f0 msgr2=0x7f342c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0039f0 0x7f342c003e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c240 tx=0x7f343c01cc50 comp rx=0 tx=0).stop 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3447f2a640 1 -- 172.21.15.3:0/2482068066 shutdown_connections 2024-09-12T12:24:08.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2482068066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f342c0039f0 0x7f342c003e10 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3447f2a640 1 -- 172.21.15.3:0/2482068066 >> 172.21.15.3:0/2482068066 conn(0x7f3440067e00 msgr2=0x7f3440068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3447f2a640 1 -- 172.21.15.3:0/2482068066 shutdown_connections 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.636+0000 7f3447f2a640 1 -- 172.21.15.3:0/2482068066 wait complete. 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 Processor -- start 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 -- start start 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0039f0 0x7f34401465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34401e6990 0x7f34401e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34401e6e10 0x7f34401e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f344015c9b0 con 0x7f34401e6990 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f344015c830 con 0x7f34401e6e10 2024-09-12T12:24:08.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f344015ba30 con 0x7f342c0039f0 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34401e6990 0x7f34401e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34401e6990 0x7f34401e89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48014/0 (socket says 172.21.15.3:48014) 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 -- 172.21.15.3:0/2996131129 learned_addr learned my addr 172.21.15.3:0/2996131129 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0039f0 0x7f34401465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34464c7640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34401e6e10 0x7f34401e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 -- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0039f0 msgr2=0x7f34401465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0039f0 0x7f34401465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 -- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34401e6e10 msgr2=0x7f34401e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34401e6e10 0x7f34401e8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 -- 172.21.15.3:0/2996131129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34401e7370 con 0x7f34401e6990 2024-09-12T12:24:08.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0039f0 0x7f34401465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34454c5640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34401e6990 0x7f34401e89e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3430010820 tx=0x7f3430010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34367fc640 1 -- 172.21.15.3:0/2996131129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343000f040 con 0x7f34401e6990 2024-09-12T12:24:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34367fc640 1 -- 172.21.15.3:0/2996131129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3430004220 con 0x7f34401e6990 2024-09-12T12:24:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34401e7650 con 0x7f34401e6990 2024-09-12T12:24:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34367fc640 1 -- 172.21.15.3:0/2996131129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343000e6f0 con 0x7f34401e6990 2024-09-12T12:24:08.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34401e7c10 con 0x7f34401e6990 2024-09-12T12:24:08.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.640+0000 7f34367fc640 1 -- 172.21.15.3:0/2996131129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f343000e890 con 0x7f34401e6990 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34367fc640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34240678c0 0x7f3424069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34367fc640 1 -- 172.21.15.3:0/2996131129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f3430085d20 con 0x7f34401e6990 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34240678c0 0x7f3424069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 Processor -- start 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- start start 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34240678c0 0x7f3424069d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f343c0ab000 tx=0x7f343c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c9cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c985d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c986290 0x55d82c986670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d82c86db60 con 0x55d82c986290 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d82c7d4c70 con 0x55d82c9cfb20 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d82c77aed0 con 0x55d82c9d0440 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c986290 0x55d82c986670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c986290 0x55d82c986670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48016/0 (socket says 172.21.15.3:48016) 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 -- 172.21.15.3:0/1501198155 learned_addr learned my addr 172.21.15.3:0/1501198155 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34454c5640 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c985d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f3445cc6640 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c9cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 -- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 msgr2=0x55d82c985d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c985d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 -- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 msgr2=0x55d82c9cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c9cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 -- 172.21.15.3:0/1501198155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82c78ce50 con 0x55d82c986290 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f3445cc6640 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c9cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34464c7640 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c986290 0x55d82c986670 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f344015ba00 tx=0x7f3438001f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76644ff70dec4415 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34237fe640 1 -- 172.21.15.3:0/1501198155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438003d90 con 0x55d82c986290 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34237fe640 1 -- 172.21.15.3:0/1501198155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34380050d0 con 0x55d82c986290 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f34237fe640 1 -- 172.21.15.3:0/1501198155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34380054c0 con 0x55d82c986290 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- 172.21.15.3:0/1501198155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c986290 msgr2=0x55d82c986670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c986290 0x55d82c986670 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f344015ba00 tx=0x7f3438001f00 comp rx=0 tx=0).stop 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- 172.21.15.3:0/1501198155 shutdown_connections 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c986290 0x55d82c986670 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c985d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 --2- 172.21.15.3:0/1501198155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c9cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- 172.21.15.3:0/1501198155 >> 172.21.15.3:0/1501198155 conn(0x55d82c98c600 msgr2=0x55d82c984630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- 172.21.15.3:0/1501198155 shutdown_connections 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.644+0000 7f344afe6000 1 -- 172.21.15.3:0/1501198155 wait complete. 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- start start 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 Processor -- start 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c98ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c990cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c991230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d82c836b00 con 0x55d82c98b390 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d82c7af360 con 0x55d82c9cfb20 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d82c7ac2d0 con 0x55d82c9d0440 2024-09-12T12:24:08.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 10 client.0 _ll_register_callbacks cb 0x55d82c971820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c991230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c991230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48028/0 (socket says 172.21.15.3:48028) 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 -- 172.21.15.3:0/67240311 learned_addr learned my addr 172.21.15.3:0/67240311 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34454c5640 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c990cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3445cc6640 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c98ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 -- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 msgr2=0x55d82c990cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c990cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 -- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 msgr2=0x55d82c98ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c98ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 -- 172.21.15.3:0/67240311 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82c77b120 con 0x55d82c98b390 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f34464c7640 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c991230 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3438009010 tx=0x7f3438001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3e3a7335e984c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 1 -- 172.21.15.3:0/67240311 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438022020 con 0x55d82c98b390 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 1 -- 172.21.15.3:0/67240311 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f3438006d60 con 0x55d82c98b390 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34380050d0 con 0x55d82c98b390 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 1 -- 172.21.15.3:0/67240311 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f343802c050 con 0x55d82c98b390 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 10 client.6534 fetch_fsmap learned FSMap version 54 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55d82c774450 con 0x55d82c98b390 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 1 -- 172.21.15.3:0/67240311 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34380050d0 con 0x55d82c98b390 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 1 -- 172.21.15.3:0/67240311 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f343800e770 con 0x55d82c98b390 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 10 client.6534 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 10 client.6534 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 20 client.6534 populate_metadata read hostname 'smithi003' 2024-09-12T12:24:08.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9de280 0x55d82c9e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 4 client.6534 mds_command: new command op to 6375 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 --> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82c76ec00 con 0x55d82c9de280 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3445cc6640 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9de280 0x55d82c9e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3445cc6640 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9de280 0x55d82c9e0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8c495d5e1ade2c server_cookie=f4b97fb729fd5d0f in_seq=0 out_seq=0 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 10 client.6534 ms_handle_connect on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 1 -- 172.21.15.3:0/67240311 <== mds.0 v2:172.21.15.3:6834/2940169617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d82c76ec00 con 0x55d82c9de280 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 10 client.6534 handle_command_reply: tid=0 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 client.6534 shutdown 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 msgr2=0x55d82c991230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c991230 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3438009010 tx=0x7f3438001520 comp rx=0 tx=0).stop 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 shutdown_connections 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9de280 0x55d82c9e0670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c991230 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9d0440 0x55d82c990cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f3420ff9640 0 client.6534 ms_handle_reset on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 --2- 172.21.15.3:0/67240311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9cfb20 0x55d82c98ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 >> 172.21.15.3:0/67240311 conn(0x55d82c986290 msgr2=0x55d82c984d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 shutdown_connections 2024-09-12T12:24:08.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.648+0000 7f344afe6000 1 -- 172.21.15.3:0/67240311 wait complete. 2024-09-12T12:24:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.652+0000 7f344afe6000 20 client.6534 trim_cache size 0 max 16384 2024-09-12T12:24:08.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 Processor -- start 2024-09-12T12:24:08.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 -- start start 2024-09-12T12:24:08.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c9cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9d0440 0x55d82c9e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3440203640 con 0x55d82c98b390 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f34400ce310 con 0x55d82c9d0440 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3440203120 con 0x55d82c9cfb20 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c9cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c9cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48042/0 (socket says 172.21.15.3:48042) 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 -- 172.21.15.3:0/2891454873 learned_addr learned my addr 172.21.15.3:0/2891454873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 -- 172.21.15.3:0/2891454873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 msgr2=0x55d82c9cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2891454873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 -- 172.21.15.3:0/2891454873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9d0440 msgr2=0x55d82c9e0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2891454873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9d0440 0x55d82c9e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 -- 172.21.15.3:0/2891454873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82c75bfd0 con 0x55d82c98b390 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.668+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2891454873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c9cec90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f343c000c10 tx=0x7f343c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6457f36382f9f46f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3422ffd640 1 -- 172.21.15.3:0/2891454873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c0a5de0 con 0x55d82c98b390 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3422ffd640 1 -- 172.21.15.3:0/2891454873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c004cc0 con 0x55d82c98b390 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3422ffd640 1 -- 172.21.15.3:0/2891454873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c0072a0 con 0x55d82c98b390 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2891454873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 msgr2=0x55d82c9cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- 172.21.15.3:0/2891454873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c9cec90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f343c000c10 tx=0x7f343c0a7000 comp rx=0 tx=0).stop 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2891454873 shutdown_connections 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- 172.21.15.3:0/2891454873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c9d0440 0x55d82c9e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- 172.21.15.3:0/2891454873 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- 172.21.15.3:0/2891454873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c98b390 0x55d82c9cec90 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2891454873 >> 172.21.15.3:0/2891454873 conn(0x55d82c986290 msgr2=0x55d82c98a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2891454873 shutdown_connections 2024-09-12T12:24:08.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2891454873 wait complete. 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- start start 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 Processor -- start 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c98b390 0x55d82c9e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9fe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c9d0440 0x55d82c9feea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d82c7d3f80 con 0x55d82c9d0440 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d82c78cca0 con 0x55d82c98b390 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55d82c7d92a0 con 0x55d82c9cfb20 2024-09-12T12:24:08.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 10 client.0 _ll_register_callbacks cb 0x7f3438001940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c9d0440 0x55d82c9feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c9d0440 0x55d82c9feea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48048/0 (socket says 172.21.15.3:48048) 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 -- 172.21.15.3:0/2892852410 learned_addr learned my addr 172.21.15.3:0/2892852410 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34454c5640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c98b390 0x55d82c9e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 -- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 msgr2=0x55d82c9fe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9fe960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 -- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c98b390 msgr2=0x55d82c9e3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c98b390 0x55d82c9e3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 -- 172.21.15.3:0/2892852410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82c857750 con 0x55d82c9d0440 2024-09-12T12:24:08.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34454c5640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c98b390 0x55d82c9e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f34464c7640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c9d0440 0x55d82c9feea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3438007dc0 tx=0x7f34380014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df65aa089969373e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3420ff9640 1 -- 172.21.15.3:0/2892852410 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438022030 con 0x55d82c9d0440 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3420ff9640 1 -- 172.21.15.3:0/2892852410 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f3438006d60 con 0x55d82c9d0440 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343c004cc0 con 0x55d82c9d0440 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3420ff9640 1 -- 172.21.15.3:0/2892852410 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3438040050 con 0x55d82c9d0440 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 10 client.6540 fetch_fsmap learned FSMap version 54 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55d82c762a30 con 0x55d82c9d0440 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3420ff9640 1 -- 172.21.15.3:0/2892852410 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c004cc0 con 0x55d82c9d0440 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3420ff9640 1 -- 172.21.15.3:0/2892852410 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f3438043020 con 0x55d82c9d0440 2024-09-12T12:24:08.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 10 client.6540 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 10 client.6540 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 20 client.6540 populate_metadata read hostname 'smithi003' 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9ff3e0 0x55d82ca429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 4 client.6540 mds_command: new command op to 6375 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 --> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] -- command(tid 0: {"prefix": "status"}) -- 0x55d82c7e23a0 con 0x55d82c9ff3e0 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.672+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9ff3e0 0x55d82ca429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3445cc6640 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9ff3e0 0x55d82ca429f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd1cd402eadc8d9 server_cookie=4bb9b8416cfa1bb5 in_seq=0 out_seq=0 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3420ff9640 10 client.6540 ms_handle_connect on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3420ff9640 1 -- 172.21.15.3:0/2892852410 <== mds.0 v2:172.21.15.3:6834/2940169617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d82c7e23a0 con 0x55d82c9ff3e0 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3420ff9640 10 client.6540 handle_command_reply: tid=0 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 client.6540 shutdown 2024-09-12T12:24:08.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c9d0440 msgr2=0x55d82c9feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c9d0440 0x55d82c9feea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3438007dc0 tx=0x7f34380014b0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 shutdown_connections 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x55d82c9ff3e0 0x55d82ca429f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d82c9d0440 0x55d82c9feea0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3420ff9640 0 client.6540 ms_handle_reset on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55d82c9cfb20 0x55d82c9fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 --2- 172.21.15.3:0/2892852410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d82c98b390 0x55d82c9e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 >> 172.21.15.3:0/2892852410 conn(0x55d82c986290 msgr2=0x55d82c9cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 shutdown_connections 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 1 -- 172.21.15.3:0/2892852410 wait complete. 2024-09-12T12:24:08.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f344afe6000 20 client.6540 trim_cache size 0 max 16384 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6375, 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:24:08.685 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.292206178000001, 2024-09-12T12:24:08.686 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 54, 2024-09-12T12:24:08.686 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 79, 2024-09-12T12:24:08.686 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 79, 2024-09-12T12:24:08.686 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.323225252 2024-09-12T12:24:08.686 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34240678c0 msgr2=0x7f3424069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34240678c0 0x7f3424069d80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f343c0ab000 tx=0x7f343c0a7070 comp rx=0 tx=0).stop 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34401e6990 msgr2=0x7f34401e89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34401e6990 0x7f34401e89e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f3430010820 tx=0x7f3430010ce0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 shutdown_connections 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f34240678c0 0x7f3424069d80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34401e6e10 0x7f34401e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34401e6990 0x7f34401e89e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 --2- 172.21.15.3:0/2996131129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f342c0039f0 0x7f34401465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 >> 172.21.15.3:0/2996131129 conn(0x7f3440067e00 msgr2=0x7f344006e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 shutdown_connections 2024-09-12T12:24:08.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.676+0000 7f3447f2a640 1 -- 172.21.15.3:0/2996131129 wait complete. 2024-09-12T12:24:08.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:24:08.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd6455fc640 1 Processor -- start 2024-09-12T12:24:08.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd6455fc640 1 -- start start 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd6455fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd6455fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd64015b620 0x7fd64015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd6455fc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6400ad0c0 con 0x7fd6400d05b0 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd6455fc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd6400ad3a0 con 0x7fd64015b620 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd6455fc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6400ad680 con 0x7fd6400ae250 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd64015b620 0x7fd64015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd64015b620 0x7fd64015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48058/0 (socket says 172.21.15.3:48058) 2024-09-12T12:24:08.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63f7fe640 1 -- 172.21.15.3:0/2998856082 learned_addr learned my addr 172.21.15.3:0/2998856082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:08.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63e7fc640 1 --2- 172.21.15.3:0/2998856082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63f7fe640 1 -- 172.21.15.3:0/2998856082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 msgr2=0x7fd6400ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:08.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63f7fe640 1 --2- 172.21.15.3:0/2998856082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63f7fe640 1 -- 172.21.15.3:0/2998856082 >> v1:172.21.15.142:6789/0 conn(0x7fd6400d05b0 legacy=0x7fd6400d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.808+0000 7fd63f7fe640 1 -- 172.21.15.3:0/2998856082 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6400ad960 con 0x7fd64015b620 2024-09-12T12:24:08.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd63f7fe640 1 --2- 172.21.15.3:0/2998856082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd64015b620 0x7fd64015bab0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd638009e50 tx=0x7fd63801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c161471cad951f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd63dffb640 1 -- 172.21.15.3:0/2998856082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380a58f0 con 0x7fd64015b620 2024-09-12T12:24:08.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd63dffb640 1 -- 172.21.15.3:0/2998856082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6380a5a90 con 0x7fd64015b620 2024-09-12T12:24:08.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd63dffb640 1 -- 172.21.15.3:0/2998856082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380a5e60 con 0x7fd64015b620 2024-09-12T12:24:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- 172.21.15.3:0/2998856082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd64015b620 msgr2=0x7fd64015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 --2- 172.21.15.3:0/2998856082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd64015b620 0x7fd64015bab0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd638009e50 tx=0x7fd63801ca00 comp rx=0 tx=0).stop 2024-09-12T12:24:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- 172.21.15.3:0/2998856082 shutdown_connections 2024-09-12T12:24:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 --2- 172.21.15.3:0/2998856082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd64015b620 0x7fd64015bab0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 --2- 172.21.15.3:0/2998856082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- 172.21.15.3:0/2998856082 >> 172.21.15.3:0/2998856082 conn(0x7fd640067e00 msgr2=0x7fd640068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:08.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- 172.21.15.3:0/2998856082 shutdown_connections 2024-09-12T12:24:08.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- 172.21.15.3:0/2998856082 wait complete. 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 Processor -- start 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- start start 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6400d05b0 0x7fd64014ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64015b620 0x7fd6400cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd64015c9b0 con 0x7fd6400d05b0 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd64015c830 con 0x7fd64015b620 2024-09-12T12:24:08.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd6455fc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd6400b31e0 con 0x7fd6400ae250 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd63e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6400d05b0 0x7fd64014ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd63e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6400d05b0 0x7fd64014ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48074/0 (socket says 172.21.15.3:48074) 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.812+0000 7fd63e7fc640 1 -- 172.21.15.3:0/4271089686 learned_addr learned my addr 172.21.15.3:0/4271089686 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63f7fe640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64015b620 0x7fd6400cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63effd640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63e7fc640 1 -- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 msgr2=0x7fd6400b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63e7fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63e7fc640 1 -- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64015b620 msgr2=0x7fd6400cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63e7fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64015b620 0x7fd6400cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63e7fc640 1 -- 172.21.15.3:0/4271089686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6400b3d40 con 0x7fd6400d05b0 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63e7fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6400d05b0 0x7fd64014ace0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd63400d800 tx=0x7fd63400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6277fe640 1 -- 172.21.15.3:0/4271089686 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63400b6c0 con 0x7fd6400d05b0 2024-09-12T12:24:08.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6277fe640 1 -- 172.21.15.3:0/4271089686 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd634019070 con 0x7fd6400d05b0 2024-09-12T12:24:08.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6400caf40 con 0x7fd6400d05b0 2024-09-12T12:24:08.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6277fe640 1 -- 172.21.15.3:0/4271089686 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63400ba60 con 0x7fd6400d05b0 2024-09-12T12:24:08.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6400b4000 con 0x7fd6400d05b0 2024-09-12T12:24:08.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6400ae6b0 con 0x7fd6400d05b0 2024-09-12T12:24:08.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6277fe640 1 -- 172.21.15.3:0/4271089686 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd63400bc00 con 0x7fd6400d05b0 2024-09-12T12:24:08.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6277fe640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd62c0678f0 0x7fd62c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:08.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd6277fe640 1 -- 172.21.15.3:0/4271089686 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fd634086050 con 0x7fd6400d05b0 2024-09-12T12:24:08.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63effd640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd62c0678f0 0x7fd62c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:08.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.816+0000 7fd63effd640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd62c0678f0 0x7fd62c069db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:08.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:08.820+0000 7fd6277fe640 1 -- 172.21.15.3:0/4271089686 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd634056c20 con 0x7fd6400d05b0 2024-09-12T12:24:09.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.028+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd64006dce0 con 0x7fd6400d05b0 2024-09-12T12:24:09.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.028+0000 7fd6277fe640 1 -- 172.21.15.3:0/4271089686 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7fd634056520 con 0x7fd6400d05b0 2024-09-12T12:24:09.033 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:24:09.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd62c0678f0 msgr2=0x7fd62c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:09.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd62c0678f0 0x7fd62c069db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).stop 2024-09-12T12:24:09.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6400d05b0 msgr2=0x7fd64014ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:09.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6400d05b0 0x7fd64014ace0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd63400d800 tx=0x7fd63400dcc0 comp rx=0 tx=0).stop 2024-09-12T12:24:09.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 shutdown_connections 2024-09-12T12:24:09.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd62c0678f0 0x7fd62c069db0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:09.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd64015b620 0x7fd6400cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:09.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6400d05b0 0x7fd64014ace0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:09.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 --2- 172.21.15.3:0/4271089686 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd6400ae250 0x7fd6400b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:09.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 >> 172.21.15.3:0/4271089686 conn(0x7fd640067e00 msgr2=0x7fd6400b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:09.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 shutdown_connections 2024-09-12T12:24:09.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09.032+0000 7fd6455fc640 1 -- 172.21.15.3:0/4271089686 wait complete. 2024-09-12T12:24:09.052 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:24:09.053 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:24:09.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:09.053 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:24:09.058 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:09.059 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42237sec preferred_lft 42237sec 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:24:09.060 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:09.061 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:24:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.062 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:24:09.062 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:09.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:09.062 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:24:09.112 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:24:09.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:09.114 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:24:09.163 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:24:09.163 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:24:09.163 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:24:09.163 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:24:09.163 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:24:09.163 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:24:09.164 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:24:09.164 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:24:09.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:24:09.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:24:09.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:24:09.164 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:24:09.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:24:09.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:09.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:09.287 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:24:09.293 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:24:09.293 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:24:09.294 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:24:09.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:09.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:24:09.350 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:24:09.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:09.351 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:09.402 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:24:09.403 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:09.403 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:09.449 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:24:09.507 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:24:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:24:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:24:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:24:09.556 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:24:09.557 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:09.558 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:09.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:09.559 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:24:09.606 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:24:09.606 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:24:09.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:24:09.667 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:09.714 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:24:09.714 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:24:09.714 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:24:09.714 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:24:09.714 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:24:09.714 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:09.715 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:09.716 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:09.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:09.717 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:24:09.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:24:09.736+0000 7fae39738580 -1 init, newargv = 0x55656f579bf0 newargc=15 2024-09-12T12:24:09.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[43057]: starting ceph client 2024-09-12T12:24:09.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[43057]: starting fuse 2024-09-12T12:24:10.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:24:10.859 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:10.907 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:24:10.907 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:24:10.907 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:24:10.907 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:24:10.907 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:24:10.907 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:24:10.907 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:10.908 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:10.909 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:10.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:10.910 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:24:10.957 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:24:10.958 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:24:10.958 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:24:10.959 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:24:10.960 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:24:10.960 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:10.960 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:24:10.960 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:24:11.085 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.43057.asok 2024-09-12T12:24:11.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:11.097 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43057.asok 2024-09-12T12:24:11.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43057.asok status 2024-09-12T12:24:11.253 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:24:11.253 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:24:11.253 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:24:11.253 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:24:11.253 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "43057", 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6552, 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6552 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:11.254 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3115748735 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3115748735 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.6552 192.168.0.1:0/3115748735", 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3115748735", 2024-09-12T12:24:11.255 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:24:11.256 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 54, 2024-09-12T12:24:11.256 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 79, 2024-09-12T12:24:11.256 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:24:11.256 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:24:11.256 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:24:11.256 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:24:11.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.267 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:24:11.268 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:11.274 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:24:11.274 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:11.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.275 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:11.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:24:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc79488a640 1 Processor -- start 2024-09-12T12:24:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc79488a640 1 -- start start 2024-09-12T12:24:11.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc79488a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc79015e830 0x7fc7900b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc79488a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7900b6bc0 0x7fc79006c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc79488a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7900b7140 con 0x7fc79006cb70 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc79488a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc79006cfe0 con 0x7fc7900b6bc0 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc79488a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc7900cb620 con 0x7fc79015e830 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78dd9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7900b6bc0 0x7fc79006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78dd9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc7900b6bc0 0x7fc79006c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48080/0 (socket says 172.21.15.3:48080) 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78dd9b640 1 -- 172.21.15.3:0/4025634094 learned_addr learned my addr 172.21.15.3:0/4025634094 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:11.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78e59c640 1 --2- 172.21.15.3:0/4025634094 >> v2:172.21.15.142:3300/0 conn(0x7fc79015e830 0x7fc7900b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:11.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78dd9b640 1 -- 172.21.15.3:0/4025634094 >> v2:172.21.15.142:3300/0 conn(0x7fc79015e830 msgr2=0x7fc7900b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:11.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78dd9b640 1 --2- 172.21.15.3:0/4025634094 >> v2:172.21.15.142:3300/0 conn(0x7fc79015e830 0x7fc7900b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78dd9b640 1 -- 172.21.15.3:0/4025634094 >> v1:172.21.15.142:6789/0 conn(0x7fc79006cb70 legacy=0x7fc7900d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:11.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.504+0000 7fc78dd9b640 1 -- 172.21.15.3:0/4025634094 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7900b7420 con 0x7fc7900b6bc0 2024-09-12T12:24:11.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc78dd9b640 1 --2- 172.21.15.3:0/4025634094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7900b6bc0 0x7fc79006c630 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc790072c20 tx=0x7fc78401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acda0dce8c1b67af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:11.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc78d59a640 1 -- 172.21.15.3:0/4025634094 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840b0070 con 0x7fc7900b6bc0 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc78d59a640 1 -- 172.21.15.3:0/4025634094 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7840ab600 con 0x7fc7900b6bc0 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc78d59a640 1 -- 172.21.15.3:0/4025634094 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7840ab9d0 con 0x7fc7900b6bc0 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- 172.21.15.3:0/4025634094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7900b6bc0 msgr2=0x7fc79006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 --2- 172.21.15.3:0/4025634094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7900b6bc0 0x7fc79006c630 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc790072c20 tx=0x7fc78401c4e0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- 172.21.15.3:0/4025634094 shutdown_connections 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 --2- 172.21.15.3:0/4025634094 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc7900b6bc0 0x7fc79006c630 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 --2- 172.21.15.3:0/4025634094 >> v2:172.21.15.142:3300/0 conn(0x7fc79015e830 0x7fc7900b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- 172.21.15.3:0/4025634094 >> 172.21.15.3:0/4025634094 conn(0x7fc7900b77c0 msgr2=0x7fc7900b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:11.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- 172.21.15.3:0/4025634094 shutdown_connections 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- 172.21.15.3:0/4025634094 wait complete. 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 Processor -- start 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- start start 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc79006cb70 0x7fc7900afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7900b6bc0 0x7fc79015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79015e830 0x7fc7900b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc79016b430 con 0x7fc79006cb70 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc79016b2b0 con 0x7fc7900b6bc0 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.508+0000 7fc79488a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc79016b5b0 con 0x7fc79015e830 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc79006cb70 0x7fc7900afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc79006cb70 0x7fc7900afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48092/0 (socket says 172.21.15.3:48092) 2024-09-12T12:24:11.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 -- 172.21.15.3:0/745055339 learned_addr learned my addr 172.21.15.3:0/745055339 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78ed9d640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79015e830 0x7fc7900b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78dd9b640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7900b6bc0 0x7fc79015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 -- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79015e830 msgr2=0x7fc7900b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79015e830 0x7fc7900b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 -- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7900b6bc0 msgr2=0x7fc79015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7900b6bc0 0x7fc79015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 -- 172.21.15.3:0/745055339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7900b0280 con 0x7fc79006cb70 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78e59c640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc79006cb70 0x7fc7900afd40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc78000d620 tx=0x7fc78000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc77effd640 1 -- 172.21.15.3:0/745055339 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78000b620 con 0x7fc79006cb70 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc77effd640 1 -- 172.21.15.3:0/745055339 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc780019070 con 0x7fc79006cb70 2024-09-12T12:24:11.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79015c6f0 con 0x7fc79006cb70 2024-09-12T12:24:11.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc77effd640 1 -- 172.21.15.3:0/745055339 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78000b9e0 con 0x7fc79006cb70 2024-09-12T12:24:11.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7900af280 con 0x7fc79006cb70 2024-09-12T12:24:11.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7900af990 con 0x7fc79006cb70 2024-09-12T12:24:11.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc77effd640 1 -- 172.21.15.3:0/745055339 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc78000bb80 con 0x7fc79006cb70 2024-09-12T12:24:11.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc77effd640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7700678c0 0x7fc770069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:11.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc77effd640 1 -- 172.21.15.3:0/745055339 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fc780086040 con 0x7fc79006cb70 2024-09-12T12:24:11.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.512+0000 7fc78dd9b640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7700678c0 0x7fc770069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:11.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.516+0000 7fc78dd9b640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7700678c0 0x7fc770069d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc7840ad000 tx=0x7fc784008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:11.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.520+0000 7fc77effd640 1 -- 172.21.15.3:0/745055339 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc780056c10 con 0x7fc79006cb70 2024-09-12T12:24:11.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc790158140 con 0x7fc79006cb70 2024-09-12T12:24:11.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc77effd640 1 -- 172.21.15.3:0/745055339 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7fc780056510 con 0x7fc79006cb70 2024-09-12T12:24:11.726 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:24:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7700678c0 msgr2=0x7fc770069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7700678c0 0x7fc770069d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc7840ad000 tx=0x7fc784008040 comp rx=0 tx=0).stop 2024-09-12T12:24:11.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc79006cb70 msgr2=0x7fc7900afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc79006cb70 0x7fc7900afd40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc78000d620 tx=0x7fc78000dae0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 shutdown_connections 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc7700678c0 0x7fc770069d80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc79015e830 0x7fc7900b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7900b6bc0 0x7fc79015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 --2- 172.21.15.3:0/745055339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc79006cb70 0x7fc7900afd40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 >> 172.21.15.3:0/745055339 conn(0x7fc7900b77c0 msgr2=0x7fc790069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 shutdown_connections 2024-09-12T12:24:11.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:11.724+0000 7fc79488a640 1 -- 172.21.15.3:0/745055339 wait complete. 2024-09-12T12:24:11.745 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:24:11.745 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:24:11.745 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:24:11.745 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:24:11.746 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:11.746 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:24:11.746 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:24:11.746 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:24:11.746 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:24:11.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.746 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:24:11.753 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:24:11.754 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42233sec preferred_lft 42233sec 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:24:11.755 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:24:11.756 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:11.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.757 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:24:11.806 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:24:11.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.808 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:24:11.861 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:24:11.862 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:24:11.863 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:24:11.863 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:24:11.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:24:11.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:24:11.863 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:24:11.863 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:24:11.948 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:24:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:24:11.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:24:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:11.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.986 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:24:11.991 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:24:11.991 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:24:11.991 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:24:11.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:11.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:24:12.051 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:24:12.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:12.052 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:12.099 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:24:12.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:12.100 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:12.150 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:24:12.204 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:12.252 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:24:12.252 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:24:12.252 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:24:12.252 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:24:12.253 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:24:12.254 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:12.255 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:12.255 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:12.255 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:12.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:12.256 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:24:12.303 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:24:12.303 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:24:12.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:24:12.364 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:12.412 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:24:12.413 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:24:12.414 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:24:12.415 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:24:12.415 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:12.415 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:12.415 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:12.415 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:12.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:12.415 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:24:12.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:24:12.427+0000 7f32fad78580 -1 init, newargv = 0x55da6afcc0d0 newargc=15 2024-09-12T12:24:12.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36417]: starting ceph client 2024-09-12T12:24:12.437 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36417]: starting fuse 2024-09-12T12:24:13.536 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:24:13.552 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:24:13.602 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:24:13.603 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:13.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:13.605 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:24:13.651 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:24:13.652 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:24:13.652 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:13.653 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:24:13.654 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:24:13.776 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.36417.asok 2024-09-12T12:24:13.780 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:24:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:13.788 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36417.asok 2024-09-12T12:24:13.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:13.788 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36417.asok status 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "36417", 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:24:13.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 5875, 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 5875 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2480759643 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:24:13.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 2480759643 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.5875 192.168.0.1:0/2480759643", 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/2480759643", 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 54, 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 79, 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:24:13.946 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:24:13.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:13.959 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:24:13.959 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:13.965 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:24:13.965 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:13.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:13.965 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:14.019 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:14.019 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:14.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:24:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16c06b0640 1 Processor -- start 2024-09-12T12:24:14.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16c06b0640 1 -- start start 2024-09-12T12:24:14.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16c06b0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16b815c710 0x7f16b80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16c06b0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16b80ae880 0x7f16b80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16c06b0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16b80b5000 con 0x7f16b8073cb0 2024-09-12T12:24:14.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16c06b0640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f16b8074090 con 0x7f16b815c710 2024-09-12T12:24:14.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16c06b0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f16b80b37c0 con 0x7f16b80ae880 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bdc4b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16b815c710 0x7f16b80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bec4d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f16b80ae880 0x7f16b80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bdc4b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f16b815c710 0x7f16b80b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48096/0 (socket says 172.21.15.3:48096) 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bdc4b640 1 -- 172.21.15.3:0/713614761 learned_addr learned my addr 172.21.15.3:0/713614761 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bdc4b640 1 -- 172.21.15.3:0/713614761 >> v2:172.21.15.142:3300/0 conn(0x7f16b80ae880 msgr2=0x7f16b80aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bdc4b640 1 --2- 172.21.15.3:0/713614761 >> v2:172.21.15.142:3300/0 conn(0x7f16b80ae880 0x7f16b80aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bdc4b640 1 -- 172.21.15.3:0/713614761 >> v1:172.21.15.142:6789/0 conn(0x7f16b8073cb0 legacy=0x7f16b80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.200+0000 7f16bdc4b640 1 -- 172.21.15.3:0/713614761 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b80b3aa0 con 0x7f16b815c710 2024-09-12T12:24:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16bdc4b640 1 --2- 172.21.15.3:0/713614761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80b30e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f16b80b46c0 tx=0x7f16a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984f75723013f260 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16bd44a640 1 -- 172.21.15.3:0/713614761 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80ab610 con 0x7f16b815c710 2024-09-12T12:24:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16bd44a640 1 -- 172.21.15.3:0/713614761 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a80b0070 con 0x7f16b815c710 2024-09-12T12:24:14.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16bd44a640 1 -- 172.21.15.3:0/713614761 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80ab9d0 con 0x7f16b815c710 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 -- 172.21.15.3:0/713614761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 msgr2=0x7f16b80b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 --2- 172.21.15.3:0/713614761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80b30e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f16b80b46c0 tx=0x7f16a801c520 comp rx=0 tx=0).stop 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 -- 172.21.15.3:0/713614761 shutdown_connections 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 --2- 172.21.15.3:0/713614761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80b30e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 --2- 172.21.15.3:0/713614761 >> v2:172.21.15.142:3300/0 conn(0x7f16b80ae880 0x7f16b80aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 -- 172.21.15.3:0/713614761 >> 172.21.15.3:0/713614761 conn(0x7f16b8067e00 msgr2=0x7f16b8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 -- 172.21.15.3:0/713614761 shutdown_connections 2024-09-12T12:24:14.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.204+0000 7f16c06b0640 1 -- 172.21.15.3:0/713614761 wait complete. 2024-09-12T12:24:14.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 Processor -- start 2024-09-12T12:24:14.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 -- start start 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b8073cb0 0x7f16b8157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b80ae880 0x7f16b80ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8168f40 con 0x7f16b815c710 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8168dc0 con 0x7f16b80ae880 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f16b81690c0 con 0x7f16b8073cb0 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48104/0 (socket says 172.21.15.3:48104) 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 -- 172.21.15.3:0/3712958272 learned_addr learned my addr 172.21.15.3:0/3712958272 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:14.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bdc4b640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b80ae880 0x7f16b80ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16be44c640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b8073cb0 0x7f16b8157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 -- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b8073cb0 msgr2=0x7f16b8157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b8073cb0 0x7f16b8157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 -- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b80ae880 msgr2=0x7f16b80ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b80ae880 0x7f16b80ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 -- 172.21.15.3:0/3712958272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b8158190 con 0x7f16b815c710 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16bec4d640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80cd0c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f16b000e960 tx=0x7f16b000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16aeffd640 1 -- 172.21.15.3:0/3712958272 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0017070 con 0x7f16b815c710 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16aeffd640 1 -- 172.21.15.3:0/3712958272 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b000c9a0 con 0x7f16b815c710 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b8158470 con 0x7f16b815c710 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16aeffd640 1 -- 172.21.15.3:0/3712958272 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b000cd70 con 0x7f16b815c710 2024-09-12T12:24:14.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b8158a30 con 0x7f16b815c710 2024-09-12T12:24:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.208+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b80aecb0 con 0x7f16b815c710 2024-09-12T12:24:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.212+0000 7f16aeffd640 1 -- 172.21.15.3:0/3712958272 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f16b001a070 con 0x7f16b815c710 2024-09-12T12:24:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.212+0000 7f16aeffd640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f169c0678c0 0x7f169c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.212+0000 7f16aeffd640 1 -- 172.21.15.3:0/3712958272 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f16b0086140 con 0x7f16b815c710 2024-09-12T12:24:14.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.212+0000 7f16be44c640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f169c0678c0 0x7f169c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.212+0000 7f16be44c640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f169c0678c0 0x7f169c069d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f16b4001d50 tx=0x7f16b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.216+0000 7f16aeffd640 1 -- 172.21.15.3:0/3712958272 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f16b0056b50 con 0x7f16b815c710 2024-09-12T12:24:14.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.396+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f16b8068360 con 0x7f16b815c710 2024-09-12T12:24:14.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.396+0000 7f16aeffd640 1 -- 172.21.15.3:0/3712958272 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) ==== 80+0+0 (secure 0 0 0) 0x7f16b0056450 con 0x7f16b815c710 2024-09-12T12:24:14.399 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:24:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f169c0678c0 msgr2=0x7f169c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f169c0678c0 0x7f169c069d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f16b4001d50 tx=0x7f16b4009040 comp rx=0 tx=0).stop 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 msgr2=0x7f16b80cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80cd0c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f16b000e960 tx=0x7f16b000ee20 comp rx=0 tx=0).stop 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 shutdown_connections 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f169c0678c0 0x7f169c069d80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16b815c710 0x7f16b80cd0c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16b80ae880 0x7f16b80ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 --2- 172.21.15.3:0/3712958272 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f16b8073cb0 0x7f16b8157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 >> 172.21.15.3:0/3712958272 conn(0x7f16b8067e00 msgr2=0x7f16b815b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 shutdown_connections 2024-09-12T12:24:14.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.400+0000 7f16c06b0640 1 -- 172.21.15.3:0/3712958272 wait complete. 2024-09-12T12:24:14.419 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:14.419 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:14.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:24:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 Processor -- start 2024-09-12T12:24:14.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 -- start start 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080d05b0 0x7f47080d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47080ae250 0x7f47080ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f47080b42f0 0x7f47080b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f47080d0ed0 con 0x7f47080ae250 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f47080ad0c0 con 0x7f47080b42f0 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f470e1a7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47080ad3a0 con 0x7f47080d05b0 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47080ae250 0x7f47080ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f47080ae250 0x7f47080ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48118/0 (socket says 172.21.15.3:48118) 2024-09-12T12:24:14.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 -- 172.21.15.3:0/1789791184 learned_addr learned my addr 172.21.15.3:0/1789791184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4707fff640 1 --2- 172.21.15.3:0/1789791184 >> v2:172.21.15.142:3300/0 conn(0x7f47080b42f0 0x7f47080b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f47077fe640 1 --2- 172.21.15.3:0/1789791184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080d05b0 0x7f47080d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 -- 172.21.15.3:0/1789791184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080d05b0 msgr2=0x7f47080d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 --2- 172.21.15.3:0/1789791184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080d05b0 0x7f47080d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 -- 172.21.15.3:0/1789791184 >> v2:172.21.15.142:3300/0 conn(0x7f47080b42f0 msgr2=0x7f47080b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 --2- 172.21.15.3:0/1789791184 >> v2:172.21.15.142:3300/0 conn(0x7f47080b42f0 0x7f47080b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4706ffd640 1 -- 172.21.15.3:0/1789791184 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47080ad680 con 0x7f47080ae250 2024-09-12T12:24:14.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.588+0000 7f4707fff640 1 --2- 172.21.15.3:0/1789791184 >> v2:172.21.15.142:3300/0 conn(0x7f47080b42f0 0x7f47080b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:14.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f4706ffd640 1 --2- 172.21.15.3:0/1789791184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f47080ae6b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f47080d1140 tx=0x7f46fc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5cbeaa1463cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f47067fc640 1 -- 172.21.15.3:0/1789791184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0b1070 con 0x7f47080ae250 2024-09-12T12:24:14.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f47067fc640 1 -- 172.21.15.3:0/1789791184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc0ac630 con 0x7f47080ae250 2024-09-12T12:24:14.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f47067fc640 1 -- 172.21.15.3:0/1789791184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0a8050 con 0x7f47080ae250 2024-09-12T12:24:14.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 -- 172.21.15.3:0/1789791184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 msgr2=0x7f47080ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 --2- 172.21.15.3:0/1789791184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f47080ae6b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f47080d1140 tx=0x7f46fc01c4f0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 -- 172.21.15.3:0/1789791184 shutdown_connections 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 --2- 172.21.15.3:0/1789791184 >> v2:172.21.15.142:3300/0 conn(0x7f47080b42f0 0x7f47080b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 --2- 172.21.15.3:0/1789791184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f47080ae6b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 --2- 172.21.15.3:0/1789791184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080d05b0 0x7f47080d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 -- 172.21.15.3:0/1789791184 >> 172.21.15.3:0/1789791184 conn(0x7f4708067e00 msgr2=0x7f4708068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 -- 172.21.15.3:0/1789791184 shutdown_connections 2024-09-12T12:24:14.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.592+0000 7f470e1a7640 1 -- 172.21.15.3:0/1789791184 wait complete. 2024-09-12T12:24:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 Processor -- start 2024-09-12T12:24:14.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 -- start start 2024-09-12T12:24:14.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f4708151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080b42f0 0x7f4708148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47080d05b0 0x7f47081489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47080aed70 con 0x7f47080ae250 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47080aebf0 con 0x7f47080d05b0 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f47081559b0 con 0x7f47080b42f0 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f4708151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f4708151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48132/0 (socket says 172.21.15.3:48132) 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 -- 172.21.15.3:0/3681899515 learned_addr learned my addr 172.21.15.3:0/3681899515 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f4707fff640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47080d05b0 0x7f47081489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f4706ffd640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080b42f0 0x7f4708148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 -- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080b42f0 msgr2=0x7f4708148460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080b42f0 0x7f4708148460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 -- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47080d05b0 msgr2=0x7f47081489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47080d05b0 0x7f47081489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 -- 172.21.15.3:0/3681899515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47080b1540 con 0x7f47080ae250 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f47077fe640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f4708151890 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c7f0 tx=0x7f46f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f46effff640 1 -- 172.21.15.3:0/3681899515 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8015070 con 0x7f47080ae250 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f46effff640 1 -- 172.21.15.3:0/3681899515 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f8004240 con 0x7f47080ae250 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47080b1820 con 0x7f47080ae250 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f46effff640 1 -- 172.21.15.3:0/3681899515 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8003620 con 0x7f47080ae250 2024-09-12T12:24:14.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4708159da0 con 0x7f47080ae250 2024-09-12T12:24:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.596+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470815c830 con 0x7f47080ae250 2024-09-12T12:24:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.600+0000 7f46effff640 1 -- 172.21.15.3:0/3681899515 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46f80039d0 con 0x7f47080ae250 2024-09-12T12:24:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.600+0000 7f46effff640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46f40678c0 0x7f46f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.600+0000 7f46effff640 1 -- 172.21.15.3:0/3681899515 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f46f8085d30 con 0x7f47080ae250 2024-09-12T12:24:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.600+0000 7f4706ffd640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46f40678c0 0x7f46f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.600+0000 7f4706ffd640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46f40678c0 0x7f46f4069d80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0ae000 tx=0x7f46fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.604+0000 7f46effff640 1 -- 172.21.15.3:0/3681899515 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f46f8056860 con 0x7f47080ae250 2024-09-12T12:24:14.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.776+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47080716f0 con 0x7f47080ae250 2024-09-12T12:24:14.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f46effff640 1 -- 172.21.15.3:0/3681899515 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) ==== 80+0+0 (secure 0 0 0) 0x7f46f8056160 con 0x7f47080ae250 2024-09-12T12:24:14.781 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46f40678c0 msgr2=0x7f46f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46f40678c0 0x7f46f4069d80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0ae000 tx=0x7f46fc008040 comp rx=0 tx=0).stop 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 msgr2=0x7f4708151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f4708151890 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f46f800c7f0 tx=0x7f46f800ccb0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 shutdown_connections 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f46f40678c0 0x7f46f4069d80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47080d05b0 0x7f47081489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47080b42f0 0x7f4708148460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 --2- 172.21.15.3:0/3681899515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47080ae250 0x7f4708151890 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 >> 172.21.15.3:0/3681899515 conn(0x7f4708067e00 msgr2=0x7f470806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 shutdown_connections 2024-09-12T12:24:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:14.780+0000 7f470e1a7640 1 -- 172.21.15.3:0/3681899515 wait complete. 2024-09-12T12:24:14.801 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:24:14.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:14.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:14.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[43057]: fuse finished with error 0 and tester_r 0 2024-09-12T12:24:14.812 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:24:17.195 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:24:17.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:24:20.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:24:20.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:20.914 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:24:20.922 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:24:20.922 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:20.930 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:24:21.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f10463e0640 1 Processor -- start 2024-09-12T12:24:21.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f10463e0640 1 -- start start 2024-09-12T12:24:21.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f10463e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 0x7f10400d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f10463e0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10400b5000 con 0x7f1040073cb0 2024-09-12T12:24:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f10463e0640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10400b3510 con 0x7f104015c710 2024-09-12T12:24:21.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f10463e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1040074090 con 0x7f10400b30e0 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1040073cb0 0x7f10400adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37582/0 (socket says 172.21.15.3:37582) 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103ffff640 1 -- 172.21.15.3:0/268842362 learned_addr learned my addr 172.21.15.3:0/268842362 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f104497d640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 0x7f10400d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1018605264 0 0) 0x7f10400b5000 con 0x7f1040073cb0 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1028003530 con 0x7f1040073cb0 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 703010219 0 0) 0x7f10400b3510 con 0x7f104015c710 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10400b5000 con 0x7f104015c710 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3105720210 0 0) 0x7f1028003530 con 0x7f1040073cb0 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 msgr2=0x7f10400d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:21.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 0x7f10400d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 >> v1:172.21.15.142:6789/0 conn(0x7f104015c710 legacy=0x7f10400ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10400b37c0 con 0x7f1040073cb0 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1030001600 con 0x7f1040073cb0 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 >> v1:172.21.15.3:6789/0 conn(0x7f1040073cb0 legacy=0x7f10400adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 0x7f1028003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028004130 0x7f1028004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1028004d50 0x7f10280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f102800bae0 con 0x7f1028004d50 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f102800bc90 con 0x7f1028004130 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.100+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f102800be90 con 0x7f10400b30e0 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1028004d50 0x7f10280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f103f7fe640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028004130 0x7f1028004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f103ffff640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 0x7f1028003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 -- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 msgr2=0x7f1028003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10400b30e0 0x7f1028003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 -- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028004130 msgr2=0x7f1028004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1028004130 0x7f1028004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 -- 172.21.15.3:0/268842362 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1028003530 con 0x7f1028004d50 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1028004d50 0x7f10280051d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1038009fc0 tx=0x7f103801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e051f0bec9eed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10380b2070 con 0x7f1028004d50 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10380a5b80 con 0x7f1028004d50 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f103effd640 1 -- 172.21.15.3:0/268842362 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038009c10 con 0x7f1028004d50 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f10463e0640 1 -- 172.21.15.3:0/268842362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1028004d50 msgr2=0x7f10280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f10463e0640 1 --2- 172.21.15.3:0/268842362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1028004d50 0x7f10280051d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1038009fc0 tx=0x7f103801cc80 comp rx=0 tx=0).stop 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f104497d640 1 -- 172.21.15.3:0/268842362 reap_dead start 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f10463e0640 1 -- 172.21.15.3:0/268842362 shutdown_connections 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f10463e0640 1 -- 172.21.15.3:0/268842362 >> 172.21.15.3:0/268842362 conn(0x7f1040067e00 msgr2=0x7f1040068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:21.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f10463e0640 1 -- 172.21.15.3:0/268842362 shutdown_connections 2024-09-12T12:24:21.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.104+0000 7f10463e0640 1 -- 172.21.15.3:0/268842362 wait complete. 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 Processor -- start 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 -- start start 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1040146730 0x7f10400cf640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10400cfb80 0x7f1040152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10401533b0 0x7f10401f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10400ad6f0 con 0x7f10400cfb80 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10400b3d80 con 0x7f10401533b0 2024-09-12T12:24:21.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f10400ae320 con 0x7f1040146730 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10400cfb80 0x7f1040152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10400cfb80 0x7f1040152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42880/0 (socket says 172.21.15.3:42880) 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 -- 172.21.15.3:0/4252466990 learned_addr learned my addr 172.21.15.3:0/4252466990 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103ffff640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1040146730 0x7f10400cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f104497d640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10401533b0 0x7f10401f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 -- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1040146730 msgr2=0x7f10400cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1040146730 0x7f10400cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 -- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10401533b0 msgr2=0x7f10401f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10401533b0 0x7f10401f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 -- 172.21.15.3:0/4252466990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1040146b10 con 0x7f10400cfb80 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103ffff640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1040146730 0x7f10400cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f104497d640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10401533b0 0x7f10401f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103f7fe640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10400cfb80 0x7f1040152e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f103400e7e0 tx=0x7f103400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103cff9640 1 -- 172.21.15.3:0/4252466990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400ee90 con 0x7f10400cfb80 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103cff9640 1 -- 172.21.15.3:0/4252466990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1034018070 con 0x7f10400cfb80 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1040153e40 con 0x7f10400cfb80 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103cff9640 1 -- 172.21.15.3:0/4252466990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400c6c0 con 0x7f10400cfb80 2024-09-12T12:24:21.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10401543c0 con 0x7f10400cfb80 2024-09-12T12:24:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f103cff9640 1 -- 172.21.15.3:0/4252466990 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f103400c860 con 0x7f10400cfb80 2024-09-12T12:24:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.108+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10400adc10 con 0x7f10400cfb80 2024-09-12T12:24:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.112+0000 7f103cff9640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f102c067a90 0x7f102c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.112+0000 7f103cff9640 1 -- 172.21.15.3:0/4252466990 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f1034085770 con 0x7f10400cfb80 2024-09-12T12:24:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.112+0000 7f103ffff640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f102c067a90 0x7f102c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.112+0000 7f103ffff640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f102c067a90 0x7f102c069f50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1030000fe0 tx=0x7f1030001ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:21.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.116+0000 7f103cff9640 1 -- 172.21.15.3:0/4252466990 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f10340562a0 con 0x7f10400cfb80 2024-09-12T12:24:21.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10401590c0 con 0x7f10400cfb80 2024-09-12T12:24:21.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f103cff9640 1 -- 172.21.15.3:0/4252466990 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) ==== 80+0+0 (secure 0 0 0) 0x7f1034055ba0 con 0x7f10400cfb80 2024-09-12T12:24:21.290 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f102c067a90 msgr2=0x7f102c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f102c067a90 0x7f102c069f50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1030000fe0 tx=0x7f1030001ab0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10400cfb80 msgr2=0x7f1040152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10400cfb80 0x7f1040152e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f103400e7e0 tx=0x7f103400eca0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 shutdown_connections 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f102c067a90 0x7f102c069f50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10401533b0 0x7f10401f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10400cfb80 0x7f1040152e70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 --2- 172.21.15.3:0/4252466990 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1040146730 0x7f10400cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 >> 172.21.15.3:0/4252466990 conn(0x7f1040067e00 msgr2=0x7f104015a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 shutdown_connections 2024-09-12T12:24:21.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.288+0000 7f10463e0640 1 -- 172.21.15.3:0/4252466990 wait complete. 2024-09-12T12:24:21.308 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:24:21.308 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:21.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:24:21.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6da125640 1 Processor -- start 2024-09-12T12:24:21.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6da125640 1 -- start start 2024-09-12T12:24:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6da125640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc6d40b7740 0x7fc6d40b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6da125640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6d40d30b0 con 0x7fc6d40d4150 2024-09-12T12:24:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6da125640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6d40d3390 con 0x7fc6d40d1190 2024-09-12T12:24:21.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6da125640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc6d40d45e0 con 0x7fc6d40b7740 2024-09-12T12:24:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d2ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc6d40d4150 0x7fc6d40cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:37594/0 (socket says 172.21.15.3:37594) 2024-09-12T12:24:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d2ffd640 1 -- 172.21.15.3:0/3487252429 learned_addr learned my addr 172.21.15.3:0/3487252429 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d3fff640 1 --2- 172.21.15.3:0/3487252429 >> v2:172.21.15.142:3300/0 conn(0x7fc6d40b7740 0x7fc6d40b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d27fc640 1 -- 172.21.15.3:0/3487252429 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698036089 0 0) 0x7fc6d40d30b0 con 0x7fc6d40d4150 2024-09-12T12:24:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d27fc640 1 -- 172.21.15.3:0/3487252429 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6bc003530 con 0x7fc6d40d4150 2024-09-12T12:24:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d27fc640 1 -- 172.21.15.3:0/3487252429 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 994424385 0 0) 0x7fc6bc003530 con 0x7fc6d40d4150 2024-09-12T12:24:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d3fff640 1 -- 172.21.15.3:0/3487252429 >> v1:172.21.15.142:6789/0 conn(0x7fc6d40d1190 legacy=0x7fc6d40d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d3fff640 1 -- 172.21.15.3:0/3487252429 >> v1:172.21.15.3:6789/0 conn(0x7fc6d40d4150 legacy=0x7fc6d40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.440+0000 7fc6d3fff640 1 -- 172.21.15.3:0/3487252429 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d40d3670 con 0x7fc6d40b7740 2024-09-12T12:24:21.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6d3fff640 1 --2- 172.21.15.3:0/3487252429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40b7740 0x7fc6d40b7bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0097c0 tx=0x7fc6cc01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d834b7a7f93b7cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:21.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6d27fc640 1 -- 172.21.15.3:0/3487252429 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0b5070 con 0x7fc6d40b7740 2024-09-12T12:24:21.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6d27fc640 1 -- 172.21.15.3:0/3487252429 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6cc0a8630 con 0x7fc6d40b7740 2024-09-12T12:24:21.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6d27fc640 1 -- 172.21.15.3:0/3487252429 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8001a00 con 0x7fc6d40b7740 2024-09-12T12:24:21.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6da125640 1 -- 172.21.15.3:0/3487252429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40b7740 msgr2=0x7fc6d40b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6da125640 1 --2- 172.21.15.3:0/3487252429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40b7740 0x7fc6d40b7bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0097c0 tx=0x7fc6cc01ebb0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6da125640 1 -- 172.21.15.3:0/3487252429 shutdown_connections 2024-09-12T12:24:21.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6da125640 1 --2- 172.21.15.3:0/3487252429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40b7740 0x7fc6d40b7bd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6da125640 1 -- 172.21.15.3:0/3487252429 >> 172.21.15.3:0/3487252429 conn(0x7fc6d4067a00 msgr2=0x7fc6d4067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:21.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6da125640 1 -- 172.21.15.3:0/3487252429 shutdown_connections 2024-09-12T12:24:21.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.444+0000 7fc6da125640 1 -- 172.21.15.3:0/3487252429 wait complete. 2024-09-12T12:24:21.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 Processor -- start 2024-09-12T12:24:21.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 -- start start 2024-09-12T12:24:21.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6d40b7740 0x7fc6d4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d40d1190 0x7fc6d40cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40d4150 0x7fc6d40ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40d2cd0 con 0x7fc6d40b7740 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40d3cc0 con 0x7fc6d40d4150 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d40d2e50 con 0x7fc6d40d1190 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6d40b7740 0x7fc6d4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6d40b7740 0x7fc6d4160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42888/0 (socket says 172.21.15.3:42888) 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 -- 172.21.15.3:0/3556931499 learned_addr learned my addr 172.21.15.3:0/3556931499 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d3fff640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40d4150 0x7fc6d40ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 -- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d40d1190 msgr2=0x7fc6d40cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d40d1190 0x7fc6d40cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 -- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40d4150 msgr2=0x7fc6d40ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40d4150 0x7fc6d40ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 -- 172.21.15.3:0/3556931499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d40ccd80 con 0x7fc6d40b7740 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d3fff640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40d4150 0x7fc6d40ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6d37fe640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6d40b7740 0x7fc6d4160d00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c500 tx=0x7fc6c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6bbfff640 1 -- 172.21.15.3:0/3556931499 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4016020 con 0x7fc6d40b7740 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6bbfff640 1 -- 172.21.15.3:0/3556931499 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6c4005220 con 0x7fc6d40b7740 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d4159e20 con 0x7fc6d40b7740 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6bbfff640 1 -- 172.21.15.3:0/3556931499 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4007030 con 0x7fc6d40b7740 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d415c270 con 0x7fc6d40b7740 2024-09-12T12:24:21.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d415e860 con 0x7fc6d40b7740 2024-09-12T12:24:21.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6bbfff640 1 -- 172.21.15.3:0/3556931499 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc6c40071f0 con 0x7fc6d40b7740 2024-09-12T12:24:21.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6bbfff640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c00678c0 0x7fc6c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:21.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.448+0000 7fc6bbfff640 1 -- 172.21.15.3:0/3556931499 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fc6c40857b0 con 0x7fc6d40b7740 2024-09-12T12:24:21.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.452+0000 7fc6d2ffd640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c00678c0 0x7fc6c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:21.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.452+0000 7fc6d2ffd640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c00678c0 0x7fc6c0069d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8001570 tx=0x7fc6c8001d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:21.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.456+0000 7fc6bbfff640 1 -- 172.21.15.3:0/3556931499 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc6c4020020 con 0x7fc6d40b7740 2024-09-12T12:24:21.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.624+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc6d40cb0c0 con 0x7fc6d40b7740 2024-09-12T12:24:21.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.624+0000 7fc6bbfff640 1 -- 172.21.15.3:0/3556931499 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) ==== 80+0+0 (secure 0 0 0) 0x7fc6c4056260 con 0x7fc6d40b7740 2024-09-12T12:24:21.628 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c00678c0 msgr2=0x7fc6c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c00678c0 0x7fc6c0069d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8001570 tx=0x7fc6c8001d30 comp rx=0 tx=0).stop 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6d40b7740 msgr2=0x7fc6d4160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6d40b7740 0x7fc6d4160d00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c500 tx=0x7fc6c400c9c0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 shutdown_connections 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc6c00678c0 0x7fc6c0069d80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d40d4150 0x7fc6d40ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6d40d1190 0x7fc6d40cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 --2- 172.21.15.3:0/3556931499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc6d40b7740 0x7fc6d4160d00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 >> 172.21.15.3:0/3556931499 conn(0x7fc6d4067a00 msgr2=0x7fc6d40b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:21.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 shutdown_connections 2024-09-12T12:24:21.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:21.628+0000 7fc6da125640 1 -- 172.21.15.3:0/3556931499 wait complete. 2024-09-12T12:24:21.646 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:24:21.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:21.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:21.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36417]: fuse finished with error 0 and tester_r 0 2024-09-12T12:24:21.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:24:27.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:24:27.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:27.763 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:24:27.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:27.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd91ff640 1 Processor -- start 2024-09-12T12:24:27.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd91ff640 1 -- start start 2024-09-12T12:24:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd91ff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40ae250 0x7fccd40ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd91ff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd415b620 0x7fccd415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd91ff640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd40ad0c0 con 0x7fccd40d05b0 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd91ff640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fccd40ad3a0 con 0x7fccd415b620 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd91ff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fccd40ad680 con 0x7fccd40ae250 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd359e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd415b620 0x7fccd415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd359e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccd415b620 0x7fccd415bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42898/0 (socket says 172.21.15.3:42898) 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.903+0000 7fccd359e640 1 -- 172.21.15.3:0/1907034830 learned_addr learned my addr 172.21.15.3:0/1907034830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd259c640 1 --2- 172.21.15.3:0/1907034830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40ae250 0x7fccd40ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd359e640 1 -- 172.21.15.3:0/1907034830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40ae250 msgr2=0x7fccd40ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd359e640 1 --2- 172.21.15.3:0/1907034830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40ae250 0x7fccd40ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd359e640 1 -- 172.21.15.3:0/1907034830 >> v1:172.21.15.142:6789/0 conn(0x7fccd40d05b0 legacy=0x7fccd40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd359e640 1 -- 172.21.15.3:0/1907034830 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd40ad960 con 0x7fccd415b620 2024-09-12T12:24:27.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd359e640 1 --2- 172.21.15.3:0/1907034830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415b620 0x7fccd415bab0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009870 tx=0x7fcccc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad7895f4d9b5a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd1d9b640 1 -- 172.21.15.3:0/1907034830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0a9610 con 0x7fccd415b620 2024-09-12T12:24:27.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd1d9b640 1 -- 172.21.15.3:0/1907034830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcccc0a97b0 con 0x7fccd415b620 2024-09-12T12:24:27.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd1d9b640 1 -- 172.21.15.3:0/1907034830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0a9b80 con 0x7fccd415b620 2024-09-12T12:24:27.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd91ff640 1 -- 172.21.15.3:0/1907034830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415b620 msgr2=0x7fccd415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.907+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1907034830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415b620 0x7fccd415bab0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009870 tx=0x7fcccc01c530 comp rx=0 tx=0).stop 2024-09-12T12:24:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- 172.21.15.3:0/1907034830 shutdown_connections 2024-09-12T12:24:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1907034830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd415b620 0x7fccd415bab0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1907034830 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40ae250 0x7fccd40ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- 172.21.15.3:0/1907034830 >> 172.21.15.3:0/1907034830 conn(0x7fccd4067e00 msgr2=0x7fccd4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- 172.21.15.3:0/1907034830 shutdown_connections 2024-09-12T12:24:27.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- 172.21.15.3:0/1907034830 wait complete. 2024-09-12T12:24:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 Processor -- start 2024-09-12T12:24:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- start start 2024-09-12T12:24:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd40ae250 0x7fccd4147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40d05b0 0x7fccd414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd415b620 0x7fccd414b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccd415c9b0 con 0x7fccd40ae250 2024-09-12T12:24:27.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fccd415c830 con 0x7fccd415b620 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fccd4167cb0 con 0x7fccd40d05b0 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd40ae250 0x7fccd4147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd40ae250 0x7fccd4147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41546/0 (socket says 172.21.15.3:41546) 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 -- 172.21.15.3:0/1222228399 learned_addr learned my addr 172.21.15.3:0/1222228399 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd259c640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40d05b0 0x7fccd414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd359e640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd415b620 0x7fccd414b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 -- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40d05b0 msgr2=0x7fccd414b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40d05b0 0x7fccd414b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 -- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd415b620 msgr2=0x7fccd414b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd415b620 0x7fccd414b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 -- 172.21.15.3:0/1222228399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd406d4c0 con 0x7fccd40ae250 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd259c640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40d05b0 0x7fccd414b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd359e640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd415b620 0x7fccd414b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd2d9d640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd40ae250 0x7fccd4147640 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c450 tx=0x7fccc400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccbb7fe640 1 -- 172.21.15.3:0/1222228399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4016020 con 0x7fccd40ae250 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccbb7fe640 1 -- 172.21.15.3:0/1222228399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc40052d0 con 0x7fccd40ae250 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd41550b0 con 0x7fccd40ae250 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccbb7fe640 1 -- 172.21.15.3:0/1222228399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4007260 con 0x7fccd40ae250 2024-09-12T12:24:27.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.911+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41501c0 con 0x7fccd40ae250 2024-09-12T12:24:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.915+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd40ae6b0 con 0x7fccd40ae250 2024-09-12T12:24:27.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.915+0000 7fccbb7fe640 1 -- 172.21.15.3:0/1222228399 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fccc4007400 con 0x7fccd40ae250 2024-09-12T12:24:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.915+0000 7fccbb7fe640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccc00678f0 0x7fccc0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.915+0000 7fccbb7fe640 1 -- 172.21.15.3:0/1222228399 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fccc40855b0 con 0x7fccd40ae250 2024-09-12T12:24:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.915+0000 7fccd259c640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccc00678f0 0x7fccc0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.915+0000 7fccd259c640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccc00678f0 0x7fccc0069db0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fccd4050200 tx=0x7fccc800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:27.919+0000 7fccbb7fe640 1 -- 172.21.15.3:0/1222228399 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fccc4020020 con 0x7fccd40ae250 2024-09-12T12:24:28.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.115+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccd4073340 con 0x7fccd40ae250 2024-09-12T12:24:28.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccbb7fe640 1 -- 172.21.15.3:0/1222228399 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5222 (secure 0 0 0) 0x7fccc4005480 con 0x7fccd40ae250 2024-09-12T12:24:28.122 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:28.122 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:57.730276+0000","modified":"2024-09-12T12:24:00.762240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2940169617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2940169617},{"type":"v1","addr":"172.21.15.3:6835","nonce":2940169617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-09-12T12:24:28.122 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 54 2024-09-12T12:24:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccc00678f0 msgr2=0x7fccc0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccc00678f0 0x7fccc0069db0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fccd4050200 tx=0x7fccc800b040 comp rx=0 tx=0).stop 2024-09-12T12:24:28.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd40ae250 msgr2=0x7fccd4147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd40ae250 0x7fccd4147640 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c450 tx=0x7fccc400c910 comp rx=0 tx=0).stop 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 shutdown_connections 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccc00678f0 0x7fccc0069db0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd415b620 0x7fccd414b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fccd40d05b0 0x7fccd414b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 --2- 172.21.15.3:0/1222228399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccd40ae250 0x7fccd4147640 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 >> 172.21.15.3:0/1222228399 conn(0x7fccd4067e00 msgr2=0x7fccd4153060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 shutdown_connections 2024-09-12T12:24:28.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.119+0000 7fccd91ff640 1 -- 172.21.15.3:0/1222228399 wait complete. 2024-09-12T12:24:28.142 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 session ls 2024-09-12T12:24:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.263+0000 7f511fb30640 1 Processor -- start 2024-09-12T12:24:28.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511fb30640 1 -- start start 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511fb30640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5118162a00 0x7f51180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511fb30640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f51180d0cd0 0x7f51180ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511fb30640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51180cc9c0 con 0x7f511806e7f0 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511fb30640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f51180cd5b0 con 0x7f51180d0cd0 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511fb30640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f51180cda40 con 0x7f5118162a00 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f51180d0cd0 0x7f51180ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f51180d0cd0 0x7f51180ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41552/0 (socket says 172.21.15.3:41552) 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 -- 172.21.15.3:0/251010554 learned_addr learned my addr 172.21.15.3:0/251010554 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511d8cc640 1 --2- 172.21.15.3:0/251010554 >> v2:172.21.15.142:3300/0 conn(0x7f5118162a00 0x7f51180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 -- 172.21.15.3:0/251010554 >> v2:172.21.15.142:3300/0 conn(0x7f5118162a00 msgr2=0x7f51180b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 --2- 172.21.15.3:0/251010554 >> v2:172.21.15.142:3300/0 conn(0x7f5118162a00 0x7f51180b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 -- 172.21.15.3:0/251010554 >> v1:172.21.15.142:6789/0 conn(0x7f511806e7f0 legacy=0x7f51180d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 -- 172.21.15.3:0/251010554 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51180ccc80 con 0x7f51180d0cd0 2024-09-12T12:24:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511e0cd640 1 --2- 172.21.15.3:0/251010554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f51180ce850 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f51180d1180 tx=0x7f511001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474d856f47b1aed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511c8ca640 1 -- 172.21.15.3:0/251010554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100a5840 con 0x7f51180d0cd0 2024-09-12T12:24:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511c8ca640 1 -- 172.21.15.3:0/251010554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51100a59e0 con 0x7f51180d0cd0 2024-09-12T12:24:28.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.267+0000 7f511c8ca640 1 -- 172.21.15.3:0/251010554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100a5db0 con 0x7f51180d0cd0 2024-09-12T12:24:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- 172.21.15.3:0/251010554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 msgr2=0x7f51180ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 --2- 172.21.15.3:0/251010554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f51180ce850 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f51180d1180 tx=0x7f511001c980 comp rx=0 tx=0).stop 2024-09-12T12:24:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- 172.21.15.3:0/251010554 shutdown_connections 2024-09-12T12:24:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 --2- 172.21.15.3:0/251010554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f51180ce850 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 --2- 172.21.15.3:0/251010554 >> v2:172.21.15.142:3300/0 conn(0x7f5118162a00 0x7f51180b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- 172.21.15.3:0/251010554 >> 172.21.15.3:0/251010554 conn(0x7f5118067e00 msgr2=0x7f5118068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- 172.21.15.3:0/251010554 shutdown_connections 2024-09-12T12:24:28.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- 172.21.15.3:0/251010554 wait complete. 2024-09-12T12:24:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 Processor -- start 2024-09-12T12:24:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- start start 2024-09-12T12:24:28.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f511806e7f0 0x7f51180ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f511814f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5118162a00 0x7f51180ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f51180cf5a0 con 0x7f51180d0cd0 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5118162de0 con 0x7f5118162a00 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f51180cf720 con 0x7f511806e7f0 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f511814f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f511814f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41568/0 (socket says 172.21.15.3:41568) 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 -- 172.21.15.3:0/4237995780 learned_addr learned my addr 172.21.15.3:0/4237995780 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511e0cd640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5118162a00 0x7f51180ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d8cc640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f511806e7f0 0x7f51180ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 -- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f511806e7f0 msgr2=0x7f51180ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f511806e7f0 0x7f51180ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 -- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5118162a00 msgr2=0x7f51180ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5118162a00 0x7f51180ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 -- 172.21.15.3:0/4237995780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511815e6d0 con 0x7f51180d0cd0 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511e0cd640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5118162a00 0x7f51180ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d8cc640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f511806e7f0 0x7f51180ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511d0cb640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f511814f460 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f510800c450 tx=0x7f510800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f510e7fc640 1 -- 172.21.15.3:0/4237995780 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108012830 con 0x7f51180d0cd0 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f510e7fc640 1 -- 172.21.15.3:0/4237995780 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5108017070 con 0x7f51180d0cd0 2024-09-12T12:24:28.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511815e9b0 con 0x7f51180d0cd0 2024-09-12T12:24:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f510e7fc640 1 -- 172.21.15.3:0/4237995780 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108012c30 con 0x7f51180d0cd0 2024-09-12T12:24:28.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.271+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511815ef70 con 0x7f51180d0cd0 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f510e7fc640 1 -- 172.21.15.3:0/4237995780 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f510801c460 con 0x7f51180d0cd0 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f510e7fc640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50fc0678c0 0x7f50fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f510e7fc640 1 -- 172.21.15.3:0/4237995780 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f5108085ac0 con 0x7f51180d0cd0 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511d8cc640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50fc0678c0 0x7f50fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 Processor -- start 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 -- start start 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511d8cc640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50fc0678c0 0x7f50fc069d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f51180af470 tx=0x7f5114008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dc7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dc7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564c91caec40 con 0x564c91dc77d0 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c91c15d50 con 0x564c91e11380 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x564c91bd0ab0 con 0x564c91e11ca0 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dc7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dc7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41580/0 (socket says 172.21.15.3:41580) 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 -- 172.21.15.3:0/2952420497 learned_addr learned my addr 172.21.15.3:0/2952420497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511d0cb640 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dc7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511d8cc640 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 -- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 msgr2=0x564c91dc7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dc7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 -- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 msgr2=0x564c91e11760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e11760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.275+0000 7f511e0cd640 1 -- 172.21.15.3:0/2952420497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c91bbc590 con 0x564c91dc77d0 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511e0cd640 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dc7bb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f511001d220 tx=0x7f5110004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447540f84a24fc81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50fb7fe640 1 -- 172.21.15.3:0/2952420497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100a5840 con 0x564c91dc77d0 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50fb7fe640 1 -- 172.21.15.3:0/2952420497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51100a59e0 con 0x564c91dc77d0 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50fb7fe640 1 -- 172.21.15.3:0/2952420497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100a5dd0 con 0x564c91dc77d0 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/2952420497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 msgr2=0x564c91dc7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dc7bb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f511001d220 tx=0x7f5110004ac0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/2952420497 shutdown_connections 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dc7bb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dc7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- 172.21.15.3:0/2952420497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e11760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/2952420497 >> 172.21.15.3:0/2952420497 conn(0x564c91dbebc0 msgr2=0x564c91dc5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/2952420497 shutdown_connections 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/2952420497 wait complete. 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- start start 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 Processor -- start 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dcc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91dfb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dfbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564c91bd4400 con 0x564c91dc77d0 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c91cd23a0 con 0x564c91e11380 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x564c91c55260 con 0x564c91e11ca0 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 10 client.0 _ll_register_callbacks cb 0x564c91c77020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:28.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dcc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dcc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41594/0 (socket says 172.21.15.3:41594) 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 -- 172.21.15.3:0/670939663 learned_addr learned my addr 172.21.15.3:0/670939663 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511e0cd640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dfbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d0cb640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91dfb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 -- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 msgr2=0x564c91dfbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dfbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 -- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 msgr2=0x564c91dfb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91dfb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 -- 172.21.15.3:0/670939663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c91bb8dd0 con 0x564c91dc77d0 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d0cb640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91dfb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f511d8cc640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dcc860 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5114004660 tx=0x7f5114004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441a136687785b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:24:28.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50f8ff9640 1 -- 172.21.15.3:0/670939663 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400f040 con 0x564c91dc77d0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50f8ff9640 1 -- 172.21.15.3:0/670939663 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f5114015c10 con 0x564c91dc77d0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51100a59e0 con 0x564c91dc77d0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50f8ff9640 1 -- 172.21.15.3:0/670939663 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5114015e10 con 0x564c91dc77d0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 10 client.6603 fetch_fsmap learned FSMap version 54 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x564c91bb5800 con 0x564c91dc77d0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50f8ff9640 1 -- 172.21.15.3:0/670939663 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51100a59e0 con 0x564c91dc77d0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f50f8ff9640 1 -- 172.21.15.3:0/670939663 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f5114031080 con 0x564c91dc77d0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 10 client.6603 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 10 client.6603 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 20 client.6603 populate_metadata read hostname 'smithi003' 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e180e0 0x564c91e184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 4 client.6603 mds_command: new command op to 6375 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.279+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 --> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c91bafe20 con 0x564c91e180e0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f511d0cb640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e180e0 0x564c91e184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f511d0cb640 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e180e0 0x564c91e184f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb2d89ade8a0a37c server_cookie=542b1e2b64e85802 in_seq=0 out_seq=0 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f50f8ff9640 10 client.6603 ms_handle_connect on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f50f8ff9640 1 -- 172.21.15.3:0/670939663 <== mds.0 v2:172.21.15.3:6834/2940169617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c91bafe20 con 0x564c91e180e0 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f50f8ff9640 10 client.6603 handle_command_reply: tid=0 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 client.6603 shutdown 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 msgr2=0x564c91dcc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dcc860 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5114004660 tx=0x7f5114004740 comp rx=0 tx=0).stop 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 shutdown_connections 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e180e0 0x564c91e184f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91e11ca0 0x564c91dfbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91dfb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 --2- 172.21.15.3:0/670939663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91dcc860 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f50f8ff9640 0 client.6603 ms_handle_reset on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 >> 172.21.15.3:0/670939663 conn(0x564c91dbebc0 msgr2=0x564c91dc6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 shutdown_connections 2024-09-12T12:24:28.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 1 -- 172.21.15.3:0/670939663 wait complete. 2024-09-12T12:24:28.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.283+0000 7f5122bec000 20 client.6603 trim_cache size 0 max 16384 2024-09-12T12:24:28.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 Processor -- start 2024-09-12T12:24:28.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- start start 2024-09-12T12:24:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91dc77d0 0x564c91e28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91e11380 0x564c91e14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f511815b840 con 0x564c91e11380 2024-09-12T12:24:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f511814edf0 con 0x564c91dc77d0 2024-09-12T12:24:28.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f511815b1d0 con 0x564c91dbebc0 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91e11380 0x564c91e14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91e11380 0x564c91e14090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41598/0 (socket says 172.21.15.3:41598) 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 -- 172.21.15.3:0/3556377892 learned_addr learned my addr 172.21.15.3:0/3556377892 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511d0cb640 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91dc77d0 0x564c91e28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511d8cc640 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 -- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 msgr2=0x564c91e286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e286e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 -- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91dc77d0 msgr2=0x564c91e28c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91dc77d0 0x564c91e28c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 -- 172.21.15.3:0/3556377892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c91bd3480 con 0x564c91e11380 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511d8cc640 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f511e0cd640 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91e11380 0x564c91e14090 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f511001d220 tx=0x7f51100a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ba88cbbbef5ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f50faffd640 1 -- 172.21.15.3:0/3556377892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100039a0 con 0x564c91e11380 2024-09-12T12:24:28.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f50faffd640 1 -- 172.21.15.3:0/3556377892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5110003b40 con 0x564c91e11380 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f50faffd640 1 -- 172.21.15.3:0/3556377892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100a5840 con 0x564c91e11380 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- 172.21.15.3:0/3556377892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91e11380 msgr2=0x564c91e14090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91e11380 0x564c91e14090 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f511001d220 tx=0x7f51100a7000 comp rx=0 tx=0).stop 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- 172.21.15.3:0/3556377892 shutdown_connections 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91e11380 0x564c91e14090 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91dc77d0 0x564c91e28c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 --2- 172.21.15.3:0/3556377892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- 172.21.15.3:0/3556377892 >> 172.21.15.3:0/3556377892 conn(0x564c91dcc500 msgr2=0x564c91dfb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- 172.21.15.3:0/3556377892 shutdown_connections 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- 172.21.15.3:0/3556377892 wait complete. 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 -- start start 2024-09-12T12:24:28.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.303+0000 7f5122bec000 1 Processor -- start 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e16a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91e859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564c91bbbf30 con 0x564c91dc77d0 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564c91bd47d0 con 0x564c91e11380 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x564c91c15060 con 0x564c91dbebc0 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 10 client.0 _ll_register_callbacks cb 0x7f5114009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91e859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91e859f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41614/0 (socket says 172.21.15.3:41614) 2024-09-12T12:24:28.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 -- 172.21.15.3:0/3479556032 learned_addr learned my addr 172.21.15.3:0/3479556032 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d8cc640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e16a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511e0cd640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 -- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 msgr2=0x564c91e16a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e16a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 -- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 msgr2=0x564c91e85f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e85f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 -- 172.21.15.3:0/3479556032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c91ba9650 con 0x564c91dc77d0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511e0cd640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d0cb640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91e859f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5108004600 tx=0x7f5108052290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfd233ab435cda8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 1 -- 172.21.15.3:0/3479556032 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108082060 con 0x564c91dc77d0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 1 -- 172.21.15.3:0/3479556032 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f5108052570 con 0x564c91dc77d0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5110003b40 con 0x564c91dc77d0 2024-09-12T12:24:28.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 1 -- 172.21.15.3:0/3479556032 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5108079050 con 0x564c91dc77d0 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 10 client.6609 fetch_fsmap learned FSMap version 54 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x564c91bad3c0 con 0x564c91dc77d0 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 1 -- 172.21.15.3:0/3479556032 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5110003b40 con 0x564c91dc77d0 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 1 -- 172.21.15.3:0/3479556032 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f510805e650 con 0x564c91dc77d0 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 10 client.6609 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 10 client.6609 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 20 client.6609 populate_metadata read hostname 'smithi003' 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e11ca0 0x564c91e86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 4 client.6609 mds_command: new command op to 6375 tid=0 multi_id=0 [{"prefix": "session ls"}] 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 --> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] -- command(tid 0: {"prefix": "session ls"}) -- 0x564c91c23480 con 0x564c91e11ca0 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d8cc640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e11ca0 0x564c91e86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511d8cc640 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e11ca0 0x564c91e86470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424d715f4b67ad11 server_cookie=b2f2c47c74bb463c in_seq=0 out_seq=0 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 10 client.6609 ms_handle_connect on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:28.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 1 -- 172.21.15.3:0/3479556032 <== mds.0 v2:172.21.15.3:6834/2940169617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c91c23480 con 0x564c91e11ca0 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 10 client.6609 handle_command_reply: tid=0 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 client.6609 shutdown 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 msgr2=0x564c91e859f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91e859f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5108004600 tx=0x7f5108052290 comp rx=0 tx=0).stop 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 shutdown_connections 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:6834/2940169617,v1:172.21.15.3:6835/2940169617] conn(0x564c91e11ca0 0x564c91e86470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c91e11380 0x564c91e85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564c91dc77d0 0x564c91e859f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 --2- 172.21.15.3:0/3479556032 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x564c91dbebc0 0x564c91e16a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 >> 172.21.15.3:0/3479556032 conn(0x564c91dcc500 msgr2=0x564c91dfb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f50f8ff9640 0 client.6609 ms_handle_reset on v2:172.21.15.3:6834/2940169617 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 shutdown_connections 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 1 -- 172.21.15.3:0/3479556032 wait complete. 2024-09-12T12:24:28.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f5122bec000 20 client.6609 trim_cache size 0 max 16384 2024-09-12T12:24:28.318 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T12:24:28.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50fc0678c0 msgr2=0x7f50fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50fc0678c0 0x7f50fc069d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f51180af470 tx=0x7f5114008040 comp rx=0 tx=0).stop 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 msgr2=0x7f511814f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f511814f460 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f510800c450 tx=0x7f510800c910 comp rx=0 tx=0).stop 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 shutdown_connections 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50fc0678c0 0x7f50fc069d80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5118162a00 0x7f51180ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f51180d0cd0 0x7f511814f460 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 --2- 172.21.15.3:0/4237995780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f511806e7f0 0x7f51180ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 >> 172.21.15.3:0/4237995780 conn(0x7f5118067e00 msgr2=0x7f511814fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 shutdown_connections 2024-09-12T12:24:28.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.307+0000 7f511fb30640 1 -- 172.21.15.3:0/4237995780 wait complete. 2024-09-12T12:24:28.329 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T12:24:28.464 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T12:24:28.464 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-12T12:24:28.464 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:24:28.465 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:24:28.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:24:28.549 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:24:28.549 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:24:28.549 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:28.549 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:28.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:24:28.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:24:28.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:28.555 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:24:28.601 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-12T12:24:28.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:24:28.602 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:24:28.602 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:24:28.602 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:28.609 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:24:28.609 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:24:28.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:28.609 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:24:28.659 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-12T12:24:28.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:24:28.660 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:24:28.660 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:28.661 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:28.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:24:28.666 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:24:28.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:28.667 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T12:24:28.713 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-12T12:24:28.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:24:28.714 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:24:28.714 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:24:28.714 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:28.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:24:28.721 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:24:28.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:28.721 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T12:24:28.767 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-12T12:24:28.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:24:28.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:28.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f5e036640 1 Processor -- start 2024-09-12T12:24:28.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f5e036640 1 -- start start 2024-09-12T12:24:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f5e036640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7f580d1190 0x7f7f580d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f5e036640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f580d1570 0x7f7f580cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f5e036640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f580d30b0 con 0x7f7f580cf950 2024-09-12T12:24:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f5e036640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7f580d3cc0 con 0x7f7f580d1570 2024-09-12T12:24:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f5e036640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7f580d3390 con 0x7f7f580d1190 2024-09-12T12:24:28.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f56ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f580d1570 0x7f7f580cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f56ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7f580d1570 0x7f7f580cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41622/0 (socket says 172.21.15.3:41622) 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f56ffd640 1 -- 172.21.15.3:0/1402115226 learned_addr learned my addr 172.21.15.3:0/1402115226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f57fff640 1 --1- 172.21.15.3:0/1402115226 >> v1:172.21.15.3:6789/0 conn(0x7f7f580cf950 0x7f7f580b7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40470/0 (socket says 172.21.15.3:40470) 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f56ffd640 1 -- 172.21.15.3:0/1402115226 >> v2:172.21.15.142:3300/0 conn(0x7f7f580d1190 msgr2=0x7f7f580d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f577fe640 1 --2- 172.21.15.3:0/1402115226 >> v2:172.21.15.142:3300/0 conn(0x7f7f580d1190 0x7f7f580d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f56ffd640 1 --2- 172.21.15.3:0/1402115226 >> v2:172.21.15.142:3300/0 conn(0x7f7f580d1190 0x7f7f580d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f56ffd640 1 -- 172.21.15.3:0/1402115226 >> v1:172.21.15.3:6789/0 conn(0x7f7f580cf950 legacy=0x7f7f580b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f567fc640 1 -- 172.21.15.3:0/1402115226 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2631639780 0 0) 0x7f7f580d30b0 con 0x7f7f580cf950 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f56ffd640 1 -- 172.21.15.3:0/1402115226 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f580d3670 con 0x7f7f580d1570 2024-09-12T12:24:28.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.899+0000 7f7f577fe640 1 --2- 172.21.15.3:0/1402115226 >> v2:172.21.15.142:3300/0 conn(0x7f7f580d1190 0x7f7f580d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f56ffd640 1 --2- 172.21.15.3:0/1402115226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f580cf410 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7f5806d9e0 tx=0x7f7f4c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c361282d31a7fce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f567fc640 1 -- 172.21.15.3:0/1402115226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0b5070 con 0x7f7f580d1570 2024-09-12T12:24:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f567fc640 1 -- 172.21.15.3:0/1402115226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4c0a8a10 con 0x7f7f580d1570 2024-09-12T12:24:28.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f567fc640 1 -- 172.21.15.3:0/1402115226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0a8de0 con 0x7f7f580d1570 2024-09-12T12:24:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- 172.21.15.3:0/1402115226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 msgr2=0x7f7f580cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1402115226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f580cf410 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7f5806d9e0 tx=0x7f7f4c01ed10 comp rx=0 tx=0).stop 2024-09-12T12:24:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- 172.21.15.3:0/1402115226 shutdown_connections 2024-09-12T12:24:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1402115226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f580cf410 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1402115226 >> v2:172.21.15.142:3300/0 conn(0x7f7f580d1190 0x7f7f580d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- 172.21.15.3:0/1402115226 >> 172.21.15.3:0/1402115226 conn(0x7f7f58067a00 msgr2=0x7f7f58067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- 172.21.15.3:0/1402115226 shutdown_connections 2024-09-12T12:24:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- 172.21.15.3:0/1402115226 wait complete. 2024-09-12T12:24:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 Processor -- start 2024-09-12T12:24:28.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- start start 2024-09-12T12:24:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f580cf950 0x7f7f581546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f580d1190 0x7f7f58160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f58161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7f580d2cd0 con 0x7f7f580d1570 2024-09-12T12:24:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f580d1a00 con 0x7f7f580d1190 2024-09-12T12:24:28.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f5e036640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7f580d2e50 con 0x7f7f580cf950 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f58161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f58161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41632/0 (socket says 172.21.15.3:41632) 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 -- 172.21.15.3:0/1160153581 learned_addr learned my addr 172.21.15.3:0/1160153581 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f56ffd640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f580d1190 0x7f7f58160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f577fe640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f580cf950 0x7f7f581546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 -- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f580cf950 msgr2=0x7f7f581546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f580cf950 0x7f7f581546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 -- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f580d1190 msgr2=0x7f7f58160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f580d1190 0x7f7f58160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.903+0000 7f7f57fff640 1 -- 172.21.15.3:0/1160153581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f58154c20 con 0x7f7f580d1570 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f57fff640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f58161180 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7f50001570 tx=0x7f7f5000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f3ffff640 1 -- 172.21.15.3:0/1160153581 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50018070 con 0x7f7f580d1570 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f3ffff640 1 -- 172.21.15.3:0/1160153581 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5000a640 con 0x7f7f580d1570 2024-09-12T12:24:28.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f3ffff640 1 -- 172.21.15.3:0/1160153581 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000aa30 con 0x7f7f580d1570 2024-09-12T12:24:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f580c9000 con 0x7f7f580d1570 2024-09-12T12:24:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f580cb9d0 con 0x7f7f580d1570 2024-09-12T12:24:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f580cb7c0 con 0x7f7f580d1570 2024-09-12T12:24:28.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f3ffff640 1 -- 172.21.15.3:0/1160153581 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7f5000abd0 con 0x7f7f580d1570 2024-09-12T12:24:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f3ffff640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7f440678c0 0x7f7f44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f3ffff640 1 -- 172.21.15.3:0/1160153581 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7f7f500853f0 con 0x7f7f580d1570 2024-09-12T12:24:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f577fe640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7f440678c0 0x7f7f44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:28.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.907+0000 7f7f577fe640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7f440678c0 0x7f7f44069d80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800a9a0 tx=0x7f7f48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:28.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:28.911+0000 7f7f3ffff640 1 -- 172.21.15.3:0/1160153581 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7f50055e90 con 0x7f7f580d1570 2024-09-12T12:24:29.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.111+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f5815f9e0 con 0x7f7f580d1570 2024-09-12T12:24:29.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f3ffff640 1 -- 172.21.15.3:0/1160153581 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5222 (secure 0 0 0) 0x7f7f50055790 con 0x7f7f580d1570 2024-09-12T12:24:29.117 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:29.117 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:57.730276+0000","modified":"2024-09-12T12:24:00.762240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2940169617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2940169617},{"type":"v1","addr":"172.21.15.3:6835","nonce":2940169617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-09-12T12:24:29.118 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 54 2024-09-12T12:24:29.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7f440678c0 msgr2=0x7f7f44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7f440678c0 0x7f7f44069d80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800a9a0 tx=0x7f7f48008040 comp rx=0 tx=0).stop 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 msgr2=0x7f7f58161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f58161180 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7f50001570 tx=0x7f7f5000cca0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 shutdown_connections 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7f440678c0 0x7f7f44069d80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f580d1570 0x7f7f58161180 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f580d1190 0x7f7f58160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 --2- 172.21.15.3:0/1160153581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f580cf950 0x7f7f581546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 >> 172.21.15.3:0/1160153581 conn(0x7f7f58067a00 msgr2=0x7f7f580b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 shutdown_connections 2024-09-12T12:24:29.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.115+0000 7f7f5e036640 1 -- 172.21.15.3:0/1160153581 wait complete. 2024-09-12T12:24:29.137 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cf348640 1 Processor -- start 2024-09-12T12:24:29.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cf348640 1 -- start start 2024-09-12T12:24:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cf348640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5c80d1190 0x7fd5c80d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cf348640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80d30b0 con 0x7fd5c80b7740 2024-09-12T12:24:29.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cf348640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80d3390 con 0x7fd5c80d1570 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cf348640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c80d3cc0 con 0x7fd5c80d1190 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cd0e4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd5c80d1190 0x7fd5c80d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cd8e5640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd5c80b7740 0x7fd5c80b7bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40478/0 (socket says 172.21.15.3:40478) 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cd8e5640 1 -- 172.21.15.3:0/244528901 learned_addr learned my addr 172.21.15.3:0/244528901 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cd0e4640 1 -- 172.21.15.3:0/244528901 >> v1:172.21.15.142:6789/0 conn(0x7fd5c80d1570 legacy=0x7fd5c80cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cd0e4640 1 -- 172.21.15.3:0/244528901 >> v1:172.21.15.3:6789/0 conn(0x7fd5c80b7740 legacy=0x7fd5c80b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5bffff640 1 -- 172.21.15.3:0/244528901 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209644027 0 0) 0x7fd5c80d30b0 con 0x7fd5c80b7740 2024-09-12T12:24:29.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.255+0000 7fd5cd0e4640 1 -- 172.21.15.3:0/244528901 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80d3670 con 0x7fd5c80d1190 2024-09-12T12:24:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cd0e4640 1 --2- 172.21.15.3:0/244528901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80d4150 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40097c0 tx=0x7fd5c401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4aca20df3ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5bffff640 1 -- 172.21.15.3:0/244528901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40b5070 con 0x7fd5c80d1190 2024-09-12T12:24:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5bffff640 1 -- 172.21.15.3:0/244528901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c40a8990 con 0x7fd5c80d1190 2024-09-12T12:24:29.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5bffff640 1 -- 172.21.15.3:0/244528901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40a8d60 con 0x7fd5c80d1190 2024-09-12T12:24:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- 172.21.15.3:0/244528901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 msgr2=0x7fd5c80d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 --2- 172.21.15.3:0/244528901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80d4150 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40097c0 tx=0x7fd5c401ec90 comp rx=0 tx=0).stop 2024-09-12T12:24:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- 172.21.15.3:0/244528901 shutdown_connections 2024-09-12T12:24:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 --2- 172.21.15.3:0/244528901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80d4150 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- 172.21.15.3:0/244528901 >> 172.21.15.3:0/244528901 conn(0x7fd5c8067a00 msgr2=0x7fd5c8067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- 172.21.15.3:0/244528901 shutdown_connections 2024-09-12T12:24:29.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- 172.21.15.3:0/244528901 wait complete. 2024-09-12T12:24:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 Processor -- start 2024-09-12T12:24:29.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- start start 2024-09-12T12:24:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5c80b7740 0x7fd5c8161330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80d1570 0x7fd5c80cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80d2cd0 con 0x7fd5c80d1190 2024-09-12T12:24:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80d1a00 con 0x7fd5c80d1570 2024-09-12T12:24:29.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cf348640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80d2e50 con 0x7fd5c80b7740 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cc8e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cc8e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41652/0 (socket says 172.21.15.3:41652) 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.259+0000 7fd5cc8e3640 1 -- 172.21.15.3:0/2632072965 learned_addr learned my addr 172.21.15.3:0/2632072965 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cd0e4640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5c80b7740 0x7fd5c8161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cd8e5640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80d1570 0x7fd5c80cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cc8e3640 1 -- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5c80b7740 msgr2=0x7fd5c8161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cc8e3640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5c80b7740 0x7fd5c8161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cc8e3640 1 -- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80d1570 msgr2=0x7fd5c80cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cc8e3640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80d1570 0x7fd5c80cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cc8e3640 1 -- 172.21.15.3:0/2632072965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80c8a30 con 0x7fd5c80d1190 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cd8e5640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80d1570 0x7fd5c80cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cc8e3640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80cc940 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c3c0 tx=0x7fd5b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5bdffb640 1 -- 172.21.15.3:0/2632072965 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8017070 con 0x7fd5c80d1190 2024-09-12T12:24:29.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5bdffb640 1 -- 172.21.15.3:0/2632072965 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b80052d0 con 0x7fd5c80d1190 2024-09-12T12:24:29.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5bdffb640 1 -- 172.21.15.3:0/2632072965 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8003840 con 0x7fd5c80d1190 2024-09-12T12:24:29.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c8161870 con 0x7fd5c80d1190 2024-09-12T12:24:29.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c80c8750 con 0x7fd5c80d1190 2024-09-12T12:24:29.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c815e860 con 0x7fd5c80d1190 2024-09-12T12:24:29.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5bdffb640 1 -- 172.21.15.3:0/2632072965 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5b801a070 con 0x7fd5c80d1190 2024-09-12T12:24:29.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5bdffb640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5ac0678f0 0x7fd5ac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5bdffb640 1 -- 172.21.15.3:0/2632072965 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fd5b8084ee0 con 0x7fd5c80d1190 2024-09-12T12:24:29.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.263+0000 7fd5cd0e4640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5ac0678f0 0x7fd5ac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.267+0000 7fd5cd0e4640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5ac0678f0 0x7fd5ac069db0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40b2000 tx=0x7fd5c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.267+0000 7fd5bdffb640 1 -- 172.21.15.3:0/2632072965 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd5c80c8a30 con 0x7fd5c80d1190 2024-09-12T12:24:29.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.471+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5c80ba4c0 con 0x7fd5c80d1190 2024-09-12T12:24:29.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.471+0000 7fd5bdffb640 1 -- 172.21.15.3:0/2632072965 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5222 (secure 0 0 0) 0x7fd5b8005480 con 0x7fd5c80d1190 2024-09-12T12:24:29.476 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:29.476 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:57.730276+0000","modified":"2024-09-12T12:24:00.762240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2940169617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2940169617},{"type":"v1","addr":"172.21.15.3:6835","nonce":2940169617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-09-12T12:24:29.477 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 54 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5ac0678f0 msgr2=0x7fd5ac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5ac0678f0 0x7fd5ac069db0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40b2000 tx=0x7fd5c4008040 comp rx=0 tx=0).stop 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 msgr2=0x7fd5c80cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80cc940 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c3c0 tx=0x7fd5b800c880 comp rx=0 tx=0).stop 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 shutdown_connections 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd5ac0678f0 0x7fd5ac069db0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5c80d1570 0x7fd5c80cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd5c80d1190 0x7fd5c80cc940 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 --2- 172.21.15.3:0/2632072965 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd5c80b7740 0x7fd5c8161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 >> 172.21.15.3:0/2632072965 conn(0x7fd5c8067a00 msgr2=0x7fd5c80b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:29.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 shutdown_connections 2024-09-12T12:24:29.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.475+0000 7fd5cf348640 1 -- 172.21.15.3:0/2632072965 wait complete. 2024-09-12T12:24:29.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:24:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee6be7640 1 Processor -- start 2024-09-12T12:24:29.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee6be7640 1 -- start start 2024-09-12T12:24:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee6be7640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdee00b7740 0x7fdee00b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee6be7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee00d30b0 con 0x7fdee00d1190 2024-09-12T12:24:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee6be7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee00d3390 con 0x7fdee00d4150 2024-09-12T12:24:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee6be7640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdee00d45e0 con 0x7fdee00b7740 2024-09-12T12:24:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdee00d1190 0x7fdee00d1570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40480/0 (socket says 172.21.15.3:40480) 2024-09-12T12:24:29.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 -- 172.21.15.3:0/1093394626 learned_addr learned my addr 172.21.15.3:0/1093394626 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3424698302 0 0) 0x7fdee00d30b0 con 0x7fdee00d1190 2024-09-12T12:24:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec8003530 con 0x7fdee00d1190 2024-09-12T12:24:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee5184640 1 --2- 172.21.15.3:0/1093394626 >> v2:172.21.15.142:3300/0 conn(0x7fdee00b7740 0x7fdee00b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2210856769 0 0) 0x7fdec8003530 con 0x7fdee00d1190 2024-09-12T12:24:29.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 >> v2:172.21.15.142:3300/0 conn(0x7fdee00b7740 msgr2=0x7fdee00b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 --2- 172.21.15.3:0/1093394626 >> v2:172.21.15.142:3300/0 conn(0x7fdee00b7740 0x7fdee00b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 >> v1:172.21.15.142:6789/0 conn(0x7fdee00d4150 legacy=0x7fdee00cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00d3670 con 0x7fdee00d1190 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fded0001680 con 0x7fdee00d1190 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 >> v1:172.21.15.3:6789/0 conn(0x7fdee00d1190 legacy=0x7fdee00d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdec80039c0 0x7fdec8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8004810 0x7fdec8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80051d0 0x7fdec800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00d45e0 con 0x7fdec80039c0 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdec800c080 con 0x7fdec80051d0 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdec800c280 con 0x7fdec8004810 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdec80039c0 0x7fdec8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee5184640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80051d0 0x7fdec800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedffff640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8004810 0x7fdec8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 -- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8004810 msgr2=0x7fdec8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8004810 0x7fdec8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 -- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80051d0 msgr2=0x7fdec800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80051d0 0x7fdec800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee4983640 1 -- 172.21.15.3:0/1093394626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec8003530 con 0x7fdec80039c0 2024-09-12T12:24:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdee5184640 1 -- 172.21.15.3:0/1093394626 reap_dead start 2024-09-12T12:24:29.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.615+0000 7fdedffff640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdec8004810 0x7fdec8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee4983640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdec80039c0 0x7fdec8003de0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fded000c210 tx=0x7fded001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eedfb9d188e7089 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0007d40 con 0x7fdec80039c0 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded0005b00 con 0x7fdec80039c0 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdedf7fe640 1 -- 172.21.15.3:0/1093394626 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0007070 con 0x7fdec80039c0 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 -- 172.21.15.3:0/1093394626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdec80039c0 msgr2=0x7fdec8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdec80039c0 0x7fdec8003de0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fded000c210 tx=0x7fded001f3b0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 -- 172.21.15.3:0/1093394626 shutdown_connections 2024-09-12T12:24:29.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 --2- 172.21.15.3:0/1093394626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdec80039c0 0x7fdec8003de0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 -- 172.21.15.3:0/1093394626 >> 172.21.15.3:0/1093394626 conn(0x7fdee0067a00 msgr2=0x7fdee0067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 -- 172.21.15.3:0/1093394626 shutdown_connections 2024-09-12T12:24:29.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 -- 172.21.15.3:0/1093394626 wait complete. 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 Processor -- start 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.619+0000 7fdee6be7640 1 -- start start 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80039c0 0x7fdee01557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee00b6d60 0x7fdee00b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee0155de0 0x7fdee00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00d2cd0 con 0x7fdee00b6d60 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00d3cc0 con 0x7fdec80039c0 2024-09-12T12:24:29.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00d2e50 con 0x7fdee0155de0 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee00b6d60 0x7fdee00b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee00b6d60 0x7fdee00b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41658/0 (socket says 172.21.15.3:41658) 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 -- 172.21.15.3:0/4100496628 learned_addr learned my addr 172.21.15.3:0/4100496628 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee4983640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80039c0 0x7fdee01557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee5184640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee0155de0 0x7fdee00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 -- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee0155de0 msgr2=0x7fdee00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee0155de0 0x7fdee00cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 -- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80039c0 msgr2=0x7fdee01557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80039c0 0x7fdee01557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 -- 172.21.15.3:0/4100496628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00b42d0 con 0x7fdee00b6d60 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedffff640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee00b6d60 0x7fdee00b5370 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fded400d4a0 tx=0x7fded400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedd7fa640 1 -- 172.21.15.3:0/4100496628 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4005280 con 0x7fdee00b6d60 2024-09-12T12:24:29.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedd7fa640 1 -- 172.21.15.3:0/4100496628 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded4005440 con 0x7fdee00b6d60 2024-09-12T12:24:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedd7fa640 1 -- 172.21.15.3:0/4100496628 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4007260 con 0x7fdee00b6d60 2024-09-12T12:24:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee00b45b0 con 0x7fdee00b6d60 2024-09-12T12:24:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee00b4b70 con 0x7fdee00b6d60 2024-09-12T12:24:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee00ba4c0 con 0x7fdee00b6d60 2024-09-12T12:24:29.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedd7fa640 1 -- 172.21.15.3:0/4100496628 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fded4019020 con 0x7fdee00b6d60 2024-09-12T12:24:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedd7fa640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdecc0678c0 0x7fdecc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdedd7fa640 1 -- 172.21.15.3:0/4100496628 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5709+0+0 (secure 0 0 0) 0x7fded40852f0 con 0x7fdee00b6d60 2024-09-12T12:24:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.623+0000 7fdee4983640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdecc0678c0 0x7fdecc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.627+0000 7fdee4983640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdecc0678c0 0x7fdecc069d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fded00af000 tx=0x7fded00aa000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.627+0000 7fdedd7fa640 1 -- 172.21.15.3:0/4100496628 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fded4055ec0 con 0x7fdee00b6d60 2024-09-12T12:24:29.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.791+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdee00c7e10 con 0x7fdee00b6d60 2024-09-12T12:24:29.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.795+0000 7fdedd7fa640 1 -- 172.21.15.3:0/4100496628 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16660 (secure 0 0 0) 0x7fded40557c0 con 0x7fdee00b6d60 2024-09-12T12:24:29.799 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:29.799 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":79,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:23:59.375425+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:55.370943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:56.721996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:24:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdecc0678c0 msgr2=0x7fdecc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdecc0678c0 0x7fdecc069d80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fded00af000 tx=0x7fded00aa000 comp rx=0 tx=0).stop 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee00b6d60 msgr2=0x7fdee00b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee00b6d60 0x7fdee00b5370 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fded400d4a0 tx=0x7fded400d960 comp rx=0 tx=0).stop 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 shutdown_connections 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdecc0678c0 0x7fdecc069d80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdee0155de0 0x7fdee00cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdee00b6d60 0x7fdee00b5370 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 --2- 172.21.15.3:0/4100496628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdec80039c0 0x7fdee01557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 >> 172.21.15.3:0/4100496628 conn(0x7fdee0067a00 msgr2=0x7fdee00c8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 shutdown_connections 2024-09-12T12:24:29.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.799+0000 7fdee6be7640 1 -- 172.21.15.3:0/4100496628 wait complete. 2024-09-12T12:24:29.819 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:24:29.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:29.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5092a5b640 1 Processor -- start 2024-09-12T12:24:29.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5092a5b640 1 -- start start 2024-09-12T12:24:29.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5092a5b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c0d05b0 0x7f508c0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5092a5b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f508c0ae250 0x7f508c0ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5092a5b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f508c15c630 con 0x7f508c0b42f0 2024-09-12T12:24:29.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5092a5b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f508c0d0ed0 con 0x7f508c0ae250 2024-09-12T12:24:29.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5092a5b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f508c0ad0c0 con 0x7f508c0d05b0 2024-09-12T12:24:29.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5090ff8640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f508c0b42f0 0x7f508c0b47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40486/0 (socket says 172.21.15.3:40486) 2024-09-12T12:24:29.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f5090ff8640 1 -- 172.21.15.3:0/2404576456 learned_addr learned my addr 172.21.15.3:0/2404576456 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:29.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508b7fe640 1 --2- 172.21.15.3:0/2404576456 >> v2:172.21.15.142:3300/0 conn(0x7f508c0ae250 0x7f508c0ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508bfff640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c0d05b0 0x7f508c0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 672153789 0 0) 0x7f50840014a0 con 0x7f508c0b42f0 2024-09-12T12:24:29.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5074003530 con 0x7f508c0b42f0 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3576091445 0 0) 0x7f5074003530 con 0x7f508c0b42f0 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c0d05b0 msgr2=0x7f508c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c0d05b0 0x7f508c0d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 >> v2:172.21.15.142:3300/0 conn(0x7f508c0ae250 msgr2=0x7f508c0ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 --2- 172.21.15.3:0/2404576456 >> v2:172.21.15.142:3300/0 conn(0x7f508c0ae250 0x7f508c0ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c0ad3a0 con 0x7f508c0b42f0 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f5084001bd0 con 0x7f508c0b42f0 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 >> v1:172.21.15.3:6789/0 conn(0x7f508c0b42f0 legacy=0x7f508c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f5074003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5074004810 0x7f5074004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c0d05b0 0x7f50740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0d0ed0 con 0x7f50740039c0 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0ad0c0 con 0x7f5074004810 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f507400b9d0 con 0x7f508c0d05b0 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508bfff640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f5074003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508bfff640 1 -- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c0d05b0 msgr2=0x7f50740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508bfff640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c0d05b0 0x7f50740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508bfff640 1 -- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5074004810 msgr2=0x7f5074004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508b7fe640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5074004810 0x7f5074004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508bfff640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5074004810 0x7f5074004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.939+0000 7f508bfff640 1 -- 172.21.15.3:0/2404576456 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5074003530 con 0x7f50740039c0 2024-09-12T12:24:29.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f508bfff640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f5074003de0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f507c009ae0 tx=0x7f507c01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d87130087e07f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0b2070 con 0x7f50740039c0 2024-09-12T12:24:29.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c0a5570 con 0x7f50740039c0 2024-09-12T12:24:29.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f508affd640 1 -- 172.21.15.3:0/2404576456 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0096b0 con 0x7f50740039c0 2024-09-12T12:24:29.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- 172.21.15.3:0/2404576456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 msgr2=0x7f5074003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 --2- 172.21.15.3:0/2404576456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f5074003de0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f507c009ae0 tx=0x7f507c01ca80 comp rx=0 tx=0).stop 2024-09-12T12:24:29.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5090ff8640 1 -- 172.21.15.3:0/2404576456 reap_dead start 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- 172.21.15.3:0/2404576456 shutdown_connections 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- 172.21.15.3:0/2404576456 >> 172.21.15.3:0/2404576456 conn(0x7f508c067e00 msgr2=0x7f508c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- 172.21.15.3:0/2404576456 shutdown_connections 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- 172.21.15.3:0/2404576456 wait complete. 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 Processor -- start 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- start start 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f508c162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c1587c0 0x7f508c14c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c06e360 0x7f508c06e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0aed70 con 0x7f50740039c0 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0aebf0 con 0x7f508c1587c0 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f5092a5b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f508c1559b0 con 0x7f508c06e360 2024-09-12T12:24:29.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.943+0000 7f508bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f508c162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f508c162140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41670/0 (socket says 172.21.15.3:41670) 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 -- 172.21.15.3:0/2576973148 learned_addr learned my addr 172.21.15.3:0/2576973148 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508b7fe640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c1587c0 0x7f508c14c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5090ff8640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c06e360 0x7f508c06e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 -- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c06e360 msgr2=0x7f508c06e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c06e360 0x7f508c06e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 -- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c1587c0 msgr2=0x7f508c14c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c1587c0 0x7f508c14c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 -- 172.21.15.3:0/2576973148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c158c40 con 0x7f50740039c0 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508bfff640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f508c162140 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f507c01c8b0 tx=0x7f5074005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5088ff9640 1 -- 172.21.15.3:0/2576973148 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0b2040 con 0x7f50740039c0 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5088ff9640 1 -- 172.21.15.3:0/2576973148 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c0a5d20 con 0x7f50740039c0 2024-09-12T12:24:29.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c1550c0 con 0x7f50740039c0 2024-09-12T12:24:29.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5088ff9640 1 -- 172.21.15.3:0/2576973148 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c004220 con 0x7f50740039c0 2024-09-12T12:24:29.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c0cd770 con 0x7f50740039c0 2024-09-12T12:24:29.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c1583a0 con 0x7f50740039c0 2024-09-12T12:24:29.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5088ff9640 1 -- 172.21.15.3:0/2576973148 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f507c003620 con 0x7f50740039c0 2024-09-12T12:24:29.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5088ff9640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50780678c0 0x7f5078069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f5088ff9640 1 -- 172.21.15.3:0/2576973148 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5738+0+0 (secure 0 0 0) 0x7f507c11f920 con 0x7f50740039c0 2024-09-12T12:24:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508b7fe640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50780678c0 0x7f5078069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:29.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.947+0000 7f508b7fe640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50780678c0 0x7f5078069d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f508c164940 tx=0x7f5080008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:29.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:29.951+0000 7f5088ff9640 1 -- 172.21.15.3:0/2576973148 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f508c1583a0 con 0x7f50740039c0 2024-09-12T12:24:30.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.151+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f508c15c830 con 0x7f50740039c0 2024-09-12T12:24:30.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5088ff9640 1 -- 172.21.15.3:0/2576973148 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5222 (secure 0 0 0) 0x7f508c158c40 con 0x7f50740039c0 2024-09-12T12:24:30.157 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:30.158 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:57.730276+0000","modified":"2024-09-12T12:24:00.762240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2940169617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2940169617},{"type":"v1","addr":"172.21.15.3:6835","nonce":2940169617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-09-12T12:24:30.158 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 54 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50780678c0 msgr2=0x7f5078069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50780678c0 0x7f5078069d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f508c164940 tx=0x7f5080008040 comp rx=0 tx=0).stop 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 msgr2=0x7f508c162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f508c162140 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f507c01c8b0 tx=0x7f5074005840 comp rx=0 tx=0).stop 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 shutdown_connections 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50780678c0 0x7f5078069d80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f508c06e360 0x7f508c06e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f508c1587c0 0x7f508c14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 --2- 172.21.15.3:0/2576973148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50740039c0 0x7f508c162140 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 >> 172.21.15.3:0/2576973148 conn(0x7f508c067e00 msgr2=0x7f508c072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:30.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 shutdown_connections 2024-09-12T12:24:30.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.155+0000 7f5092a5b640 1 -- 172.21.15.3:0/2576973148 wait complete. 2024-09-12T12:24:30.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:30.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f37c07640 1 Processor -- start 2024-09-12T12:24:30.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f37c07640 1 -- start start 2024-09-12T12:24:30.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f37c07640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f300d1190 0x7f1f300d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f37c07640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f300d30b0 con 0x7f1f300d1570 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f37c07640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f300d3390 con 0x7f1f300b7740 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f37c07640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1f300d3cc0 con 0x7f1f300d1190 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f359a3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1f300d1190 0x7f1f300d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f351a2640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1f300d1570 0x7f1f300cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40494/0 (socket says 172.21.15.3:40494) 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f351a2640 1 -- 172.21.15.3:0/3833158359 learned_addr learned my addr 172.21.15.3:0/3833158359 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f349a1640 1 -- 172.21.15.3:0/3833158359 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636221546 0 0) 0x7f1f300d30b0 con 0x7f1f300d1570 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f349a1640 1 -- 172.21.15.3:0/3833158359 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f1c003530 con 0x7f1f300d1570 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f359a3640 1 -- 172.21.15.3:0/3833158359 >> v1:172.21.15.142:6789/0 conn(0x7f1f300b7740 legacy=0x7f1f300b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f359a3640 1 -- 172.21.15.3:0/3833158359 >> v1:172.21.15.3:6789/0 conn(0x7f1f300d1570 legacy=0x7f1f300cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.287+0000 7f1f359a3640 1 -- 172.21.15.3:0/3833158359 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f300d3670 con 0x7f1f300d1190 2024-09-12T12:24:30.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f359a3640 1 --2- 172.21.15.3:0/3833158359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f300d4150 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1f200097c0 tx=0x7f1f2001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e036b3bc4dcce077 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f349a1640 1 -- 172.21.15.3:0/3833158359 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f200b5070 con 0x7f1f300d1190 2024-09-12T12:24:30.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f349a1640 1 -- 172.21.15.3:0/3833158359 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f200a8950 con 0x7f1f300d1190 2024-09-12T12:24:30.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f349a1640 1 -- 172.21.15.3:0/3833158359 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f200a8d00 con 0x7f1f300d1190 2024-09-12T12:24:30.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 -- 172.21.15.3:0/3833158359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 msgr2=0x7f1f300d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 --2- 172.21.15.3:0/3833158359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f300d4150 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f1f200097c0 tx=0x7f1f2001ec50 comp rx=0 tx=0).stop 2024-09-12T12:24:30.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 -- 172.21.15.3:0/3833158359 shutdown_connections 2024-09-12T12:24:30.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 --2- 172.21.15.3:0/3833158359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f300d4150 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 -- 172.21.15.3:0/3833158359 >> 172.21.15.3:0/3833158359 conn(0x7f1f30067a00 msgr2=0x7f1f30067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:30.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 -- 172.21.15.3:0/3833158359 shutdown_connections 2024-09-12T12:24:30.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 -- 172.21.15.3:0/3833158359 wait complete. 2024-09-12T12:24:30.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 Processor -- start 2024-09-12T12:24:30.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 -- start start 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f300b7740 0x7f1f30154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f301555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f300d1570 0x7f1f30160df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.291+0000 7f1f37c07640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1f300d2cd0 con 0x7f1f300d1190 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f37c07640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f300d1a00 con 0x7f1f300b7740 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f37c07640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f300d2e50 con 0x7f1f300d1570 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f301555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f301555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41688/0 (socket says 172.21.15.3:41688) 2024-09-12T12:24:30.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 -- 172.21.15.3:0/484592745 learned_addr learned my addr 172.21.15.3:0/484592745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 -- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f300d1570 msgr2=0x7f1f30160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f300d1570 0x7f1f30160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 -- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f300b7740 msgr2=0x7f1f30154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f300b7740 0x7f1f30154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 -- 172.21.15.3:0/484592745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f30155fd0 con 0x7f1f300d1190 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f351a2640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f301555f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c3c0 tx=0x7f1f2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f267fc640 1 -- 172.21.15.3:0/484592745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c017070 con 0x7f1f300d1190 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f267fc640 1 -- 172.21.15.3:0/484592745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2c005cf0 con 0x7f1f300d1190 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f301562b0 con 0x7f1f300d1190 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f267fc640 1 -- 172.21.15.3:0/484592745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0072f0 con 0x7f1f300d1190 2024-09-12T12:24:30.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f30156870 con 0x7f1f300d1190 2024-09-12T12:24:30.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f300ba4c0 con 0x7f1f300d1190 2024-09-12T12:24:30.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f267fc640 1 -- 172.21.15.3:0/484592745 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f2c01a070 con 0x7f1f300d1190 2024-09-12T12:24:30.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f267fc640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f140678f0 0x7f1f14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f267fc640 1 -- 172.21.15.3:0/484592745 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5738+0+0 (secure 0 0 0) 0x7f1f2c084fc0 con 0x7f1f300d1190 2024-09-12T12:24:30.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f359a3640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f140678f0 0x7f1f14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.295+0000 7f1f359a3640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f140678f0 0x7f1f14069db0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1f200b2000 tx=0x7f1f20004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.299+0000 7f1f267fc640 1 -- 172.21.15.3:0/484592745 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1f2c055b70 con 0x7f1f300d1190 2024-09-12T12:24:30.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.499+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f3015d670 con 0x7f1f300d1190 2024-09-12T12:24:30.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f267fc640 1 -- 172.21.15.3:0/484592745 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5222 (secure 0 0 0) 0x7f1f2c055470 con 0x7f1f300d1190 2024-09-12T12:24:30.506 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:30.506 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":54,"btime":"2024-09-12T12:24:01:391708+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:23:57.730276+0000","modified":"2024-09-12T12:24:00.762240+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2940169617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2940169617},{"type":"v1","addr":"172.21.15.3:6835","nonce":2940169617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-09-12T12:24:30.507 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 54 2024-09-12T12:24:30.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f140678f0 msgr2=0x7f1f14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f140678f0 0x7f1f14069db0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1f200b2000 tx=0x7f1f20004740 comp rx=0 tx=0).stop 2024-09-12T12:24:30.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 msgr2=0x7f1f301555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f301555f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c3c0 tx=0x7f1f2c00c880 comp rx=0 tx=0).stop 2024-09-12T12:24:30.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 shutdown_connections 2024-09-12T12:24:30.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f140678f0 0x7f1f14069db0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f300d1570 0x7f1f30160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f300d1190 0x7f1f301555f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 --2- 172.21.15.3:0/484592745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f300b7740 0x7f1f30154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 >> 172.21.15.3:0/484592745 conn(0x7f1f30067a00 msgr2=0x7f1f3015c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:30.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 shutdown_connections 2024-09-12T12:24:30.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.503+0000 7f1f37c07640 1 -- 172.21.15.3:0/484592745 wait complete. 2024-09-12T12:24:30.525 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:24:30.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.631+0000 7f67dc1e2640 1 Processor -- start 2024-09-12T12:24:30.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.631+0000 7f67dc1e2640 1 -- start start 2024-09-12T12:24:30.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67dc1e2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67d415c710 0x7f67d40b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67dc1e2640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67d40ae880 0x7f67d40aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67dc1e2640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d40b5000 con 0x7f67d4073cb0 2024-09-12T12:24:30.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67dc1e2640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f67d4074090 con 0x7f67d415c710 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67dc1e2640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f67d40b37c0 con 0x7f67d40ae880 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67d415c710 0x7f67d40b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f67d415c710 0x7f67d40b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41704/0 (socket says 172.21.15.3:41704) 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 -- 172.21.15.3:0/3462330378 learned_addr learned my addr 172.21.15.3:0/3462330378 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67da77f640 1 --2- 172.21.15.3:0/3462330378 >> v2:172.21.15.142:3300/0 conn(0x7f67d40ae880 0x7f67d40aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 -- 172.21.15.3:0/3462330378 >> v2:172.21.15.142:3300/0 conn(0x7f67d40ae880 msgr2=0x7f67d40aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 --2- 172.21.15.3:0/3462330378 >> v2:172.21.15.142:3300/0 conn(0x7f67d40ae880 0x7f67d40aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 -- 172.21.15.3:0/3462330378 >> v1:172.21.15.142:6789/0 conn(0x7f67d4073cb0 legacy=0x7f67d40adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:30.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 -- 172.21.15.3:0/3462330378 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d40b3aa0 con 0x7f67d415c710 2024-09-12T12:24:30.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d977d640 1 --2- 172.21.15.3:0/3462330378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d40b30e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f67d40b46c0 tx=0x7f67c401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd435510ea01bd5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d8f7c640 1 -- 172.21.15.3:0/3462330378 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b1070 con 0x7f67d415c710 2024-09-12T12:24:30.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d8f7c640 1 -- 172.21.15.3:0/3462330378 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c40ac630 con 0x7f67d415c710 2024-09-12T12:24:30.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.635+0000 7f67d8f7c640 1 -- 172.21.15.3:0/3462330378 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40a8050 con 0x7f67d415c710 2024-09-12T12:24:30.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- 172.21.15.3:0/3462330378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 msgr2=0x7f67d40b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/3462330378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d40b30e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f67d40b46c0 tx=0x7f67c401c4e0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- 172.21.15.3:0/3462330378 shutdown_connections 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/3462330378 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d40b30e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/3462330378 >> v2:172.21.15.142:3300/0 conn(0x7f67d40ae880 0x7f67d40aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- 172.21.15.3:0/3462330378 >> 172.21.15.3:0/3462330378 conn(0x7f67d4067e00 msgr2=0x7f67d4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- 172.21.15.3:0/3462330378 shutdown_connections 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- 172.21.15.3:0/3462330378 wait complete. 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 Processor -- start 2024-09-12T12:24:30.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- start start 2024-09-12T12:24:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67d4073cb0 0x7f67d4156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d40ae880 0x7f67d41555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4168f40 con 0x7f67d415c710 2024-09-12T12:24:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4168dc0 con 0x7f67d40ae880 2024-09-12T12:24:30.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67d41690c0 con 0x7f67d4073cb0 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d4152eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41710/0 (socket says 172.21.15.3:41710) 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 -- 172.21.15.3:0/506519832 learned_addr learned my addr 172.21.15.3:0/506519832 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67d977d640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d40ae880 0x7f67d41555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67d9f7e640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67d4073cb0 0x7f67d4156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 -- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67d4073cb0 msgr2=0x7f67d4156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67d4073cb0 0x7f67d4156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 -- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d40ae880 msgr2=0x7f67d41555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d40ae880 0x7f67d41555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 -- 172.21.15.3:0/506519832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d41547e0 con 0x7f67d415c710 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67d977d640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d40ae880 0x7f67d41555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67da77f640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d4152eb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00cd70 tx=0x7f67cc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67ca7fc640 1 -- 172.21.15.3:0/506519832 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc003ba0 con 0x7f67d415c710 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67ca7fc640 1 -- 172.21.15.3:0/506519832 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67cc003d40 con 0x7f67d415c710 2024-09-12T12:24:30.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67ca7fc640 1 -- 172.21.15.3:0/506519832 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc004240 con 0x7f67d415c710 2024-09-12T12:24:30.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d4154ac0 con 0x7f67d415c710 2024-09-12T12:24:30.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.639+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d4155080 con 0x7f67d415c710 2024-09-12T12:24:30.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.643+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d40aecb0 con 0x7f67d415c710 2024-09-12T12:24:30.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.643+0000 7f67ca7fc640 1 -- 172.21.15.3:0/506519832 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67cc0043e0 con 0x7f67d415c710 2024-09-12T12:24:30.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.643+0000 7f67ca7fc640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b80678c0 0x7f67b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.643+0000 7f67ca7fc640 1 -- 172.21.15.3:0/506519832 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5738+0+0 (secure 0 0 0) 0x7f67cc086700 con 0x7f67d415c710 2024-09-12T12:24:30.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.643+0000 7f67d9f7e640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b80678c0 0x7f67b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.643+0000 7f67d9f7e640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b80678c0 0x7f67b8069d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67d4155cf0 tx=0x7f67d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.647+0000 7f67ca7fc640 1 -- 172.21.15.3:0/506519832 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f67cc01d340 con 0x7f67d415c710 2024-09-12T12:24:30.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.811+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67d406bd70 con 0x7f67d415c710 2024-09-12T12:24:30.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.811+0000 7f67ca7fc640 1 -- 172.21.15.3:0/506519832 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16710 (secure 0 0 0) 0x7f67cc0572b0 con 0x7f67d415c710 2024-09-12T12:24:30.817 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:30.817 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":80,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:24:29.861857+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:23:55.370943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-12T12:23:56.721996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.27","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:24:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b80678c0 msgr2=0x7f67b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b80678c0 0x7f67b8069d80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67d4155cf0 tx=0x7f67d0009040 comp rx=0 tx=0).stop 2024-09-12T12:24:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 msgr2=0x7f67d4152eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d4152eb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00cd70 tx=0x7f67cc00a660 comp rx=0 tx=0).stop 2024-09-12T12:24:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 shutdown_connections 2024-09-12T12:24:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b80678c0 0x7f67b8069d80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67d415c710 0x7f67d4152eb0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d40ae880 0x7f67d41555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 --2- 172.21.15.3:0/506519832 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67d4073cb0 0x7f67d4156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 >> 172.21.15.3:0/506519832 conn(0x7f67d4067e00 msgr2=0x7f67d40681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 shutdown_connections 2024-09-12T12:24:30.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.815+0000 7f67dc1e2640 1 -- 172.21.15.3:0/506519832 wait complete. 2024-09-12T12:24:30.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T12:24:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa753d8c640 1 Processor -- start 2024-09-12T12:24:30.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa753d8c640 1 -- start start 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa753d8c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa74c070c30 0x7fa74c0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa753d8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 0x7fa74c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa753d8c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa74c0afb40 con 0x7fa74c15f990 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa753d8c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa74c0b0710 con 0x7fa74c070c30 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa753d8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c0afde0 con 0x7fa74c0d08c0 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa751b28640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa74c15f990 0x7fa74c070190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40506/0 (socket says 172.21.15.3:40506) 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa751b28640 1 -- 172.21.15.3:0/4268461417 learned_addr learned my addr 172.21.15.3:0/4268461417 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa752329640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 0x7fa74c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa751327640 1 --2- 172.21.15.3:0/4268461417 >> v2:172.21.15.142:3300/0 conn(0x7fa74c070c30 0x7fa74c0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1289170558 0 0) 0x7fa74c0afb40 con 0x7fa74c15f990 2024-09-12T12:24:30.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa738003530 con 0x7fa74c15f990 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 996391100 0 0) 0x7fa738003530 con 0x7fa74c15f990 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 msgr2=0x7fa74c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 0x7fa74c0b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 >> v2:172.21.15.142:3300/0 conn(0x7fa74c070c30 msgr2=0x7fa74c0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 --2- 172.21.15.3:0/4268461417 >> v2:172.21.15.142:3300/0 conn(0x7fa74c070c30 0x7fa74c0d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c0b00c0 con 0x7fa74c15f990 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fa73c001680 con 0x7fa74c15f990 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 >> v1:172.21.15.3:6789/0 conn(0x7fa74c15f990 legacy=0x7fa74c070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 0x7fa7380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa738003f40 0x7fa738004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa738004d50 0x7fa7380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c0b0710 con 0x7fa738004d50 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c0afde0 con 0x7fa738003f40 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa73800ba20 con 0x7fa74c0d08c0 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa752329640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa738004d50 0x7fa7380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa751b28640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 0x7fa7380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa751327640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa738003f40 0x7fa738004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa752329640 1 -- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 msgr2=0x7fa7380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa752329640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c0d08c0 0x7fa7380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa752329640 1 -- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa738003f40 msgr2=0x7fa738004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa752329640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa738003f40 0x7fa738004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa752329640 1 -- 172.21.15.3:0/4268461417 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738003530 con 0x7fa738004d50 2024-09-12T12:24:30.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.951+0000 7fa751327640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa738003f40 0x7fa738004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa752329640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa738004d50 0x7fa7380051d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e170 tx=0x7fa74401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b20594fc519986 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7440b2070 con 0x7fa738004d50 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7440a5b60 con 0x7fa738004d50 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa750b26640 1 -- 172.21.15.3:0/4268461417 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744009c10 con 0x7fa738004d50 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- 172.21.15.3:0/4268461417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa738004d50 msgr2=0x7fa7380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 --2- 172.21.15.3:0/4268461417 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa738004d50 0x7fa7380051d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e170 tx=0x7fa74401cc60 comp rx=0 tx=0).stop 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa752329640 1 -- 172.21.15.3:0/4268461417 reap_dead start 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- 172.21.15.3:0/4268461417 shutdown_connections 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- 172.21.15.3:0/4268461417 >> 172.21.15.3:0/4268461417 conn(0x7fa74c0b3ea0 msgr2=0x7fa74c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- 172.21.15.3:0/4268461417 shutdown_connections 2024-09-12T12:24:30.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- 172.21.15.3:0/4268461417 wait complete. 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 Processor -- start 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- start start 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa74c1559d0 0x7fa74c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c06d8c0 0x7fa74c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74c0cab60 0x7fa74c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c15fef0 con 0x7fa74c0cab60 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c15fd70 con 0x7fa74c1559d0 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa753d8c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa74c0cccb0 con 0x7fa74c06d8c0 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa752329640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74c0cab60 0x7fa74c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa752329640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74c0cab60 0x7fa74c06c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41720/0 (socket says 172.21.15.3:41720) 2024-09-12T12:24:30.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.955+0000 7fa752329640 1 -- 172.21.15.3:0/872206040 learned_addr learned my addr 172.21.15.3:0/872206040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa751327640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c06d8c0 0x7fa74c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa751b28640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa74c1559d0 0x7fa74c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa752329640 1 -- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c06d8c0 msgr2=0x7fa74c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa752329640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c06d8c0 0x7fa74c0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa752329640 1 -- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa74c1559d0 msgr2=0x7fa74c06d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa752329640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa74c1559d0 0x7fa74c06d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa752329640 1 -- 172.21.15.3:0/872206040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c0c9580 con 0x7fa74c0cab60 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa752329640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74c0cab60 0x7fa74c06c4f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e170 tx=0x7fa744009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa7427fc640 1 -- 172.21.15.3:0/872206040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7440b2040 con 0x7fa74c0cab60 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa7427fc640 1 -- 172.21.15.3:0/872206040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7440a5960 con 0x7fa74c0cab60 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c0c9860 con 0x7fa74c0cab60 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa7427fc640 1 -- 172.21.15.3:0/872206040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744009c10 con 0x7fa74c0cab60 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c0c9e20 con 0x7fa74c0cab60 2024-09-12T12:24:30.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa7427fc640 1 -- 172.21.15.3:0/872206040 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa7440be020 con 0x7fa74c0cab60 2024-09-12T12:24:30.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74c0d0380 con 0x7fa74c0cab60 2024-09-12T12:24:30.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa7427fc640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7300678c0 0x7fa730069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:30.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa7427fc640 1 -- 172.21.15.3:0/872206040 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5738+0+0 (secure 0 0 0) 0x7fa74411fcf0 con 0x7fa74c0cab60 2024-09-12T12:24:30.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa751b28640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7300678c0 0x7fa730069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:30.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.959+0000 7fa751b28640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7300678c0 0x7fa730069d80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa73c005400 tx=0x7fa73c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:30.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:30.963+0000 7fa7427fc640 1 -- 172.21.15.3:0/872206040 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa7440c5020 con 0x7fa74c0cab60 2024-09-12T12:24:31.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.167+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa74c0ceb70 con 0x7fa74c0cab60 2024-09-12T12:24:31.173 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:24:31.171+0000 7f0a5d435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:24:31.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.171+0000 7fa7427fc640 1 -- 172.21.15.3:0/872206040 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) ==== 161+0+0 (secure 0 0 0) 0x7fa7440169f0 con 0x7fa74c0cab60 2024-09-12T12:24:31.176 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:24:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7300678c0 msgr2=0x7fa730069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7300678c0 0x7fa730069d80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa73c005400 tx=0x7fa73c009040 comp rx=0 tx=0).stop 2024-09-12T12:24:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74c0cab60 msgr2=0x7fa74c06c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:31.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74c0cab60 0x7fa74c06c4f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e170 tx=0x7fa744009750 comp rx=0 tx=0).stop 2024-09-12T12:24:31.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 shutdown_connections 2024-09-12T12:24:31.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa7300678c0 0x7fa730069d80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa74c0cab60 0x7fa74c06c4f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa74c06d8c0 0x7fa74c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 --2- 172.21.15.3:0/872206040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa74c1559d0 0x7fa74c06d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 >> 172.21.15.3:0/872206040 conn(0x7fa74c0b3ea0 msgr2=0x7fa74c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 shutdown_connections 2024-09-12T12:24:31.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.175+0000 7fa753d8c640 1 -- 172.21.15.3:0/872206040 wait complete. 2024-09-12T12:24:31.182 INFO:tasks.ceph.mds.a.smithi003.stderr: -1640> 2024-09-12T12:24:00.332+0000 7fceeb5c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:24:31.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:24:31.214 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:24:31.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.311+0000 7f8efb716640 1 Processor -- start 2024-09-12T12:24:31.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.311+0000 7f8efb716640 1 -- start start 2024-09-12T12:24:31.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8efb716640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ef415c210 0x7f8ef406e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:31.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8efb716640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8ef40adf60 0x7f8ef415feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:31.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8efb716640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef4160600 con 0x7f8ef40d05c0 2024-09-12T12:24:31.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8efb716640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef41608e0 con 0x7f8ef415c210 2024-09-12T12:24:31.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8efb716640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef4160bc0 con 0x7f8ef40adf60 2024-09-12T12:24:31.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ef415c210 0x7f8ef406e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:31.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ef415c210 0x7f8ef406e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41766/0 (socket says 172.21.15.3:41766) 2024-09-12T12:24:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 -- 172.21.15.3:0/1482477256 learned_addr learned my addr 172.21.15.3:0/1482477256 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef8cb1640 1 --2- 172.21.15.3:0/1482477256 >> v2:172.21.15.142:3300/0 conn(0x7f8ef40adf60 0x7f8ef415feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 -- 172.21.15.3:0/1482477256 >> v2:172.21.15.142:3300/0 conn(0x7f8ef40adf60 msgr2=0x7f8ef415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 --2- 172.21.15.3:0/1482477256 >> v2:172.21.15.142:3300/0 conn(0x7f8ef40adf60 0x7f8ef415feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 -- 172.21.15.3:0/1482477256 >> v1:172.21.15.3:6789/0 conn(0x7f8ef40d05c0 legacy=0x7f8ef40ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 -- 172.21.15.3:0/1482477256 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef40cec00 con 0x7f8ef415c210 2024-09-12T12:24:31.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ee7fff640 1 -- 172.21.15.3:0/1482477256 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094962535 0 0) 0x7f8ef4160600 con 0x7f8ef40d05c0 2024-09-12T12:24:31.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ef94b2640 1 --2- 172.21.15.3:0/1482477256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef406e450 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40b1d20 tx=0x7f8ef001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d400c9a62f6da5ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:31.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ee7fff640 1 -- 172.21.15.3:0/1482477256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00b1070 con 0x7f8ef415c210 2024-09-12T12:24:31.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ee7fff640 1 -- 172.21.15.3:0/1482477256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef00ac600 con 0x7f8ef415c210 2024-09-12T12:24:31.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.315+0000 7f8ee7fff640 1 -- 172.21.15.3:0/1482477256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00ac9d0 con 0x7f8ef415c210 2024-09-12T12:24:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/1482477256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 msgr2=0x7f8ef406e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 --2- 172.21.15.3:0/1482477256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef406e450 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40b1d20 tx=0x7f8ef001c490 comp rx=0 tx=0).stop 2024-09-12T12:24:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/1482477256 shutdown_connections 2024-09-12T12:24:31.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 --2- 172.21.15.3:0/1482477256 >> v2:172.21.15.142:3300/0 conn(0x7f8ef40adf60 0x7f8ef415feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 --2- 172.21.15.3:0/1482477256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef406e450 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/1482477256 >> 172.21.15.3:0/1482477256 conn(0x7f8ef4067e00 msgr2=0x7f8ef4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/1482477256 shutdown_connections 2024-09-12T12:24:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/1482477256 wait complete. 2024-09-12T12:24:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 Processor -- start 2024-09-12T12:24:31.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- start start 2024-09-12T12:24:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ef40adf60 0x7f8ef415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef40d05c0 0x7f8ef413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef4140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef41695e0 con 0x7f8ef415c210 2024-09-12T12:24:31.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4169460 con 0x7f8ef40d05c0 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef4169760 con 0x7f8ef40adf60 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef4140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef4140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41776/0 (socket says 172.21.15.3:41776) 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 -- 172.21.15.3:0/285737352 learned_addr learned my addr 172.21.15.3:0/285737352 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef94b2640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ef40adf60 0x7f8ef415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef8cb1640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef40d05c0 0x7f8ef413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 -- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ef40adf60 msgr2=0x7f8ef415b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ef40adf60 0x7f8ef415b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 -- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef40d05c0 msgr2=0x7f8ef413ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef40d05c0 0x7f8ef413ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 -- 172.21.15.3:0/285737352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef40b1a50 con 0x7f8ef415c210 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef8cb1640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef40d05c0 0x7f8ef413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:31.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ef9cb3640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef4140500 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c780 tx=0x7f8eec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ee5ffb640 1 -- 172.21.15.3:0/285737352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec003db0 con 0x7f8ef415c210 2024-09-12T12:24:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ee5ffb640 1 -- 172.21.15.3:0/285737352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec005070 con 0x7f8ef415c210 2024-09-12T12:24:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef40b1d30 con 0x7f8ef415c210 2024-09-12T12:24:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8ee5ffb640 1 -- 172.21.15.3:0/285737352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec005460 con 0x7f8ef415c210 2024-09-12T12:24:31.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef40b22f0 con 0x7f8ef415c210 2024-09-12T12:24:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.319+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef406e450 con 0x7f8ef415c210 2024-09-12T12:24:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.323+0000 7f8ee5ffb640 1 -- 172.21.15.3:0/285737352 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8eec005600 con 0x7f8ef415c210 2024-09-12T12:24:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.323+0000 7f8ee5ffb640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ed80678c0 0x7f8ed8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.323+0000 7f8ee5ffb640 1 -- 172.21.15.3:0/285737352 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5824+0+0 (secure 0 0 0) 0x7f8eec085c60 con 0x7f8ef415c210 2024-09-12T12:24:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.323+0000 7f8ef94b2640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ed80678c0 0x7f8ed8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:31.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.323+0000 7f8ef94b2640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ed80678c0 0x7f8ed8069d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00ae000 tx=0x7f8ef0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:31.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.327+0000 7f8ee5ffb640 1 -- 172.21.15.3:0/285737352 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8eec0567c0 con 0x7f8ef415c210 2024-09-12T12:24:31.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:31.519+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8ef4072100 con 0x7f8ef415c210 2024-09-12T12:24:32.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.175+0000 7f8ee5ffb640 1 -- 172.21.15.3:0/285737352 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) ==== 104+0+0 (secure 0 0 0) 0x7f8eec0560c0 con 0x7f8ef415c210 2024-09-12T12:24:32.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ed80678c0 msgr2=0x7f8ed8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ed80678c0 0x7f8ed8069d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00ae000 tx=0x7f8ef0008040 comp rx=0 tx=0).stop 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 msgr2=0x7f8ef4140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef4140500 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00c780 tx=0x7f8eec00cc40 comp rx=0 tx=0).stop 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 shutdown_connections 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ed80678c0 0x7f8ed8069d80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ef415c210 0x7f8ef4140500 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ef40d05c0 0x7f8ef413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 --2- 172.21.15.3:0/285737352 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ef40adf60 0x7f8ef415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 >> 172.21.15.3:0/285737352 conn(0x7f8ef4067e00 msgr2=0x7f8ef406dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 shutdown_connections 2024-09-12T12:24:32.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.179+0000 7f8efb716640 1 -- 172.21.15.3:0/285737352 wait complete. 2024-09-12T12:24:32.206 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:24:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.379+0000 7fa580f62640 1 Processor -- start 2024-09-12T12:24:32.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.379+0000 7fa580f62640 1 -- start start 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa580f62640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa580f62640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa57c06d8f0 0x7fa57c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa580f62640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c0ad380 con 0x7fa57c0b4720 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa580f62640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa57c0b4b00 con 0x7fa57c06d8f0 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa580f62640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c06ddd0 con 0x7fa57c0ae130 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa57c06d8f0 0x7fa57c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa57c06d8f0 0x7fa57c0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41786/0 (socket says 172.21.15.3:41786) 2024-09-12T12:24:32.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 -- 172.21.15.3:0/2616585751 learned_addr learned my addr 172.21.15.3:0/2616585751 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 -- 172.21.15.3:0/2616585751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 msgr2=0x7fa57c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa579d9b640 1 --2- 172.21.15.3:0/2616585751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 --2- 172.21.15.3:0/2616585751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c15b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 -- 172.21.15.3:0/2616585751 >> v1:172.21.15.142:6789/0 conn(0x7fa57c0b4720 legacy=0x7fa57c0ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:32.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 -- 172.21.15.3:0/2616585751 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c06e0b0 con 0x7fa57c06d8f0 2024-09-12T12:24:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57ad9d640 1 --2- 172.21.15.3:0/2616585751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c0b30e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa574009870 tx=0x7fa57401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0543cfbf39104d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57959a640 1 -- 172.21.15.3:0/2616585751 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740b0070 con 0x7fa57c06d8f0 2024-09-12T12:24:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57959a640 1 -- 172.21.15.3:0/2616585751 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa574004230 con 0x7fa57c06d8f0 2024-09-12T12:24:32.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.383+0000 7fa57959a640 1 -- 172.21.15.3:0/2616585751 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574003620 con 0x7fa57c06d8f0 2024-09-12T12:24:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- 172.21.15.3:0/2616585751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 msgr2=0x7fa57c0b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 --2- 172.21.15.3:0/2616585751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c0b30e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa574009870 tx=0x7fa57401c520 comp rx=0 tx=0).stop 2024-09-12T12:24:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- 172.21.15.3:0/2616585751 shutdown_connections 2024-09-12T12:24:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 --2- 172.21.15.3:0/2616585751 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c0b30e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 --2- 172.21.15.3:0/2616585751 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- 172.21.15.3:0/2616585751 >> 172.21.15.3:0/2616585751 conn(0x7fa57c067bd0 msgr2=0x7fa57c067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:32.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- 172.21.15.3:0/2616585751 shutdown_connections 2024-09-12T12:24:32.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- 172.21.15.3:0/2616585751 wait complete. 2024-09-12T12:24:32.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 Processor -- start 2024-09-12T12:24:32.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- start start 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c09a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa57c0b4720 0x7fa57c09cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c06ee70 con 0x7fa57c06d8f0 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c0b3620 con 0x7fa57c0b4720 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c14b740 con 0x7fa57c0ae130 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c09a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c09a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41790/0 (socket says 172.21.15.3:41790) 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 -- 172.21.15.3:0/2297453235 learned_addr learned my addr 172.21.15.3:0/2297453235 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa579d9b640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:32.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57ad9d640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa57c0b4720 0x7fa57c09cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 -- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 msgr2=0x7fa57c14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 -- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa57c0b4720 msgr2=0x7fa57c09cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa57c0b4720 0x7fa57c09cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 -- 172.21.15.3:0/2297453235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c13dbd0 con 0x7fa57c06d8f0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa579d9b640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa57a59c640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c09a1d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ba80 tx=0x7fa56c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa56affd640 1 -- 172.21.15.3:0/2297453235 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c019070 con 0x7fa57c06d8f0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa56affd640 1 -- 172.21.15.3:0/2297453235 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c00cc90 con 0x7fa57c06d8f0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c073bd0 con 0x7fa57c06d8f0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa56affd640 1 -- 172.21.15.3:0/2297453235 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c014680 con 0x7fa57c06d8f0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.387+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c0738f0 con 0x7fa57c06d8f0 2024-09-12T12:24:32.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.391+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c0b30e0 con 0x7fa57c06d8f0 2024-09-12T12:24:32.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.391+0000 7fa56affd640 1 -- 172.21.15.3:0/2297453235 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa56c01c070 con 0x7fa57c06d8f0 2024-09-12T12:24:32.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.391+0000 7fa56affd640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa55c0678c0 0x7fa55c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:32.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.391+0000 7fa56affd640 1 -- 172.21.15.3:0/2297453235 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5824+0+0 (secure 0 0 0) 0x7fa56c085a90 con 0x7fa57c06d8f0 2024-09-12T12:24:32.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.391+0000 7fa579d9b640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa55c0678c0 0x7fa55c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:32.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.391+0000 7fa579d9b640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa55c0678c0 0x7fa55c069d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:32.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.395+0000 7fa56affd640 1 -- 172.21.15.3:0/2297453235 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa57c13dbd0 con 0x7fa57c06d8f0 2024-09-12T12:24:32.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:32.563+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa57c06f6e0 con 0x7fa57c06d8f0 2024-09-12T12:24:33.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.183+0000 7fa56affd640 1 -- 172.21.15.3:0/2297453235 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) ==== 181+0+0 (secure 0 0 0) 0x7fa56c0565f0 con 0x7fa57c06d8f0 2024-09-12T12:24:33.189 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:24:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa55c0678c0 msgr2=0x7fa55c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa55c0678c0 0x7fa55c069d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa570006fd0 tx=0x7fa570008040 comp rx=0 tx=0).stop 2024-09-12T12:24:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 msgr2=0x7fa57c09a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c09a1d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ba80 tx=0x7fa56c00bf40 comp rx=0 tx=0).stop 2024-09-12T12:24:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 shutdown_connections 2024-09-12T12:24:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa55c0678c0 0x7fa55c069d80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa57c0b4720 0x7fa57c09cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa57c0ae130 0x7fa57c14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 --2- 172.21.15.3:0/2297453235 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa57c06d8f0 0x7fa57c09a1d0 secure :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ba80 tx=0x7fa56c00bf40 comp rx=0 tx=0).stop 2024-09-12T12:24:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 >> 172.21.15.3:0/2297453235 conn(0x7fa57c067bd0 msgr2=0x7fa57c1515f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 shutdown_connections 2024-09-12T12:24:33.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.191+0000 7fa580f62640 1 -- 172.21.15.3:0/2297453235 wait complete. 2024-09-12T12:24:33.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:24:33.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7ea492f640 1 Processor -- start 2024-09-12T12:24:33.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7ea492f640 1 -- start start 2024-09-12T12:24:33.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7ea492f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ea00cad40 0x7f7ea00cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:33.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7ea492f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00d0d30 0x7f7ea00cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:33.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7ea492f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea006d5a0 con 0x7f7ea00b38a0 2024-09-12T12:24:33.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7ea492f640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea00cc2b0 con 0x7f7ea00cad40 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7ea492f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea00cb120 con 0x7f7ea00d0d30 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9e59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ea00cad40 0x7f7ea00cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9e59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f7ea00cad40 0x7f7ea00cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41804/0 (socket says 172.21.15.3:41804) 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9e59c640 1 -- 172.21.15.3:0/2062579867 learned_addr learned my addr 172.21.15.3:0/2062579867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9e59c640 1 -- 172.21.15.3:0/2062579867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00d0d30 msgr2=0x7f7ea00cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9ed9d640 1 --2- 172.21.15.3:0/2062579867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00d0d30 0x7f7ea00cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9e59c640 1 --2- 172.21.15.3:0/2062579867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00d0d30 0x7f7ea00cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9e59c640 1 -- 172.21.15.3:0/2062579867 >> v1:172.21.15.142:6789/0 conn(0x7f7ea00b38a0 legacy=0x7f7ea015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:33.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.351+0000 7f7e9e59c640 1 -- 172.21.15.3:0/2062579867 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea00cc500 con 0x7f7ea00cad40 2024-09-12T12:24:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7e9e59c640 1 --2- 172.21.15.3:0/2062579867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00cee20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000bb10 tx=0x7f7e9001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25d7c02657eeaf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7e9d59a640 1 -- 172.21.15.3:0/2062579867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e900b4070 con 0x7f7ea00cad40 2024-09-12T12:24:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7e9d59a640 1 -- 172.21.15.3:0/2062579867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e900a7ae0 con 0x7f7ea00cad40 2024-09-12T12:24:33.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7e9d59a640 1 -- 172.21.15.3:0/2062579867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e900af440 con 0x7f7ea00cad40 2024-09-12T12:24:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 -- 172.21.15.3:0/2062579867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 msgr2=0x7f7ea00cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 --2- 172.21.15.3:0/2062579867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00cee20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000bb10 tx=0x7f7e9001eb30 comp rx=0 tx=0).stop 2024-09-12T12:24:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 -- 172.21.15.3:0/2062579867 shutdown_connections 2024-09-12T12:24:33.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 --2- 172.21.15.3:0/2062579867 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00d0d30 0x7f7ea00cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 --2- 172.21.15.3:0/2062579867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00cee20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 -- 172.21.15.3:0/2062579867 >> 172.21.15.3:0/2062579867 conn(0x7f7ea00b2d80 msgr2=0x7f7ea00b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 -- 172.21.15.3:0/2062579867 shutdown_connections 2024-09-12T12:24:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 -- 172.21.15.3:0/2062579867 wait complete. 2024-09-12T12:24:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 Processor -- start 2024-09-12T12:24:33.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.355+0000 7f7ea492f640 1 -- start start 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00b38a0 0x7f7ea009a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ea00d0d30 0x7f7ea00b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea00ce5b0 con 0x7f7ea00cad40 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea00cd080 con 0x7f7ea00d0d30 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea00c8320 con 0x7f7ea00b38a0 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41812/0 (socket says 172.21.15.3:41812) 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 -- 172.21.15.3:0/3009635116 learned_addr learned my addr 172.21.15.3:0/3009635116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9e59c640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00b38a0 0x7f7ea009a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9ed9d640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ea00d0d30 0x7f7ea00b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:33.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 -- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00b38a0 msgr2=0x7f7ea009a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00b38a0 0x7f7ea009a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 -- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ea00d0d30 msgr2=0x7f7ea00b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ea00d0d30 0x7f7ea00b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 -- 172.21.15.3:0/3009635116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea009aef0 con 0x7f7ea00cad40 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e9dd9b640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00b6250 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400ba50 tx=0x7f7e9400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e8effd640 1 -- 172.21.15.3:0/3009635116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94019070 con 0x7f7ea00cad40 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e8effd640 1 -- 172.21.15.3:0/3009635116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9400cc20 con 0x7f7ea00cad40 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea00c2d10 con 0x7f7ea00cad40 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e8effd640 1 -- 172.21.15.3:0/3009635116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94014680 con 0x7f7ea00cad40 2024-09-12T12:24:33.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea00c32d0 con 0x7f7ea00cad40 2024-09-12T12:24:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea015ee10 con 0x7f7ea00cad40 2024-09-12T12:24:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e8effd640 1 -- 172.21.15.3:0/3009635116 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7e9401c070 con 0x7f7ea00cad40 2024-09-12T12:24:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e8effd640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7e800678f0 0x7f7e80069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.359+0000 7f7e8effd640 1 -- 172.21.15.3:0/3009635116 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5386+0+0 (secure 0 0 0) 0x7f7e94085770 con 0x7f7ea00cad40 2024-09-12T12:24:33.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.363+0000 7f7e9e59c640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7e800678f0 0x7f7e80069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:33.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.363+0000 7f7e9e59c640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7e800678f0 0x7f7e80069db0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7e900b1000 tx=0x7f7e900a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:33.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.363+0000 7f7e8effd640 1 -- 172.21.15.3:0/3009635116 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7e94055e30 con 0x7f7ea00cad40 2024-09-12T12:24:33.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:33.527+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7ea00b71d0 con 0x7f7ea00cad40 2024-09-12T12:24:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.195+0000 7f7e8effd640 1 -- 172.21.15.3:0/3009635116 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) ==== 169+0+0 (secure 0 0 0) 0x7f7e9405a130 con 0x7f7ea00cad40 2024-09-12T12:24:34.199 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T12:24:34.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7e800678f0 msgr2=0x7f7e80069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7e800678f0 0x7f7e80069db0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7e900b1000 tx=0x7f7e900a9070 comp rx=0 tx=0).stop 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 msgr2=0x7f7ea00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00b6250 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400ba50 tx=0x7f7e9400bf10 comp rx=0 tx=0).stop 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 shutdown_connections 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7e800678f0 0x7f7e80069db0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ea00d0d30 0x7f7ea00b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea00cad40 0x7f7ea00b6250 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 --2- 172.21.15.3:0/3009635116 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ea00b38a0 0x7f7ea009a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 >> 172.21.15.3:0/3009635116 conn(0x7f7ea00b2d80 msgr2=0x7f7ea0067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 shutdown_connections 2024-09-12T12:24:34.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.199+0000 7f7ea492f640 1 -- 172.21.15.3:0/3009635116 wait complete. 2024-09-12T12:24:34.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T12:24:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 Processor -- start 2024-09-12T12:24:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 -- start start 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd13815e610 0x7fd1380d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1380d3f80 0x7fd1380d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1380d3a30 0x7fd1380d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd13815e9f0 con 0x7fd1380d3f80 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd138072590 con 0x7fd13815e610 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd13daf5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd138072870 con 0x7fd1380d3a30 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd136ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1380d3f80 0x7fd1380d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd136ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd1380d3f80 0x7fd1380d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41818/0 (socket says 172.21.15.3:41818) 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.351+0000 7fd136ffd640 1 -- 172.21.15.3:0/1281880523 learned_addr learned my addr 172.21.15.3:0/1281880523 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd136ffd640 1 -- 172.21.15.3:0/1281880523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1380d3a30 msgr2=0x7fd1380d27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd136ffd640 1 --2- 172.21.15.3:0/1281880523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1380d3a30 0x7fd1380d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd136ffd640 1 -- 172.21.15.3:0/1281880523 >> v2:172.21.15.142:3300/0 conn(0x7fd13815e610 msgr2=0x7fd1380d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd136ffd640 1 --2- 172.21.15.3:0/1281880523 >> v2:172.21.15.142:3300/0 conn(0x7fd13815e610 0x7fd1380d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd136ffd640 1 -- 172.21.15.3:0/1281880523 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd138072b50 con 0x7fd1380d3f80 2024-09-12T12:24:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd136ffd640 1 --2- 172.21.15.3:0/1281880523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380d1430 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009870 tx=0x7fd12c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d27cabbca413b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd1367fc640 1 -- 172.21.15.3:0/1281880523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0b0070 con 0x7fd1380d3f80 2024-09-12T12:24:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd1367fc640 1 -- 172.21.15.3:0/1281880523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c0ab600 con 0x7fd1380d3f80 2024-09-12T12:24:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd1367fc640 1 -- 172.21.15.3:0/1281880523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0ab9d0 con 0x7fd1380d3f80 2024-09-12T12:24:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 -- 172.21.15.3:0/1281880523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 msgr2=0x7fd1380d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 --2- 172.21.15.3:0/1281880523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380d1430 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009870 tx=0x7fd12c01c1d0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 -- 172.21.15.3:0/1281880523 shutdown_connections 2024-09-12T12:24:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 --2- 172.21.15.3:0/1281880523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1380d3a30 0x7fd1380d27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 --2- 172.21.15.3:0/1281880523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380d1430 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 --2- 172.21.15.3:0/1281880523 >> v2:172.21.15.142:3300/0 conn(0x7fd13815e610 0x7fd1380d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 -- 172.21.15.3:0/1281880523 >> 172.21.15.3:0/1281880523 conn(0x7fd1380b6040 msgr2=0x7fd1380b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 -- 172.21.15.3:0/1281880523 shutdown_connections 2024-09-12T12:24:34.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.355+0000 7fd13daf5640 1 -- 172.21.15.3:0/1281880523 wait complete. 2024-09-12T12:24:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 Processor -- start 2024-09-12T12:24:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 -- start start 2024-09-12T12:24:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1380d3a30 0x7fd1380ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd13815e610 0x7fd138157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd1380b7c70 con 0x7fd1380d3f80 2024-09-12T12:24:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd1380b7af0 con 0x7fd1380d3a30 2024-09-12T12:24:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd1380b7df0 con 0x7fd13815e610 2024-09-12T12:24:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41820/0 (socket says 172.21.15.3:41820) 2024-09-12T12:24:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 -- 172.21.15.3:0/3162030789 learned_addr learned my addr 172.21.15.3:0/3162030789 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd137fff640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd13815e610 0x7fd138157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd1377fe640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1380d3a30 0x7fd1380ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 -- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd13815e610 msgr2=0x7fd138157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd13815e610 0x7fd138157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 -- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1380d3a30 msgr2=0x7fd1380ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1380d3a30 0x7fd1380ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 -- 172.21.15.3:0/3162030789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1380cd220 con 0x7fd1380d3f80 2024-09-12T12:24:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd137fff640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd13815e610 0x7fd138157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd136ffd640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380cc8b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0ae040 tx=0x7fd12c01aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd11ffff640 1 -- 172.21.15.3:0/3162030789 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0ba020 con 0x7fd1380d3f80 2024-09-12T12:24:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd11ffff640 1 -- 172.21.15.3:0/3162030789 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c0b6640 con 0x7fd1380d3f80 2024-09-12T12:24:34.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1380cb850 con 0x7fd1380d3f80 2024-09-12T12:24:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd11ffff640 1 -- 172.21.15.3:0/3162030789 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0b0040 con 0x7fd1380d3f80 2024-09-12T12:24:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1380cbe10 con 0x7fd1380d3f80 2024-09-12T12:24:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd138161b40 con 0x7fd1380d3f80 2024-09-12T12:24:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd11ffff640 1 -- 172.21.15.3:0/3162030789 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd12c0bd020 con 0x7fd1380d3f80 2024-09-12T12:24:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.359+0000 7fd11ffff640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1240678f0 0x7fd124069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.363+0000 7fd11ffff640 1 -- 172.21.15.3:0/3162030789 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7fd12c11fd40 con 0x7fd1380d3f80 2024-09-12T12:24:34.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.363+0000 7fd1377fe640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1240678f0 0x7fd124069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.363+0000 7fd1377fe640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1240678f0 0x7fd124069db0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd12800a790 tx=0x7fd128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:34.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.367+0000 7fd11ffff640 1 -- 172.21.15.3:0/3162030789 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd12c0c4030 con 0x7fd1380d3f80 2024-09-12T12:24:34.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.531+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd1380cf310 con 0x7fd1380d3f80 2024-09-12T12:24:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd11ffff640 1 -- 172.21.15.3:0/3162030789 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fd12c016110 con 0x7fd1380d3f80 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1240678f0 msgr2=0x7fd124069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1240678f0 0x7fd124069db0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd12800a790 tx=0x7fd128008040 comp rx=0 tx=0).stop 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 msgr2=0x7fd1380cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380cc8b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0ae040 tx=0x7fd12c01aa10 comp rx=0 tx=0).stop 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 shutdown_connections 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd1240678f0 0x7fd124069db0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd13815e610 0x7fd138157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1380d3f80 0x7fd1380cc8b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 --2- 172.21.15.3:0/3162030789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1380d3a30 0x7fd1380ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 >> 172.21.15.3:0/3162030789 conn(0x7fd1380b6040 msgr2=0x7fd1380b9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 shutdown_connections 2024-09-12T12:24:34.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.535+0000 7fd13daf5640 1 -- 172.21.15.3:0/3162030789 wait complete. 2024-09-12T12:24:34.558 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop' 2024-09-12T12:24:34.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f56eed640 1 Processor -- start 2024-09-12T12:24:34.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f56eed640 1 -- start start 2024-09-12T12:24:34.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f56eed640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f500d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f56eed640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f500b7740 0x7f1f500b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f56eed640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f500d30b0 con 0x7f1f500d1570 2024-09-12T12:24:34.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f56eed640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f1f500d3cc0 con 0x7f1f500b7740 2024-09-12T12:24:34.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f56eed640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f500d3390 con 0x7f1f500d1190 2024-09-12T12:24:34.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f54c89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f500d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f5548a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1f500b7740 0x7f1f500b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f54c89640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f500d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:46182/0 (socket says 172.21.15.3:46182) 2024-09-12T12:24:34.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f54c89640 1 -- 172.21.15.3:0/2335790397 learned_addr learned my addr 172.21.15.3:0/2335790397 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:34.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f54c89640 1 -- 172.21.15.3:0/2335790397 >> v2:172.21.15.142:3300/0 conn(0x7f1f500b7740 msgr2=0x7f1f500b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:34.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f54c89640 1 --2- 172.21.15.3:0/2335790397 >> v2:172.21.15.142:3300/0 conn(0x7f1f500b7740 0x7f1f500b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f54c89640 1 -- 172.21.15.3:0/2335790397 >> v1:172.21.15.142:6789/0 conn(0x7f1f500d1570 legacy=0x7f1f500cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.671+0000 7f1f54c89640 1 -- 172.21.15.3:0/2335790397 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f500d3670 con 0x7f1f500d1190 2024-09-12T12:24:34.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f54c89640 1 --2- 172.21.15.3:0/2335790397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f500d4150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009870 tx=0x7f1f4001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f065351b4b2eba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:34.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f4f7fe640 1 -- 172.21.15.3:0/2335790397 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f400b5070 con 0x7f1f500d1190 2024-09-12T12:24:34.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f4f7fe640 1 -- 172.21.15.3:0/2335790397 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f400a8a50 con 0x7f1f500d1190 2024-09-12T12:24:34.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f4f7fe640 1 -- 172.21.15.3:0/2335790397 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f400a8e20 con 0x7f1f500d1190 2024-09-12T12:24:34.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 -- 172.21.15.3:0/2335790397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 msgr2=0x7f1f500d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2335790397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f500d4150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009870 tx=0x7f1f4001ed50 comp rx=0 tx=0).stop 2024-09-12T12:24:34.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 -- 172.21.15.3:0/2335790397 shutdown_connections 2024-09-12T12:24:34.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2335790397 >> v2:172.21.15.142:3300/0 conn(0x7f1f500b7740 0x7f1f500b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2335790397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f500d4150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 -- 172.21.15.3:0/2335790397 >> 172.21.15.3:0/2335790397 conn(0x7f1f50067a00 msgr2=0x7f1f50067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:34.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 -- 172.21.15.3:0/2335790397 shutdown_connections 2024-09-12T12:24:34.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 -- 172.21.15.3:0/2335790397 wait complete. 2024-09-12T12:24:34.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 Processor -- start 2024-09-12T12:24:34.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.675+0000 7f1f56eed640 1 -- start start 2024-09-12T12:24:34.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f500b7740 0x7f1f501547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f501f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f500d1570 0x7f1f500cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500d2cd0 con 0x7f1f500d1570 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500d19d0 con 0x7f1f500b7740 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1f500d2e50 con 0x7f1f500d1190 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f500d1570 0x7f1f500cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f500d1570 0x7f1f500cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41832/0 (socket says 172.21.15.3:41832) 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 -- 172.21.15.3:0/2611366570 learned_addr learned my addr 172.21.15.3:0/2611366570 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f4ffff640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f501f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f54c89640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f500b7740 0x7f1f501547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 -- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 msgr2=0x7f1f501f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f501f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 -- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f500b7740 msgr2=0x7f1f501547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f500b7740 0x7f1f501547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:34.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 -- 172.21.15.3:0/2611366570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f501eb000 con 0x7f1f500d1570 2024-09-12T12:24:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f5548a640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f500d1570 0x7f1f500cce50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800efc0 tx=0x7f1f4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f4d7fa640 1 -- 172.21.15.3:0/2611366570 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48018020 con 0x7f1f500d1570 2024-09-12T12:24:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f4d7fa640 1 -- 172.21.15.3:0/2611366570 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f48014480 con 0x7f1f500d1570 2024-09-12T12:24:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f4d7fa640 1 -- 172.21.15.3:0/2611366570 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48014870 con 0x7f1f500d1570 2024-09-12T12:24:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f501f8e40 con 0x7f1f500d1570 2024-09-12T12:24:34.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f501e88f0 con 0x7f1f500d1570 2024-09-12T12:24:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5015e860 con 0x7f1f500d1570 2024-09-12T12:24:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f4d7fa640 1 -- 172.21.15.3:0/2611366570 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f4801b020 con 0x7f1f500d1570 2024-09-12T12:24:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f4d7fa640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f3c0678c0 0x7f1f3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f4d7fa640 1 -- 172.21.15.3:0/2611366570 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f1f48086780 con 0x7f1f500d1570 2024-09-12T12:24:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.679+0000 7f1f54c89640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f3c0678c0 0x7f1f3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:34.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.683+0000 7f1f54c89640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f3c0678c0 0x7f1f3c069d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1f400b2000 tx=0x7f1f400aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:34.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.683+0000 7f1f4d7fa640 1 -- 172.21.15.3:0/2611366570 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f1f48022030 con 0x7f1f500d1570 2024-09-12T12:24:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:34.923+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]} v 0) -- 0x7f1f500ba4c0 con 0x7f1f500d1570 2024-09-12T12:24:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.199+0000 7f1f4d7fa640 1 -- 172.21.15.3:0/2611366570 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_tell_session_drop"]}]=0 v252) ==== 145+0+0 (secure 0 0 0) 0x7f1f48010040 con 0x7f1f500d1570 2024-09-12T12:24:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f3c0678c0 msgr2=0x7f1f3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f3c0678c0 0x7f1f3c069d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1f400b2000 tx=0x7f1f400aa070 comp rx=0 tx=0).stop 2024-09-12T12:24:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f500d1570 msgr2=0x7f1f500cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f500d1570 0x7f1f500cce50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800efc0 tx=0x7f1f4800c490 comp rx=0 tx=0).stop 2024-09-12T12:24:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 shutdown_connections 2024-09-12T12:24:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1f3c0678c0 0x7f1f3c069d80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1f500d1570 0x7f1f500cce50 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1f500d1190 0x7f1f501f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 --2- 172.21.15.3:0/2611366570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f500b7740 0x7f1f501547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 >> 172.21.15.3:0/2611366570 conn(0x7f1f50067a00 msgr2=0x7f1f500b5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 shutdown_connections 2024-09-12T12:24:35.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.203+0000 7f1f56eed640 1 -- 172.21.15.3:0/2611366570 wait complete. 2024-09-12T12:24:35.225 INFO:tasks.cephfs_test_runner:That when a `tell` command is sent using the python CLI, ... ok 2024-09-12T12:24:35.226 INFO:tasks.cephfs_test_runner:Starting test: test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) That when many sessions are updated, they are correctly 2024-09-12T12:24:35.226 INFO:tasks.cephfs_test_runner:test_version_splitting (tasks.cephfs.test_sessionmap.TestSessionMap) 2024-09-12T12:24:35.226 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:24:35.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 Processor -- start 2024-09-12T12:24:35.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 -- start start 2024-09-12T12:24:35.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb20c0b0920 0x7fb20c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb20c0b1770 0x7fb20c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb20c0b2120 con 0x7fb20c0b0920 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb20c0cc5b0 con 0x7fb20c0b1770 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.347+0000 7fb210f80640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb20c15ec80 con 0x7fb20c0afff0 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb20c0b0920 0x7fb20c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb20c0b0920 0x7fb20c0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41836/0 (socket says 172.21.15.3:41836) 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 -- 172.21.15.3:0/1164536237 learned_addr learned my addr 172.21.15.3:0/1164536237 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb20a59c640 1 --2- 172.21.15.3:0/1164536237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb20ad9d640 1 --2- 172.21.15.3:0/1164536237 >> v2:172.21.15.142:3300/0 conn(0x7fb20c0b1770 0x7fb20c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 -- 172.21.15.3:0/1164536237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 msgr2=0x7fb20c0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 --2- 172.21.15.3:0/1164536237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c0ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 -- 172.21.15.3:0/1164536237 >> v2:172.21.15.142:3300/0 conn(0x7fb20c0b1770 msgr2=0x7fb20c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 --2- 172.21.15.3:0/1164536237 >> v2:172.21.15.142:3300/0 conn(0x7fb20c0b1770 0x7fb20c0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 -- 172.21.15.3:0/1164536237 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c0cb9a0 con 0x7fb20c0b0920 2024-09-12T12:24:35.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb20ad9d640 1 --2- 172.21.15.3:0/1164536237 >> v2:172.21.15.142:3300/0 conn(0x7fb20c0b1770 0x7fb20c0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:35.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb209d9b640 1 --2- 172.21.15.3:0/1164536237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b0920 0x7fb20c0ce520 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb20c072170 tx=0x7fb20001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fc54a98e449658 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb20959a640 1 -- 172.21.15.3:0/1164536237 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000a7070 con 0x7fb20c0b0920 2024-09-12T12:24:35.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb20959a640 1 -- 172.21.15.3:0/1164536237 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2000b0070 con 0x7fb20c0b0920 2024-09-12T12:24:35.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb20959a640 1 -- 172.21.15.3:0/1164536237 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000ab890 con 0x7fb20c0b0920 2024-09-12T12:24:35.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 -- 172.21.15.3:0/1164536237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b0920 msgr2=0x7fb20c0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 --2- 172.21.15.3:0/1164536237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b0920 0x7fb20c0ce520 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb20c072170 tx=0x7fb20001c490 comp rx=0 tx=0).stop 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 -- 172.21.15.3:0/1164536237 shutdown_connections 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 --2- 172.21.15.3:0/1164536237 >> v2:172.21.15.142:3300/0 conn(0x7fb20c0b1770 0x7fb20c0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 --2- 172.21.15.3:0/1164536237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b0920 0x7fb20c0ce520 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 --2- 172.21.15.3:0/1164536237 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 -- 172.21.15.3:0/1164536237 >> 172.21.15.3:0/1164536237 conn(0x7fb20c0731b0 msgr2=0x7fb20c0735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 -- 172.21.15.3:0/1164536237 shutdown_connections 2024-09-12T12:24:35.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.351+0000 7fb210f80640 1 -- 172.21.15.3:0/1164536237 wait complete. 2024-09-12T12:24:35.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 Processor -- start 2024-09-12T12:24:35.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 -- start start 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c09a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b1770 0x7fb20c0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb20c15ac20 0x7fb20c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c0cebe0 con 0x7fb20c0b1770 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c0cea60 con 0x7fb20c15ac20 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb20c0cd230 con 0x7fb20c0afff0 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b1770 0x7fb20c0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b1770 0x7fb20c0b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41842/0 (socket says 172.21.15.3:41842) 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 -- 172.21.15.3:0/2645907614 learned_addr learned my addr 172.21.15.3:0/2645907614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb20ad9d640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb20c15ac20 0x7fb20c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb20a59c640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c09a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 -- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 msgr2=0x7fb20c09a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c09a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 -- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb20c15ac20 msgr2=0x7fb20c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb20c15ac20 0x7fb20c0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 -- 172.21.15.3:0/2645907614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c0b1010 con 0x7fb20c0b1770 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb20a59c640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c09a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb209d9b640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b1770 0x7fb20c0b3500 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb2000ad040 tx=0x7fb20001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb1faffd640 1 -- 172.21.15.3:0/2645907614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000be5f0 con 0x7fb20c0b1770 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb1faffd640 1 -- 172.21.15.3:0/2645907614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2000be7b0 con 0x7fb20c0b1770 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c159210 con 0x7fb20c0b1770 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb1faffd640 1 -- 172.21.15.3:0/2645907614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000beba0 con 0x7fb20c0b1770 2024-09-12T12:24:35.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c09ab90 con 0x7fb20c0b1770 2024-09-12T12:24:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.355+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c0cd600 con 0x7fb20c0b1770 2024-09-12T12:24:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.359+0000 7fb1faffd640 1 -- 172.21.15.3:0/2645907614 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb2000b43f0 con 0x7fb20c0b1770 2024-09-12T12:24:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.359+0000 7fb1faffd640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb1ec0678f0 0x7fb1ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.359+0000 7fb1faffd640 1 -- 172.21.15.3:0/2645907614 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7fb200120850 con 0x7fb20c0b1770 2024-09-12T12:24:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.359+0000 7fb20a59c640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb1ec0678f0 0x7fb1ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.359+0000 7fb20a59c640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb1ec0678f0 0x7fb1ec069db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0c82a0 tx=0x7fb1fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.363+0000 7fb1faffd640 1 -- 172.21.15.3:0/2645907614 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb2000c4060 con 0x7fb20c0b1770 2024-09-12T12:24:35.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.519+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb20c0d04e0 con 0x7fb20c0b1770 2024-09-12T12:24:35.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.519+0000 7fb1faffd640 1 -- 172.21.15.3:0/2645907614 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+12971 (secure 0 0 0) 0x7fb200016110 con 0x7fb20c0b1770 2024-09-12T12:24:35.524 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:35.524 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":84,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:24:34.189703+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/2940169617":"2024-09-13T12:24:31.173378+0000","172.21.15.3:6834/2940169617":"2024-09-13T12:24:31.173378+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:24:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb1ec0678f0 msgr2=0x7fb1ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb1ec0678f0 0x7fb1ec069db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0c82a0 tx=0x7fb1fc008040 comp rx=0 tx=0).stop 2024-09-12T12:24:35.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b1770 msgr2=0x7fb20c0b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b1770 0x7fb20c0b3500 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb2000ad040 tx=0x7fb20001acd0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 shutdown_connections 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb1ec0678f0 0x7fb1ec069db0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb20c15ac20 0x7fb20c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb20c0b1770 0x7fb20c0b3500 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 --2- 172.21.15.3:0/2645907614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb20c0afff0 0x7fb20c09a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 >> 172.21.15.3:0/2645907614 conn(0x7fb20c0731b0 msgr2=0x7fb20c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 shutdown_connections 2024-09-12T12:24:35.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.523+0000 7fb210f80640 1 -- 172.21.15.3:0/2645907614 wait complete. 2024-09-12T12:24:35.542 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:19:37.735209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:24:35.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T12:24:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a9eac640 1 Processor -- start 2024-09-12T12:24:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a9eac640 1 -- start start 2024-09-12T12:24:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a9eac640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 0x7fa5a40cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a9eac640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a40d10d0 con 0x7fa5a41663f0 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a9eac640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a40d13b0 con 0x7fa5a415e750 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a9eac640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40cfb20 con 0x7fa5a40cf170 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a3fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa5a41663f0 0x7fa5a41687e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40536/0 (socket says 172.21.15.3:40536) 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a3fff640 1 -- 172.21.15.3:0/3058678024 learned_addr learned my addr 172.21.15.3:0/3058678024 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a2ffd640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 0x7fa5a40cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639223512 0 0) 0x7fa5a40d10d0 con 0x7fa5a41663f0 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa58c003530 con 0x7fa5a41663f0 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1409222402 0 0) 0x7fa58c003530 con 0x7fa5a41663f0 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 msgr2=0x7fa5a40cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 0x7fa5a40cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 >> v1:172.21.15.142:6789/0 conn(0x7fa5a415e750 legacy=0x7fa5a40d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40d1690 con 0x7fa5a41663f0 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fa59c001680 con 0x7fa5a41663f0 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 >> v1:172.21.15.3:6789/0 conn(0x7fa5a41663f0 legacy=0x7fa5a41687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa58c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58c004810 0x7fa58c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 0x7fa58c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40cfb20 con 0x7fa58c0039f0 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c00bb40 con 0x7fa58c004810 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa58c00bd40 con 0x7fa5a40cf170 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a37fe640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa58c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a2ffd640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58c004810 0x7fa58c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 0x7fa58c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a37fe640 1 -- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 msgr2=0x7fa58c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a37fe640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a40cf170 0x7fa58c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a37fe640 1 -- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58c004810 msgr2=0x7fa58c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a37fe640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa58c004810 0x7fa58c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.667+0000 7fa5a37fe640 1 -- 172.21.15.3:0/3058678024 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c003530 con 0x7fa58c0039f0 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a37fe640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa58c003e10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa594001e40 tx=0x7fa59401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54b0369ca7101cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594001a90 con 0x7fa58c0039f0 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594003b80 con 0x7fa58c0039f0 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a27fc640 1 -- 172.21.15.3:0/3058678024 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594005080 con 0x7fa58c0039f0 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 -- 172.21.15.3:0/3058678024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 msgr2=0x7fa58c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 --2- 172.21.15.3:0/3058678024 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa58c003e10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa594001e40 tx=0x7fa59401c740 comp rx=0 tx=0).stop 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a3fff640 1 -- 172.21.15.3:0/3058678024 reap_dead start 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 -- 172.21.15.3:0/3058678024 shutdown_connections 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 -- 172.21.15.3:0/3058678024 >> 172.21.15.3:0/3058678024 conn(0x7fa5a4067a00 msgr2=0x7fa5a4067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 -- 172.21.15.3:0/3058678024 shutdown_connections 2024-09-12T12:24:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 -- 172.21.15.3:0/3058678024 wait complete. 2024-09-12T12:24:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 Processor -- start 2024-09-12T12:24:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.671+0000 7fa5a9eac640 1 -- start start 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa5a4158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a415aa20 0x7fa5a4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4158b70 0x7fa5a40ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40b6e10 con 0x7fa58c0039f0 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40d1c50 con 0x7fa5a4158b70 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40d4290 con 0x7fa5a415aa20 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa5a4158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa5a4158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41854/0 (socket says 172.21.15.3:41854) 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 -- 172.21.15.3:0/1129899207 learned_addr learned my addr 172.21.15.3:0/1129899207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a2ffd640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a415aa20 0x7fa5a4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4158b70 0x7fa5a40ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 -- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a415aa20 msgr2=0x7fa5a4157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a415aa20 0x7fa5a4157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 -- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4158b70 msgr2=0x7fa5a40ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4158b70 0x7fa5a40ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 -- 172.21.15.3:0/1129899207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a41f4750 con 0x7fa58c0039f0 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a3fff640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4158b70 0x7fa5a40ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a37fe640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa5a4158590 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa594000f90 tx=0x7fa58c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa58bfff640 1 -- 172.21.15.3:0/1129899207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5940bd070 con 0x7fa58c0039f0 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa58bfff640 1 -- 172.21.15.3:0/1129899207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594001260 con 0x7fa58c0039f0 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a415aeb0 con 0x7fa58c0039f0 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa58bfff640 1 -- 172.21.15.3:0/1129899207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594007dc0 con 0x7fa58c0039f0 2024-09-12T12:24:35.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a415b470 con 0x7fa58c0039f0 2024-09-12T12:24:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a415eb30 con 0x7fa58c0039f0 2024-09-12T12:24:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa58bfff640 1 -- 172.21.15.3:0/1129899207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa5940b1040 con 0x7fa58c0039f0 2024-09-12T12:24:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa58bfff640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5900678c0 0x7fa590069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa58bfff640 1 -- 172.21.15.3:0/1129899207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7fa59411f770 con 0x7fa58c0039f0 2024-09-12T12:24:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.675+0000 7fa5a2ffd640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5900678c0 0x7fa590069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.679+0000 7fa5a2ffd640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5900678c0 0x7fa590069d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40b7df0 tx=0x7fa598008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.679+0000 7fa58bfff640 1 -- 172.21.15.3:0/1129899207 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa5940ad020 con 0x7fa58c0039f0 2024-09-12T12:24:35.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.843+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5a40d2700 con 0x7fa58c0039f0 2024-09-12T12:24:35.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.843+0000 7fa58bfff640 1 -- 172.21.15.3:0/1129899207 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7fa594016450 con 0x7fa58c0039f0 2024-09-12T12:24:35.849 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T12:24:35.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5900678c0 msgr2=0x7fa590069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5900678c0 0x7fa590069d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40b7df0 tx=0x7fa598008040 comp rx=0 tx=0).stop 2024-09-12T12:24:35.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 msgr2=0x7fa5a4158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa5a4158590 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa594000f90 tx=0x7fa58c005840 comp rx=0 tx=0).stop 2024-09-12T12:24:35.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 shutdown_connections 2024-09-12T12:24:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa5900678c0 0x7fa590069d80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa5a4158b70 0x7fa5a40ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa5a415aa20 0x7fa5a4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 --2- 172.21.15.3:0/1129899207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa58c0039f0 0x7fa5a4158590 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 >> 172.21.15.3:0/1129899207 conn(0x7fa5a4067a00 msgr2=0x7fa5a41591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 shutdown_connections 2024-09-12T12:24:35.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.847+0000 7fa5a9eac640 1 -- 172.21.15.3:0/1129899207 wait complete. 2024-09-12T12:24:35.868 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-09-12T12:24:35.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a89c1640 1 Processor -- start 2024-09-12T12:24:35.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a89c1640 1 -- start start 2024-09-12T12:24:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a89c1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f06a40ae250 0x7f06a40ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a89c1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 0x7f06a415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a89c1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a40ad0c0 con 0x7f06a40d05b0 2024-09-12T12:24:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a89c1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f06a40ad3a0 con 0x7f06a40ae250 2024-09-12T12:24:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a89c1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06a40ad680 con 0x7f06a415b620 2024-09-12T12:24:35.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a259c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f06a40d05b0 0x7f06a40d0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40540/0 (socket says 172.21.15.3:40540) 2024-09-12T12:24:35.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a259c640 1 -- 172.21.15.3:0/717402747 learned_addr learned my addr 172.21.15.3:0/717402747 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:35.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a1d9b640 1 --2- 172.21.15.3:0/717402747 >> v2:172.21.15.142:3300/0 conn(0x7f06a40ae250 0x7f06a40ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a2d9d640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 0x7f06a415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3514721715 0 0) 0x7f06a40ad0c0 con 0x7f06a40d05b0 2024-09-12T12:24:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f068c003530 con 0x7f06a40d05b0 2024-09-12T12:24:35.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2305377244 0 0) 0x7f068c003530 con 0x7f06a40d05b0 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 msgr2=0x7f06a415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 0x7f06a415bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 >> v2:172.21.15.142:3300/0 conn(0x7f06a40ae250 msgr2=0x7f06a40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 --2- 172.21.15.3:0/717402747 >> v2:172.21.15.142:3300/0 conn(0x7f06a40ae250 0x7f06a40ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a40ad960 con 0x7f06a40d05b0 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a1d9b640 1 --2- 172.21.15.3:0/717402747 >> v2:172.21.15.142:3300/0 conn(0x7f06a40ae250 0x7f06a40ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f0694001680 con 0x7f06a40d05b0 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 >> v1:172.21.15.3:6789/0 conn(0x7f06a40d05b0 legacy=0x7f06a40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 0x7f068c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c003f40 0x7f068c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f068c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40ad3a0 con 0x7f068c004d50 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40ad680 con 0x7f068c003f40 2024-09-12T12:24:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f068c00b9d0 con 0x7f06a415b620 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a2d9d640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f068c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a1d9b640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c003f40 0x7f068c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a2d9d640 1 -- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 msgr2=0x7f068c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a259c640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 0x7f068c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a2d9d640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 0x7f068c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a2d9d640 1 -- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c003f40 msgr2=0x7f068c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a2d9d640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c003f40 0x7f068c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a2d9d640 1 -- 172.21.15.3:0/717402747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c003530 con 0x7f068c004d50 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a1d9b640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f068c003f40 0x7f068c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.979+0000 7f06a259c640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a415b620 0x7f068c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a2d9d640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f068c0051d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f069c009fc0 tx=0x7f069c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc16f2441ca8ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0b2070 con 0x7f068c004d50 2024-09-12T12:24:35.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c0a5c90 con 0x7f068c004d50 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a159a640 1 -- 172.21.15.3:0/717402747 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c009c10 con 0x7f068c004d50 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 -- 172.21.15.3:0/717402747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 msgr2=0x7f068c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 --2- 172.21.15.3:0/717402747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f068c0051d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f069c009fc0 tx=0x7f069c01cc20 comp rx=0 tx=0).stop 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a2d9d640 1 -- 172.21.15.3:0/717402747 reap_dead start 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 -- 172.21.15.3:0/717402747 shutdown_connections 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 -- 172.21.15.3:0/717402747 >> 172.21.15.3:0/717402747 conn(0x7f06a4067e00 msgr2=0x7f06a4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 -- 172.21.15.3:0/717402747 shutdown_connections 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 -- 172.21.15.3:0/717402747 wait complete. 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 Processor -- start 2024-09-12T12:24:35.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.983+0000 7f06a89c1640 1 -- start start 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f06a40cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a40b27d0 0x7f06a40cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a41595f0 0x7f06a4099580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40aeda0 con 0x7f068c004d50 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06a40aec20 con 0x7f06a41595f0 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f06a41559b0 con 0x7f06a40b27d0 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f06a40cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f06a40cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41880/0 (socket says 172.21.15.3:41880) 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 -- 172.21.15.3:0/599848595 learned_addr learned my addr 172.21.15.3:0/599848595 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a1d9b640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a40b27d0 0x7f06a40cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a2d9d640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a41595f0 0x7f06a4099580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 -- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a40b27d0 msgr2=0x7f06a40cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a40b27d0 0x7f06a40cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 -- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a41595f0 msgr2=0x7f06a4099580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a41595f0 0x7f06a4099580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 -- 172.21.15.3:0/599848595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4154790 con 0x7f068c004d50 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a259c640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f06a40cf6e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0694001210 tx=0x7f069400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f0692ffd640 1 -- 172.21.15.3:0/599848595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694019070 con 0x7f068c004d50 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f0692ffd640 1 -- 172.21.15.3:0/599848595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06940074e0 con 0x7f068c004d50 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a40ce370 con 0x7f068c004d50 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f0692ffd640 1 -- 172.21.15.3:0/599848595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940049f0 con 0x7f068c004d50 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a40ce920 con 0x7f068c004d50 2024-09-12T12:24:35.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a415c830 con 0x7f068c004d50 2024-09-12T12:24:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f0692ffd640 1 -- 172.21.15.3:0/599848595 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0694004b90 con 0x7f068c004d50 2024-09-12T12:24:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f0692ffd640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06840678c0 0x7f0684069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f0692ffd640 1 -- 172.21.15.3:0/599848595 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f06940856f0 con 0x7f068c004d50 2024-09-12T12:24:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.987+0000 7f06a1d9b640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06840678c0 0x7f0684069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:35.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.991+0000 7f06a1d9b640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06840678c0 0x7f0684069d80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0698009cd0 tx=0x7f0698008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:35.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:35.991+0000 7f0692ffd640 1 -- 172.21.15.3:0/599848595 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f06940564d0 con 0x7f068c004d50 2024-09-12T12:24:36.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:36.211+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7f06a4154fb0 con 0x7f068c004d50 2024-09-12T12:24:37.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.211+0000 7f0692ffd640 1 -- 172.21.15.3:0/599848595 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v254) ==== 148+0+0 (secure 0 0 0) 0x7f0694055dd0 con 0x7f068c004d50 2024-09-12T12:24:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06840678c0 msgr2=0x7f0684069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06840678c0 0x7f0684069d80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0698009cd0 tx=0x7f0698008040 comp rx=0 tx=0).stop 2024-09-12T12:24:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 msgr2=0x7f06a40cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f06a40cf6e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0694001210 tx=0x7f069400df70 comp rx=0 tx=0).stop 2024-09-12T12:24:37.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 shutdown_connections 2024-09-12T12:24:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f06840678c0 0x7f0684069d80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06a41595f0 0x7f06a4099580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f06a40b27d0 0x7f06a40cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 --2- 172.21.15.3:0/599848595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f068c004d50 0x7f06a40cf6e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 >> 172.21.15.3:0/599848595 conn(0x7f06a4067e00 msgr2=0x7f06a4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 shutdown_connections 2024-09-12T12:24:37.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.215+0000 7f06a89c1640 1 -- 172.21.15.3:0/599848595 wait complete. 2024-09-12T12:24:37.238 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-12T12:24:37.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf788a8640 1 Processor -- start 2024-09-12T12:24:37.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf788a8640 1 -- start start 2024-09-12T12:24:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf788a8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7415f1d0 0x7fbf740cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf788a8640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7406ee70 con 0x7fbf7406ea90 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf788a8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf740cb410 con 0x7fbf7415ed20 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf788a8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf740cf050 con 0x7fbf7415f1d0 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7259c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbf7406ea90 0x7fbf7415e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:40542/0 (socket says 172.21.15.3:40542) 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7259c640 1 -- 172.21.15.3:0/3553127790 learned_addr learned my addr 172.21.15.3:0/3553127790 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf72d9d640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7415f1d0 0x7fbf740cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2571799292 0 0) 0x7fbf7406ee70 con 0x7fbf7406ea90 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf60003530 con 0x7fbf7406ea90 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 469112571 0 0) 0x7fbf60003530 con 0x7fbf7406ea90 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7415f1d0 msgr2=0x7fbf740cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7415f1d0 0x7fbf740cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 >> v1:172.21.15.142:6789/0 conn(0x7fbf7415ed20 legacy=0x7fbf7406d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740cce20 con 0x7fbf7406ea90 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fbf6c001680 con 0x7fbf7406ea90 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 >> v1:172.21.15.3:6789/0 conn(0x7fbf7406ea90 legacy=0x7fbf7415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600039f0 0x7fbf60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf60004810 0x7fbf60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7415f1d0 0x7fbf60005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6000bb00 con 0x7fbf60004810 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6000bcb0 con 0x7fbf600039f0 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6000beb0 con 0x7fbf7415f1d0 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf71d9b640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf60004810 0x7fbf60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf7259c640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600039f0 0x7fbf60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf71d9b640 1 -- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7415f1d0 msgr2=0x7fbf60005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf71d9b640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf7415f1d0 0x7fbf60005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf71d9b640 1 -- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600039f0 msgr2=0x7fbf60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:37.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf71d9b640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf600039f0 0x7fbf60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.367+0000 7fbf71d9b640 1 -- 172.21.15.3:0/3553127790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf60003530 con 0x7fbf60004810 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf71d9b640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf60004810 0x7fbf60004c90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c001e70 tx=0x7fbf5c01c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a54ffb99b70839e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c001a90 con 0x7fbf60004810 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c003b80 con 0x7fbf60004810 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf7159a640 1 -- 172.21.15.3:0/3553127790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c005080 con 0x7fbf60004810 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 -- 172.21.15.3:0/3553127790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf60004810 msgr2=0x7fbf60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 --2- 172.21.15.3:0/3553127790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf60004810 0x7fbf60004c90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c001e70 tx=0x7fbf5c01c750 comp rx=0 tx=0).stop 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf72d9d640 1 -- 172.21.15.3:0/3553127790 reap_dead start 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 -- 172.21.15.3:0/3553127790 shutdown_connections 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 -- 172.21.15.3:0/3553127790 >> 172.21.15.3:0/3553127790 conn(0x7fbf740d0a70 msgr2=0x7fbf740d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:37.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 -- 172.21.15.3:0/3553127790 shutdown_connections 2024-09-12T12:24:37.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 -- 172.21.15.3:0/3553127790 wait complete. 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 Processor -- start 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.371+0000 7fbf788a8640 1 -- start start 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf60004810 0x7fbf7414ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf740b5740 0x7fbf741555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf741667a0 0x7fbf74166c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74074c10 con 0x7fbf740b5740 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf740ccad0 con 0x7fbf741667a0 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbf74074d90 con 0x7fbf60004810 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf740b5740 0x7fbf741555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf740b5740 0x7fbf741555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41910/0 (socket says 172.21.15.3:41910) 2024-09-12T12:24:37.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 -- 172.21.15.3:0/2687399296 learned_addr learned my addr 172.21.15.3:0/2687399296 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 -- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf60004810 msgr2=0x7fbf7414ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf60004810 0x7fbf7414ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 -- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf741667a0 msgr2=0x7fbf74166c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf741667a0 0x7fbf74166c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 -- 172.21.15.3:0/2687399296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf740b5f20 con 0x7fbf740b5740 2024-09-12T12:24:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf71d9b640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf740b5740 0x7fbf741555a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c01cc30 tx=0x7fbf60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf6affd640 1 -- 172.21.15.3:0/2687399296 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0bd070 con 0x7fbf740b5740 2024-09-12T12:24:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf6affd640 1 -- 172.21.15.3:0/2687399296 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c005690 con 0x7fbf740b5740 2024-09-12T12:24:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf6affd640 1 -- 172.21.15.3:0/2687399296 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c001a90 con 0x7fbf740b5740 2024-09-12T12:24:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf74150150 con 0x7fbf740b5740 2024-09-12T12:24:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf74150710 con 0x7fbf740b5740 2024-09-12T12:24:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf74075180 con 0x7fbf740b5740 2024-09-12T12:24:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf6affd640 1 -- 172.21.15.3:0/2687399296 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf5c0b1040 con 0x7fbf740b5740 2024-09-12T12:24:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf6affd640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbf540678f0 0x7fbf54069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.375+0000 7fbf6affd640 1 -- 172.21.15.3:0/2687399296 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7fbf5c11f5a0 con 0x7fbf740b5740 2024-09-12T12:24:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.379+0000 7fbf7259c640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbf540678f0 0x7fbf54069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.379+0000 7fbf7259c640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbf540678f0 0x7fbf54069db0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c001210 tx=0x7fbf6c001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.383+0000 7fbf6affd640 1 -- 172.21.15.3:0/2687399296 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbf5c0ad020 con 0x7fbf740b5740 2024-09-12T12:24:37.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.543+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbf740b1160 con 0x7fbf740b5740 2024-09-12T12:24:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf6affd640 1 -- 172.21.15.3:0/2687399296 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fbf5c016490 con 0x7fbf740b5740 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbf540678f0 msgr2=0x7fbf54069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbf540678f0 0x7fbf54069db0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c001210 tx=0x7fbf6c001120 comp rx=0 tx=0).stop 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf740b5740 msgr2=0x7fbf741555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf740b5740 0x7fbf741555a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c01cc30 tx=0x7fbf60005840 comp rx=0 tx=0).stop 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 shutdown_connections 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbf540678f0 0x7fbf54069db0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf741667a0 0x7fbf74166c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbf740b5740 0x7fbf741555a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 --2- 172.21.15.3:0/2687399296 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbf60004810 0x7fbf7414ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.551+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 >> 172.21.15.3:0/2687399296 conn(0x7fbf740d0a70 msgr2=0x7fbf740ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:37.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.555+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 shutdown_connections 2024-09-12T12:24:37.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.555+0000 7fbf788a8640 1 -- 172.21.15.3:0/2687399296 wait complete. 2024-09-12T12:24:37.574 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:24:37.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:24:37.627 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:24:37.684 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:24:37.685 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:37.685 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:37.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:24:37.691 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:24:37.691 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:37.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:24:37.697 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:37.697 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:37.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:24:37.738 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:24:37.738 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:37.744 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:24:37.744 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-12T12:24:37.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.911+0000 7f101ad84640 1 Processor -- start 2024-09-12T12:24:37.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.911+0000 7f101ad84640 1 -- start start 2024-09-12T12:24:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f101ad84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 0x7f10140d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f101ad84640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10140b5000 con 0x7f101415c710 2024-09-12T12:24:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f101ad84640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10140b3510 con 0x7f1014073cb0 2024-09-12T12:24:37.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f101ad84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1014074090 con 0x7f10140b30e0 2024-09-12T12:24:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f101415c710 0x7f10140ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53324/0 (socket says 172.21.15.3:53324) 2024-09-12T12:24:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 -- 172.21.15.3:0/2582840459 learned_addr learned my addr 172.21.15.3:0/2582840459 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1019321640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 0x7f10140d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176124780 0 0) 0x7f10140b5000 con 0x7f101415c710 2024-09-12T12:24:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ffc003530 con 0x7f101415c710 2024-09-12T12:24:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 935683629 0 0) 0x7f0ffc003530 con 0x7f101415c710 2024-09-12T12:24:37.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 msgr2=0x7f10140d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 0x7f10140d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 >> v1:172.21.15.142:6789/0 conn(0x7f1014073cb0 legacy=0x7f10140adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10140b37c0 con 0x7f101415c710 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1008001680 con 0x7f101415c710 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 >> v1:172.21.15.3:6789/0 conn(0x7f101415c710 legacy=0x7f10140ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 0x7f0ffc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ffc003f70 0x7f0ffc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc004d50 0x7f0ffc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc00ba80 con 0x7f0ffc003f70 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc00bc30 con 0x7f0ffc004d50 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0ffc00be30 con 0x7f10140b30e0 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ffc003f70 0x7f0ffc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 -- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 msgr2=0x7f0ffc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1019321640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc004d50 0x7f0ffc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1018b20640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 0x7f0ffc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 0x7f0ffc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 -- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc004d50 msgr2=0x7f0ffc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc004d50 0x7f0ffc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 -- 172.21.15.3:0/2582840459 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc003530 con 0x7f0ffc003f70 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1019321640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ffc004d50 0x7f0ffc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:37.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1018b20640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10140b30e0 0x7f0ffc0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:37.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.915+0000 7f1013fff640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ffc003f70 0x7f0ffc004810 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f100800c240 tx=0x7f100801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce76245222d0c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:37.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008007d20 con 0x7f0ffc003f70 2024-09-12T12:24:37.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1008005ae0 con 0x7f0ffc003f70 2024-09-12T12:24:37.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f10137fe640 1 -- 172.21.15.3:0/2582840459 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008007070 con 0x7f0ffc003f70 2024-09-12T12:24:37.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- 172.21.15.3:0/2582840459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ffc003f70 msgr2=0x7f0ffc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 --2- 172.21.15.3:0/2582840459 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ffc003f70 0x7f0ffc004810 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f100800c240 tx=0x7f100801f3f0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1019321640 1 -- 172.21.15.3:0/2582840459 reap_dead start 2024-09-12T12:24:37.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- 172.21.15.3:0/2582840459 shutdown_connections 2024-09-12T12:24:37.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- 172.21.15.3:0/2582840459 >> 172.21.15.3:0/2582840459 conn(0x7f1014067e00 msgr2=0x7f1014068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:37.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- 172.21.15.3:0/2582840459 shutdown_connections 2024-09-12T12:24:37.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- 172.21.15.3:0/2582840459 wait complete. 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 Processor -- start 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- start start 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc003f70 0x7f10141471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10141f1bc0 0x7f10141f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10140cdb50 0x7f10140ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10140ad6f0 con 0x7f10141f1bc0 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f10140b3d80 con 0x7f10140cdb50 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f10140ae320 con 0x7f0ffc003f70 2024-09-12T12:24:37.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10141f1bc0 0x7f10141f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10141f1bc0 0x7f10141f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48924/0 (socket says 172.21.15.3:48924) 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1018b20640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc003f70 0x7f10141471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 -- 172.21.15.3:0/3174576903 learned_addr learned my addr 172.21.15.3:0/3174576903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1019321640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10140cdb50 0x7f10140ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 -- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc003f70 msgr2=0x7f10141471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc003f70 0x7f10141471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 -- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10140cdb50 msgr2=0x7f10140ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10140cdb50 0x7f10140ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:37.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 -- 172.21.15.3:0/3174576903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10141f2580 con 0x7f10141f1bc0 2024-09-12T12:24:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1019321640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10140cdb50 0x7f10140ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f1013fff640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10141f1bc0 0x7f10141f2040 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f100800c240 tx=0x7f0ffc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f10117fa640 1 -- 172.21.15.3:0/3174576903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10080047f0 con 0x7f10141f1bc0 2024-09-12T12:24:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f10117fa640 1 -- 172.21.15.3:0/3174576903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1008004990 con 0x7f10141f1bc0 2024-09-12T12:24:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10140ce880 con 0x7f10141f1bc0 2024-09-12T12:24:37.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f10117fa640 1 -- 172.21.15.3:0/3174576903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008004d40 con 0x7f10141f1bc0 2024-09-12T12:24:37.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.919+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1014147f20 con 0x7f10141f1bc0 2024-09-12T12:24:37.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.923+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10140ceb60 con 0x7f10141f1bc0 2024-09-12T12:24:37.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.923+0000 7f10117fa640 1 -- 172.21.15.3:0/3174576903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1008007070 con 0x7f10141f1bc0 2024-09-12T12:24:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.923+0000 7f10117fa640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10000678f0 0x7f1000069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.923+0000 7f10117fa640 1 -- 172.21.15.3:0/3174576903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f1008122f90 con 0x7f10141f1bc0 2024-09-12T12:24:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.923+0000 7f1018b20640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10000678f0 0x7f1000069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:37.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.923+0000 7f1018b20640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10000678f0 0x7f1000069db0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1004007fd0 tx=0x7f1004009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:37.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:37.927+0000 7f10117fa640 1 -- 172.21.15.3:0/3174576903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f10080b7070 con 0x7f10141f1bc0 2024-09-12T12:24:38.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.127+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10140adc10 con 0x7f10141f1bc0 2024-09-12T12:24:38.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.127+0000 7f10117fa640 1 -- 172.21.15.3:0/3174576903 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+3908 (secure 0 0 0) 0x7f1008019100 con 0x7f10141f1bc0 2024-09-12T12:24:38.132 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:38.133 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":56,"btime":"2024-09-12T12:24:32:176156+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/196493794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":196493794},{"type":"v1","addr":"172.21.15.142:6835","nonce":196493794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1488157440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1488157440},{"type":"v1","addr":"172.21.15.142:6837","nonce":1488157440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/2920554416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":2920554416},{"type":"v1","addr":"172.21.15.3:6833","nonce":2920554416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":6666,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3211643738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3211643738},{"type":"v1","addr":"172.21.15.3:6835","nonce":3211643738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[]} 2024-09-12T12:24:38.133 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 56 2024-09-12T12:24:38.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10000678f0 msgr2=0x7f1000069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:38.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10000678f0 0x7f1000069db0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1004007fd0 tx=0x7f1004009040 comp rx=0 tx=0).stop 2024-09-12T12:24:38.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10141f1bc0 msgr2=0x7f10141f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:38.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10141f1bc0 0x7f10141f2040 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f100800c240 tx=0x7f0ffc005840 comp rx=0 tx=0).stop 2024-09-12T12:24:38.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 shutdown_connections 2024-09-12T12:24:38.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f10000678f0 0x7f1000069db0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10140cdb50 0x7f10140ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10141f1bc0 0x7f10141f2040 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 --2- 172.21.15.3:0/3174576903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ffc003f70 0x7f10141471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 >> 172.21.15.3:0/3174576903 conn(0x7f1014067e00 msgr2=0x7f1014073330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 shutdown_connections 2024-09-12T12:24:38.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.131+0000 7f101ad84640 1 -- 172.21.15.3:0/3174576903 wait complete. 2024-09-12T12:24:38.152 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:24:38.153 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-12T12:24:38.153 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:24:38.153 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:24:38.153 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:24:38.153 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-12T12:24:38.153 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:24:38.153 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:24:38.154 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:24:38.154 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-12T12:24:38.154 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:24:38.154 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:24:38.154 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:24:38.154 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-12T12:24:38.154 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:24:38.155 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:24:38.164 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:24:38.164 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:24:38.165 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:24:38.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:24:38.171 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:24:38.172 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:24:38.174 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:24:38.175 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:24:38.206 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:24:38.207 DEBUG:teuthology.parallel:result is None 2024-09-12T12:24:38.207 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:24:38.207 DEBUG:teuthology.parallel:result is None 2024-09-12T12:24:38.220 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:24:38.220 DEBUG:teuthology.parallel:result is None 2024-09-12T12:24:38.221 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:24:38.221 DEBUG:teuthology.parallel:result is None 2024-09-12T12:24:38.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-12T12:24:38.269 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:24:38.269 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:24:38.289 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:24:38.294 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:24:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d3301f640 1 Processor -- start 2024-09-12T12:24:38.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d3301f640 1 -- start start 2024-09-12T12:24:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d3301f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0d1190 0x7f4d2c0d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d3301f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4d2c0d1570 0x7f4d2c0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d3301f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2c0d30b0 con 0x7f4d2c0cf950 2024-09-12T12:24:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d3301f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4d2c0d3cc0 con 0x7f4d2c0d1570 2024-09-12T12:24:38.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d3301f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c0d3390 con 0x7f4d2c0d1190 2024-09-12T12:24:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d315bc640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f4d2c0cf950 0x7f4d2c0b7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:39376/0 (socket says 172.21.15.3:39376) 2024-09-12T12:24:38.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d315bc640 1 -- 172.21.15.3:0/2333435736 learned_addr learned my addr 172.21.15.3:0/2333435736 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d30dbb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0d1190 0x7f4d2c0d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d2bfff640 1 --2- 172.21.15.3:0/2333435736 >> v2:172.21.15.142:3300/0 conn(0x7f4d2c0d1570 0x7f4d2c0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d30dbb640 1 -- 172.21.15.3:0/2333435736 >> v2:172.21.15.142:3300/0 conn(0x7f4d2c0d1570 msgr2=0x7f4d2c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d30dbb640 1 --2- 172.21.15.3:0/2333435736 >> v2:172.21.15.142:3300/0 conn(0x7f4d2c0d1570 0x7f4d2c0cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d30dbb640 1 -- 172.21.15.3:0/2333435736 >> v1:172.21.15.142:6789/0 conn(0x7f4d2c0cf950 legacy=0x7f4d2c0b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d2b7fe640 1 -- 172.21.15.3:0/2333435736 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259853133 0 0) 0x7f4d2c0d30b0 con 0x7f4d2c0cf950 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d30dbb640 1 -- 172.21.15.3:0/2333435736 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c0d3670 con 0x7f4d2c0d1190 2024-09-12T12:24:38.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.339+0000 7f4d2bfff640 1 --2- 172.21.15.3:0/2333435736 >> v2:172.21.15.142:3300/0 conn(0x7f4d2c0d1570 0x7f4d2c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:38.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d30dbb640 1 --2- 172.21.15.3:0/2333435736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0d1190 0x7f4d2c0d4150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009d90 tx=0x7f4d1c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93248efe3dc3076 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d2b7fe640 1 -- 172.21.15.3:0/2333435736 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0b2070 con 0x7f4d2c0d1190 2024-09-12T12:24:38.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d2b7fe640 1 -- 172.21.15.3:0/2333435736 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1c0a5a20 con 0x7f4d2c0d1190 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d2b7fe640 1 -- 172.21.15.3:0/2333435736 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0a5df0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 -- 172.21.15.3:0/2333435736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0d1190 msgr2=0x7f4d2c0d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 --2- 172.21.15.3:0/2333435736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0d1190 0x7f4d2c0d4150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c009d90 tx=0x7f4d1c01c940 comp rx=0 tx=0).stop 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 -- 172.21.15.3:0/2333435736 shutdown_connections 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 --2- 172.21.15.3:0/2333435736 >> v2:172.21.15.142:3300/0 conn(0x7f4d2c0d1570 0x7f4d2c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 --2- 172.21.15.3:0/2333435736 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0d1190 0x7f4d2c0d4150 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 -- 172.21.15.3:0/2333435736 >> 172.21.15.3:0/2333435736 conn(0x7f4d2c067a00 msgr2=0x7f4d2c067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 -- 172.21.15.3:0/2333435736 shutdown_connections 2024-09-12T12:24:38.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 -- 172.21.15.3:0/2333435736 wait complete. 2024-09-12T12:24:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 Processor -- start 2024-09-12T12:24:38.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 -- start start 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.343+0000 7f4d3301f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0cf950 0x7f4d2c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d2c0d1190 0x7f4d2c15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d2c0d1570 0x7f4d2c15d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c0b83e0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c0b8260 con 0x7f4d2c0d1570 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c15e860 con 0x7f4d2c0cf950 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d2c0d1190 0x7f4d2c15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d2c0d1190 0x7f4d2c15d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48968/0 (socket says 172.21.15.3:48968) 2024-09-12T12:24:38.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 -- 172.21.15.3:0/3983299841 learned_addr learned my addr 172.21.15.3:0/3983299841 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d30dbb640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0cf950 0x7f4d2c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d315bc640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d2c0d1570 0x7f4d2c15d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 -- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0cf950 msgr2=0x7f4d2c1501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0cf950 0x7f4d2c1501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 -- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d2c0d1570 msgr2=0x7f4d2c15d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d2c0d1570 0x7f4d2c15d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 -- 172.21.15.3:0/3983299841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c0cd2f0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d30dbb640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0cf950 0x7f4d2c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d2bfff640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d2c0d1190 0x7f4d2c15d470 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e960 tx=0x7f4d1400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d297fa640 1 -- 172.21.15.3:0/3983299841 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14017070 con 0x7f4d2c0d1190 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d297fa640 1 -- 172.21.15.3:0/3983299841 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1400ca00 con 0x7f4d2c0d1190 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d297fa640 1 -- 172.21.15.3:0/3983299841 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400cdd0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c0cd5d0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c0cdb90 con 0x7f4d2c0d1190 2024-09-12T12:24:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d2c0ba4c0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d297fa640 1 -- 172.21.15.3:0/3983299841 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d1401a070 con 0x7f4d2c0d1190 2024-09-12T12:24:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d297fa640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4d0c0678c0 0x7f4d0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:38.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d297fa640 1 -- 172.21.15.3:0/3983299841 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 4943+0+0 (secure 0 0 0) 0x7f4d14085bb0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d30dbb640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4d0c0678c0 0x7f4d0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:38.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.347+0000 7f4d30dbb640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4d0c0678c0 0x7f4d0c069d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0af000 tx=0x7f4d1c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:38.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.351+0000 7f4d297fa640 1 -- 172.21.15.3:0/3983299841 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4d140563a0 con 0x7f4d2c0d1190 2024-09-12T12:24:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:38.527+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4d2c0cf410 con 0x7f4d2c0d1190 2024-09-12T12:24:39.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.583+0000 7f4d297fa640 1 -- 172.21.15.3:0/3983299841 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) ==== 97+0+0 (secure 0 0 0) 0x7f4d1405a6a0 con 0x7f4d2c0d1190 2024-09-12T12:24:39.588 INFO:teuthology.orchestra.run.smithi003.stderr: removed all blocklist entries 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4d0c0678c0 msgr2=0x7f4d0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4d0c0678c0 0x7f4d0c069d80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0af000 tx=0x7f4d1c0a7070 comp rx=0 tx=0).stop 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d2c0d1190 msgr2=0x7f4d2c15d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d2c0d1190 0x7f4d2c15d470 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e960 tx=0x7f4d1400ee20 comp rx=0 tx=0).stop 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 shutdown_connections 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4d0c0678c0 0x7f4d0c069d80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d2c0d1570 0x7f4d2c15d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d2c0d1190 0x7f4d2c15d470 secure :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e960 tx=0x7f4d1400ee20 comp rx=0 tx=0).stop 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 --2- 172.21.15.3:0/3983299841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4d2c0cf950 0x7f4d2c1501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 >> 172.21.15.3:0/3983299841 conn(0x7f4d2c067a00 msgr2=0x7f4d2c156c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 shutdown_connections 2024-09-12T12:24:39.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.587+0000 7f4d3301f640 1 -- 172.21.15.3:0/3983299841 wait complete. 2024-09-12T12:24:39.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-12T12:24:39.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab1638a640 1 Processor -- start 2024-09-12T12:24:39.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab1638a640 1 -- start start 2024-09-12T12:24:39.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab1638a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fab1015c710 0x7fab100b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab1638a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab100b5000 con 0x7fab10073cb0 2024-09-12T12:24:39.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab1638a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab100b37c0 con 0x7fab100ae880 2024-09-12T12:24:39.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab1638a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fab10074090 con 0x7fab1015c710 2024-09-12T12:24:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fab10073cb0 0x7fab100adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53354/0 (socket says 172.21.15.3:53354) 2024-09-12T12:24:39.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0ffff640 1 -- 172.21.15.3:0/2389267925 learned_addr learned my addr 172.21.15.3:0/2389267925 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0f7fe640 1 --2- 172.21.15.3:0/2389267925 >> v2:172.21.15.142:3300/0 conn(0x7fab1015c710 0x7fab100b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471308145 0 0) 0x7fab100b5000 con 0x7fab10073cb0 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faafc003530 con 0x7fab10073cb0 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4223083056 0 0) 0x7faafc003530 con 0x7fab10073cb0 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 >> v2:172.21.15.142:3300/0 conn(0x7fab1015c710 msgr2=0x7fab100b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 --2- 172.21.15.3:0/2389267925 >> v2:172.21.15.142:3300/0 conn(0x7fab1015c710 0x7fab100b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 >> v1:172.21.15.142:6789/0 conn(0x7fab100ae880 legacy=0x7fab100d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab100b3aa0 con 0x7fab10073cb0 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7faaf8001a00 con 0x7fab10073cb0 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0f7fe640 1 --2- 172.21.15.3:0/2389267925 >> v2:172.21.15.142:3300/0 conn(0x7fab1015c710 0x7fab100b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 >> v1:172.21.15.3:6789/0 conn(0x7fab10073cb0 legacy=0x7fab100adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc0039c0 0x7faafc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc004810 0x7faafc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faafc0051d0 0x7faafc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faafc00c0f0 con 0x7faafc0051d0 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faafc00c2a0 con 0x7faafc0039c0 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faafc00c4a0 con 0x7faafc004810 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab14927640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faafc0051d0 0x7faafc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0ffff640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc0039c0 0x7faafc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab14927640 1 -- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc004810 msgr2=0x7faafc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0f7fe640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc004810 0x7faafc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab14927640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc004810 0x7faafc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab14927640 1 -- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc0039c0 msgr2=0x7faafc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab14927640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc0039c0 0x7faafc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab14927640 1 -- 172.21.15.3:0/2389267925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc003530 con 0x7faafc0051d0 2024-09-12T12:24:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0ffff640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faafc0039c0 0x7faafc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:39.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.739+0000 7fab0f7fe640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc004810 0x7faafc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab14927640 1 -- 172.21.15.3:0/2389267925 reap_dead start 2024-09-12T12:24:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab14927640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faafc0051d0 0x7faafc00b9d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fab08001780 tx=0x7fab0801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff9aa526055d293 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080013a0 con 0x7faafc0051d0 2024-09-12T12:24:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab080a9070 con 0x7faafc0051d0 2024-09-12T12:24:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab0effd640 1 -- 172.21.15.3:0/2389267925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08003600 con 0x7faafc0051d0 2024-09-12T12:24:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 -- 172.21.15.3:0/2389267925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faafc0051d0 msgr2=0x7faafc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faafc0051d0 0x7faafc00b9d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fab08001780 tx=0x7fab0801c420 comp rx=0 tx=0).stop 2024-09-12T12:24:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 -- 172.21.15.3:0/2389267925 shutdown_connections 2024-09-12T12:24:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 --2- 172.21.15.3:0/2389267925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faafc0051d0 0x7faafc00b9d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 -- 172.21.15.3:0/2389267925 >> 172.21.15.3:0/2389267925 conn(0x7fab10067e00 msgr2=0x7fab10068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 -- 172.21.15.3:0/2389267925 shutdown_connections 2024-09-12T12:24:39.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 -- 172.21.15.3:0/2389267925 wait complete. 2024-09-12T12:24:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.743+0000 7fab1638a640 1 Processor -- start 2024-09-12T12:24:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 -- start start 2024-09-12T12:24:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0051d0 0x7fab10158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab100c9c50 0x7fab100cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab100cbe30 0x7fab100d0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab100aecb0 con 0x7fab100cbe30 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fab100ad6f0 con 0x7fab100c9c50 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fab100aee30 con 0x7faafc0051d0 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab100cbe30 0x7fab100d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab100cbe30 0x7fab100d0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48996/0 (socket says 172.21.15.3:48996) 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 -- 172.21.15.3:0/2127396586 learned_addr learned my addr 172.21.15.3:0/2127396586 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0f7fe640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab100c9c50 0x7fab100cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0ffff640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0051d0 0x7fab10158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 -- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0051d0 msgr2=0x7fab10158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0051d0 0x7fab10158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 -- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab100c9c50 msgr2=0x7fab100cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab100c9c50 0x7fab100cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 -- 172.21.15.3:0/2127396586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab10158eb0 con 0x7fab100cbe30 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0f7fe640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab100c9c50 0x7fab100cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0ffff640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0051d0 0x7fab10158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab14927640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab100cbe30 0x7fab100d0550 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fab0801c900 tx=0x7faafc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0cff9640 1 -- 172.21.15.3:0/2127396586 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080013a0 con 0x7fab100cbe30 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0cff9640 1 -- 172.21.15.3:0/2127396586 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab080050d0 con 0x7fab100cbe30 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab100ca0e0 con 0x7fab100cbe30 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0cff9640 1 -- 172.21.15.3:0/2127396586 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08007a40 con 0x7fab100cbe30 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab101f4110 con 0x7fab100cbe30 2024-09-12T12:24:39.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab100b30e0 con 0x7fab100cbe30 2024-09-12T12:24:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0cff9640 1 -- 172.21.15.3:0/2127396586 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab080ab070 con 0x7fab100cbe30 2024-09-12T12:24:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0cff9640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faaf00678f0 0x7faaf0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.747+0000 7fab0cff9640 1 -- 172.21.15.3:0/2127396586 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 4857+0+0 (secure 0 0 0) 0x7fab0811edc0 con 0x7fab100cbe30 2024-09-12T12:24:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.751+0000 7fab0ffff640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faaf00678f0 0x7faaf0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:39.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.751+0000 7fab0ffff640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faaf00678f0 0x7faaf0069db0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faaf8001590 tx=0x7faaf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:39.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.751+0000 7fab0cff9640 1 -- 172.21.15.3:0/2127396586 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fab10158eb0 con 0x7fab100cbe30 2024-09-12T12:24:39.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.975+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fab100add50 con 0x7fab100cbe30 2024-09-12T12:24:39.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.975+0000 7fab0cff9640 1 -- 172.21.15.3:0/2127396586 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fab08016130 con 0x7fab100cbe30 2024-09-12T12:24:39.979 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:39.979 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "auth_dump": [ 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.a", 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmYGZnGhAAM9wRr66OPbtFJ36z5DV4Fg==", 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.980 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.b", 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbIkfKRAAesWC0dq9HWrUnAhujNb3WA==", 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.c", 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKmUbIxAAbuXCeY6VK7+NUJxVepJ8Gg==", 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.981 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mds.d", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmEkTILhAATU/Z/XFtSNGExVDjmRkrHw==", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:24:39.982 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.0", 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAs3OJm4JL6CBAAiCuhac5FRE0xgbS1rIZu7w==", 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.983 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.1", 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAv3OJm2SgoGBAAgLLD3Pt7g+oXNtNtmVwwbA==", 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.2", 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAy3OJmao6iNxAAGdRg22FzD86bNM0YOls1Qw==", 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.984 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.3", 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA23OJmjjDuHBAATP58kNL0lMMaCSxxIFQ1Dg==", 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.4", 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA83OJmftw/JBAAjdxgvkIjK3TqX5kFAPUNkA==", 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.986 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.5", 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQA/3OJmRv9AMxAAwlySRhewDq1HNyu4yMy0HQ==", 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.6", 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBD3OJmgtErCBAAIlVEnsvNB8bybwgXYnT2jQ==", 2024-09-12T12:24:39.987 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "osd.7", 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQBG3OJmGhPPFhAAaPxzlT4MYzjC4yT96ThSpA==", 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow profile osd", 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile osd", 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.0", 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA==", 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow r", 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.1", 2024-09-12T12:24:39.989 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmzoL2ABAAo2GnmEwZb5rJGw1bpeRrVw==", 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.2", 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmVFveORAAa/YKmCYQw532MMQtHvffOw==", 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.990 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.3", 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAp3OJmEr4rBhAAeqx+bHormS5C8VfXZ/pwNA==", 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow", 2024-09-12T12:24:39.991 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow r", 2024-09-12T12:24:39.992 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow rw", 2024-09-12T12:24:39.992 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow rwx" 2024-09-12T12:24:39.992 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.992 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.992 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.992 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "client.admin", 2024-09-12T12:24:39.992 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmKNVOAxAAMmejHiBys/ZYsGdB0oiroQ==", 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr": "allow *", 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow *", 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.x", 2024-09-12T12:24:39.993 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJmNW74FhAA3YxyVFhGK34CX675JDCClA==", 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": "mgr.y", 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "key": "AQAo3OJm8QVOFBAAL6ijRNsEzaJUt4PXBr6ySg==", 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": { 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": "allow *", 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "mon": "allow profile mgr", 2024-09-12T12:24:39.994 INFO:teuthology.orchestra.run.smithi003.stdout: "osd": "allow *" 2024-09-12T12:24:39.995 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.995 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:39.995 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:24:39.995 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:24:39.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faaf00678f0 msgr2=0x7faaf0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faaf00678f0 0x7faaf0069db0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faaf8001590 tx=0x7faaf8009040 comp rx=0 tx=0).stop 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab100cbe30 msgr2=0x7fab100d0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab100cbe30 0x7fab100d0550 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fab0801c900 tx=0x7faafc005840 comp rx=0 tx=0).stop 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 shutdown_connections 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7faaf00678f0 0x7faaf0069db0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab100cbe30 0x7fab100d0550 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fab100c9c50 0x7fab100cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 --2- 172.21.15.3:0/2127396586 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faafc0051d0 0x7fab10158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 >> 172.21.15.3:0/2127396586 conn(0x7fab10067e00 msgr2=0x7fab1015a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 shutdown_connections 2024-09-12T12:24:39.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:39.979+0000 7fab1638a640 1 -- 172.21.15.3:0/2127396586 wait complete. 2024-09-12T12:24:40.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-12T12:24:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f218719c640 1 Processor -- start 2024-09-12T12:24:40.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f218719c640 1 -- start start 2024-09-12T12:24:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f218719c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f21800b3f40 0x7f21800af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f218719c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218015fad0 con 0x7f2180158b10 2024-09-12T12:24:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f218719c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218015eec0 con 0x7f21800b3ab0 2024-09-12T12:24:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f218719c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f218015f1a0 con 0x7f21800b3f40 2024-09-12T12:24:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2184f38640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2180158b10 0x7f2180162b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53366/0 (socket says 172.21.15.3:53366) 2024-09-12T12:24:40.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2184f38640 1 -- 172.21.15.3:0/563080224 learned_addr learned my addr 172.21.15.3:0/563080224 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:40.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2185739640 1 --2- 172.21.15.3:0/563080224 >> v2:172.21.15.142:3300/0 conn(0x7f21800b3f40 0x7f21800af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4134893511 0 0) 0x7f218015fad0 con 0x7f2180158b10 2024-09-12T12:24:40.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2168003530 con 0x7f2180158b10 2024-09-12T12:24:40.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1137851772 0 0) 0x7f2168003530 con 0x7f2180158b10 2024-09-12T12:24:40.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 >> v2:172.21.15.142:3300/0 conn(0x7f21800b3f40 msgr2=0x7f21800af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 --2- 172.21.15.3:0/563080224 >> v2:172.21.15.142:3300/0 conn(0x7f21800b3f40 0x7f21800af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 >> v1:172.21.15.142:6789/0 conn(0x7f21800b3ab0 legacy=0x7f21800aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218015f480 con 0x7f2180158b10 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f2178001a00 con 0x7f2180158b10 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 >> v1:172.21.15.3:6789/0 conn(0x7f2180158b10 legacy=0x7f2180162b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680039c0 0x7f2168003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2168004810 0x7f2168004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21680051d0 0x7f216800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f218015f1a0 con 0x7f21680051d0 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f216800c070 con 0x7f21680039c0 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f216800c270 con 0x7f2168004810 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2185739640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21680051d0 0x7f216800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f217ffff640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2168004810 0x7f2168004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2185739640 1 -- 172.21.15.3:0/563080224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2168004810 msgr2=0x7f2168004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:40.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2185739640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2168004810 0x7f2168004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2185739640 1 -- 172.21.15.3:0/563080224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680039c0 msgr2=0x7f2168003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:40.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2185739640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21680039c0 0x7f2168003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.115+0000 7f2185739640 1 -- 172.21.15.3:0/563080224 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2168003530 con 0x7f21680051d0 2024-09-12T12:24:40.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f2185739640 1 -- 172.21.15.3:0/563080224 reap_dead start 2024-09-12T12:24:40.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f2185739640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21680051d0 0x7f216800b9d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2170009ae0 tx=0x7f217001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebc343b91246d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:40.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21700096b0 con 0x7f21680051d0 2024-09-12T12:24:40.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2170004020 con 0x7f21680051d0 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f217f7fe640 1 -- 172.21.15.3:0/563080224 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170004410 con 0x7f21680051d0 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 -- 172.21.15.3:0/563080224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21680051d0 msgr2=0x7f216800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21680051d0 0x7f216800b9d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2170009ae0 tx=0x7f217001cab0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 -- 172.21.15.3:0/563080224 shutdown_connections 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 --2- 172.21.15.3:0/563080224 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21680051d0 0x7f216800b9d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 -- 172.21.15.3:0/563080224 >> 172.21.15.3:0/563080224 conn(0x7f21800b27b0 msgr2=0x7f21800b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 -- 172.21.15.3:0/563080224 shutdown_connections 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 -- 172.21.15.3:0/563080224 wait complete. 2024-09-12T12:24:40.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 Processor -- start 2024-09-12T12:24:40.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.119+0000 7f218719c640 1 -- start start 2024-09-12T12:24:40.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21680051d0 0x7f21800cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2180147d90 0x7f2180146320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218006e950 0x7f218006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2180158ef0 con 0x7f218006e950 2024-09-12T12:24:40.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21800b4340 con 0x7f2180147d90 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21800b5bf0 con 0x7f21680051d0 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218006e950 0x7f218006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218006e950 0x7f218006edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49020/0 (socket says 172.21.15.3:49020) 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 -- 172.21.15.3:0/1119092946 learned_addr learned my addr 172.21.15.3:0/1119092946 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f217ffff640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2180147d90 0x7f2180146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2184f38640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21680051d0 0x7f21800cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 -- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21680051d0 msgr2=0x7f21800cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21680051d0 0x7f21800cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 -- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2180147d90 msgr2=0x7f2180146320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2180147d90 0x7f2180146320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 -- 172.21.15.3:0/1119092946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2180146b90 con 0x7f218006e950 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f2185739640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218006e950 0x7f218006edd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f21700984c0 tx=0x7f2168005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f217d7fa640 1 -- 172.21.15.3:0/1119092946 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21700b2070 con 0x7f218006e950 2024-09-12T12:24:40.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f217d7fa640 1 -- 172.21.15.3:0/1119092946 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21700bb070 con 0x7f218006e950 2024-09-12T12:24:40.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f217d7fa640 1 -- 172.21.15.3:0/1119092946 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21700096b0 con 0x7f218006e950 2024-09-12T12:24:40.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2180146e70 con 0x7f218006e950 2024-09-12T12:24:40.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2180147430 con 0x7f218006e950 2024-09-12T12:24:40.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218015eb20 con 0x7f218006e950 2024-09-12T12:24:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f217d7fa640 1 -- 172.21.15.3:0/1119092946 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f21700a5cf0 con 0x7f218006e950 2024-09-12T12:24:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.123+0000 7f217d7fa640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f216c0678c0 0x7f216c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.127+0000 7f217d7fa640 1 -- 172.21.15.3:0/1119092946 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 4857+0+0 (secure 0 0 0) 0x7f217011ff00 con 0x7f218006e950 2024-09-12T12:24:40.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.127+0000 7f2184f38640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f216c0678c0 0x7f216c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.127+0000 7f2184f38640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f216c0678c0 0x7f216c069d80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2178001570 tx=0x7f2178007e50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:40.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.127+0000 7f217d7fa640 1 -- 172.21.15.3:0/1119092946 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f21700c4020 con 0x7f218006e950 2024-09-12T12:24:40.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.295+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f21800cd290 con 0x7f218006e950 2024-09-12T12:24:40.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.295+0000 7f217d7fa640 1 -- 172.21.15.3:0/1119092946 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) ==== 84+0+65 (secure 0 0 0) 0x7f21700167c0 con 0x7f218006e950 2024-09-12T12:24:40.299 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:40.300 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T12:24:40.300 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:24:40.300 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T12:24:40.301 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T12:24:40.301 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:40.301 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T12:24:40.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f216c0678c0 msgr2=0x7f216c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f216c0678c0 0x7f216c069d80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2178001570 tx=0x7f2178007e50 comp rx=0 tx=0).stop 2024-09-12T12:24:40.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218006e950 msgr2=0x7f218006edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218006e950 0x7f218006edd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f21700984c0 tx=0x7f2168005840 comp rx=0 tx=0).stop 2024-09-12T12:24:40.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 shutdown_connections 2024-09-12T12:24:40.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f216c0678c0 0x7f216c069d80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f218006e950 0x7f218006edd0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2180147d90 0x7f2180146320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 --2- 172.21.15.3:0/1119092946 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21680051d0 0x7f21800cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 >> 172.21.15.3:0/1119092946 conn(0x7f21800b27b0 msgr2=0x7f21800b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:40.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 shutdown_connections 2024-09-12T12:24:40.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.299+0000 7f218719c640 1 -- 172.21.15.3:0/1119092946 wait complete. 2024-09-12T12:24:40.320 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:24:40.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:24:40.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.451+0000 7f73ff989640 1 Processor -- start 2024-09-12T12:24:40.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.451+0000 7f73ff989640 1 -- start start 2024-09-12T12:24:40.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f73f81523f0 0x7f73f80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f80ad160 con 0x7f73f815c910 2024-09-12T12:24:40.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73f80ad440 con 0x7f73f80d08e0 2024-09-12T12:24:40.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f73f80d0d30 con 0x7f73f81523f0 2024-09-12T12:24:40.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fd725640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f73f81523f0 0x7f73f80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fcf24640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f73f815c910 0x7f73f80ae2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53376/0 (socket says 172.21.15.3:53376) 2024-09-12T12:24:40.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fd725640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f73f81523f0 0x7f73f80d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49026/0 (socket says 172.21.15.3:49026) 2024-09-12T12:24:40.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fcf24640 1 -- 172.21.15.3:0/520987462 learned_addr learned my addr 172.21.15.3:0/520987462 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:40.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fd725640 1 -- 172.21.15.3:0/520987462 >> v1:172.21.15.142:6789/0 conn(0x7f73f80d08e0 legacy=0x7f73f815ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fd725640 1 -- 172.21.15.3:0/520987462 >> v1:172.21.15.3:6789/0 conn(0x7f73f815c910 legacy=0x7f73f80ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fd725640 1 -- 172.21.15.3:0/520987462 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f80ad720 con 0x7f73f81523f0 2024-09-12T12:24:40.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73fd725640 1 --2- 172.21.15.3:0/520987462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f81523f0 0x7f73f80d03a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f73f40097c0 tx=0x7f73f401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999f7bc60472825b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:40.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73effff640 1 -- 172.21.15.3:0/520987462 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40a7070 con 0x7f73f81523f0 2024-09-12T12:24:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73effff640 1 -- 172.21.15.3:0/520987462 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f40b0070 con 0x7f73f81523f0 2024-09-12T12:24:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73effff640 1 -- 172.21.15.3:0/520987462 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40ab890 con 0x7f73f81523f0 2024-09-12T12:24:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- 172.21.15.3:0/520987462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f81523f0 msgr2=0x7f73f80d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 --2- 172.21.15.3:0/520987462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f81523f0 0x7f73f80d03a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f73f40097c0 tx=0x7f73f401c470 comp rx=0 tx=0).stop 2024-09-12T12:24:40.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- 172.21.15.3:0/520987462 shutdown_connections 2024-09-12T12:24:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 --2- 172.21.15.3:0/520987462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f81523f0 0x7f73f80d03a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- 172.21.15.3:0/520987462 >> 172.21.15.3:0/520987462 conn(0x7f73f8067bd0 msgr2=0x7f73f8067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- 172.21.15.3:0/520987462 shutdown_connections 2024-09-12T12:24:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.455+0000 7f73ff989640 1 -- 172.21.15.3:0/520987462 wait complete. 2024-09-12T12:24:40.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 Processor -- start 2024-09-12T12:24:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 -- start start 2024-09-12T12:24:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73f80d08e0 0x7f73f80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73f81523f0 0x7f73f80c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f815c910 0x7f73f815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f73f815ea60 con 0x7f73f815c910 2024-09-12T12:24:40.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f73f815e8e0 con 0x7f73f81523f0 2024-09-12T12:24:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f73f81695b0 con 0x7f73f80d08e0 2024-09-12T12:24:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f815c910 0x7f73f815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f815c910 0x7f73f815a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49042/0 (socket says 172.21.15.3:49042) 2024-09-12T12:24:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 -- 172.21.15.3:0/3281418176 learned_addr learned my addr 172.21.15.3:0/3281418176 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:40.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fd725640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73f80d08e0 0x7f73f80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fcf24640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73f81523f0 0x7f73f80c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 -- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73f80d08e0 msgr2=0x7f73f80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73f80d08e0 0x7f73f80714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 -- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73f81523f0 msgr2=0x7f73f80c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:40.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73f81523f0 0x7f73f80c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 -- 172.21.15.3:0/3281418176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f80cb7b0 con 0x7f73f815c910 2024-09-12T12:24:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fcf24640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73f81523f0 0x7f73f80c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73fdf26640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f815c910 0x7f73f815a4f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c730 tx=0x7f73f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73edffb640 1 -- 172.21.15.3:0/3281418176 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0003b80 con 0x7f73f815c910 2024-09-12T12:24:40.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73edffb640 1 -- 172.21.15.3:0/3281418176 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f0003d20 con 0x7f73f815c910 2024-09-12T12:24:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73edffb640 1 -- 172.21.15.3:0/3281418176 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f000aa30 con 0x7f73f815c910 2024-09-12T12:24:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f80cba90 con 0x7f73f815c910 2024-09-12T12:24:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f80cc050 con 0x7f73f815c910 2024-09-12T12:24:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.459+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f80d03a0 con 0x7f73f815c910 2024-09-12T12:24:40.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.463+0000 7f73edffb640 1 -- 172.21.15.3:0/3281418176 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f73f000abd0 con 0x7f73f815c910 2024-09-12T12:24:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.463+0000 7f73edffb640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f73dc0678c0 0x7f73dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.463+0000 7f73edffb640 1 -- 172.21.15.3:0/3281418176 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 4857+0+0 (secure 0 0 0) 0x7f73f00858a0 con 0x7f73f815c910 2024-09-12T12:24:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.463+0000 7f73fd725640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f73dc0678c0 0x7f73dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.463+0000 7f73fd725640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f73dc0678c0 0x7f73dc069d80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f73f40af000 tx=0x7f73f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.467+0000 7f73edffb640 1 -- 172.21.15.3:0/3281418176 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f73f0056690 con 0x7f73f815c910 2024-09-12T12:24:40.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:40.631+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f73f80cc380 con 0x7f73f815c910 2024-09-12T12:24:41.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.591+0000 7f73edffb640 1 -- 172.21.15.3:0/3281418176 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) ==== 138+0+0 (secure 0 0 0) 0x7f73f0055f90 con 0x7f73f815c910 2024-09-12T12:24:41.593 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T12:24:41.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f73dc0678c0 msgr2=0x7f73dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f73dc0678c0 0x7f73dc069d80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f73f40af000 tx=0x7f73f4008040 comp rx=0 tx=0).stop 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f815c910 msgr2=0x7f73f815a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f815c910 0x7f73f815a4f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f73f000c730 tx=0x7f73f000cbf0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 shutdown_connections 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f73dc0678c0 0x7f73dc069d80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f815c910 0x7f73f815a4f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73f81523f0 0x7f73f80c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 --2- 172.21.15.3:0/3281418176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73f80d08e0 0x7f73f80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 >> 172.21.15.3:0/3281418176 conn(0x7f73f8067bd0 msgr2=0x7f73f80736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 shutdown_connections 2024-09-12T12:24:41.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.595+0000 7f73ff989640 1 -- 172.21.15.3:0/3281418176 wait complete. 2024-09-12T12:24:41.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:24:41.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc881002640 1 Processor -- start 2024-09-12T12:24:41.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc881002640 1 -- start start 2024-09-12T12:24:41.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc881002640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c0ccbd0 0x7fc87c0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc881002640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc87c06e670 0x7fc87c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc881002640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc87c0cd4f0 con 0x7fc87c15ad40 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc881002640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc87c0b53e0 con 0x7fc87c06e670 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc881002640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc87c0b4b40 con 0x7fc87c0ccbd0 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87b59e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc87c06e670 0x7fc87c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87b59e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc87c06e670 0x7fc87c06cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49058/0 (socket says 172.21.15.3:49058) 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87b59e640 1 -- 172.21.15.3:0/3286235925 learned_addr learned my addr 172.21.15.3:0/3286235925 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87ad9d640 1 --2- 172.21.15.3:0/3286235925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c0ccbd0 0x7fc87c0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87b59e640 1 -- 172.21.15.3:0/3286235925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c0ccbd0 msgr2=0x7fc87c0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87b59e640 1 --2- 172.21.15.3:0/3286235925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c0ccbd0 0x7fc87c0ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87b59e640 1 -- 172.21.15.3:0/3286235925 >> v1:172.21.15.142:6789/0 conn(0x7fc87c15ad40 legacy=0x7fc87c15b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87b59e640 1 -- 172.21.15.3:0/3286235925 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c0b4e20 con 0x7fc87c06e670 2024-09-12T12:24:41.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.763+0000 7fc87ad9d640 1 --2- 172.21.15.3:0/3286235925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c0ccbd0 0x7fc87c0ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:41.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc87b59e640 1 --2- 172.21.15.3:0/3286235925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c06e670 0x7fc87c06cd50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc874009e50 tx=0x7fc87401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec41b32be9d6424 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc879d9b640 1 -- 172.21.15.3:0/3286235925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740a57a0 con 0x7fc87c06e670 2024-09-12T12:24:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc879d9b640 1 -- 172.21.15.3:0/3286235925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8740a5940 con 0x7fc87c06e670 2024-09-12T12:24:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc879d9b640 1 -- 172.21.15.3:0/3286235925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740a5d10 con 0x7fc87c06e670 2024-09-12T12:24:41.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 -- 172.21.15.3:0/3286235925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c06e670 msgr2=0x7fc87c06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 --2- 172.21.15.3:0/3286235925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c06e670 0x7fc87c06cd50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc874009e50 tx=0x7fc87401ca30 comp rx=0 tx=0).stop 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 -- 172.21.15.3:0/3286235925 shutdown_connections 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 --2- 172.21.15.3:0/3286235925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c06e670 0x7fc87c06cd50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 --2- 172.21.15.3:0/3286235925 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c0ccbd0 0x7fc87c0ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 -- 172.21.15.3:0/3286235925 >> 172.21.15.3:0/3286235925 conn(0x7fc87c067e00 msgr2=0x7fc87c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 -- 172.21.15.3:0/3286235925 shutdown_connections 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 -- 172.21.15.3:0/3286235925 wait complete. 2024-09-12T12:24:41.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 Processor -- start 2024-09-12T12:24:41.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.767+0000 7fc881002640 1 -- start start 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c06e670 0x7fc87c073900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c0ccbd0 0x7fc87c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c15ad40 0x7fc87c1faab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c06ec40 con 0x7fc87c15ad40 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c06eac0 con 0x7fc87c0ccbd0 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc87c0cfdd0 con 0x7fc87c06e670 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c15ad40 0x7fc87c1faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c15ad40 0x7fc87c1faab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49064/0 (socket says 172.21.15.3:49064) 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 -- 172.21.15.3:0/3450704523 learned_addr learned my addr 172.21.15.3:0/3450704523 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87a59c640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c0ccbd0 0x7fc87c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87ad9d640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c06e670 0x7fc87c073900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 -- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c06e670 msgr2=0x7fc87c073900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c06e670 0x7fc87c073900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 -- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c0ccbd0 msgr2=0x7fc87c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c0ccbd0 0x7fc87c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 -- 172.21.15.3:0/3450704523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c09ae30 con 0x7fc87c15ad40 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87a59c640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c0ccbd0 0x7fc87c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc87b59e640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c15ad40 0x7fc87c1faab0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc874009e50 tx=0x7fc8740197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc8637fe640 1 -- 172.21.15.3:0/3450704523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740bc020 con 0x7fc87c15ad40 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc8637fe640 1 -- 172.21.15.3:0/3450704523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8740b8690 con 0x7fc87c15ad40 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c0b26c0 con 0x7fc87c15ad40 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc8637fe640 1 -- 172.21.15.3:0/3450704523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740b8a80 con 0x7fc87c15ad40 2024-09-12T12:24:41.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1513a0 con 0x7fc87c15ad40 2024-09-12T12:24:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87c09abc0 con 0x7fc87c15ad40 2024-09-12T12:24:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.771+0000 7fc8637fe640 1 -- 172.21.15.3:0/3450704523 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8740bf020 con 0x7fc87c15ad40 2024-09-12T12:24:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.775+0000 7fc8637fe640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8680678c0 0x7fc868069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.775+0000 7fc8637fe640 1 -- 172.21.15.3:0/3450704523 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5227+0+0 (secure 0 0 0) 0x7fc874120d50 con 0x7fc87c15ad40 2024-09-12T12:24:41.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.775+0000 7fc87ad9d640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8680678c0 0x7fc868069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.775+0000 7fc87ad9d640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8680678c0 0x7fc868069d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc86c006fd0 tx=0x7fc86c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.779+0000 7fc8637fe640 1 -- 172.21.15.3:0/3450704523 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc8740c6030 con 0x7fc87c15ad40 2024-09-12T12:24:41.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:41.943+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc87c06cd50 con 0x7fc87c15ad40 2024-09-12T12:24:42.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.603+0000 7fc8637fe640 1 -- 172.21.15.3:0/3450704523 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) ==== 170+0+0 (secure 0 0 0) 0x7fc874016620 con 0x7fc87c15ad40 2024-09-12T12:24:42.606 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T12:24:42.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8680678c0 msgr2=0x7fc868069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8680678c0 0x7fc868069d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc86c006fd0 tx=0x7fc86c008040 comp rx=0 tx=0).stop 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c15ad40 msgr2=0x7fc87c1faab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c15ad40 0x7fc87c1faab0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc874009e50 tx=0x7fc8740197d0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 shutdown_connections 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8680678c0 0x7fc868069d80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc87c15ad40 0x7fc87c1faab0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc87c0ccbd0 0x7fc87c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 --2- 172.21.15.3:0/3450704523 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc87c06e670 0x7fc87c073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 >> 172.21.15.3:0/3450704523 conn(0x7fc87c067e00 msgr2=0x7fc87c15cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 shutdown_connections 2024-09-12T12:24:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.607+0000 7fc881002640 1 -- 172.21.15.3:0/3450704523 wait complete. 2024-09-12T12:24:42.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:24:42.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 Processor -- start 2024-09-12T12:24:42.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 -- start start 2024-09-12T12:24:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8ad4161b40 0x7f8ad4067a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:42.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad4161f20 0x7f8ad4067f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ad40b1220 0x7f8ad4073220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40b1030 con 0x7f8ad40b1220 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad40d21d0 con 0x7f8ad4161b40 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad9e21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad40b0d50 con 0x7f8ad4161f20 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ad40b1220 0x7f8ad4073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8ad40b1220 0x7f8ad4073220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49078/0 (socket says 172.21.15.3:49078) 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 -- 172.21.15.3:0/2885279963 learned_addr learned my addr 172.21.15.3:0/2885279963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad2ffd640 1 --2- 172.21.15.3:0/2885279963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad4161f20 0x7f8ad4067f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad37fe640 1 --2- 172.21.15.3:0/2885279963 >> v2:172.21.15.142:3300/0 conn(0x7f8ad4161b40 0x7f8ad4067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 -- 172.21.15.3:0/2885279963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad4161f20 msgr2=0x7f8ad4067f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:42.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 --2- 172.21.15.3:0/2885279963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad4161f20 0x7f8ad4067f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 -- 172.21.15.3:0/2885279963 >> v2:172.21.15.142:3300/0 conn(0x7f8ad4161b40 msgr2=0x7f8ad4067a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 --2- 172.21.15.3:0/2885279963 >> v2:172.21.15.142:3300/0 conn(0x7f8ad4161b40 0x7f8ad4067a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad3fff640 1 -- 172.21.15.3:0/2885279963 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40d1ef0 con 0x7f8ad40b1220 2024-09-12T12:24:42.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.763+0000 7f8ad37fe640 1 --2- 172.21.15.3:0/2885279963 >> v2:172.21.15.142:3300/0 conn(0x7f8ad4161b40 0x7f8ad4067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:42.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad3fff640 1 --2- 172.21.15.3:0/2885279963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad40b1220 0x7f8ad4073220 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4071470 tx=0x7f8acc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87108c1c027d2a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:42.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad27fc640 1 -- 172.21.15.3:0/2885279963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0b0070 con 0x7f8ad40b1220 2024-09-12T12:24:42.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad27fc640 1 -- 172.21.15.3:0/2885279963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8acc0ab600 con 0x7f8ad40b1220 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad27fc640 1 -- 172.21.15.3:0/2885279963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0ab9d0 con 0x7f8ad40b1220 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 -- 172.21.15.3:0/2885279963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad40b1220 msgr2=0x7f8ad4073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/2885279963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad40b1220 0x7f8ad4073220 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4071470 tx=0x7f8acc01c490 comp rx=0 tx=0).stop 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 -- 172.21.15.3:0/2885279963 shutdown_connections 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/2885279963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad40b1220 0x7f8ad4073220 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/2885279963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad4161f20 0x7f8ad4067f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/2885279963 >> v2:172.21.15.142:3300/0 conn(0x7f8ad4161b40 0x7f8ad4067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 -- 172.21.15.3:0/2885279963 >> 172.21.15.3:0/2885279963 conn(0x7f8ad40d12c0 msgr2=0x7f8ad40d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:42.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 -- 172.21.15.3:0/2885279963 shutdown_connections 2024-09-12T12:24:42.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 -- 172.21.15.3:0/2885279963 wait complete. 2024-09-12T12:24:42.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 Processor -- start 2024-09-12T12:24:42.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.767+0000 7f8ad9e21640 1 -- start start 2024-09-12T12:24:42.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad40b1220 0x7f8ad40aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad4161b40 0x7f8ad4156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4161f20 0x7f8ad4157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40b1b00 con 0x7f8ad4161f20 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40b1980 con 0x7f8ad4161b40 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad40b1c80 con 0x7f8ad40b1220 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4161f20 0x7f8ad4157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4161f20 0x7f8ad4157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49088/0 (socket says 172.21.15.3:49088) 2024-09-12T12:24:42.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 -- 172.21.15.3:0/3554061776 learned_addr learned my addr 172.21.15.3:0/3554061776 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:42.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad37fe640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad40b1220 0x7f8ad40aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:42.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad2ffd640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad4161b40 0x7f8ad4156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:42.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 -- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad40b1220 msgr2=0x7f8ad40aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:42.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad40b1220 0x7f8ad40aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 -- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad4161b40 msgr2=0x7f8ad4156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:42.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad4161b40 0x7f8ad4156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 -- 172.21.15.3:0/3554061776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40af2b0 con 0x7f8ad4161f20 2024-09-12T12:24:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad3fff640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4161f20 0x7f8ad4157520 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8acc01ab40 tx=0x7f8acc019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8acbfff640 1 -- 172.21.15.3:0/3554061776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0b0040 con 0x7f8ad4161f20 2024-09-12T12:24:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8acbfff640 1 -- 172.21.15.3:0/3554061776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8acc0b9070 con 0x7f8ad4161f20 2024-09-12T12:24:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad409ab60 con 0x7f8ad4161f20 2024-09-12T12:24:42.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8acbfff640 1 -- 172.21.15.3:0/3554061776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0b4630 con 0x7f8ad4161f20 2024-09-12T12:24:42.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad40af830 con 0x7f8ad4161f20 2024-09-12T12:24:42.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad4073220 con 0x7f8ad4161f20 2024-09-12T12:24:42.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8acbfff640 1 -- 172.21.15.3:0/3554061776 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8acc0b47d0 con 0x7f8ad4161f20 2024-09-12T12:24:42.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8acbfff640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ab80678c0 0x7f8ab8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:42.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.771+0000 7f8acbfff640 1 -- 172.21.15.3:0/3554061776 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5609+0+0 (secure 0 0 0) 0x7f8acc11f730 con 0x7f8ad4161f20 2024-09-12T12:24:42.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.775+0000 7f8ad37fe640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ab80678c0 0x7f8ab8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:42.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.775+0000 7f8ad37fe640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ab80678c0 0x7f8ab8069d80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4157ba0 tx=0x7f8ac4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:42.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.779+0000 7f8acbfff640 1 -- 172.21.15.3:0/3554061776 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8acc016110 con 0x7f8ad4161f20 2024-09-12T12:24:42.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:42.983+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8ad40c4b10 con 0x7f8ad4161f20 2024-09-12T12:24:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.667+0000 7f8acbfff640 1 -- 172.21.15.3:0/3554061776 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v58) ==== 174+0+0 (secure 0 0 0) 0x7f8acc0f03f0 con 0x7f8ad4161f20 2024-09-12T12:24:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 14 and data pool 15 2024-09-12T12:24:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ab80678c0 msgr2=0x7f8ab8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ab80678c0 0x7f8ab8069d80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4157ba0 tx=0x7f8ac4008040 comp rx=0 tx=0).stop 2024-09-12T12:24:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4161f20 msgr2=0x7f8ad4157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4161f20 0x7f8ad4157520 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8acc01ab40 tx=0x7f8acc019230 comp rx=0 tx=0).stop 2024-09-12T12:24:43.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 shutdown_connections 2024-09-12T12:24:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8ab80678c0 0x7f8ab8069d80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4161f20 0x7f8ad4157520 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ad4161b40 0x7f8ad4156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 --2- 172.21.15.3:0/3554061776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8ad40b1220 0x7f8ad40aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 >> 172.21.15.3:0/3554061776 conn(0x7f8ad40d12c0 msgr2=0x7f8ad4067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 shutdown_connections 2024-09-12T12:24:43.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.671+0000 7f8ad9e21640 1 -- 172.21.15.3:0/3554061776 wait complete. 2024-09-12T12:24:43.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:24:43.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf99018640 1 Processor -- start 2024-09-12T12:24:43.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf99018640 1 -- start start 2024-09-12T12:24:43.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf99018640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940d4150 0x7fdf940cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:43.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf99018640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf940cf950 0x7fdf940b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf99018640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf940d45b0 con 0x7fdf940d1190 2024-09-12T12:24:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf99018640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdf940d3cc0 con 0x7fdf940cf950 2024-09-12T12:24:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf99018640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf940d30b0 con 0x7fdf940d4150 2024-09-12T12:24:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9259c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940d4150 0x7fdf940cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9359e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdf940cf950 0x7fdf940b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9259c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940d4150 0x7fdf940cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:47940/0 (socket says 172.21.15.3:47940) 2024-09-12T12:24:43.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9259c640 1 -- 172.21.15.3:0/1398066025 learned_addr learned my addr 172.21.15.3:0/1398066025 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9259c640 1 -- 172.21.15.3:0/1398066025 >> v2:172.21.15.142:3300/0 conn(0x7fdf940cf950 msgr2=0x7fdf940b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9259c640 1 --2- 172.21.15.3:0/1398066025 >> v2:172.21.15.142:3300/0 conn(0x7fdf940cf950 0x7fdf940b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9259c640 1 -- 172.21.15.3:0/1398066025 >> v1:172.21.15.142:6789/0 conn(0x7fdf940d1190 legacy=0x7fdf940d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:43.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.819+0000 7fdf9259c640 1 -- 172.21.15.3:0/1398066025 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf940d3390 con 0x7fdf940d4150 2024-09-12T12:24:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf9259c640 1 --2- 172.21.15.3:0/1398066025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940d4150 0x7fdf940cf410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009870 tx=0x7fdf8801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd6212e7ee1b754 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf91d9b640 1 -- 172.21.15.3:0/1398066025 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880a95f0 con 0x7fdf940d4150 2024-09-12T12:24:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf91d9b640 1 -- 172.21.15.3:0/1398066025 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf880a9790 con 0x7fdf940d4150 2024-09-12T12:24:43.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf91d9b640 1 -- 172.21.15.3:0/1398066025 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880a9b60 con 0x7fdf940d4150 2024-09-12T12:24:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 -- 172.21.15.3:0/1398066025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940d4150 msgr2=0x7fdf940cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 --2- 172.21.15.3:0/1398066025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940d4150 0x7fdf940cf410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009870 tx=0x7fdf8801c530 comp rx=0 tx=0).stop 2024-09-12T12:24:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 -- 172.21.15.3:0/1398066025 shutdown_connections 2024-09-12T12:24:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 --2- 172.21.15.3:0/1398066025 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940d4150 0x7fdf940cf410 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 --2- 172.21.15.3:0/1398066025 >> v2:172.21.15.142:3300/0 conn(0x7fdf940cf950 0x7fdf940b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 -- 172.21.15.3:0/1398066025 >> 172.21.15.3:0/1398066025 conn(0x7fdf94067a00 msgr2=0x7fdf94067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:43.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 -- 172.21.15.3:0/1398066025 shutdown_connections 2024-09-12T12:24:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 -- 172.21.15.3:0/1398066025 wait complete. 2024-09-12T12:24:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 Processor -- start 2024-09-12T12:24:43.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.823+0000 7fdf99018640 1 -- start start 2024-09-12T12:24:43.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940cf950 0x7fdf940c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:43.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf940d1190 0x7fdf9415ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:43.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf940d4150 0x7fdf940ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:43.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9415e9e0 con 0x7fdf940d1190 2024-09-12T12:24:43.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9415e860 con 0x7fdf940d4150 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9416b170 con 0x7fdf940cf950 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf940d1190 0x7fdf9415ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf940d1190 0x7fdf9415ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49094/0 (socket says 172.21.15.3:49094) 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 -- 172.21.15.3:0/3680789466 learned_addr learned my addr 172.21.15.3:0/3680789466 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf92d9d640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940cf950 0x7fdf940c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9359e640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf940d4150 0x7fdf940ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 -- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940cf950 msgr2=0x7fdf940c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940cf950 0x7fdf940c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 -- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf940d4150 msgr2=0x7fdf940ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf940d4150 0x7fdf940ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 -- 172.21.15.3:0/3680789466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf940ba2b0 con 0x7fdf940d1190 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf92d9d640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940cf950 0x7fdf940c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9359e640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf940d4150 0x7fdf940ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:43.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf9259c640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf940d1190 0x7fdf9415ebb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdf8801abe0 tx=0x7fdf880192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:43.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf7b7fe640 1 -- 172.21.15.3:0/3680789466 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880bb640 con 0x7fdf940d1190 2024-09-12T12:24:43.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf7b7fe640 1 -- 172.21.15.3:0/3680789466 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf880bb800 con 0x7fdf940d1190 2024-09-12T12:24:43.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf940cb180 con 0x7fdf940d1190 2024-09-12T12:24:43.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf7b7fe640 1 -- 172.21.15.3:0/3680789466 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880bbbf0 con 0x7fdf940d1190 2024-09-12T12:24:43.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf941542c0 con 0x7fdf940d1190 2024-09-12T12:24:43.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf940b7740 con 0x7fdf940d1190 2024-09-12T12:24:43.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf7b7fe640 1 -- 172.21.15.3:0/3680789466 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf880c43f0 con 0x7fdf940d1190 2024-09-12T12:24:43.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf7b7fe640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf800678c0 0x7fdf80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:43.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf7b7fe640 1 -- 172.21.15.3:0/3680789466 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fdf881202d0 con 0x7fdf940d1190 2024-09-12T12:24:43.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.827+0000 7fdf92d9d640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf800678c0 0x7fdf80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:43.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.831+0000 7fdf92d9d640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf800678c0 0x7fdf80069d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdf840016f0 tx=0x7fdf84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:43.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.831+0000 7fdf7b7fe640 1 -- 172.21.15.3:0/3680789466 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdf940cb180 con 0x7fdf940d1190 2024-09-12T12:24:44.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:43.999+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf940ba4c0 con 0x7fdf940d1190 2024-09-12T12:24:44.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf7b7fe640 1 -- 172.21.15.3:0/3680789466 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16621 (secure 0 0 0) 0x7fdf880ad020 con 0x7fdf940d1190 2024-09-12T12:24:44.005 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:44.005 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:24:42.988832+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:24:40.677600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-12T12:24:41.990474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:24:44.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf800678c0 msgr2=0x7fdf80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf800678c0 0x7fdf80069d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdf840016f0 tx=0x7fdf84009040 comp rx=0 tx=0).stop 2024-09-12T12:24:44.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf940d1190 msgr2=0x7fdf9415ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf940d1190 0x7fdf9415ebb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdf8801abe0 tx=0x7fdf880192d0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 shutdown_connections 2024-09-12T12:24:44.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdf800678c0 0x7fdf80069d80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf940d4150 0x7fdf940ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf940d1190 0x7fdf9415ebb0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 --2- 172.21.15.3:0/3680789466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf940cf950 0x7fdf940c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 >> 172.21.15.3:0/3680789466 conn(0x7fdf94067a00 msgr2=0x7fdf940b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 shutdown_connections 2024-09-12T12:24:44.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.003+0000 7fdf99018640 1 -- 172.21.15.3:0/3680789466 wait complete. 2024-09-12T12:24:44.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:24:44.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b6508e640 1 Processor -- start 2024-09-12T12:24:44.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b6508e640 1 -- start start 2024-09-12T12:24:44.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b6508e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f8b6015f990 0x7f8b60070190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b6508e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b60070c30 0x7f8b600d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b6508e640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b600afb40 con 0x7f8b600d08c0 2024-09-12T12:24:44.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b6508e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8b600d0d30 con 0x7f8b60070c30 2024-09-12T12:24:44.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b6508e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f8b600b0710 con 0x7f8b6015f990 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5e59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b60070c30 0x7f8b600d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5e59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b60070c30 0x7f8b600d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49096/0 (socket says 172.21.15.3:49096) 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5e59c640 1 -- 172.21.15.3:0/3701903623 learned_addr learned my addr 172.21.15.3:0/3701903623 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5ed9d640 1 --2- 172.21.15.3:0/3701903623 >> v2:172.21.15.142:3300/0 conn(0x7f8b6015f990 0x7f8b60070190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5e59c640 1 -- 172.21.15.3:0/3701903623 >> v2:172.21.15.142:3300/0 conn(0x7f8b6015f990 msgr2=0x7f8b60070190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5e59c640 1 --2- 172.21.15.3:0/3701903623 >> v2:172.21.15.142:3300/0 conn(0x7f8b6015f990 0x7f8b60070190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5e59c640 1 -- 172.21.15.3:0/3701903623 >> v1:172.21.15.142:6789/0 conn(0x7f8b600d08c0 legacy=0x7f8b6006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:44.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.143+0000 7f8b5e59c640 1 -- 172.21.15.3:0/3701903623 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b600afde0 con 0x7f8b60070c30 2024-09-12T12:24:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5e59c640 1 --2- 172.21.15.3:0/3701903623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b60070c30 0x7f8b600d0380 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8b60072aa0 tx=0x7f8b5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ecf1622003a7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5dd9b640 1 -- 172.21.15.3:0/3701903623 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540b0070 con 0x7f8b60070c30 2024-09-12T12:24:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5dd9b640 1 -- 172.21.15.3:0/3701903623 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b540ab600 con 0x7f8b60070c30 2024-09-12T12:24:44.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5dd9b640 1 -- 172.21.15.3:0/3701903623 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540ab9d0 con 0x7f8b60070c30 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- 172.21.15.3:0/3701903623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b60070c30 msgr2=0x7f8b600d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 --2- 172.21.15.3:0/3701903623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b60070c30 0x7f8b600d0380 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8b60072aa0 tx=0x7f8b5401c420 comp rx=0 tx=0).stop 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- 172.21.15.3:0/3701903623 shutdown_connections 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 --2- 172.21.15.3:0/3701903623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b60070c30 0x7f8b600d0380 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 --2- 172.21.15.3:0/3701903623 >> v2:172.21.15.142:3300/0 conn(0x7f8b6015f990 0x7f8b60070190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- 172.21.15.3:0/3701903623 >> 172.21.15.3:0/3701903623 conn(0x7f8b600b3ea0 msgr2=0x7f8b600b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- 172.21.15.3:0/3701903623 shutdown_connections 2024-09-12T12:24:44.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- 172.21.15.3:0/3701903623 wait complete. 2024-09-12T12:24:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 Processor -- start 2024-09-12T12:24:44.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- start start 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b60070c30 0x7f8b600c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b600d08c0 0x7f8b60157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b6015f990 0x7f8b600b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6015fd70 con 0x7f8b6015f990 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b600706d0 con 0x7f8b60070c30 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6015fef0 con 0x7f8b600d08c0 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b6015f990 0x7f8b600b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b6015f990 0x7f8b600b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49110/0 (socket says 172.21.15.3:49110) 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 -- 172.21.15.3:0/1556092319 learned_addr learned my addr 172.21.15.3:0/1556092319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:44.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5e59c640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b600d08c0 0x7f8b60157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5ed9d640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b60070c30 0x7f8b600c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 -- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b600d08c0 msgr2=0x7f8b60157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b600d08c0 0x7f8b60157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 -- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b60070c30 msgr2=0x7f8b600c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b60070c30 0x7f8b600c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 -- 172.21.15.3:0/1556092319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b600c7d70 con 0x7f8b6015f990 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b5f59e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b6015f990 0x7f8b600b5a90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c780 tx=0x7f8b5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b477fe640 1 -- 172.21.15.3:0/1556092319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58004b70 con 0x7f8b6015f990 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b477fe640 1 -- 172.21.15.3:0/1556092319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b58004d10 con 0x7f8b6015f990 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b477fe640 1 -- 172.21.15.3:0/1556092319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800aa90 con 0x7f8b6015f990 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.147+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b600c8050 con 0x7f8b6015f990 2024-09-12T12:24:44.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.151+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b600c8610 con 0x7f8b6015f990 2024-09-12T12:24:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.151+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b600cccb0 con 0x7f8b6015f990 2024-09-12T12:24:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.151+0000 7f8b477fe640 1 -- 172.21.15.3:0/1556092319 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8b5800ac30 con 0x7f8b6015f990 2024-09-12T12:24:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.151+0000 7f8b477fe640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8b4c0678c0 0x7f8b4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.151+0000 7f8b477fe640 1 -- 172.21.15.3:0/1556092319 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8b58085f10 con 0x7f8b6015f990 2024-09-12T12:24:44.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.151+0000 7f8b5ed9d640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8b4c0678c0 0x7f8b4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.151+0000 7f8b5ed9d640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8b4c0678c0 0x7f8b4c069d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b930 tx=0x7f8b50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:44.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.155+0000 7f8b477fe640 1 -- 172.21.15.3:0/1556092319 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8b58020070 con 0x7f8b6015f990 2024-09-12T12:24:44.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.323+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b600cddc0 con 0x7f8b6015f990 2024-09-12T12:24:44.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.323+0000 7f8b477fe640 1 -- 172.21.15.3:0/1556092319 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16621 (secure 0 0 0) 0x7f8b58056ae0 con 0x7f8b6015f990 2024-09-12T12:24:44.328 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:44.328 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":92,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:24:42.988832+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:24:40.677600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-12T12:24:41.990474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8b4c0678c0 msgr2=0x7f8b4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8b4c0678c0 0x7f8b4c069d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b930 tx=0x7f8b50008040 comp rx=0 tx=0).stop 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b6015f990 msgr2=0x7f8b600b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b6015f990 0x7f8b600b5a90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c780 tx=0x7f8b5800cc40 comp rx=0 tx=0).stop 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 shutdown_connections 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8b4c0678c0 0x7f8b4c069d80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b6015f990 0x7f8b600b5a90 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b600d08c0 0x7f8b60157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 --2- 172.21.15.3:0/1556092319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b60070c30 0x7f8b600c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 >> 172.21.15.3:0/1556092319 conn(0x7f8b600b3ea0 msgr2=0x7f8b6006f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 shutdown_connections 2024-09-12T12:24:44.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.327+0000 7f8b6508e640 1 -- 172.21.15.3:0/1556092319 wait complete. 2024-09-12T12:24:44.348 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:24:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc274e3640 1 Processor -- start 2024-09-12T12:24:44.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc274e3640 1 -- start start 2024-09-12T12:24:44.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc274e3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcc200cf5e0 0x7fcc20169b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc274e3640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc200d10d0 con 0x7fcc200cf170 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc274e3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc200d13b0 con 0x7fcc2015e750 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc274e3640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcc200d1690 con 0x7fcc200cf5e0 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc24a7e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcc200cf170 0x7fcc201663f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53390/0 (socket says 172.21.15.3:53390) 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc24a7e640 1 -- 172.21.15.3:0/1043119351 learned_addr learned my addr 172.21.15.3:0/1043119351 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc25a80640 1 --2- 172.21.15.3:0/1043119351 >> v2:172.21.15.3:3300/0 conn(0x7fcc200cf5e0 0x7fcc20169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc25a80640 1 -- 172.21.15.3:0/1043119351 >> v1:172.21.15.142:6789/0 conn(0x7fcc2015e750 legacy=0x7fcc200d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc25a80640 1 -- 172.21.15.3:0/1043119351 >> v1:172.21.15.3:6789/0 conn(0x7fcc200cf170 legacy=0x7fcc201663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.479+0000 7fcc25a80640 1 -- 172.21.15.3:0/1043119351 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc200d1970 con 0x7fcc200cf5e0 2024-09-12T12:24:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc25a80640 1 --2- 172.21.15.3:0/1043119351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc20169b20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009d90 tx=0x7fcc1801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d46d1dfb419e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc13fff640 1 -- 172.21.15.3:0/1043119351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc180b4070 con 0x7fcc200cf5e0 2024-09-12T12:24:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc13fff640 1 -- 172.21.15.3:0/1043119351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc180af600 con 0x7fcc200cf5e0 2024-09-12T12:24:44.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc13fff640 1 -- 172.21.15.3:0/1043119351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc180af9d0 con 0x7fcc200cf5e0 2024-09-12T12:24:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 -- 172.21.15.3:0/1043119351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 msgr2=0x7fcc20169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 --2- 172.21.15.3:0/1043119351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc20169b20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009d90 tx=0x7fcc1801f150 comp rx=0 tx=0).stop 2024-09-12T12:24:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 -- 172.21.15.3:0/1043119351 shutdown_connections 2024-09-12T12:24:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 --2- 172.21.15.3:0/1043119351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc20169b20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 -- 172.21.15.3:0/1043119351 >> 172.21.15.3:0/1043119351 conn(0x7fcc20067a00 msgr2=0x7fcc20067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 -- 172.21.15.3:0/1043119351 shutdown_connections 2024-09-12T12:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 -- 172.21.15.3:0/1043119351 wait complete. 2024-09-12T12:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 Processor -- start 2024-09-12T12:24:44.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.483+0000 7fcc274e3640 1 -- start start 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc200cf170 0x7fcc20160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc200c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc2015e750 0x7fcc2014b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200b6e10 con 0x7fcc200cf5e0 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200cfa50 con 0x7fcc200cf170 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcc200d4290 con 0x7fcc2015e750 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc200c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc200c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49122/0 (socket says 172.21.15.3:49122) 2024-09-12T12:24:44.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 -- 172.21.15.3:0/4222273103 learned_addr learned my addr 172.21.15.3:0/4222273103 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc2527f640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc200cf170 0x7fcc20160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 -- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc2015e750 msgr2=0x7fcc2014b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc2015e750 0x7fcc2014b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 -- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc200cf170 msgr2=0x7fcc20160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc200cf170 0x7fcc20160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 -- 172.21.15.3:0/4222273103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc200c97d0 con 0x7fcc200cf5e0 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc24a7e640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc200c90d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00c500 tx=0x7fcc1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc11ffb640 1 -- 172.21.15.3:0/4222273103 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c017070 con 0x7fcc200cf5e0 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc11ffb640 1 -- 172.21.15.3:0/4222273103 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc1c005cf0 con 0x7fcc200cf5e0 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc200c9ab0 con 0x7fcc200cf5e0 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc11ffb640 1 -- 172.21.15.3:0/4222273103 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0072f0 con 0x7fcc200cf5e0 2024-09-12T12:24:44.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc200ca070 con 0x7fcc200cf5e0 2024-09-12T12:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc200d2700 con 0x7fcc200cf5e0 2024-09-12T12:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc11ffb640 1 -- 172.21.15.3:0/4222273103 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc1c01a070 con 0x7fcc200cf5e0 2024-09-12T12:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc11ffb640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcc040678c0 0x7fcc04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc11ffb640 1 -- 172.21.15.3:0/4222273103 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fcc1c084f30 con 0x7fcc200cf5e0 2024-09-12T12:24:44.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.487+0000 7fcc2527f640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcc040678c0 0x7fcc04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:44.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.491+0000 7fcc2527f640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcc040678c0 0x7fcc04069d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcc14000f70 tx=0x7fcc14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:44.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.491+0000 7fcc11ffb640 1 -- 172.21.15.3:0/4222273103 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcc1c0554b0 con 0x7fcc200cf5e0 2024-09-12T12:24:44.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:44.699+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcc200b4b90 con 0x7fcc200cf5e0 2024-09-12T12:24:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.659+0000 7fcc11ffb640 1 -- 172.21.15.3:0/4222273103 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) ==== 118+0+0 (secure 0 0 0) 0x7fcc1c0597b0 con 0x7fcc200cf5e0 2024-09-12T12:24:45.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcc040678c0 msgr2=0x7fcc04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcc040678c0 0x7fcc04069d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcc14000f70 tx=0x7fcc14009040 comp rx=0 tx=0).stop 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 msgr2=0x7fcc200c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc200c90d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00c500 tx=0x7fcc1c00c9c0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 shutdown_connections 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcc040678c0 0x7fcc04069d80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc2015e750 0x7fcc2014b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcc200cf5e0 0x7fcc200c90d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 --2- 172.21.15.3:0/4222273103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc200cf170 0x7fcc20160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 >> 172.21.15.3:0/4222273103 conn(0x7fcc20067a00 msgr2=0x7fcc200cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 shutdown_connections 2024-09-12T12:24:45.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.663+0000 7fcc274e3640 1 -- 172.21.15.3:0/4222273103 wait complete. 2024-09-12T12:24:45.687 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-12T12:24:45.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-12T12:24:45.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.815+0000 7fd862129640 1 Processor -- start 2024-09-12T12:24:45.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.815+0000 7fd862129640 1 -- start start 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd862129640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd85c15e750 0x7fd85c0d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd862129640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd85c0cf880 0x7fd85c168400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd862129640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd85c0d10d0 con 0x7fd85c0d40c0 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd862129640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd85c0d13b0 con 0x7fd85c0cf880 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd862129640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fd85c0d1690 con 0x7fd85c15e750 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd85c0cf880 0x7fd85c168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd85c0cf880 0x7fd85c168400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49130/0 (socket says 172.21.15.3:49130) 2024-09-12T12:24:45.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 -- 172.21.15.3:0/2819260871 learned_addr learned my addr 172.21.15.3:0/2819260871 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85affd640 1 --1- 172.21.15.3:0/2819260871 >> v1:172.21.15.3:6789/0 conn(0x7fd85c0d40c0 0x7fd85c0cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53396/0 (socket says 172.21.15.3:53396) 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85b7fe640 1 --2- 172.21.15.3:0/2819260871 >> v2:172.21.15.142:3300/0 conn(0x7fd85c15e750 0x7fd85c0d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 -- 172.21.15.3:0/2819260871 >> v2:172.21.15.142:3300/0 conn(0x7fd85c15e750 msgr2=0x7fd85c0d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 --2- 172.21.15.3:0/2819260871 >> v2:172.21.15.142:3300/0 conn(0x7fd85c15e750 0x7fd85c0d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 -- 172.21.15.3:0/2819260871 >> v1:172.21.15.3:6789/0 conn(0x7fd85c0d40c0 legacy=0x7fd85c0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 -- 172.21.15.3:0/2819260871 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c0d1970 con 0x7fd85c0cf880 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85b7fe640 1 --2- 172.21.15.3:0/2819260871 >> v2:172.21.15.142:3300/0 conn(0x7fd85c15e750 0x7fd85c0d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:45.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85bfff640 1 --2- 172.21.15.3:0/2819260871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c168400 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd85c08d1c0 tx=0x7fd85401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36a925f05b84566 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85a7fc640 1 -- 172.21.15.3:0/2819260871 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540b4070 con 0x7fd85c0cf880 2024-09-12T12:24:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85a7fc640 1 -- 172.21.15.3:0/2819260871 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8540af400 con 0x7fd85c0cf880 2024-09-12T12:24:45.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd85a7fc640 1 -- 172.21.15.3:0/2819260871 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540af7d0 con 0x7fd85c0cf880 2024-09-12T12:24:45.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd862129640 1 -- 172.21.15.3:0/2819260871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 msgr2=0x7fd85c168400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:45.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.819+0000 7fd862129640 1 --2- 172.21.15.3:0/2819260871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c168400 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd85c08d1c0 tx=0x7fd85401f1e0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- 172.21.15.3:0/2819260871 shutdown_connections 2024-09-12T12:24:45.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 --2- 172.21.15.3:0/2819260871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c168400 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 --2- 172.21.15.3:0/2819260871 >> v2:172.21.15.142:3300/0 conn(0x7fd85c15e750 0x7fd85c0d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- 172.21.15.3:0/2819260871 >> 172.21.15.3:0/2819260871 conn(0x7fd85c067a00 msgr2=0x7fd85c067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:45.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- 172.21.15.3:0/2819260871 shutdown_connections 2024-09-12T12:24:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- 172.21.15.3:0/2819260871 wait complete. 2024-09-12T12:24:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 Processor -- start 2024-09-12T12:24:45.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- start start 2024-09-12T12:24:45.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c14cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:45.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd85c0d40c0 0x7fd85c159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:45.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd85c15e750 0x7fd85c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c15eb30 con 0x7fd85c0cf880 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c0b6e10 con 0x7fd85c0d40c0 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd85c15ecb0 con 0x7fd85c15e750 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c14cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c14cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49146/0 (socket says 172.21.15.3:49146) 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 -- 172.21.15.3:0/2588427540 learned_addr learned my addr 172.21.15.3:0/2588427540 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85bfff640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd85c15e750 0x7fd85c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85affd640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd85c0d40c0 0x7fd85c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 -- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd85c15e750 msgr2=0x7fd85c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd85c15e750 0x7fd85c15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 -- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd85c0d40c0 msgr2=0x7fd85c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:45.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd85c0d40c0 0x7fd85c159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 -- 172.21.15.3:0/2588427540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c15bef0 con 0x7fd85c0cf880 2024-09-12T12:24:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd85b7fe640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c14cb80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c7f0 tx=0x7fd84c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd853fff640 1 -- 172.21.15.3:0/2588427540 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c015070 con 0x7fd85c0cf880 2024-09-12T12:24:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd853fff640 1 -- 172.21.15.3:0/2588427540 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c007820 con 0x7fd85c0cf880 2024-09-12T12:24:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c15c1d0 con 0x7fd85c0cf880 2024-09-12T12:24:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd853fff640 1 -- 172.21.15.3:0/2588427540 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c007c10 con 0x7fd85c0cf880 2024-09-12T12:24:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.823+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c1f4b30 con 0x7fd85c0cf880 2024-09-12T12:24:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.827+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c0d3b80 con 0x7fd85c0cf880 2024-09-12T12:24:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.827+0000 7fd853fff640 1 -- 172.21.15.3:0/2588427540 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd84c018070 con 0x7fd85c0cf880 2024-09-12T12:24:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.827+0000 7fd853fff640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd83c0678c0 0x7fd83c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.827+0000 7fd853fff640 1 -- 172.21.15.3:0/2588427540 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fd84c085ec0 con 0x7fd85c0cf880 2024-09-12T12:24:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.827+0000 7fd85affd640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd83c0678c0 0x7fd83c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:45.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.827+0000 7fd85affd640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd83c0678c0 0x7fd83c069d80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd85c158fd0 tx=0x7fd844007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:45.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:45.831+0000 7fd853fff640 1 -- 172.21.15.3:0/2588427540 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd84c056970 con 0x7fd85c0cf880 2024-09-12T12:24:46.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.031+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd85c1f4d00 con 0x7fd85c0cf880 2024-09-12T12:24:46.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.667+0000 7fd853fff640 1 -- 172.21.15.3:0/2588427540 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) ==== 122+0+0 (secure 0 0 0) 0x7fd84c056270 con 0x7fd85c0cf880 2024-09-12T12:24:46.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd83c0678c0 msgr2=0x7fd83c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:46.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd83c0678c0 0x7fd83c069d80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd85c158fd0 tx=0x7fd844007cd0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 msgr2=0x7fd85c14cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c14cb80 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00c7f0 tx=0x7fd84c00ccb0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 shutdown_connections 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd83c0678c0 0x7fd83c069d80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd85c15e750 0x7fd85c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd85c0d40c0 0x7fd85c159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 --2- 172.21.15.3:0/2588427540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd85c0cf880 0x7fd85c14cb80 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 >> 172.21.15.3:0/2588427540 conn(0x7fd85c067a00 msgr2=0x7fd85c0b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 shutdown_connections 2024-09-12T12:24:46.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.671+0000 7fd862129640 1 -- 172.21.15.3:0/2588427540 wait complete. 2024-09-12T12:24:46.696 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-12T12:24:46.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:46.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3faa6f8640 1 Processor -- start 2024-09-12T12:24:46.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3faa6f8640 1 -- start start 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3faa6f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3faa6f8640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3fa40d1570 0x7f3fa40cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3faa6f8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fa40d30b0 con 0x7f3fa40cf950 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3faa6f8640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3fa40d3cc0 con 0x7f3fa40d1570 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3faa6f8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa40d3390 con 0x7f3fa40d1190 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40d4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:48004/0 (socket says 172.21.15.3:48004) 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa3fff640 1 -- 172.21.15.3:0/3929745161 learned_addr learned my addr 172.21.15.3:0/3929745161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:46.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa37fe640 1 --2- 172.21.15.3:0/3929745161 >> v2:172.21.15.142:3300/0 conn(0x7f3fa40d1570 0x7f3fa40cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa3fff640 1 -- 172.21.15.3:0/3929745161 >> v2:172.21.15.142:3300/0 conn(0x7f3fa40d1570 msgr2=0x7f3fa40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa3fff640 1 --2- 172.21.15.3:0/3929745161 >> v2:172.21.15.142:3300/0 conn(0x7f3fa40d1570 0x7f3fa40cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa3fff640 1 -- 172.21.15.3:0/3929745161 >> v1:172.21.15.142:6789/0 conn(0x7f3fa40cf950 legacy=0x7f3fa40b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa3fff640 1 -- 172.21.15.3:0/3929745161 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa40d3670 con 0x7f3fa40d1190 2024-09-12T12:24:46.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.811+0000 7f3fa37fe640 1 --2- 172.21.15.3:0/3929745161 >> v2:172.21.15.142:3300/0 conn(0x7f3fa40d1570 0x7f3fa40cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3fa3fff640 1 --2- 172.21.15.3:0/3929745161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40d4150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009870 tx=0x7f3f9401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d567bff1c73a40b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3fa2ffd640 1 -- 172.21.15.3:0/3929745161 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f940a9610 con 0x7f3fa40d1190 2024-09-12T12:24:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3fa2ffd640 1 -- 172.21.15.3:0/3929745161 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f940a97b0 con 0x7f3fa40d1190 2024-09-12T12:24:46.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3fa2ffd640 1 -- 172.21.15.3:0/3929745161 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f940a9b80 con 0x7f3fa40d1190 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- 172.21.15.3:0/3929745161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 msgr2=0x7f3fa40d4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/3929745161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40d4150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009870 tx=0x7f3f9401c530 comp rx=0 tx=0).stop 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- 172.21.15.3:0/3929745161 shutdown_connections 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/3929745161 >> v2:172.21.15.142:3300/0 conn(0x7f3fa40d1570 0x7f3fa40cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/3929745161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40d4150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- 172.21.15.3:0/3929745161 >> 172.21.15.3:0/3929745161 conn(0x7f3fa4067a00 msgr2=0x7f3fa4067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- 172.21.15.3:0/3929745161 shutdown_connections 2024-09-12T12:24:46.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- 172.21.15.3:0/3929745161 wait complete. 2024-09-12T12:24:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 Processor -- start 2024-09-12T12:24:46.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- start start 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cf950 0x7f3fa40c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa40d1570 0x7f3fa40b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40b83e0 con 0x7f3fa40cf950 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa40b8260 con 0x7f3fa40d1570 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.815+0000 7f3faa6f8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa415e860 con 0x7f3fa40d1190 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cf950 0x7f3fa40c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cf950 0x7f3fa40c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49150/0 (socket says 172.21.15.3:49150) 2024-09-12T12:24:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 -- 172.21.15.3:0/361424938 learned_addr learned my addr 172.21.15.3:0/361424938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa37fe640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa8c95640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa40d1570 0x7f3fa40b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 -- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 msgr2=0x7f3fa40c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 -- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa40d1570 msgr2=0x7f3fa40b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa40d1570 0x7f3fa40b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 -- 172.21.15.3:0/361424938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa40ca490 con 0x7f3fa40cf950 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa3fff640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cf950 0x7f3fa40c82b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3f940ab040 tx=0x7f3f9401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa0ff9640 1 -- 172.21.15.3:0/361424938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f940a9610 con 0x7f3fa40cf950 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa40ca770 con 0x7f3fa40cf950 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa0ff9640 1 -- 172.21.15.3:0/361424938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f940bc670 con 0x7f3fa40cf950 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa0ff9640 1 -- 172.21.15.3:0/361424938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f940bca60 con 0x7f3fa40cf950 2024-09-12T12:24:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa40cad30 con 0x7f3fa40cf950 2024-09-12T12:24:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa40cf410 con 0x7f3fa40cf950 2024-09-12T12:24:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa0ff9640 1 -- 172.21.15.3:0/361424938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3f940bcc00 con 0x7f3fa40cf950 2024-09-12T12:24:46.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa0ff9640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3f900678c0 0x7f3f90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa0ff9640 1 -- 172.21.15.3:0/361424938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f3f94120260 con 0x7f3fa40cf950 2024-09-12T12:24:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa37fe640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3f900678c0 0x7f3f90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.819+0000 7f3fa37fe640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3f900678c0 0x7f3f90069d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009580 tx=0x7f3f98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:46.823+0000 7f3fa0ff9640 1 -- 172.21.15.3:0/361424938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3f940c3030 con 0x7f3fa40cf950 2024-09-12T12:24:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.023+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fa40ba4c0 con 0x7f3fa40cf950 2024-09-12T12:24:47.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.023+0000 7f3fa0ff9640 1 -- 172.21.15.3:0/361424938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5216 (secure 0 0 0) 0x7f3f940ad020 con 0x7f3fa40cf950 2024-09-12T12:24:47.028 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:47.028 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":62,"btime":"2024-09-12T12:24:46:664139+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5992,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/95103479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":95103479},{"type":"v1","addr":"172.21.15.142:6835","nonce":95103479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/599679496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":599679496},{"type":"v1","addr":"172.21.15.142:6837","nonce":599679496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1759200537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1759200537},{"type":"v1","addr":"172.21.15.3:6834","nonce":1759200537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:24:46.035035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2492687290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2492687290},{"type":"v1","addr":"172.21.15.3:6835","nonce":2492687290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":7}]} 2024-09-12T12:24:47.028 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 62 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3f900678c0 msgr2=0x7f3f90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3f900678c0 0x7f3f90069d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009580 tx=0x7f3f98008040 comp rx=0 tx=0).stop 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cf950 msgr2=0x7f3fa40c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cf950 0x7f3fa40c82b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3f940ab040 tx=0x7f3f9401ad70 comp rx=0 tx=0).stop 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 shutdown_connections 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f3f900678c0 0x7f3f90069d80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fa40d1570 0x7f3fa40b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3fa40d1190 0x7f3fa40c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 --2- 172.21.15.3:0/361424938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fa40cf950 0x7f3fa40c82b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 >> 172.21.15.3:0/361424938 conn(0x7f3fa4067a00 msgr2=0x7f3fa415b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 shutdown_connections 2024-09-12T12:24:47.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.027+0000 7f3faa6f8640 1 -- 172.21.15.3:0/361424938 wait complete. 2024-09-12T12:24:47.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:24:47.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e8746640 1 Processor -- start 2024-09-12T12:24:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e8746640 1 -- start start 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e8746640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5e00d4150 0x7fb5e00cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e8746640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb5e00cf950 0x7fb5e00b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e8746640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e00d30b0 con 0x7fb5e00d1190 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e8746640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e00d45e0 con 0x7fb5e00d4150 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e8746640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e00d3cc0 con 0x7fb5e00cf950 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e5ce1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5e00d4150 0x7fb5e00cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e5ce1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb5e00d4150 0x7fb5e00cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49166/0 (socket says 172.21.15.3:49166) 2024-09-12T12:24:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/1790915527 learned_addr learned my addr 172.21.15.3:0/1790915527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e6ce3640 1 --2- 172.21.15.3:0/1790915527 >> v2:172.21.15.142:3300/0 conn(0x7fb5e00cf950 0x7fb5e00b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/1790915527 >> v2:172.21.15.142:3300/0 conn(0x7fb5e00cf950 msgr2=0x7fb5e00b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e5ce1640 1 --2- 172.21.15.3:0/1790915527 >> v2:172.21.15.142:3300/0 conn(0x7fb5e00cf950 0x7fb5e00b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/1790915527 >> v1:172.21.15.142:6789/0 conn(0x7fb5e00d1190 legacy=0x7fb5e00d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.171+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/1790915527 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e00d3390 con 0x7fb5e00d4150 2024-09-12T12:24:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 --2- 172.21.15.3:0/1790915527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d4150 0x7fb5e00cf410 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb5e008d710 tx=0x7fb5d001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad527d3acff02308 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e54e0640 1 -- 172.21.15.3:0/1790915527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00b5070 con 0x7fb5e00d4150 2024-09-12T12:24:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e54e0640 1 -- 172.21.15.3:0/1790915527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d00a89d0 con 0x7fb5e00d4150 2024-09-12T12:24:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e54e0640 1 -- 172.21.15.3:0/1790915527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00a8da0 con 0x7fb5e00d4150 2024-09-12T12:24:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- 172.21.15.3:0/1790915527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d4150 msgr2=0x7fb5e00cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 --2- 172.21.15.3:0/1790915527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d4150 0x7fb5e00cf410 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb5e008d710 tx=0x7fb5d001ecd0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- 172.21.15.3:0/1790915527 shutdown_connections 2024-09-12T12:24:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 --2- 172.21.15.3:0/1790915527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d4150 0x7fb5e00cf410 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 --2- 172.21.15.3:0/1790915527 >> v2:172.21.15.142:3300/0 conn(0x7fb5e00cf950 0x7fb5e00b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- 172.21.15.3:0/1790915527 >> 172.21.15.3:0/1790915527 conn(0x7fb5e0067a00 msgr2=0x7fb5e0067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- 172.21.15.3:0/1790915527 shutdown_connections 2024-09-12T12:24:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- 172.21.15.3:0/1790915527 wait complete. 2024-09-12T12:24:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 Processor -- start 2024-09-12T12:24:47.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- start start 2024-09-12T12:24:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5e00cf950 0x7fb5e0155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d1190 0x7fb5e015d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e00d4150 0x7fb5e015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e00d2cd0 con 0x7fb5e00d1190 2024-09-12T12:24:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e00d3950 con 0x7fb5e00d4150 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e8746640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e00d2e50 con 0x7fb5e00cf950 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d1190 0x7fb5e015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d1190 0x7fb5e015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49174/0 (socket says 172.21.15.3:49174) 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/2773930593 learned_addr learned my addr 172.21.15.3:0/2773930593 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e6ce3640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e00d4150 0x7fb5e015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5e00cf950 msgr2=0x7fb5e0155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5e00cf950 0x7fb5e0155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e00d4150 msgr2=0x7fb5e015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e00d4150 0x7fb5e015d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 -- 172.21.15.3:0/2773930593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e0156410 con 0x7fb5e00d1190 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5e5ce1640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d1190 0x7fb5e015d100 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00b3040 tx=0x7fb5d001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.175+0000 7fb5d6ffd640 1 -- 172.21.15.3:0/2773930593 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00c0070 con 0x7fb5e00d1190 2024-09-12T12:24:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5d6ffd640 1 -- 172.21.15.3:0/2773930593 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d00b0900 con 0x7fb5e00d1190 2024-09-12T12:24:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e00cba10 con 0x7fb5e00d1190 2024-09-12T12:24:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5d6ffd640 1 -- 172.21.15.3:0/2773930593 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00b5040 con 0x7fb5e00d1190 2024-09-12T12:24:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e00cbfd0 con 0x7fb5e00d1190 2024-09-12T12:24:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e015e860 con 0x7fb5e00d1190 2024-09-12T12:24:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5d6ffd640 1 -- 172.21.15.3:0/2773930593 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5d00c3070 con 0x7fb5e00d1190 2024-09-12T12:24:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5d6ffd640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678f0 0x7fb5c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5d6ffd640 1 -- 172.21.15.3:0/2773930593 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fb5d01229e0 con 0x7fb5e00d1190 2024-09-12T12:24:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5e64e2640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678f0 0x7fb5c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.179+0000 7fb5e64e2640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678f0 0x7fb5c4069db0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc003cf0 tx=0x7fb5dc0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.183+0000 7fb5d6ffd640 1 -- 172.21.15.3:0/2773930593 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb5d0018830 con 0x7fb5e00d1190 2024-09-12T12:24:47.266 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:24:47.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:24:47.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.343+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5e00ba4c0 con 0x7fb5e00d1190 2024-09-12T12:24:47.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.343+0000 7fb5d6ffd640 1 -- 172.21.15.3:0/2773930593 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16608 (secure 0 0 0) 0x7fb5d00f35b0 con 0x7fb5e00d1190 2024-09-12T12:24:47.349 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:47.349 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":93,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:24:44.609154+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:24:40.677600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-12T12:24:41.990474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:24:47.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678f0 msgr2=0x7fb5c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678f0 0x7fb5c4069db0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc003cf0 tx=0x7fb5dc0079c0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d1190 msgr2=0x7fb5e015d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d1190 0x7fb5e015d100 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00b3040 tx=0x7fb5d001ba70 comp rx=0 tx=0).stop 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 shutdown_connections 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c40678f0 0x7fb5c4069db0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e00d4150 0x7fb5e015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e00d1190 0x7fb5e015d100 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 --2- 172.21.15.3:0/2773930593 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5e00cf950 0x7fb5e0155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 >> 172.21.15.3:0/2773930593 conn(0x7fb5e0067a00 msgr2=0x7fb5e00b75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 shutdown_connections 2024-09-12T12:24:47.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.347+0000 7fb5e8746640 1 -- 172.21.15.3:0/2773930593 wait complete. 2024-09-12T12:24:47.367 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:24:47.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.515+0000 7fdee096c640 1 Processor -- start 2024-09-12T12:24:47.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdee096c640 1 -- start start 2024-09-12T12:24:47.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdee096c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdedc073910 0x7fdedc06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdee096c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdedc0af870 0x7fdedc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdee096c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdedc0b4f70 con 0x7fdedc15b510 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdee096c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdedc06f630 con 0x7fdedc073910 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdee096c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdedc06f910 con 0x7fdedc0af870 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdeda59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdedc15b510 0x7fdedc073200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:53408/0 (socket says 172.21.15.3:53408) 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdeda59c640 1 -- 172.21.15.3:0/2414406398 learned_addr learned my addr 172.21.15.3:0/2414406398 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded9d9b640 1 --2- 172.21.15.3:0/2414406398 >> v2:172.21.15.3:3300/0 conn(0x7fdedc073910 0x7fdedc06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fdedad9d640 1 --2- 172.21.15.3:0/2414406398 >> v2:172.21.15.142:3300/0 conn(0x7fdedc0af870 0x7fdedc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded959a640 1 -- 172.21.15.3:0/2414406398 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2831031307 0 0) 0x7fdedc0b4f70 con 0x7fdedc15b510 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded959a640 1 -- 172.21.15.3:0/2414406398 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec4005740 con 0x7fdedc15b510 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded9d9b640 1 -- 172.21.15.3:0/2414406398 >> v2:172.21.15.142:3300/0 conn(0x7fdedc0af870 msgr2=0x7fdedc06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded9d9b640 1 --2- 172.21.15.3:0/2414406398 >> v2:172.21.15.142:3300/0 conn(0x7fdedc0af870 0x7fdedc06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded9d9b640 1 -- 172.21.15.3:0/2414406398 >> v1:172.21.15.3:6789/0 conn(0x7fdedc15b510 legacy=0x7fdedc073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded9d9b640 1 -- 172.21.15.3:0/2414406398 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc06fbf0 con 0x7fdedc073910 2024-09-12T12:24:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded9d9b640 1 --2- 172.21.15.3:0/2414406398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc06deb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0694b0 tx=0x7fded001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e573c9581383f8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded959a640 1 -- 172.21.15.3:0/2414406398 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00ab650 con 0x7fdedc073910 2024-09-12T12:24:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded959a640 1 -- 172.21.15.3:0/2414406398 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded00b0070 con 0x7fdedc073910 2024-09-12T12:24:47.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.519+0000 7fded959a640 1 -- 172.21.15.3:0/2414406398 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded00aba10 con 0x7fdedc073910 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- 172.21.15.3:0/2414406398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 msgr2=0x7fdedc06deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 --2- 172.21.15.3:0/2414406398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc06deb0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0694b0 tx=0x7fded001c450 comp rx=0 tx=0).stop 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- 172.21.15.3:0/2414406398 shutdown_connections 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 --2- 172.21.15.3:0/2414406398 >> v2:172.21.15.142:3300/0 conn(0x7fdedc0af870 0x7fdedc06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 --2- 172.21.15.3:0/2414406398 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc06deb0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- 172.21.15.3:0/2414406398 >> 172.21.15.3:0/2414406398 conn(0x7fdedc0b2d80 msgr2=0x7fdedc0b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- 172.21.15.3:0/2414406398 shutdown_connections 2024-09-12T12:24:47.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- 172.21.15.3:0/2414406398 wait complete. 2024-09-12T12:24:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 Processor -- start 2024-09-12T12:24:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- start start 2024-09-12T12:24:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdedc0af870 0x7fdedc09a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc15b510 0x7fdedc152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc067a00 con 0x7fdedc073910 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdedc06f350 con 0x7fdedc15b510 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdedc067b80 con 0x7fdedc0af870 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc1553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49192/0 (socket says 172.21.15.3:49192) 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 -- 172.21.15.3:0/698048058 learned_addr learned my addr 172.21.15.3:0/698048058 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 -- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdedc0af870 msgr2=0x7fdedc09a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdedc0af870 0x7fdedc09a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdedad9d640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc15b510 0x7fdedc152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 -- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc15b510 msgr2=0x7fdedc152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc15b510 0x7fdedc152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 -- 172.21.15.3:0/698048058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdedc068940 con 0x7fdedc073910 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdedad9d640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc15b510 0x7fdedc152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdeda59c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc1553b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00caa0 tx=0x7fdecc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdecaffd640 1 -- 172.21.15.3:0/698048058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc017070 con 0x7fdedc073910 2024-09-12T12:24:47.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdecaffd640 1 -- 172.21.15.3:0/698048058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdecc00ac00 con 0x7fdedc073910 2024-09-12T12:24:47.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdedc1558f0 con 0x7fdedc073910 2024-09-12T12:24:47.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdecaffd640 1 -- 172.21.15.3:0/698048058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc012400 con 0x7fdedc073910 2024-09-12T12:24:47.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.523+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdedc15e630 con 0x7fdedc073910 2024-09-12T12:24:47.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.527+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdedc073200 con 0x7fdedc073910 2024-09-12T12:24:47.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.527+0000 7fdecaffd640 1 -- 172.21.15.3:0/698048058 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdecc01a070 con 0x7fdedc073910 2024-09-12T12:24:47.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.527+0000 7fdecaffd640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdebc0678c0 0x7fdebc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.527+0000 7fdecaffd640 1 -- 172.21.15.3:0/698048058 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fdecc085a90 con 0x7fdedc073910 2024-09-12T12:24:47.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.527+0000 7fded9d9b640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdebc0678c0 0x7fdebc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.527+0000 7fded9d9b640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdebc0678c0 0x7fdebc069d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fded00af000 tx=0x7fded0004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.531+0000 7fdecaffd640 1 -- 172.21.15.3:0/698048058 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdecc0565d0 con 0x7fdedc073910 2024-09-12T12:24:47.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.687+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdedc1467b0 con 0x7fdebc0678c0 2024-09-12T12:24:47.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.703+0000 7fdecaffd640 1 -- 172.21.15.3:0/698048058 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247424 (secure 0 0 0) 0x7fdedc1467b0 con 0x7fdebc0678c0 2024-09-12T12:24:47.710 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:47.712 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdebc0678c0 msgr2=0x7fdebc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdebc0678c0 0x7fdebc069d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fded00af000 tx=0x7fded0004740 comp rx=0 tx=0).stop 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 msgr2=0x7fdedc1553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc1553b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00caa0 tx=0x7fdecc00cf60 comp rx=0 tx=0).stop 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 shutdown_connections 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdebc0678c0 0x7fdebc069d80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdedc15b510 0x7fdedc152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdedc0af870 0x7fdedc09a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 --2- 172.21.15.3:0/698048058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdedc073910 0x7fdedc1553b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 >> 172.21.15.3:0/698048058 conn(0x7fdedc0b2d80 msgr2=0x7fdedc067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 shutdown_connections 2024-09-12T12:24:47.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.707+0000 7fdee096c640 1 -- 172.21.15.3:0/698048058 wait complete. 2024-09-12T12:24:47.727 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-12T12:24:45.757073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259412,"kb_used_data":9856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715308,"statfs":{"total":773094113280,"available":772828475392,"internally_reserved":0,"allocated":10092544,"data_stored":4952198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.160122"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.619162+0000","last_change":"2024-09-12T12:24:42.619162+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.619162+0000","last_undegraded":"2024-09-12T12:24:42.619162+0000","last_fullsized":"2024-09-12T12:24:42.619162+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631858+0000","last_change":"2024-09-12T12:24:42.631858+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631858+0000","last_undegraded":"2024-09-12T12:24:42.631858+0000","last_fullsized":"2024-09-12T12:24:42.631858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.616173+0000","last_change":"2024-09-12T12:24:42.616173+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.616173+0000","last_undegraded":"2024-09-12T12:24:42.616173+0000","last_fullsized":"2024-09-12T12:24:42.616173+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673541+0000","last_change":"2024-09-12T12:24:43.665791+0000","last_active":"2024-09-12T12:24:43.673541+0000","last_peered":"2024-09-12T12:24:43.673541+0000","last_clean":"2024-09-12T12:24:43.673541+0000","last_became_active":"2024-09-12T12:24:43.663819+0000","last_became_peered":"2024-09-12T12:24:43.663819+0000","last_unstale":"2024-09-12T12:24:43.673541+0000","last_undegraded":"2024-09-12T12:24:43.673541+0000","last_fullsized":"2024-09-12T12:24:43.673541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:34.993627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.624704+0000","last_change":"2024-09-12T12:24:42.624704+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.624704+0000","last_undegraded":"2024-09-12T12:24:42.624704+0000","last_fullsized":"2024-09-12T12:24:42.624704+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.623865+0000","last_change":"2024-09-12T12:24:42.623865+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.623865+0000","last_undegraded":"2024-09-12T12:24:42.623865+0000","last_fullsized":"2024-09-12T12:24:42.623865+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.623041+0000","last_change":"2024-09-12T12:24:42.623041+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.623041+0000","last_undegraded":"2024-09-12T12:24:42.623041+0000","last_fullsized":"2024-09-12T12:24:42.623041+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622450+0000","last_change":"2024-09-12T12:24:42.622450+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622450+0000","last_undegraded":"2024-09-12T12:24:42.622450+0000","last_fullsized":"2024-09-12T12:24:42.622450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.625422+0000","last_change":"2024-09-12T12:24:42.625422+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.625422+0000","last_undegraded":"2024-09-12T12:24:42.625422+0000","last_fullsized":"2024-09-12T12:24:42.625422+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622314+0000","last_change":"2024-09-12T12:24:42.622314+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622314+0000","last_undegraded":"2024-09-12T12:24:42.622314+0000","last_fullsized":"2024-09-12T12:24:42.622314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.640512+0000","last_change":"2024-09-12T12:24:42.640512+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.640512+0000","last_undegraded":"2024-09-12T12:24:42.640512+0000","last_fullsized":"2024-09-12T12:24:42.640512+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.619803+0000","last_change":"2024-09-12T12:24:42.619803+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.619803+0000","last_undegraded":"2024-09-12T12:24:42.619803+0000","last_fullsized":"2024-09-12T12:24:42.619803+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.671577+0000","last_change":"2024-09-12T12:24:43.653070+0000","last_active":"2024-09-12T12:24:43.671577+0000","last_peered":"2024-09-12T12:24:43.671577+0000","last_clean":"2024-09-12T12:24:43.671577+0000","last_became_active":"2024-09-12T12:24:43.651753+0000","last_became_peered":"2024-09-12T12:24:43.651753+0000","last_unstale":"2024-09-12T12:24:43.671577+0000","last_undegraded":"2024-09-12T12:24:43.671577+0000","last_fullsized":"2024-09-12T12:24:43.671577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:55.597998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.620767+0000","last_change":"2024-09-12T12:24:42.620767+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.620767+0000","last_undegraded":"2024-09-12T12:24:42.620767+0000","last_fullsized":"2024-09-12T12:24:42.620767+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673880+0000","last_change":"2024-09-12T12:24:43.670224+0000","last_active":"2024-09-12T12:24:43.673880+0000","last_peered":"2024-09-12T12:24:43.673880+0000","last_clean":"2024-09-12T12:24:43.673880+0000","last_became_active":"2024-09-12T12:24:43.669622+0000","last_became_peered":"2024-09-12T12:24:43.669622+0000","last_unstale":"2024-09-12T12:24:43.673880+0000","last_undegraded":"2024-09-12T12:24:43.673880+0000","last_fullsized":"2024-09-12T12:24:43.673880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:52:53.453156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.630255+0000","last_change":"2024-09-12T12:24:42.630255+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.630255+0000","last_undegraded":"2024-09-12T12:24:42.630255+0000","last_fullsized":"2024-09-12T12:24:42.630255+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626138+0000","last_change":"2024-09-12T12:24:42.626138+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626138+0000","last_undegraded":"2024-09-12T12:24:42.626138+0000","last_fullsized":"2024-09-12T12:24:42.626138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673828+0000","last_change":"2024-09-12T12:24:43.666474+0000","last_active":"2024-09-12T12:24:43.673828+0000","last_peered":"2024-09-12T12:24:43.673828+0000","last_clean":"2024-09-12T12:24:43.673828+0000","last_became_active":"2024-09-12T12:24:43.665798+0000","last_became_peered":"2024-09-12T12:24:43.665798+0000","last_unstale":"2024-09-12T12:24:43.673828+0000","last_undegraded":"2024-09-12T12:24:43.673828+0000","last_fullsized":"2024-09-12T12:24:43.673828+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:28.095510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618704+0000","last_change":"2024-09-12T12:24:42.618704+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618704+0000","last_undegraded":"2024-09-12T12:24:42.618704+0000","last_fullsized":"2024-09-12T12:24:42.618704+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618743+0000","last_change":"2024-09-12T12:24:42.618743+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618743+0000","last_undegraded":"2024-09-12T12:24:42.618743+0000","last_fullsized":"2024-09-12T12:24:42.618743+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.623618+0000","last_change":"2024-09-12T12:24:42.623618+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.623618+0000","last_undegraded":"2024-09-12T12:24:42.623618+0000","last_fullsized":"2024-09-12T12:24:42.623618+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631303+0000","last_change":"2024-09-12T12:24:42.631303+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631303+0000","last_undegraded":"2024-09-12T12:24:42.631303+0000","last_fullsized":"2024-09-12T12:24:42.631303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622827+0000","last_change":"2024-09-12T12:24:42.622827+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622827+0000","last_undegraded":"2024-09-12T12:24:42.622827+0000","last_fullsized":"2024-09-12T12:24:42.622827+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.624651+0000","last_change":"2024-09-12T12:24:42.624651+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.624651+0000","last_undegraded":"2024-09-12T12:24:42.624651+0000","last_fullsized":"2024-09-12T12:24:42.624651+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.623417+0000","last_change":"2024-09-12T12:24:42.623417+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.623417+0000","last_undegraded":"2024-09-12T12:24:42.623417+0000","last_fullsized":"2024-09-12T12:24:42.623417+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627753+0000","last_change":"2024-09-12T12:24:42.627753+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627753+0000","last_undegraded":"2024-09-12T12:24:42.627753+0000","last_fullsized":"2024-09-12T12:24:42.627753+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.639314+0000","last_change":"2024-09-12T12:24:42.639314+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.639314+0000","last_undegraded":"2024-09-12T12:24:42.639314+0000","last_fullsized":"2024-09-12T12:24:42.639314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626492+0000","last_change":"2024-09-12T12:24:42.626492+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626492+0000","last_undegraded":"2024-09-12T12:24:42.626492+0000","last_fullsized":"2024-09-12T12:24:42.626492+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627728+0000","last_change":"2024-09-12T12:24:42.627728+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627728+0000","last_undegraded":"2024-09-12T12:24:42.627728+0000","last_fullsized":"2024-09-12T12:24:42.627728+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.620147+0000","last_change":"2024-09-12T12:24:42.620147+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.620147+0000","last_undegraded":"2024-09-12T12:24:42.620147+0000","last_fullsized":"2024-09-12T12:24:42.620147+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673604+0000","last_change":"2024-09-12T12:24:43.664785+0000","last_active":"2024-09-12T12:24:43.673604+0000","last_peered":"2024-09-12T12:24:43.673604+0000","last_clean":"2024-09-12T12:24:43.673604+0000","last_became_active":"2024-09-12T12:24:43.663968+0000","last_became_peered":"2024-09-12T12:24:43.663968+0000","last_unstale":"2024-09-12T12:24:43.673604+0000","last_undegraded":"2024-09-12T12:24:43.673604+0000","last_fullsized":"2024-09-12T12:24:43.673604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:01.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.616056+0000","last_change":"2024-09-12T12:24:42.616056+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.616056+0000","last_undegraded":"2024-09-12T12:24:42.616056+0000","last_fullsized":"2024-09-12T12:24:42.616056+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.623349+0000","last_change":"2024-09-12T12:24:42.623349+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.623349+0000","last_undegraded":"2024-09-12T12:24:42.623349+0000","last_fullsized":"2024-09-12T12:24:42.623349+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646506+0000","last_change":"2024-09-12T12:24:42.641566+0000","last_active":"2024-09-12T12:24:42.646506+0000","last_peered":"2024-09-12T12:24:42.646506+0000","last_clean":"2024-09-12T12:24:42.646506+0000","last_became_active":"2024-09-12T12:24:42.641294+0000","last_became_peered":"2024-09-12T12:24:42.641294+0000","last_unstale":"2024-09-12T12:24:42.646506+0000","last_undegraded":"2024-09-12T12:24:42.646506+0000","last_fullsized":"2024-09-12T12:24:42.646506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:28.668303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673591+0000","last_change":"2024-09-12T12:24:43.665125+0000","last_active":"2024-09-12T12:24:43.673591+0000","last_peered":"2024-09-12T12:24:43.673591+0000","last_clean":"2024-09-12T12:24:43.673591+0000","last_became_active":"2024-09-12T12:24:43.664317+0000","last_became_peered":"2024-09-12T12:24:43.664317+0000","last_unstale":"2024-09-12T12:24:43.673591+0000","last_undegraded":"2024-09-12T12:24:43.673591+0000","last_fullsized":"2024-09-12T12:24:43.673591+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:46.448481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.654309+0000","last_change":"2024-09-12T12:24:42.653011+0000","last_active":"2024-09-12T12:24:42.654309+0000","last_peered":"2024-09-12T12:24:42.654309+0000","last_clean":"2024-09-12T12:24:42.654309+0000","last_became_active":"2024-09-12T12:24:42.652145+0000","last_became_peered":"2024-09-12T12:24:42.652145+0000","last_unstale":"2024-09-12T12:24:42.654309+0000","last_undegraded":"2024-09-12T12:24:42.654309+0000","last_fullsized":"2024-09-12T12:24:42.654309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:31.011712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618617+0000","last_change":"2024-09-12T12:24:42.618617+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618617+0000","last_undegraded":"2024-09-12T12:24:42.618617+0000","last_fullsized":"2024-09-12T12:24:42.618617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.649894+0000","last_change":"2024-09-12T12:24:42.646442+0000","last_active":"2024-09-12T12:24:42.649894+0000","last_peered":"2024-09-12T12:24:42.649894+0000","last_clean":"2024-09-12T12:24:42.649894+0000","last_became_active":"2024-09-12T12:24:42.643894+0000","last_became_peered":"2024-09-12T12:24:42.643894+0000","last_unstale":"2024-09-12T12:24:42.649894+0000","last_undegraded":"2024-09-12T12:24:42.649894+0000","last_fullsized":"2024-09-12T12:24:42.649894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:23:30.708412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.616986+0000","last_change":"2024-09-12T12:24:42.616986+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.616986+0000","last_undegraded":"2024-09-12T12:24:42.616986+0000","last_fullsized":"2024-09-12T12:24:42.616986+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646215+0000","last_change":"2024-09-12T12:24:42.637641+0000","last_active":"2024-09-12T12:24:42.646215+0000","last_peered":"2024-09-12T12:24:42.646215+0000","last_clean":"2024-09-12T12:24:42.646215+0000","last_became_active":"2024-09-12T12:24:42.637338+0000","last_became_peered":"2024-09-12T12:24:42.637338+0000","last_unstale":"2024-09-12T12:24:42.646215+0000","last_undegraded":"2024-09-12T12:24:42.646215+0000","last_fullsized":"2024-09-12T12:24:42.646215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:15.407617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.625028+0000","last_change":"2024-09-12T12:24:42.625028+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.625028+0000","last_undegraded":"2024-09-12T12:24:42.625028+0000","last_fullsized":"2024-09-12T12:24:42.625028+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.639340+0000","last_change":"2024-09-12T12:24:42.634480+0000","last_active":"2024-09-12T12:24:42.639340+0000","last_peered":"2024-09-12T12:24:42.639340+0000","last_clean":"2024-09-12T12:24:42.639340+0000","last_became_active":"2024-09-12T12:24:42.633367+0000","last_became_peered":"2024-09-12T12:24:42.633367+0000","last_unstale":"2024-09-12T12:24:42.639340+0000","last_undegraded":"2024-09-12T12:24:42.639340+0000","last_fullsized":"2024-09-12T12:24:42.639340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:53:38.692766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.628403+0000","last_change":"2024-09-12T12:24:42.628403+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.628403+0000","last_undegraded":"2024-09-12T12:24:42.628403+0000","last_fullsized":"2024-09-12T12:24:42.628403+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646203+0000","last_change":"2024-09-12T12:24:42.637461+0000","last_active":"2024-09-12T12:24:42.646203+0000","last_peered":"2024-09-12T12:24:42.646203+0000","last_clean":"2024-09-12T12:24:42.646203+0000","last_became_active":"2024-09-12T12:24:42.637031+0000","last_became_peered":"2024-09-12T12:24:42.637031+0000","last_unstale":"2024-09-12T12:24:42.646203+0000","last_undegraded":"2024-09-12T12:24:42.646203+0000","last_fullsized":"2024-09-12T12:24:42.646203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:34:47.172517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673878+0000","last_change":"2024-09-12T12:24:43.670586+0000","last_active":"2024-09-12T12:24:43.673878+0000","last_peered":"2024-09-12T12:24:43.673878+0000","last_clean":"2024-09-12T12:24:43.673878+0000","last_became_active":"2024-09-12T12:24:43.669772+0000","last_became_peered":"2024-09-12T12:24:43.669772+0000","last_unstale":"2024-09-12T12:24:43.673878+0000","last_undegraded":"2024-09-12T12:24:43.673878+0000","last_fullsized":"2024-09-12T12:24:43.673878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:43.570019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.651820+0000","last_change":"2024-09-12T12:24:42.648919+0000","last_active":"2024-09-12T12:24:42.651820+0000","last_peered":"2024-09-12T12:24:42.651820+0000","last_clean":"2024-09-12T12:24:42.651820+0000","last_became_active":"2024-09-12T12:24:42.648039+0000","last_became_peered":"2024-09-12T12:24:42.648039+0000","last_unstale":"2024-09-12T12:24:42.651820+0000","last_undegraded":"2024-09-12T12:24:42.651820+0000","last_fullsized":"2024-09-12T12:24:42.651820+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:19.161545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.621181+0000","last_change":"2024-09-12T12:24:42.621181+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.621181+0000","last_undegraded":"2024-09-12T12:24:42.621181+0000","last_fullsized":"2024-09-12T12:24:42.621181+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.628800+0000","last_change":"2024-09-12T12:24:42.625151+0000","last_active":"2024-09-12T12:24:42.628800+0000","last_peered":"2024-09-12T12:24:42.628800+0000","last_clean":"2024-09-12T12:24:42.628800+0000","last_became_active":"2024-09-12T12:24:42.624745+0000","last_became_peered":"2024-09-12T12:24:42.624745+0000","last_unstale":"2024-09-12T12:24:42.628800+0000","last_undegraded":"2024-09-12T12:24:42.628800+0000","last_fullsized":"2024-09-12T12:24:42.628800+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:03.026035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627022+0000","last_change":"2024-09-12T12:24:42.627022+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627022+0000","last_undegraded":"2024-09-12T12:24:42.627022+0000","last_fullsized":"2024-09-12T12:24:42.627022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.642596+0000","last_change":"2024-09-12T12:24:42.637921+0000","last_active":"2024-09-12T12:24:42.642596+0000","last_peered":"2024-09-12T12:24:42.642596+0000","last_clean":"2024-09-12T12:24:42.642596+0000","last_became_active":"2024-09-12T12:24:42.637229+0000","last_became_peered":"2024-09-12T12:24:42.637229+0000","last_unstale":"2024-09-12T12:24:42.642596+0000","last_undegraded":"2024-09-12T12:24:42.642596+0000","last_fullsized":"2024-09-12T12:24:42.642596+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:00.662422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.619432+0000","last_change":"2024-09-12T12:24:42.619432+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.619432+0000","last_undegraded":"2024-09-12T12:24:42.619432+0000","last_fullsized":"2024-09-12T12:24:42.619432+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.635866+0000","last_change":"2024-09-12T12:24:42.641455+0000","last_active":"2024-09-12T12:24:43.635866+0000","last_peered":"2024-09-12T12:24:43.635866+0000","last_clean":"2024-09-12T12:24:43.635866+0000","last_became_active":"2024-09-12T12:24:42.641145+0000","last_became_peered":"2024-09-12T12:24:42.641145+0000","last_unstale":"2024-09-12T12:24:43.635866+0000","last_undegraded":"2024-09-12T12:24:43.635866+0000","last_fullsized":"2024-09-12T12:24:43.635866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:58.912932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.625511+0000","last_change":"2024-09-12T12:24:42.625511+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.625511+0000","last_undegraded":"2024-09-12T12:24:42.625511+0000","last_fullsized":"2024-09-12T12:24:42.625511+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.654255+0000","last_change":"2024-09-12T12:24:42.652504+0000","last_active":"2024-09-12T12:24:42.654255+0000","last_peered":"2024-09-12T12:24:42.654255+0000","last_clean":"2024-09-12T12:24:42.654255+0000","last_became_active":"2024-09-12T12:24:42.651709+0000","last_became_peered":"2024-09-12T12:24:42.651709+0000","last_unstale":"2024-09-12T12:24:42.654255+0000","last_undegraded":"2024-09-12T12:24:42.654255+0000","last_fullsized":"2024-09-12T12:24:42.654255+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:11:58.467178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.624095+0000","last_change":"2024-09-12T12:24:42.624095+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.624095+0000","last_undegraded":"2024-09-12T12:24:42.624095+0000","last_fullsized":"2024-09-12T12:24:42.624095+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.650787+0000","last_change":"2024-09-12T12:24:42.643454+0000","last_active":"2024-09-12T12:24:42.650787+0000","last_peered":"2024-09-12T12:24:42.650787+0000","last_clean":"2024-09-12T12:24:42.650787+0000","last_became_active":"2024-09-12T12:24:42.642649+0000","last_became_peered":"2024-09-12T12:24:42.642649+0000","last_unstale":"2024-09-12T12:24:42.650787+0000","last_undegraded":"2024-09-12T12:24:42.650787+0000","last_fullsized":"2024-09-12T12:24:42.650787+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:44.456619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.666620+0000","last_change":"2024-09-12T12:24:43.647560+0000","last_active":"2024-09-12T12:24:43.666620+0000","last_peered":"2024-09-12T12:24:43.666620+0000","last_clean":"2024-09-12T12:24:43.666620+0000","last_became_active":"2024-09-12T12:24:43.646824+0000","last_became_peered":"2024-09-12T12:24:43.646824+0000","last_unstale":"2024-09-12T12:24:43.666620+0000","last_undegraded":"2024-09-12T12:24:43.666620+0000","last_fullsized":"2024-09-12T12:24:43.666620+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:35:20.924656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.638555+0000","last_change":"2024-09-12T12:24:42.637238+0000","last_active":"2024-09-12T12:24:42.638555+0000","last_peered":"2024-09-12T12:24:42.638555+0000","last_clean":"2024-09-12T12:24:42.638555+0000","last_became_active":"2024-09-12T12:24:42.636898+0000","last_became_peered":"2024-09-12T12:24:42.636898+0000","last_unstale":"2024-09-12T12:24:42.638555+0000","last_undegraded":"2024-09-12T12:24:42.638555+0000","last_fullsized":"2024-09-12T12:24:42.638555+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:26:01.359131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.615784+0000","last_change":"2024-09-12T12:24:42.615784+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.615784+0000","last_undegraded":"2024-09-12T12:24:42.615784+0000","last_fullsized":"2024-09-12T12:24:42.615784+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.633316+0000","last_change":"2024-09-12T12:24:42.627572+0000","last_active":"2024-09-12T12:24:42.633316+0000","last_peered":"2024-09-12T12:24:42.633316+0000","last_clean":"2024-09-12T12:24:42.633316+0000","last_became_active":"2024-09-12T12:24:42.627257+0000","last_became_peered":"2024-09-12T12:24:42.627257+0000","last_unstale":"2024-09-12T12:24:42.633316+0000","last_undegraded":"2024-09-12T12:24:42.633316+0000","last_fullsized":"2024-09-12T12:24:42.633316+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:20:17.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618424+0000","last_change":"2024-09-12T12:24:42.618424+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618424+0000","last_undegraded":"2024-09-12T12:24:42.618424+0000","last_fullsized":"2024-09-12T12:24:42.618424+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.650718+0000","last_change":"2024-09-12T12:24:42.643511+0000","last_active":"2024-09-12T12:24:42.650718+0000","last_peered":"2024-09-12T12:24:42.650718+0000","last_clean":"2024-09-12T12:24:42.650718+0000","last_became_active":"2024-09-12T12:24:42.642768+0000","last_became_peered":"2024-09-12T12:24:42.642768+0000","last_unstale":"2024-09-12T12:24:42.650718+0000","last_undegraded":"2024-09-12T12:24:42.650718+0000","last_fullsized":"2024-09-12T12:24:42.650718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:30.848175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618615+0000","last_change":"2024-09-12T12:24:42.618615+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618615+0000","last_undegraded":"2024-09-12T12:24:42.618615+0000","last_fullsized":"2024-09-12T12:24:42.618615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.649341+0000","last_change":"2024-09-12T12:24:42.642474+0000","last_active":"2024-09-12T12:24:42.649341+0000","last_peered":"2024-09-12T12:24:42.649341+0000","last_clean":"2024-09-12T12:24:42.649341+0000","last_became_active":"2024-09-12T12:24:42.642172+0000","last_became_peered":"2024-09-12T12:24:42.642172+0000","last_unstale":"2024-09-12T12:24:42.649341+0000","last_undegraded":"2024-09-12T12:24:42.649341+0000","last_fullsized":"2024-09-12T12:24:42.649341+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:38.836913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622219+0000","last_change":"2024-09-12T12:24:42.622219+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622219+0000","last_undegraded":"2024-09-12T12:24:42.622219+0000","last_fullsized":"2024-09-12T12:24:42.622219+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.648897+0000","last_change":"2024-09-12T12:24:42.640318+0000","last_active":"2024-09-12T12:24:42.648897+0000","last_peered":"2024-09-12T12:24:42.648897+0000","last_clean":"2024-09-12T12:24:42.648897+0000","last_became_active":"2024-09-12T12:24:42.639593+0000","last_became_peered":"2024-09-12T12:24:42.639593+0000","last_unstale":"2024-09-12T12:24:42.648897+0000","last_undegraded":"2024-09-12T12:24:42.648897+0000","last_fullsized":"2024-09-12T12:24:42.648897+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:22.351172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626380+0000","last_change":"2024-09-12T12:24:42.626380+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626380+0000","last_undegraded":"2024-09-12T12:24:42.626380+0000","last_fullsized":"2024-09-12T12:24:42.626380+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.641095+0000","last_change":"2024-09-12T12:24:42.637205+0000","last_active":"2024-09-12T12:24:42.641095+0000","last_peered":"2024-09-12T12:24:42.641095+0000","last_clean":"2024-09-12T12:24:42.641095+0000","last_became_active":"2024-09-12T12:24:42.636321+0000","last_became_peered":"2024-09-12T12:24:42.636321+0000","last_unstale":"2024-09-12T12:24:42.641095+0000","last_undegraded":"2024-09-12T12:24:42.641095+0000","last_fullsized":"2024-09-12T12:24:42.641095+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:18.210648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622854+0000","last_change":"2024-09-12T12:24:42.622854+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622854+0000","last_undegraded":"2024-09-12T12:24:42.622854+0000","last_fullsized":"2024-09-12T12:24:42.622854+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646718+0000","last_change":"2024-09-12T12:24:42.642838+0000","last_active":"2024-09-12T12:24:42.646718+0000","last_peered":"2024-09-12T12:24:42.646718+0000","last_clean":"2024-09-12T12:24:42.646718+0000","last_became_active":"2024-09-12T12:24:42.642310+0000","last_became_peered":"2024-09-12T12:24:42.642310+0000","last_unstale":"2024-09-12T12:24:42.646718+0000","last_undegraded":"2024-09-12T12:24:42.646718+0000","last_fullsized":"2024-09-12T12:24:42.646718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:52:42.944961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.619708+0000","last_change":"2024-09-12T12:24:42.619708+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.619708+0000","last_undegraded":"2024-09-12T12:24:42.619708+0000","last_fullsized":"2024-09-12T12:24:42.619708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646394+0000","last_change":"2024-09-12T12:24:42.637602+0000","last_active":"2024-09-12T12:24:42.646394+0000","last_peered":"2024-09-12T12:24:42.646394+0000","last_clean":"2024-09-12T12:24:42.646394+0000","last_became_active":"2024-09-12T12:24:42.637106+0000","last_became_peered":"2024-09-12T12:24:42.637106+0000","last_unstale":"2024-09-12T12:24:42.646394+0000","last_undegraded":"2024-09-12T12:24:42.646394+0000","last_fullsized":"2024-09-12T12:24:42.646394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:02.019478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622973+0000","last_change":"2024-09-12T12:24:42.622973+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622973+0000","last_undegraded":"2024-09-12T12:24:42.622973+0000","last_fullsized":"2024-09-12T12:24:42.622973+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646333+0000","last_change":"2024-09-12T12:24:42.639853+0000","last_active":"2024-09-12T12:24:42.646333+0000","last_peered":"2024-09-12T12:24:42.646333+0000","last_clean":"2024-09-12T12:24:42.646333+0000","last_became_active":"2024-09-12T12:24:42.639201+0000","last_became_peered":"2024-09-12T12:24:42.639201+0000","last_unstale":"2024-09-12T12:24:42.646333+0000","last_undegraded":"2024-09-12T12:24:42.646333+0000","last_fullsized":"2024-09-12T12:24:42.646333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:54.814076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.620278+0000","last_change":"2024-09-12T12:24:42.620278+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.620278+0000","last_undegraded":"2024-09-12T12:24:42.620278+0000","last_fullsized":"2024-09-12T12:24:42.620278+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.650911+0000","last_change":"2024-09-12T12:24:42.647349+0000","last_active":"2024-09-12T12:24:42.650911+0000","last_peered":"2024-09-12T12:24:42.650911+0000","last_clean":"2024-09-12T12:24:42.650911+0000","last_became_active":"2024-09-12T12:24:42.646472+0000","last_became_peered":"2024-09-12T12:24:42.646472+0000","last_unstale":"2024-09-12T12:24:42.650911+0000","last_undegraded":"2024-09-12T12:24:42.650911+0000","last_fullsized":"2024-09-12T12:24:42.650911+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:02.983014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626987+0000","last_change":"2024-09-12T12:24:42.626987+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626987+0000","last_undegraded":"2024-09-12T12:24:42.626987+0000","last_fullsized":"2024-09-12T12:24:42.626987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646621+0000","last_change":"2024-09-12T12:24:42.637645+0000","last_active":"2024-09-12T12:24:42.646621+0000","last_peered":"2024-09-12T12:24:42.646621+0000","last_clean":"2024-09-12T12:24:42.646621+0000","last_became_active":"2024-09-12T12:24:42.637099+0000","last_became_peered":"2024-09-12T12:24:42.637099+0000","last_unstale":"2024-09-12T12:24:42.646621+0000","last_undegraded":"2024-09-12T12:24:42.646621+0000","last_fullsized":"2024-09-12T12:24:42.646621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:44.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626273+0000","last_change":"2024-09-12T12:24:42.626273+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626273+0000","last_undegraded":"2024-09-12T12:24:42.626273+0000","last_fullsized":"2024-09-12T12:24:42.626273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646395+0000","last_change":"2024-09-12T12:24:42.639106+0000","last_active":"2024-09-12T12:24:42.646395+0000","last_peered":"2024-09-12T12:24:42.646395+0000","last_clean":"2024-09-12T12:24:42.646395+0000","last_became_active":"2024-09-12T12:24:42.638341+0000","last_became_peered":"2024-09-12T12:24:42.638341+0000","last_unstale":"2024-09-12T12:24:42.646395+0000","last_undegraded":"2024-09-12T12:24:42.646395+0000","last_fullsized":"2024-09-12T12:24:42.646395+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:06:47.285785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.617004+0000","last_change":"2024-09-12T12:24:42.617004+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.617004+0000","last_undegraded":"2024-09-12T12:24:42.617004+0000","last_fullsized":"2024-09-12T12:24:42.617004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646503+0000","last_change":"2024-09-12T12:24:42.641239+0000","last_active":"2024-09-12T12:24:42.646503+0000","last_peered":"2024-09-12T12:24:42.646503+0000","last_clean":"2024-09-12T12:24:42.646503+0000","last_became_active":"2024-09-12T12:24:42.638459+0000","last_became_peered":"2024-09-12T12:24:42.638459+0000","last_unstale":"2024-09-12T12:24:42.646503+0000","last_undegraded":"2024-09-12T12:24:42.646503+0000","last_fullsized":"2024-09-12T12:24:42.646503+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:28.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618020+0000","last_change":"2024-09-12T12:24:42.618020+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618020+0000","last_undegraded":"2024-09-12T12:24:42.618020+0000","last_fullsized":"2024-09-12T12:24:42.618020+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.652548+0000","last_change":"2024-09-12T12:24:42.647803+0000","last_active":"2024-09-12T12:24:42.652548+0000","last_peered":"2024-09-12T12:24:42.652548+0000","last_clean":"2024-09-12T12:24:42.652548+0000","last_became_active":"2024-09-12T12:24:42.647329+0000","last_became_peered":"2024-09-12T12:24:42.647329+0000","last_unstale":"2024-09-12T12:24:42.652548+0000","last_undegraded":"2024-09-12T12:24:42.652548+0000","last_fullsized":"2024-09-12T12:24:42.652548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:02:03.882345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622668+0000","last_change":"2024-09-12T12:24:42.622668+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622668+0000","last_undegraded":"2024-09-12T12:24:42.622668+0000","last_fullsized":"2024-09-12T12:24:42.622668+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.649746+0000","last_change":"2024-09-12T12:24:42.644303+0000","last_active":"2024-09-12T12:24:42.649746+0000","last_peered":"2024-09-12T12:24:42.649746+0000","last_clean":"2024-09-12T12:24:42.649746+0000","last_became_active":"2024-09-12T12:24:42.643559+0000","last_became_peered":"2024-09-12T12:24:42.643559+0000","last_unstale":"2024-09-12T12:24:42.649746+0000","last_undegraded":"2024-09-12T12:24:42.649746+0000","last_fullsized":"2024-09-12T12:24:42.649746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:30.300204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618479+0000","last_change":"2024-09-12T12:24:42.618479+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618479+0000","last_undegraded":"2024-09-12T12:24:42.618479+0000","last_fullsized":"2024-09-12T12:24:42.618479+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646546+0000","last_change":"2024-09-12T12:24:42.642157+0000","last_active":"2024-09-12T12:24:42.646546+0000","last_peered":"2024-09-12T12:24:42.646546+0000","last_clean":"2024-09-12T12:24:42.646546+0000","last_became_active":"2024-09-12T12:24:42.637757+0000","last_became_peered":"2024-09-12T12:24:42.637757+0000","last_unstale":"2024-09-12T12:24:42.646546+0000","last_undegraded":"2024-09-12T12:24:42.646546+0000","last_fullsized":"2024-09-12T12:24:42.646546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:53.864687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.617331+0000","last_change":"2024-09-12T12:24:42.617331+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.617331+0000","last_undegraded":"2024-09-12T12:24:42.617331+0000","last_fullsized":"2024-09-12T12:24:42.617331+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.640907+0000","last_change":"2024-09-12T12:24:42.636916+0000","last_active":"2024-09-12T12:24:42.640907+0000","last_peered":"2024-09-12T12:24:42.640907+0000","last_clean":"2024-09-12T12:24:42.640907+0000","last_became_active":"2024-09-12T12:24:42.634574+0000","last_became_peered":"2024-09-12T12:24:42.634574+0000","last_unstale":"2024-09-12T12:24:42.640907+0000","last_undegraded":"2024-09-12T12:24:42.640907+0000","last_fullsized":"2024-09-12T12:24:42.640907+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:23.588413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.618591+0000","last_change":"2024-09-12T12:24:42.618591+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.618591+0000","last_undegraded":"2024-09-12T12:24:42.618591+0000","last_fullsized":"2024-09-12T12:24:42.618591+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.641004+0000","last_change":"2024-09-12T12:24:42.625186+0000","last_active":"2024-09-12T12:24:43.641004+0000","last_peered":"2024-09-12T12:24:43.641004+0000","last_clean":"2024-09-12T12:24:43.641004+0000","last_became_active":"2024-09-12T12:24:42.624773+0000","last_became_peered":"2024-09-12T12:24:42.624773+0000","last_unstale":"2024-09-12T12:24:43.641004+0000","last_undegraded":"2024-09-12T12:24:43.641004+0000","last_fullsized":"2024-09-12T12:24:43.641004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:29.271497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626042+0000","last_change":"2024-09-12T12:24:42.626042+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626042+0000","last_undegraded":"2024-09-12T12:24:42.626042+0000","last_fullsized":"2024-09-12T12:24:42.626042+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.644283+0000","last_change":"2024-09-12T12:24:42.638683+0000","last_active":"2024-09-12T12:24:42.644283+0000","last_peered":"2024-09-12T12:24:42.644283+0000","last_clean":"2024-09-12T12:24:42.644283+0000","last_became_active":"2024-09-12T12:24:42.638385+0000","last_became_peered":"2024-09-12T12:24:42.638385+0000","last_unstale":"2024-09-12T12:24:42.644283+0000","last_undegraded":"2024-09-12T12:24:42.644283+0000","last_fullsized":"2024-09-12T12:24:42.644283+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:06:17.389134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622609+0000","last_change":"2024-09-12T12:24:42.622609+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622609+0000","last_undegraded":"2024-09-12T12:24:42.622609+0000","last_fullsized":"2024-09-12T12:24:42.622609+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.651371+0000","last_change":"2024-09-12T12:24:42.649739+0000","last_active":"2024-09-12T12:24:42.651371+0000","last_peered":"2024-09-12T12:24:42.651371+0000","last_clean":"2024-09-12T12:24:42.651371+0000","last_became_active":"2024-09-12T12:24:42.648926+0000","last_became_peered":"2024-09-12T12:24:42.648926+0000","last_unstale":"2024-09-12T12:24:42.651371+0000","last_undegraded":"2024-09-12T12:24:42.651371+0000","last_fullsized":"2024-09-12T12:24:42.651371+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:58:48.982458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":298,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.641244+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:24:42.641244+0000","last_peered":"2024-09-12T12:24:42.641244+0000","last_clean":"2024-09-12T12:24:42.641244+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:24:42.641244+0000","last_undegraded":"2024-09-12T12:24:42.641244+0000","last_fullsized":"2024-09-12T12:24:42.641244+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1122304,"data_stored":476139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542209,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1060232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1110016,"data_stored":469864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1110016,"data_stored":469864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542208,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1110016,"data_stored":469864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1122304,"data_stored":476139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1110016,"data_stored":469864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542207,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1060232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:47.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:47.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc106b25640 1 Processor -- start 2024-09-12T12:24:47.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc106b25640 1 -- start start 2024-09-12T12:24:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc106b25640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 0x7fc1000b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc106b25640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1000ae880 0x7fc1000aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc106b25640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1000b5000 con 0x7fc100073cb0 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc106b25640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fc100074090 con 0x7fc1000ae880 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc106b25640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc1000b37c0 con 0x7fc10015c710 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc1050c2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1000ae880 0x7fc1000aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc1050c2640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fc1000ae880 0x7fc1000aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49196/0 (socket says 172.21.15.3:49196) 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc1050c2640 1 -- 172.21.15.3:0/2460641336 learned_addr learned my addr 172.21.15.3:0/2460641336 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc1050c2640 1 -- 172.21.15.3:0/2460641336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 msgr2=0x7fc1000b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc1050c2640 1 --2- 172.21.15.3:0/2460641336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 0x7fc1000b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc1050c2640 1 -- 172.21.15.3:0/2460641336 >> v1:172.21.15.142:6789/0 conn(0x7fc100073cb0 legacy=0x7fc1000adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:47.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.887+0000 7fc1050c2640 1 -- 172.21.15.3:0/2460641336 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1000b3aa0 con 0x7fc1000ae880 2024-09-12T12:24:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc1050c2640 1 --2- 172.21.15.3:0/2460641336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc1000aecd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e50 tx=0x7fc0f801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42c0b853ea3bd28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0ff7fe640 1 -- 172.21.15.3:0/2460641336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80a5780 con 0x7fc1000ae880 2024-09-12T12:24:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0ff7fe640 1 -- 172.21.15.3:0/2460641336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f80a5920 con 0x7fc1000ae880 2024-09-12T12:24:47.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0ff7fe640 1 -- 172.21.15.3:0/2460641336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80a5cf0 con 0x7fc1000ae880 2024-09-12T12:24:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- 172.21.15.3:0/2460641336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 msgr2=0x7fc1000aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 --2- 172.21.15.3:0/2460641336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc1000aecd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e50 tx=0x7fc0f801c9f0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- 172.21.15.3:0/2460641336 shutdown_connections 2024-09-12T12:24:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 --2- 172.21.15.3:0/2460641336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc1000aecd0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 --2- 172.21.15.3:0/2460641336 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 0x7fc1000b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- 172.21.15.3:0/2460641336 >> 172.21.15.3:0/2460641336 conn(0x7fc100067e00 msgr2=0x7fc100068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- 172.21.15.3:0/2460641336 shutdown_connections 2024-09-12T12:24:47.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- 172.21.15.3:0/2460641336 wait complete. 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 Processor -- start 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- start start 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100073cb0 0x7fc1001fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc10015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 0x7fc100149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc100166db0 con 0x7fc1000ae880 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc10006ece0 con 0x7fc100073cb0 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc100166f30 con 0x7fc10015c710 2024-09-12T12:24:47.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc10015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc10015bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49206/0 (socket says 172.21.15.3:49206) 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 -- 172.21.15.3:0/2588705123 learned_addr learned my addr 172.21.15.3:0/2588705123 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc1048c1640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100073cb0 0x7fc1001fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc1050c2640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 0x7fc100149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 -- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 msgr2=0x7fc100149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 0x7fc100149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 -- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100073cb0 msgr2=0x7fc1001fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100073cb0 0x7fc1001fb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 -- 172.21.15.3:0/2588705123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc100154090 con 0x7fc1000ae880 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fffff640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc10015bcb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400b410 tx=0x7fc0f400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fd7fa640 1 -- 172.21.15.3:0/2588705123 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4015020 con 0x7fc1000ae880 2024-09-12T12:24:47.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fd7fa640 1 -- 172.21.15.3:0/2588705123 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f4011830 con 0x7fc1000ae880 2024-09-12T12:24:47.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc100154370 con 0x7fc1000ae880 2024-09-12T12:24:47.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.891+0000 7fc0fd7fa640 1 -- 172.21.15.3:0/2588705123 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4011c20 con 0x7fc1000ae880 2024-09-12T12:24:47.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.895+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc100154930 con 0x7fc1000ae880 2024-09-12T12:24:47.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.895+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1000aecd0 con 0x7fc1000ae880 2024-09-12T12:24:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.895+0000 7fc0fd7fa640 1 -- 172.21.15.3:0/2588705123 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc0f4018020 con 0x7fc1000ae880 2024-09-12T12:24:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.895+0000 7fc0fd7fa640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0ec0678f0 0x7fc0ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.895+0000 7fc0fd7fa640 1 -- 172.21.15.3:0/2588705123 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fc0f4085bc0 con 0x7fc1000ae880 2024-09-12T12:24:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.895+0000 7fc1048c1640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0ec0678f0 0x7fc0ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:47.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.895+0000 7fc1048c1640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0ec0678f0 0x7fc0ec069db0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc10015f6c0 tx=0x7fc0f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:47.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:47.899+0000 7fc0fd7fa640 1 -- 172.21.15.3:0/2588705123 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc0f401f020 con 0x7fc1000ae880 2024-09-12T12:24:48.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.059+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1000b0b40 con 0x7fc0ec0678f0 2024-09-12T12:24:48.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.075+0000 7fc0fd7fa640 1 -- 172.21.15.3:0/2588705123 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251323 (secure 0 0 0) 0x7fc1000b0b40 con 0x7fc0ec0678f0 2024-09-12T12:24:48.082 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:48.083 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0ec0678f0 msgr2=0x7fc0ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0ec0678f0 0x7fc0ec069db0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc10015f6c0 tx=0x7fc0f0009040 comp rx=0 tx=0).stop 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 msgr2=0x7fc10015bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc10015bcb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400b410 tx=0x7fc0f400b8d0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 shutdown_connections 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc0ec0678f0 0x7fc0ec069db0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc10015c710 0x7fc100149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1000ae880 0x7fc10015bcb0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 --2- 172.21.15.3:0/2588705123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc100073cb0 0x7fc1001fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 >> 172.21.15.3:0/2588705123 conn(0x7fc100067e00 msgr2=0x7fc10015ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:48.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 shutdown_connections 2024-09-12T12:24:48.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.079+0000 7fc106b25640 1 -- 172.21.15.3:0/2588705123 wait complete. 2024-09-12T12:24:48.098 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-12T12:24:47.757681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259648,"kb_used_data":10104,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715072,"statfs":{"total":773094113280,"available":772828233728,"internally_reserved":0,"allocated":10346496,"data_stored":5064627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001901"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616311+0000","last_change":"2024-09-12T12:24:43.638576+0000","last_active":"2024-09-12T12:24:44.616311+0000","last_peered":"2024-09-12T12:24:44.616311+0000","last_clean":"2024-09-12T12:24:44.616311+0000","last_became_active":"2024-09-12T12:24:43.637883+0000","last_became_peered":"2024-09-12T12:24:43.637883+0000","last_unstale":"2024-09-12T12:24:44.616311+0000","last_undegraded":"2024-09-12T12:24:44.616311+0000","last_fullsized":"2024-09-12T12:24:44.616311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:31.746764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631858+0000","last_change":"2024-09-12T12:24:42.631858+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631858+0000","last_undegraded":"2024-09-12T12:24:42.631858+0000","last_fullsized":"2024-09-12T12:24:42.631858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616160+0000","last_change":"2024-09-12T12:24:43.656461+0000","last_active":"2024-09-12T12:24:44.616160+0000","last_peered":"2024-09-12T12:24:44.616160+0000","last_clean":"2024-09-12T12:24:44.616160+0000","last_became_active":"2024-09-12T12:24:43.655523+0000","last_became_peered":"2024-09-12T12:24:43.655523+0000","last_unstale":"2024-09-12T12:24:44.616160+0000","last_undegraded":"2024-09-12T12:24:44.616160+0000","last_fullsized":"2024-09-12T12:24:44.616160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:45.830898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673541+0000","last_change":"2024-09-12T12:24:43.665791+0000","last_active":"2024-09-12T12:24:43.673541+0000","last_peered":"2024-09-12T12:24:43.673541+0000","last_clean":"2024-09-12T12:24:43.673541+0000","last_became_active":"2024-09-12T12:24:43.663819+0000","last_became_peered":"2024-09-12T12:24:43.663819+0000","last_unstale":"2024-09-12T12:24:43.673541+0000","last_undegraded":"2024-09-12T12:24:43.673541+0000","last_fullsized":"2024-09-12T12:24:43.673541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:34.993627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703214+0000","last_change":"2024-09-12T12:24:43.651132+0000","last_active":"2024-09-12T12:24:45.703214+0000","last_peered":"2024-09-12T12:24:45.703214+0000","last_clean":"2024-09-12T12:24:45.703214+0000","last_became_active":"2024-09-12T12:24:43.650439+0000","last_became_peered":"2024-09-12T12:24:43.650439+0000","last_unstale":"2024-09-12T12:24:45.703214+0000","last_undegraded":"2024-09-12T12:24:45.703214+0000","last_fullsized":"2024-09-12T12:24:45.703214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:44.191710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621336+0000","last_change":"2024-09-12T12:24:43.650634+0000","last_active":"2024-09-12T12:24:44.621336+0000","last_peered":"2024-09-12T12:24:44.621336+0000","last_clean":"2024-09-12T12:24:44.621336+0000","last_became_active":"2024-09-12T12:24:43.649447+0000","last_became_peered":"2024-09-12T12:24:43.649447+0000","last_unstale":"2024-09-12T12:24:44.621336+0000","last_undegraded":"2024-09-12T12:24:44.621336+0000","last_fullsized":"2024-09-12T12:24:44.621336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:56.893292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621208+0000","last_change":"2024-09-12T12:24:43.651354+0000","last_active":"2024-09-12T12:24:44.621208+0000","last_peered":"2024-09-12T12:24:44.621208+0000","last_clean":"2024-09-12T12:24:44.621208+0000","last_became_active":"2024-09-12T12:24:43.650570+0000","last_became_peered":"2024-09-12T12:24:43.650570+0000","last_unstale":"2024-09-12T12:24:44.621208+0000","last_undegraded":"2024-09-12T12:24:44.621208+0000","last_fullsized":"2024-09-12T12:24:44.621208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622450+0000","last_change":"2024-09-12T12:24:42.622450+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622450+0000","last_undegraded":"2024-09-12T12:24:42.622450+0000","last_fullsized":"2024-09-12T12:24:42.622450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904799+0000","last_change":"2024-09-12T12:24:43.652939+0000","last_active":"2024-09-12T12:24:44.904799+0000","last_peered":"2024-09-12T12:24:44.904799+0000","last_clean":"2024-09-12T12:24:44.904799+0000","last_became_active":"2024-09-12T12:24:43.651482+0000","last_became_peered":"2024-09-12T12:24:43.651482+0000","last_unstale":"2024-09-12T12:24:44.904799+0000","last_undegraded":"2024-09-12T12:24:44.904799+0000","last_fullsized":"2024-09-12T12:24:44.904799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702479+0000","last_change":"2024-09-12T12:24:43.637408+0000","last_active":"2024-09-12T12:24:45.702479+0000","last_peered":"2024-09-12T12:24:45.702479+0000","last_clean":"2024-09-12T12:24:45.702479+0000","last_became_active":"2024-09-12T12:24:43.636447+0000","last_became_peered":"2024-09-12T12:24:43.636447+0000","last_unstale":"2024-09-12T12:24:45.702479+0000","last_undegraded":"2024-09-12T12:24:45.702479+0000","last_fullsized":"2024-09-12T12:24:45.702479+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621413+0000","last_change":"2024-09-12T12:24:43.649963+0000","last_active":"2024-09-12T12:24:44.621413+0000","last_peered":"2024-09-12T12:24:44.621413+0000","last_clean":"2024-09-12T12:24:44.621413+0000","last_became_active":"2024-09-12T12:24:43.649206+0000","last_became_peered":"2024-09-12T12:24:43.649206+0000","last_unstale":"2024-09-12T12:24:44.621413+0000","last_undegraded":"2024-09-12T12:24:44.621413+0000","last_fullsized":"2024-09-12T12:24:44.621413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:36:44.276769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904389+0000","last_change":"2024-09-12T12:24:43.661406+0000","last_active":"2024-09-12T12:24:44.904389+0000","last_peered":"2024-09-12T12:24:44.904389+0000","last_clean":"2024-09-12T12:24:44.904389+0000","last_became_active":"2024-09-12T12:24:43.660623+0000","last_became_peered":"2024-09-12T12:24:43.660623+0000","last_unstale":"2024-09-12T12:24:44.904389+0000","last_undegraded":"2024-09-12T12:24:44.904389+0000","last_fullsized":"2024-09-12T12:24:44.904389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:16.413303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.671577+0000","last_change":"2024-09-12T12:24:43.653070+0000","last_active":"2024-09-12T12:24:43.671577+0000","last_peered":"2024-09-12T12:24:43.671577+0000","last_clean":"2024-09-12T12:24:43.671577+0000","last_became_active":"2024-09-12T12:24:43.651753+0000","last_became_peered":"2024-09-12T12:24:43.651753+0000","last_unstale":"2024-09-12T12:24:43.671577+0000","last_undegraded":"2024-09-12T12:24:43.671577+0000","last_fullsized":"2024-09-12T12:24:43.671577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:55.597998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683623+0000","last_change":"2024-09-12T12:24:43.622364+0000","last_active":"2024-09-12T12:24:44.683623+0000","last_peered":"2024-09-12T12:24:44.683623+0000","last_clean":"2024-09-12T12:24:44.683623+0000","last_became_active":"2024-09-12T12:24:43.620763+0000","last_became_peered":"2024-09-12T12:24:43.620763+0000","last_unstale":"2024-09-12T12:24:44.683623+0000","last_undegraded":"2024-09-12T12:24:44.683623+0000","last_fullsized":"2024-09-12T12:24:44.683623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:42.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673880+0000","last_change":"2024-09-12T12:24:43.670224+0000","last_active":"2024-09-12T12:24:43.673880+0000","last_peered":"2024-09-12T12:24:43.673880+0000","last_clean":"2024-09-12T12:24:43.673880+0000","last_became_active":"2024-09-12T12:24:43.669622+0000","last_became_peered":"2024-09-12T12:24:43.669622+0000","last_unstale":"2024-09-12T12:24:43.673880+0000","last_undegraded":"2024-09-12T12:24:43.673880+0000","last_fullsized":"2024-09-12T12:24:43.673880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:52:53.453156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621285+0000","last_change":"2024-09-12T12:24:43.658919+0000","last_active":"2024-09-12T12:24:44.621285+0000","last_peered":"2024-09-12T12:24:44.621285+0000","last_clean":"2024-09-12T12:24:44.621285+0000","last_became_active":"2024-09-12T12:24:43.658279+0000","last_became_peered":"2024-09-12T12:24:43.658279+0000","last_unstale":"2024-09-12T12:24:44.621285+0000","last_undegraded":"2024-09-12T12:24:44.621285+0000","last_fullsized":"2024-09-12T12:24:44.621285+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:40.554937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626138+0000","last_change":"2024-09-12T12:24:42.626138+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626138+0000","last_undegraded":"2024-09-12T12:24:42.626138+0000","last_fullsized":"2024-09-12T12:24:42.626138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673828+0000","last_change":"2024-09-12T12:24:43.666474+0000","last_active":"2024-09-12T12:24:43.673828+0000","last_peered":"2024-09-12T12:24:43.673828+0000","last_clean":"2024-09-12T12:24:43.673828+0000","last_became_active":"2024-09-12T12:24:43.665798+0000","last_became_peered":"2024-09-12T12:24:43.665798+0000","last_unstale":"2024-09-12T12:24:43.673828+0000","last_undegraded":"2024-09-12T12:24:43.673828+0000","last_fullsized":"2024-09-12T12:24:43.673828+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:28.095510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683840+0000","last_change":"2024-09-12T12:24:43.647026+0000","last_active":"2024-09-12T12:24:44.683840+0000","last_peered":"2024-09-12T12:24:44.683840+0000","last_clean":"2024-09-12T12:24:44.683840+0000","last_became_active":"2024-09-12T12:24:43.646064+0000","last_became_peered":"2024-09-12T12:24:43.646064+0000","last_unstale":"2024-09-12T12:24:44.683840+0000","last_undegraded":"2024-09-12T12:24:44.683840+0000","last_fullsized":"2024-09-12T12:24:44.683840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:38.167692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904536+0000","last_change":"2024-09-12T12:24:43.666544+0000","last_active":"2024-09-12T12:24:44.904536+0000","last_peered":"2024-09-12T12:24:44.904536+0000","last_clean":"2024-09-12T12:24:44.904536+0000","last_became_active":"2024-09-12T12:24:43.665661+0000","last_became_peered":"2024-09-12T12:24:43.665661+0000","last_unstale":"2024-09-12T12:24:44.904536+0000","last_undegraded":"2024-09-12T12:24:44.904536+0000","last_fullsized":"2024-09-12T12:24:44.904536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:22.604865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684069+0000","last_change":"2024-09-12T12:24:43.645911+0000","last_active":"2024-09-12T12:24:44.684069+0000","last_peered":"2024-09-12T12:24:44.684069+0000","last_clean":"2024-09-12T12:24:44.684069+0000","last_became_active":"2024-09-12T12:24:43.645127+0000","last_became_peered":"2024-09-12T12:24:43.645127+0000","last_unstale":"2024-09-12T12:24:44.684069+0000","last_undegraded":"2024-09-12T12:24:44.684069+0000","last_fullsized":"2024-09-12T12:24:44.684069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:01.666655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631303+0000","last_change":"2024-09-12T12:24:42.631303+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631303+0000","last_undegraded":"2024-09-12T12:24:42.631303+0000","last_fullsized":"2024-09-12T12:24:42.631303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622827+0000","last_change":"2024-09-12T12:24:42.622827+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622827+0000","last_undegraded":"2024-09-12T12:24:42.622827+0000","last_fullsized":"2024-09-12T12:24:42.622827+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621737+0000","last_change":"2024-09-12T12:24:43.646502+0000","last_active":"2024-09-12T12:24:44.621737+0000","last_peered":"2024-09-12T12:24:44.621737+0000","last_clean":"2024-09-12T12:24:44.621737+0000","last_became_active":"2024-09-12T12:24:43.645053+0000","last_became_peered":"2024-09-12T12:24:43.645053+0000","last_unstale":"2024-09-12T12:24:44.621737+0000","last_undegraded":"2024-09-12T12:24:44.621737+0000","last_fullsized":"2024-09-12T12:24:44.621737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:51:43.421188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703778+0000","last_change":"2024-09-12T12:24:43.645720+0000","last_active":"2024-09-12T12:24:45.703778+0000","last_peered":"2024-09-12T12:24:45.703778+0000","last_clean":"2024-09-12T12:24:45.703778+0000","last_became_active":"2024-09-12T12:24:43.644960+0000","last_became_peered":"2024-09-12T12:24:43.644960+0000","last_unstale":"2024-09-12T12:24:45.703778+0000","last_undegraded":"2024-09-12T12:24:45.703778+0000","last_fullsized":"2024-09-12T12:24:45.703778+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:07:07.741088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627753+0000","last_change":"2024-09-12T12:24:42.627753+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627753+0000","last_undegraded":"2024-09-12T12:24:42.627753+0000","last_fullsized":"2024-09-12T12:24:42.627753+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622169+0000","last_change":"2024-09-12T12:24:43.661393+0000","last_active":"2024-09-12T12:24:44.622169+0000","last_peered":"2024-09-12T12:24:44.622169+0000","last_clean":"2024-09-12T12:24:44.622169+0000","last_became_active":"2024-09-12T12:24:43.660705+0000","last_became_peered":"2024-09-12T12:24:43.660705+0000","last_unstale":"2024-09-12T12:24:44.622169+0000","last_undegraded":"2024-09-12T12:24:44.622169+0000","last_fullsized":"2024-09-12T12:24:44.622169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:34:20.041489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703551+0000","last_change":"2024-09-12T12:24:43.659047+0000","last_active":"2024-09-12T12:24:45.703551+0000","last_peered":"2024-09-12T12:24:45.703551+0000","last_clean":"2024-09-12T12:24:45.703551+0000","last_became_active":"2024-09-12T12:24:43.658354+0000","last_became_peered":"2024-09-12T12:24:43.658354+0000","last_unstale":"2024-09-12T12:24:45.703551+0000","last_undegraded":"2024-09-12T12:24:45.703551+0000","last_fullsized":"2024-09-12T12:24:45.703551+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:22.035037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627728+0000","last_change":"2024-09-12T12:24:42.627728+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627728+0000","last_undegraded":"2024-09-12T12:24:42.627728+0000","last_fullsized":"2024-09-12T12:24:42.627728+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684067+0000","last_change":"2024-09-12T12:24:43.645157+0000","last_active":"2024-09-12T12:24:44.684067+0000","last_peered":"2024-09-12T12:24:44.684067+0000","last_clean":"2024-09-12T12:24:44.684067+0000","last_became_active":"2024-09-12T12:24:43.644341+0000","last_became_peered":"2024-09-12T12:24:43.644341+0000","last_unstale":"2024-09-12T12:24:44.684067+0000","last_undegraded":"2024-09-12T12:24:44.684067+0000","last_fullsized":"2024-09-12T12:24:44.684067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:39.841147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673604+0000","last_change":"2024-09-12T12:24:43.664785+0000","last_active":"2024-09-12T12:24:43.673604+0000","last_peered":"2024-09-12T12:24:43.673604+0000","last_clean":"2024-09-12T12:24:43.673604+0000","last_became_active":"2024-09-12T12:24:43.663968+0000","last_became_peered":"2024-09-12T12:24:43.663968+0000","last_unstale":"2024-09-12T12:24:43.673604+0000","last_undegraded":"2024-09-12T12:24:43.673604+0000","last_fullsized":"2024-09-12T12:24:43.673604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:01.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615264+0000","last_change":"2024-09-12T12:24:43.661779+0000","last_active":"2024-09-12T12:24:44.615264+0000","last_peered":"2024-09-12T12:24:44.615264+0000","last_clean":"2024-09-12T12:24:44.615264+0000","last_became_active":"2024-09-12T12:24:43.660239+0000","last_became_peered":"2024-09-12T12:24:43.660239+0000","last_unstale":"2024-09-12T12:24:44.615264+0000","last_undegraded":"2024-09-12T12:24:44.615264+0000","last_fullsized":"2024-09-12T12:24:44.615264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:55.818309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683905+0000","last_change":"2024-09-12T12:24:43.648961+0000","last_active":"2024-09-12T12:24:44.683905+0000","last_peered":"2024-09-12T12:24:44.683905+0000","last_clean":"2024-09-12T12:24:44.683905+0000","last_became_active":"2024-09-12T12:24:43.648231+0000","last_became_peered":"2024-09-12T12:24:43.648231+0000","last_unstale":"2024-09-12T12:24:44.683905+0000","last_undegraded":"2024-09-12T12:24:44.683905+0000","last_fullsized":"2024-09-12T12:24:44.683905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:04.030268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646506+0000","last_change":"2024-09-12T12:24:42.641566+0000","last_active":"2024-09-12T12:24:42.646506+0000","last_peered":"2024-09-12T12:24:42.646506+0000","last_clean":"2024-09-12T12:24:42.646506+0000","last_became_active":"2024-09-12T12:24:42.641294+0000","last_became_peered":"2024-09-12T12:24:42.641294+0000","last_unstale":"2024-09-12T12:24:42.646506+0000","last_undegraded":"2024-09-12T12:24:42.646506+0000","last_fullsized":"2024-09-12T12:24:42.646506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:28.668303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673591+0000","last_change":"2024-09-12T12:24:43.665125+0000","last_active":"2024-09-12T12:24:43.673591+0000","last_peered":"2024-09-12T12:24:43.673591+0000","last_clean":"2024-09-12T12:24:43.673591+0000","last_became_active":"2024-09-12T12:24:43.664317+0000","last_became_peered":"2024-09-12T12:24:43.664317+0000","last_unstale":"2024-09-12T12:24:43.673591+0000","last_undegraded":"2024-09-12T12:24:43.673591+0000","last_fullsized":"2024-09-12T12:24:43.673591+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:46.448481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621763+0000","last_change":"2024-09-12T12:24:42.653011+0000","last_active":"2024-09-12T12:24:44.621763+0000","last_peered":"2024-09-12T12:24:44.621763+0000","last_clean":"2024-09-12T12:24:44.621763+0000","last_became_active":"2024-09-12T12:24:42.652145+0000","last_became_peered":"2024-09-12T12:24:42.652145+0000","last_unstale":"2024-09-12T12:24:44.621763+0000","last_undegraded":"2024-09-12T12:24:44.621763+0000","last_fullsized":"2024-09-12T12:24:44.621763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:16.133515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615814+0000","last_change":"2024-09-12T12:24:43.651221+0000","last_active":"2024-09-12T12:24:44.615814+0000","last_peered":"2024-09-12T12:24:44.615814+0000","last_clean":"2024-09-12T12:24:44.615814+0000","last_became_active":"2024-09-12T12:24:43.650519+0000","last_became_peered":"2024-09-12T12:24:43.650519+0000","last_unstale":"2024-09-12T12:24:44.615814+0000","last_undegraded":"2024-09-12T12:24:44.615814+0000","last_fullsized":"2024-09-12T12:24:44.615814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:03.987247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.649894+0000","last_change":"2024-09-12T12:24:42.646442+0000","last_active":"2024-09-12T12:24:42.649894+0000","last_peered":"2024-09-12T12:24:42.649894+0000","last_clean":"2024-09-12T12:24:42.649894+0000","last_became_active":"2024-09-12T12:24:42.643894+0000","last_became_peered":"2024-09-12T12:24:42.643894+0000","last_unstale":"2024-09-12T12:24:42.649894+0000","last_undegraded":"2024-09-12T12:24:42.649894+0000","last_fullsized":"2024-09-12T12:24:42.649894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:23:30.708412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.905160+0000","last_change":"2024-09-12T12:24:43.646657+0000","last_active":"2024-09-12T12:24:44.905160+0000","last_peered":"2024-09-12T12:24:44.905160+0000","last_clean":"2024-09-12T12:24:44.905160+0000","last_became_active":"2024-09-12T12:24:43.645851+0000","last_became_peered":"2024-09-12T12:24:43.645851+0000","last_unstale":"2024-09-12T12:24:44.905160+0000","last_undegraded":"2024-09-12T12:24:44.905160+0000","last_fullsized":"2024-09-12T12:24:44.905160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646215+0000","last_change":"2024-09-12T12:24:42.637641+0000","last_active":"2024-09-12T12:24:42.646215+0000","last_peered":"2024-09-12T12:24:42.646215+0000","last_clean":"2024-09-12T12:24:42.646215+0000","last_became_active":"2024-09-12T12:24:42.637338+0000","last_became_peered":"2024-09-12T12:24:42.637338+0000","last_unstale":"2024-09-12T12:24:42.646215+0000","last_undegraded":"2024-09-12T12:24:42.646215+0000","last_fullsized":"2024-09-12T12:24:42.646215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:15.407617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703339+0000","last_change":"2024-09-12T12:24:43.644547+0000","last_active":"2024-09-12T12:24:45.703339+0000","last_peered":"2024-09-12T12:24:45.703339+0000","last_clean":"2024-09-12T12:24:45.703339+0000","last_became_active":"2024-09-12T12:24:43.643836+0000","last_became_peered":"2024-09-12T12:24:43.643836+0000","last_unstale":"2024-09-12T12:24:45.703339+0000","last_undegraded":"2024-09-12T12:24:45.703339+0000","last_fullsized":"2024-09-12T12:24:45.703339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:31.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616266+0000","last_change":"2024-09-12T12:24:42.634480+0000","last_active":"2024-09-12T12:24:44.616266+0000","last_peered":"2024-09-12T12:24:44.616266+0000","last_clean":"2024-09-12T12:24:44.616266+0000","last_became_active":"2024-09-12T12:24:42.633367+0000","last_became_peered":"2024-09-12T12:24:42.633367+0000","last_unstale":"2024-09-12T12:24:44.616266+0000","last_undegraded":"2024-09-12T12:24:44.616266+0000","last_fullsized":"2024-09-12T12:24:44.616266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:19.812219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621856+0000","last_change":"2024-09-12T12:24:43.641831+0000","last_active":"2024-09-12T12:24:44.621856+0000","last_peered":"2024-09-12T12:24:44.621856+0000","last_clean":"2024-09-12T12:24:44.621856+0000","last_became_active":"2024-09-12T12:24:43.641063+0000","last_became_peered":"2024-09-12T12:24:43.641063+0000","last_unstale":"2024-09-12T12:24:44.621856+0000","last_undegraded":"2024-09-12T12:24:44.621856+0000","last_fullsized":"2024-09-12T12:24:44.621856+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:46.151100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646203+0000","last_change":"2024-09-12T12:24:42.637461+0000","last_active":"2024-09-12T12:24:42.646203+0000","last_peered":"2024-09-12T12:24:42.646203+0000","last_clean":"2024-09-12T12:24:42.646203+0000","last_became_active":"2024-09-12T12:24:42.637031+0000","last_became_peered":"2024-09-12T12:24:42.637031+0000","last_unstale":"2024-09-12T12:24:42.646203+0000","last_undegraded":"2024-09-12T12:24:42.646203+0000","last_fullsized":"2024-09-12T12:24:42.646203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:34:47.172517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673878+0000","last_change":"2024-09-12T12:24:43.670586+0000","last_active":"2024-09-12T12:24:43.673878+0000","last_peered":"2024-09-12T12:24:43.673878+0000","last_clean":"2024-09-12T12:24:43.673878+0000","last_became_active":"2024-09-12T12:24:43.669772+0000","last_became_peered":"2024-09-12T12:24:43.669772+0000","last_unstale":"2024-09-12T12:24:43.673878+0000","last_undegraded":"2024-09-12T12:24:43.673878+0000","last_fullsized":"2024-09-12T12:24:43.673878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:43.570019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702529+0000","last_change":"2024-09-12T12:24:42.648919+0000","last_active":"2024-09-12T12:24:45.702529+0000","last_peered":"2024-09-12T12:24:45.702529+0000","last_clean":"2024-09-12T12:24:45.702529+0000","last_became_active":"2024-09-12T12:24:42.648039+0000","last_became_peered":"2024-09-12T12:24:42.648039+0000","last_unstale":"2024-09-12T12:24:45.702529+0000","last_undegraded":"2024-09-12T12:24:45.702529+0000","last_fullsized":"2024-09-12T12:24:45.702529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:51.735892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903858+0000","last_change":"2024-09-12T12:24:43.650075+0000","last_active":"2024-09-12T12:24:44.903858+0000","last_peered":"2024-09-12T12:24:44.903858+0000","last_clean":"2024-09-12T12:24:44.903858+0000","last_became_active":"2024-09-12T12:24:43.649215+0000","last_became_peered":"2024-09-12T12:24:43.649215+0000","last_unstale":"2024-09-12T12:24:44.903858+0000","last_undegraded":"2024-09-12T12:24:44.903858+0000","last_fullsized":"2024-09-12T12:24:44.903858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:30.006601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615299+0000","last_change":"2024-09-12T12:24:42.625151+0000","last_active":"2024-09-12T12:24:44.615299+0000","last_peered":"2024-09-12T12:24:44.615299+0000","last_clean":"2024-09-12T12:24:44.615299+0000","last_became_active":"2024-09-12T12:24:42.624745+0000","last_became_peered":"2024-09-12T12:24:42.624745+0000","last_unstale":"2024-09-12T12:24:44.615299+0000","last_undegraded":"2024-09-12T12:24:44.615299+0000","last_fullsized":"2024-09-12T12:24:44.615299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:21.399432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627022+0000","last_change":"2024-09-12T12:24:42.627022+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627022+0000","last_undegraded":"2024-09-12T12:24:42.627022+0000","last_fullsized":"2024-09-12T12:24:42.627022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616802+0000","last_change":"2024-09-12T12:24:42.637921+0000","last_active":"2024-09-12T12:24:44.616802+0000","last_peered":"2024-09-12T12:24:44.616802+0000","last_clean":"2024-09-12T12:24:44.616802+0000","last_became_active":"2024-09-12T12:24:42.637229+0000","last_became_peered":"2024-09-12T12:24:42.637229+0000","last_unstale":"2024-09-12T12:24:44.616802+0000","last_undegraded":"2024-09-12T12:24:44.616802+0000","last_fullsized":"2024-09-12T12:24:44.616802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:59.256685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904941+0000","last_change":"2024-09-12T12:24:43.658926+0000","last_active":"2024-09-12T12:24:44.904941+0000","last_peered":"2024-09-12T12:24:44.904941+0000","last_clean":"2024-09-12T12:24:44.904941+0000","last_became_active":"2024-09-12T12:24:43.658247+0000","last_became_peered":"2024-09-12T12:24:43.658247+0000","last_unstale":"2024-09-12T12:24:44.904941+0000","last_undegraded":"2024-09-12T12:24:44.904941+0000","last_fullsized":"2024-09-12T12:24:44.904941+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.635866+0000","last_change":"2024-09-12T12:24:42.641455+0000","last_active":"2024-09-12T12:24:43.635866+0000","last_peered":"2024-09-12T12:24:43.635866+0000","last_clean":"2024-09-12T12:24:43.635866+0000","last_became_active":"2024-09-12T12:24:42.641145+0000","last_became_peered":"2024-09-12T12:24:42.641145+0000","last_unstale":"2024-09-12T12:24:43.635866+0000","last_undegraded":"2024-09-12T12:24:43.635866+0000","last_fullsized":"2024-09-12T12:24:43.635866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:58.912932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703939+0000","last_change":"2024-09-12T12:24:43.647661+0000","last_active":"2024-09-12T12:24:45.703939+0000","last_peered":"2024-09-12T12:24:45.703939+0000","last_clean":"2024-09-12T12:24:45.703939+0000","last_became_active":"2024-09-12T12:24:43.646874+0000","last_became_peered":"2024-09-12T12:24:43.646874+0000","last_unstale":"2024-09-12T12:24:45.703939+0000","last_undegraded":"2024-09-12T12:24:45.703939+0000","last_fullsized":"2024-09-12T12:24:45.703939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622442+0000","last_change":"2024-09-12T12:24:42.652504+0000","last_active":"2024-09-12T12:24:44.622442+0000","last_peered":"2024-09-12T12:24:44.622442+0000","last_clean":"2024-09-12T12:24:44.622442+0000","last_became_active":"2024-09-12T12:24:42.651709+0000","last_became_peered":"2024-09-12T12:24:42.651709+0000","last_unstale":"2024-09-12T12:24:44.622442+0000","last_undegraded":"2024-09-12T12:24:44.622442+0000","last_fullsized":"2024-09-12T12:24:44.622442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:44.399415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622185+0000","last_change":"2024-09-12T12:24:43.659018+0000","last_active":"2024-09-12T12:24:44.622185+0000","last_peered":"2024-09-12T12:24:44.622185+0000","last_clean":"2024-09-12T12:24:44.622185+0000","last_became_active":"2024-09-12T12:24:43.658369+0000","last_became_peered":"2024-09-12T12:24:43.658369+0000","last_unstale":"2024-09-12T12:24:44.622185+0000","last_undegraded":"2024-09-12T12:24:44.622185+0000","last_fullsized":"2024-09-12T12:24:44.622185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:26.274835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"92'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904658+0000","last_change":"2024-09-12T12:24:42.643454+0000","last_active":"2024-09-12T12:24:44.904658+0000","last_peered":"2024-09-12T12:24:44.904658+0000","last_clean":"2024-09-12T12:24:44.904658+0000","last_became_active":"2024-09-12T12:24:42.642649+0000","last_became_peered":"2024-09-12T12:24:42.642649+0000","last_unstale":"2024-09-12T12:24:44.904658+0000","last_undegraded":"2024-09-12T12:24:44.904658+0000","last_fullsized":"2024-09-12T12:24:44.904658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:30.300204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.666620+0000","last_change":"2024-09-12T12:24:43.647560+0000","last_active":"2024-09-12T12:24:43.666620+0000","last_peered":"2024-09-12T12:24:43.666620+0000","last_clean":"2024-09-12T12:24:43.666620+0000","last_became_active":"2024-09-12T12:24:43.646824+0000","last_became_peered":"2024-09-12T12:24:43.646824+0000","last_unstale":"2024-09-12T12:24:43.666620+0000","last_undegraded":"2024-09-12T12:24:43.666620+0000","last_fullsized":"2024-09-12T12:24:43.666620+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:35:20.924656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684213+0000","last_change":"2024-09-12T12:24:42.637238+0000","last_active":"2024-09-12T12:24:44.684213+0000","last_peered":"2024-09-12T12:24:44.684213+0000","last_clean":"2024-09-12T12:24:44.684213+0000","last_became_active":"2024-09-12T12:24:42.636898+0000","last_became_peered":"2024-09-12T12:24:42.636898+0000","last_unstale":"2024-09-12T12:24:44.684213+0000","last_undegraded":"2024-09-12T12:24:44.684213+0000","last_fullsized":"2024-09-12T12:24:44.684213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:06.749490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616014+0000","last_change":"2024-09-12T12:24:43.657062+0000","last_active":"2024-09-12T12:24:44.616014+0000","last_peered":"2024-09-12T12:24:44.616014+0000","last_clean":"2024-09-12T12:24:44.616014+0000","last_became_active":"2024-09-12T12:24:43.654265+0000","last_became_peered":"2024-09-12T12:24:43.654265+0000","last_unstale":"2024-09-12T12:24:44.616014+0000","last_undegraded":"2024-09-12T12:24:44.616014+0000","last_fullsized":"2024-09-12T12:24:44.616014+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:06.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.685093+0000","last_change":"2024-09-12T12:24:42.627572+0000","last_active":"2024-09-12T12:24:44.685093+0000","last_peered":"2024-09-12T12:24:44.685093+0000","last_clean":"2024-09-12T12:24:44.685093+0000","last_became_active":"2024-09-12T12:24:42.627257+0000","last_became_peered":"2024-09-12T12:24:42.627257+0000","last_unstale":"2024-09-12T12:24:44.685093+0000","last_undegraded":"2024-09-12T12:24:44.685093+0000","last_fullsized":"2024-09-12T12:24:44.685093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:04.397782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903797+0000","last_change":"2024-09-12T12:24:43.658133+0000","last_active":"2024-09-12T12:24:44.903797+0000","last_peered":"2024-09-12T12:24:44.903797+0000","last_clean":"2024-09-12T12:24:44.903797+0000","last_became_active":"2024-09-12T12:24:43.657012+0000","last_became_peered":"2024-09-12T12:24:43.657012+0000","last_unstale":"2024-09-12T12:24:44.903797+0000","last_undegraded":"2024-09-12T12:24:44.903797+0000","last_fullsized":"2024-09-12T12:24:44.903797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:17.137749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903854+0000","last_change":"2024-09-12T12:24:42.643511+0000","last_active":"2024-09-12T12:24:44.903854+0000","last_peered":"2024-09-12T12:24:44.903854+0000","last_clean":"2024-09-12T12:24:44.903854+0000","last_became_active":"2024-09-12T12:24:42.642768+0000","last_became_peered":"2024-09-12T12:24:42.642768+0000","last_unstale":"2024-09-12T12:24:44.903854+0000","last_undegraded":"2024-09-12T12:24:44.903854+0000","last_fullsized":"2024-09-12T12:24:44.903854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:23.588413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683454+0000","last_change":"2024-09-12T12:24:43.636996+0000","last_active":"2024-09-12T12:24:44.683454+0000","last_peered":"2024-09-12T12:24:44.683454+0000","last_clean":"2024-09-12T12:24:44.683454+0000","last_became_active":"2024-09-12T12:24:43.636232+0000","last_became_peered":"2024-09-12T12:24:43.636232+0000","last_unstale":"2024-09-12T12:24:44.683454+0000","last_undegraded":"2024-09-12T12:24:44.683454+0000","last_fullsized":"2024-09-12T12:24:44.683454+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:56:16.348629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615379+0000","last_change":"2024-09-12T12:24:42.642474+0000","last_active":"2024-09-12T12:24:44.615379+0000","last_peered":"2024-09-12T12:24:44.615379+0000","last_clean":"2024-09-12T12:24:44.615379+0000","last_became_active":"2024-09-12T12:24:42.642172+0000","last_became_peered":"2024-09-12T12:24:42.642172+0000","last_unstale":"2024-09-12T12:24:44.615379+0000","last_undegraded":"2024-09-12T12:24:44.615379+0000","last_fullsized":"2024-09-12T12:24:44.615379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:39.580983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703705+0000","last_change":"2024-09-12T12:24:43.639166+0000","last_active":"2024-09-12T12:24:45.703705+0000","last_peered":"2024-09-12T12:24:45.703705+0000","last_clean":"2024-09-12T12:24:45.703705+0000","last_became_active":"2024-09-12T12:24:43.638391+0000","last_became_peered":"2024-09-12T12:24:43.638391+0000","last_unstale":"2024-09-12T12:24:45.703705+0000","last_undegraded":"2024-09-12T12:24:45.703705+0000","last_fullsized":"2024-09-12T12:24:45.703705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:30:37.318619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904436+0000","last_change":"2024-09-12T12:24:42.640318+0000","last_active":"2024-09-12T12:24:44.904436+0000","last_peered":"2024-09-12T12:24:44.904436+0000","last_clean":"2024-09-12T12:24:44.904436+0000","last_became_active":"2024-09-12T12:24:42.639593+0000","last_became_peered":"2024-09-12T12:24:42.639593+0000","last_unstale":"2024-09-12T12:24:44.904436+0000","last_undegraded":"2024-09-12T12:24:44.904436+0000","last_fullsized":"2024-09-12T12:24:44.904436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:18.210648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702964+0000","last_change":"2024-09-12T12:24:43.653241+0000","last_active":"2024-09-12T12:24:45.702964+0000","last_peered":"2024-09-12T12:24:45.702964+0000","last_clean":"2024-09-12T12:24:45.702964+0000","last_became_active":"2024-09-12T12:24:43.652511+0000","last_became_peered":"2024-09-12T12:24:43.652511+0000","last_unstale":"2024-09-12T12:24:45.702964+0000","last_undegraded":"2024-09-12T12:24:45.702964+0000","last_fullsized":"2024-09-12T12:24:45.702964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:45.460852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702785+0000","last_change":"2024-09-12T12:24:42.637205+0000","last_active":"2024-09-12T12:24:45.702785+0000","last_peered":"2024-09-12T12:24:45.702785+0000","last_clean":"2024-09-12T12:24:45.702785+0000","last_became_active":"2024-09-12T12:24:42.636321+0000","last_became_peered":"2024-09-12T12:24:42.636321+0000","last_unstale":"2024-09-12T12:24:45.702785+0000","last_undegraded":"2024-09-12T12:24:45.702785+0000","last_fullsized":"2024-09-12T12:24:45.702785+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622470+0000","last_change":"2024-09-12T12:24:43.649792+0000","last_active":"2024-09-12T12:24:44.622470+0000","last_peered":"2024-09-12T12:24:44.622470+0000","last_clean":"2024-09-12T12:24:44.622470+0000","last_became_active":"2024-09-12T12:24:43.648913+0000","last_became_peered":"2024-09-12T12:24:43.648913+0000","last_unstale":"2024-09-12T12:24:44.622470+0000","last_undegraded":"2024-09-12T12:24:44.622470+0000","last_fullsized":"2024-09-12T12:24:44.622470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:51:34.860957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646718+0000","last_change":"2024-09-12T12:24:42.642838+0000","last_active":"2024-09-12T12:24:42.646718+0000","last_peered":"2024-09-12T12:24:42.646718+0000","last_clean":"2024-09-12T12:24:42.646718+0000","last_became_active":"2024-09-12T12:24:42.642310+0000","last_became_peered":"2024-09-12T12:24:42.642310+0000","last_unstale":"2024-09-12T12:24:42.646718+0000","last_undegraded":"2024-09-12T12:24:42.646718+0000","last_fullsized":"2024-09-12T12:24:42.646718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:52:42.944961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684507+0000","last_change":"2024-09-12T12:24:43.652450+0000","last_active":"2024-09-12T12:24:44.684507+0000","last_peered":"2024-09-12T12:24:44.684507+0000","last_clean":"2024-09-12T12:24:44.684507+0000","last_became_active":"2024-09-12T12:24:43.651643+0000","last_became_peered":"2024-09-12T12:24:43.651643+0000","last_unstale":"2024-09-12T12:24:44.684507+0000","last_undegraded":"2024-09-12T12:24:44.684507+0000","last_fullsized":"2024-09-12T12:24:44.684507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:20.816453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646394+0000","last_change":"2024-09-12T12:24:42.637602+0000","last_active":"2024-09-12T12:24:42.646394+0000","last_peered":"2024-09-12T12:24:42.646394+0000","last_clean":"2024-09-12T12:24:42.646394+0000","last_became_active":"2024-09-12T12:24:42.637106+0000","last_became_peered":"2024-09-12T12:24:42.637106+0000","last_unstale":"2024-09-12T12:24:42.646394+0000","last_undegraded":"2024-09-12T12:24:42.646394+0000","last_fullsized":"2024-09-12T12:24:42.646394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:02.019478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684325+0000","last_change":"2024-09-12T12:24:43.652172+0000","last_active":"2024-09-12T12:24:44.684325+0000","last_peered":"2024-09-12T12:24:44.684325+0000","last_clean":"2024-09-12T12:24:44.684325+0000","last_became_active":"2024-09-12T12:24:43.651461+0000","last_became_peered":"2024-09-12T12:24:43.651461+0000","last_unstale":"2024-09-12T12:24:44.684325+0000","last_undegraded":"2024-09-12T12:24:44.684325+0000","last_fullsized":"2024-09-12T12:24:44.684325+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:24.797814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703619+0000","last_change":"2024-09-12T12:24:42.639853+0000","last_active":"2024-09-12T12:24:45.703619+0000","last_peered":"2024-09-12T12:24:45.703619+0000","last_clean":"2024-09-12T12:24:45.703619+0000","last_became_active":"2024-09-12T12:24:42.639201+0000","last_became_peered":"2024-09-12T12:24:42.639201+0000","last_unstale":"2024-09-12T12:24:45.703619+0000","last_undegraded":"2024-09-12T12:24:45.703619+0000","last_fullsized":"2024-09-12T12:24:45.703619+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:23.071254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683367+0000","last_change":"2024-09-12T12:24:43.656139+0000","last_active":"2024-09-12T12:24:44.683367+0000","last_peered":"2024-09-12T12:24:44.683367+0000","last_clean":"2024-09-12T12:24:44.683367+0000","last_became_active":"2024-09-12T12:24:43.651199+0000","last_became_peered":"2024-09-12T12:24:43.651199+0000","last_unstale":"2024-09-12T12:24:44.683367+0000","last_undegraded":"2024-09-12T12:24:44.683367+0000","last_fullsized":"2024-09-12T12:24:44.683367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:15:43.831842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702539+0000","last_change":"2024-09-12T12:24:42.647349+0000","last_active":"2024-09-12T12:24:45.702539+0000","last_peered":"2024-09-12T12:24:45.702539+0000","last_clean":"2024-09-12T12:24:45.702539+0000","last_became_active":"2024-09-12T12:24:42.646472+0000","last_became_peered":"2024-09-12T12:24:42.646472+0000","last_unstale":"2024-09-12T12:24:45.702539+0000","last_undegraded":"2024-09-12T12:24:45.702539+0000","last_fullsized":"2024-09-12T12:24:45.702539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:05.139726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703187+0000","last_change":"2024-09-12T12:24:43.637022+0000","last_active":"2024-09-12T12:24:45.703187+0000","last_peered":"2024-09-12T12:24:45.703187+0000","last_clean":"2024-09-12T12:24:45.703187+0000","last_became_active":"2024-09-12T12:24:43.636260+0000","last_became_peered":"2024-09-12T12:24:43.636260+0000","last_unstale":"2024-09-12T12:24:45.703187+0000","last_undegraded":"2024-09-12T12:24:45.703187+0000","last_fullsized":"2024-09-12T12:24:45.703187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:15.925952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646621+0000","last_change":"2024-09-12T12:24:42.637645+0000","last_active":"2024-09-12T12:24:42.646621+0000","last_peered":"2024-09-12T12:24:42.646621+0000","last_clean":"2024-09-12T12:24:42.646621+0000","last_became_active":"2024-09-12T12:24:42.637099+0000","last_became_peered":"2024-09-12T12:24:42.637099+0000","last_unstale":"2024-09-12T12:24:42.646621+0000","last_undegraded":"2024-09-12T12:24:42.646621+0000","last_fullsized":"2024-09-12T12:24:42.646621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:44.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626273+0000","last_change":"2024-09-12T12:24:42.626273+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626273+0000","last_undegraded":"2024-09-12T12:24:42.626273+0000","last_fullsized":"2024-09-12T12:24:42.626273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703333+0000","last_change":"2024-09-12T12:24:42.639106+0000","last_active":"2024-09-12T12:24:45.703333+0000","last_peered":"2024-09-12T12:24:45.703333+0000","last_clean":"2024-09-12T12:24:45.703333+0000","last_became_active":"2024-09-12T12:24:42.638341+0000","last_became_peered":"2024-09-12T12:24:42.638341+0000","last_unstale":"2024-09-12T12:24:45.703333+0000","last_undegraded":"2024-09-12T12:24:45.703333+0000","last_fullsized":"2024-09-12T12:24:45.703333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:44.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616629+0000","last_change":"2024-09-12T12:24:43.655680+0000","last_active":"2024-09-12T12:24:44.616629+0000","last_peered":"2024-09-12T12:24:44.616629+0000","last_clean":"2024-09-12T12:24:44.616629+0000","last_became_active":"2024-09-12T12:24:43.654815+0000","last_became_peered":"2024-09-12T12:24:43.654815+0000","last_unstale":"2024-09-12T12:24:44.616629+0000","last_undegraded":"2024-09-12T12:24:44.616629+0000","last_fullsized":"2024-09-12T12:24:44.616629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:23.355405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646503+0000","last_change":"2024-09-12T12:24:42.641239+0000","last_active":"2024-09-12T12:24:42.646503+0000","last_peered":"2024-09-12T12:24:42.646503+0000","last_clean":"2024-09-12T12:24:42.646503+0000","last_became_active":"2024-09-12T12:24:42.638459+0000","last_became_peered":"2024-09-12T12:24:42.638459+0000","last_unstale":"2024-09-12T12:24:42.646503+0000","last_undegraded":"2024-09-12T12:24:42.646503+0000","last_fullsized":"2024-09-12T12:24:42.646503+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:28.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616100+0000","last_change":"2024-09-12T12:24:43.650775+0000","last_active":"2024-09-12T12:24:44.616100+0000","last_peered":"2024-09-12T12:24:44.616100+0000","last_clean":"2024-09-12T12:24:44.616100+0000","last_became_active":"2024-09-12T12:24:43.650001+0000","last_became_peered":"2024-09-12T12:24:43.650001+0000","last_unstale":"2024-09-12T12:24:44.616100+0000","last_undegraded":"2024-09-12T12:24:44.616100+0000","last_fullsized":"2024-09-12T12:24:44.616100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:24.075487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.652548+0000","last_change":"2024-09-12T12:24:42.647803+0000","last_active":"2024-09-12T12:24:42.652548+0000","last_peered":"2024-09-12T12:24:42.652548+0000","last_clean":"2024-09-12T12:24:42.652548+0000","last_became_active":"2024-09-12T12:24:42.647329+0000","last_became_peered":"2024-09-12T12:24:42.647329+0000","last_unstale":"2024-09-12T12:24:42.652548+0000","last_undegraded":"2024-09-12T12:24:42.652548+0000","last_fullsized":"2024-09-12T12:24:42.652548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:02:03.882345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702515+0000","last_change":"2024-09-12T12:24:43.648046+0000","last_active":"2024-09-12T12:24:45.702515+0000","last_peered":"2024-09-12T12:24:45.702515+0000","last_clean":"2024-09-12T12:24:45.702515+0000","last_became_active":"2024-09-12T12:24:43.647327+0000","last_became_peered":"2024-09-12T12:24:43.647327+0000","last_unstale":"2024-09-12T12:24:45.702515+0000","last_undegraded":"2024-09-12T12:24:45.702515+0000","last_fullsized":"2024-09-12T12:24:45.702515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.704161+0000","last_change":"2024-09-12T12:24:42.644303+0000","last_active":"2024-09-12T12:24:45.704161+0000","last_peered":"2024-09-12T12:24:45.704161+0000","last_clean":"2024-09-12T12:24:45.704161+0000","last_became_active":"2024-09-12T12:24:42.643559+0000","last_became_peered":"2024-09-12T12:24:42.643559+0000","last_unstale":"2024-09-12T12:24:45.704161+0000","last_undegraded":"2024-09-12T12:24:45.704161+0000","last_fullsized":"2024-09-12T12:24:45.704161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:52.789860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684880+0000","last_change":"2024-09-12T12:24:43.637804+0000","last_active":"2024-09-12T12:24:44.684880+0000","last_peered":"2024-09-12T12:24:44.684880+0000","last_clean":"2024-09-12T12:24:44.684880+0000","last_became_active":"2024-09-12T12:24:43.636765+0000","last_became_peered":"2024-09-12T12:24:43.636765+0000","last_unstale":"2024-09-12T12:24:44.684880+0000","last_undegraded":"2024-09-12T12:24:44.684880+0000","last_fullsized":"2024-09-12T12:24:44.684880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:22.403665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646546+0000","last_change":"2024-09-12T12:24:42.642157+0000","last_active":"2024-09-12T12:24:42.646546+0000","last_peered":"2024-09-12T12:24:42.646546+0000","last_clean":"2024-09-12T12:24:42.646546+0000","last_became_active":"2024-09-12T12:24:42.637757+0000","last_became_peered":"2024-09-12T12:24:42.637757+0000","last_unstale":"2024-09-12T12:24:42.646546+0000","last_undegraded":"2024-09-12T12:24:42.646546+0000","last_fullsized":"2024-09-12T12:24:42.646546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:53.864687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904303+0000","last_change":"2024-09-12T12:24:43.643084+0000","last_active":"2024-09-12T12:24:44.904303+0000","last_peered":"2024-09-12T12:24:44.904303+0000","last_clean":"2024-09-12T12:24:44.904303+0000","last_became_active":"2024-09-12T12:24:43.642362+0000","last_became_peered":"2024-09-12T12:24:43.642362+0000","last_unstale":"2024-09-12T12:24:44.904303+0000","last_undegraded":"2024-09-12T12:24:44.904303+0000","last_fullsized":"2024-09-12T12:24:44.904303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:45.403648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702332+0000","last_change":"2024-09-12T12:24:42.636916+0000","last_active":"2024-09-12T12:24:45.702332+0000","last_peered":"2024-09-12T12:24:45.702332+0000","last_clean":"2024-09-12T12:24:45.702332+0000","last_became_active":"2024-09-12T12:24:42.634574+0000","last_became_peered":"2024-09-12T12:24:42.634574+0000","last_unstale":"2024-09-12T12:24:45.702332+0000","last_undegraded":"2024-09-12T12:24:45.702332+0000","last_fullsized":"2024-09-12T12:24:45.702332+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:33.717766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683795+0000","last_change":"2024-09-12T12:24:43.644818+0000","last_active":"2024-09-12T12:24:44.683795+0000","last_peered":"2024-09-12T12:24:44.683795+0000","last_clean":"2024-09-12T12:24:44.683795+0000","last_became_active":"2024-09-12T12:24:43.644041+0000","last_became_peered":"2024-09-12T12:24:43.644041+0000","last_unstale":"2024-09-12T12:24:44.683795+0000","last_undegraded":"2024-09-12T12:24:44.683795+0000","last_fullsized":"2024-09-12T12:24:44.683795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:58.101698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.641004+0000","last_change":"2024-09-12T12:24:42.625186+0000","last_active":"2024-09-12T12:24:43.641004+0000","last_peered":"2024-09-12T12:24:43.641004+0000","last_clean":"2024-09-12T12:24:43.641004+0000","last_became_active":"2024-09-12T12:24:42.624773+0000","last_became_peered":"2024-09-12T12:24:42.624773+0000","last_unstale":"2024-09-12T12:24:43.641004+0000","last_undegraded":"2024-09-12T12:24:43.641004+0000","last_fullsized":"2024-09-12T12:24:43.641004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:29.271497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702272+0000","last_change":"2024-09-12T12:24:43.647973+0000","last_active":"2024-09-12T12:24:45.702272+0000","last_peered":"2024-09-12T12:24:45.702272+0000","last_clean":"2024-09-12T12:24:45.702272+0000","last_became_active":"2024-09-12T12:24:43.647185+0000","last_became_peered":"2024-09-12T12:24:43.647185+0000","last_unstale":"2024-09-12T12:24:45.702272+0000","last_undegraded":"2024-09-12T12:24:45.702272+0000","last_fullsized":"2024-09-12T12:24:45.702272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683462+0000","last_change":"2024-09-12T12:24:42.638683+0000","last_active":"2024-09-12T12:24:44.683462+0000","last_peered":"2024-09-12T12:24:44.683462+0000","last_clean":"2024-09-12T12:24:44.683462+0000","last_became_active":"2024-09-12T12:24:42.638385+0000","last_became_peered":"2024-09-12T12:24:42.638385+0000","last_unstale":"2024-09-12T12:24:44.683462+0000","last_undegraded":"2024-09-12T12:24:44.683462+0000","last_fullsized":"2024-09-12T12:24:44.683462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.197995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684036+0000","last_change":"2024-09-12T12:24:43.647413+0000","last_active":"2024-09-12T12:24:44.684036+0000","last_peered":"2024-09-12T12:24:44.684036+0000","last_clean":"2024-09-12T12:24:44.684036+0000","last_became_active":"2024-09-12T12:24:43.646570+0000","last_became_peered":"2024-09-12T12:24:43.646570+0000","last_unstale":"2024-09-12T12:24:44.684036+0000","last_undegraded":"2024-09-12T12:24:44.684036+0000","last_fullsized":"2024-09-12T12:24:44.684036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:53:39.696999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621547+0000","last_change":"2024-09-12T12:24:42.649739+0000","last_active":"2024-09-12T12:24:44.621547+0000","last_peered":"2024-09-12T12:24:44.621547+0000","last_clean":"2024-09-12T12:24:44.621547+0000","last_became_active":"2024-09-12T12:24:42.648926+0000","last_became_peered":"2024-09-12T12:24:42.648926+0000","last_unstale":"2024-09-12T12:24:44.621547+0000","last_undegraded":"2024-09-12T12:24:44.621547+0000","last_fullsized":"2024-09-12T12:24:44.621547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:29.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":302,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621655+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:24:44.621655+0000","last_peered":"2024-09-12T12:24:44.621655+0000","last_clean":"2024-09-12T12:24:44.621655+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:24:44.621655+0000","last_undegraded":"2024-09-12T12:24:44.621655+0000","last_fullsized":"2024-09-12T12:24:44.621655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1122304,"data_stored":476139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1748992,"data_stored":1079694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":489284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":490950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":489248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":489214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542207,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1060232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:48.102 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:48.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fc46a640 1 Processor -- start 2024-09-12T12:24:48.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fc46a640 1 -- start start 2024-09-12T12:24:48.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fc46a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f66f40d1190 0x7f66f40d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fc46a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 0x7f66f40b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fc46a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f40d30b0 con 0x7f66f40d1570 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fc46a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f66f40d3cc0 con 0x7f66f40d1190 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fc46a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66f40d3390 con 0x7f66f40b7740 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f66f40d1570 0x7f66f40cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35248/0 (socket says 172.21.15.3:35248) 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 -- 172.21.15.3:0/615216451 learned_addr learned my addr 172.21.15.3:0/615216451 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fa206640 1 --2- 172.21.15.3:0/615216451 >> v2:172.21.15.142:3300/0 conn(0x7f66f40d1190 0x7f66f40d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66faa07640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 0x7f66f40b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3905875567 0 0) 0x7f66f40d30b0 con 0x7f66f40d1570 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66e0003530 con 0x7f66f40d1570 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3814690253 0 0) 0x7f66e0003530 con 0x7f66f40d1570 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 msgr2=0x7f66f40b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 0x7f66f40b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 >> v2:172.21.15.142:3300/0 conn(0x7f66f40d1190 msgr2=0x7f66f40d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 --2- 172.21.15.3:0/615216451 >> v2:172.21.15.142:3300/0 conn(0x7f66f40d1190 0x7f66f40d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f40d3670 con 0x7f66f40d1570 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f66e4001680 con 0x7f66f40d1570 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 >> v1:172.21.15.3:6789/0 conn(0x7f66f40d1570 legacy=0x7f66f40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 0x7f66e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66e0003f40 0x7f66e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40d3cc0 con 0x7f66e0003f40 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40d3390 con 0x7f66e0004d50 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66e000ba00 con 0x7f66f40b7740 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66e0003f40 0x7f66e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66fa206640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 0x7f66e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66faa07640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 -- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 msgr2=0x7f66e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f40b7740 0x7f66e00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 -- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0004d50 msgr2=0x7f66e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66f9a05640 1 -- 172.21.15.3:0/615216451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e0003530 con 0x7f66e0003f40 2024-09-12T12:24:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.251+0000 7f66faa07640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66f9a05640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66e0003f40 0x7f66e0004810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f66e4001060 tx=0x7f66e401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eba0aaa953f04ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4007d80 con 0x7f66e0003f40 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e40a9020 con 0x7f66e0003f40 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66f9204640 1 -- 172.21.15.3:0/615216451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e40a5ba0 con 0x7f66e0003f40 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- 172.21.15.3:0/615216451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66e0003f40 msgr2=0x7f66e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 --2- 172.21.15.3:0/615216451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66e0003f40 0x7f66e0004810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f66e4001060 tx=0x7f66e401cca0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66faa07640 1 -- 172.21.15.3:0/615216451 reap_dead start 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- 172.21.15.3:0/615216451 shutdown_connections 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- 172.21.15.3:0/615216451 >> 172.21.15.3:0/615216451 conn(0x7f66f4067a00 msgr2=0x7f66f4067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- 172.21.15.3:0/615216451 shutdown_connections 2024-09-12T12:24:48.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- 172.21.15.3:0/615216451 wait complete. 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 Processor -- start 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- start start 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0003f40 0x7f66f40c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f415b510 0x7f66f415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f41e8480 0x7f66f41e8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40d2cd0 con 0x7f66f41e8480 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f66f40d1a00 con 0x7f66e0003f40 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66fc46a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f66f40d2e50 con 0x7f66f415b510 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66faa07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f41e8480 0x7f66f41e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66faa07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f41e8480 0x7f66f41e8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56712/0 (socket says 172.21.15.3:56712) 2024-09-12T12:24:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.255+0000 7f66faa07640 1 -- 172.21.15.3:0/2360224892 learned_addr learned my addr 172.21.15.3:0/2360224892 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66f9a05640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f415b510 0x7f66f415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66fa206640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0003f40 0x7f66f40c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66faa07640 1 -- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f415b510 msgr2=0x7f66f415ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66faa07640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f415b510 0x7f66f415ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66faa07640 1 -- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0003f40 msgr2=0x7f66f40c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66faa07640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0003f40 0x7f66f40c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66faa07640 1 -- 172.21.15.3:0/2360224892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f40c8e60 con 0x7f66f41e8480 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66fa206640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0003f40 0x7f66f40c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66f9a05640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f415b510 0x7f66f415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66faa07640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f41e8480 0x7f66f41e8900 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f66f40d05e0 tx=0x7f66f0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66eaffd640 1 -- 172.21.15.3:0/2360224892 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f000f040 con 0x7f66f41e8480 2024-09-12T12:24:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66eaffd640 1 -- 172.21.15.3:0/2360224892 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f001c070 con 0x7f66f41e8480 2024-09-12T12:24:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f40c9140 con 0x7f66f41e8480 2024-09-12T12:24:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66eaffd640 1 -- 172.21.15.3:0/2360224892 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f000ea90 con 0x7f66f41e8480 2024-09-12T12:24:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41e7c80 con 0x7f66f41e8480 2024-09-12T12:24:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f40c9310 con 0x7f66f41e8480 2024-09-12T12:24:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66eaffd640 1 -- 172.21.15.3:0/2360224892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f66f000ec30 con 0x7f66f41e8480 2024-09-12T12:24:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66eaffd640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66d80678c0 0x7f66d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66eaffd640 1 -- 172.21.15.3:0/2360224892 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f66f0085380 con 0x7f66f41e8480 2024-09-12T12:24:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66fa206640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66d80678c0 0x7f66d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.259+0000 7f66fa206640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66d80678c0 0x7f66d8069d80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f66f40ce6c0 tx=0x7f66ec00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.263+0000 7f66eaffd640 1 -- 172.21.15.3:0/2360224892 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f66f40c9310 con 0x7f66f41e8480 2024-09-12T12:24:48.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.415+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66f40c8c10 con 0x7f66d80678c0 2024-09-12T12:24:48.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.435+0000 7f66eaffd640 1 -- 172.21.15.3:0/2360224892 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251323 (secure 0 0 0) 0x7f66f40c8c10 con 0x7f66d80678c0 2024-09-12T12:24:48.440 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66d80678c0 msgr2=0x7f66d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66d80678c0 0x7f66d8069d80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f66f40ce6c0 tx=0x7f66ec00b040 comp rx=0 tx=0).stop 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f41e8480 msgr2=0x7f66f41e8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f41e8480 0x7f66f41e8900 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f66f40d05e0 tx=0x7f66f0010c20 comp rx=0 tx=0).stop 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 shutdown_connections 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f66d80678c0 0x7f66d8069d80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66f41e8480 0x7f66f41e8900 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66f415b510 0x7f66f415ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 --2- 172.21.15.3:0/2360224892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66e0003f40 0x7f66f40c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 >> 172.21.15.3:0/2360224892 conn(0x7f66f4067a00 msgr2=0x7f66f40b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 shutdown_connections 2024-09-12T12:24:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.439+0000 7f66fc46a640 1 -- 172.21.15.3:0/2360224892 wait complete. 2024-09-12T12:24:48.458 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-12T12:24:47.757681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259648,"kb_used_data":10104,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715072,"statfs":{"total":773094113280,"available":772828233728,"internally_reserved":0,"allocated":10346496,"data_stored":5064627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001901"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616311+0000","last_change":"2024-09-12T12:24:43.638576+0000","last_active":"2024-09-12T12:24:44.616311+0000","last_peered":"2024-09-12T12:24:44.616311+0000","last_clean":"2024-09-12T12:24:44.616311+0000","last_became_active":"2024-09-12T12:24:43.637883+0000","last_became_peered":"2024-09-12T12:24:43.637883+0000","last_unstale":"2024-09-12T12:24:44.616311+0000","last_undegraded":"2024-09-12T12:24:44.616311+0000","last_fullsized":"2024-09-12T12:24:44.616311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:31.746764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631858+0000","last_change":"2024-09-12T12:24:42.631858+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631858+0000","last_undegraded":"2024-09-12T12:24:42.631858+0000","last_fullsized":"2024-09-12T12:24:42.631858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616160+0000","last_change":"2024-09-12T12:24:43.656461+0000","last_active":"2024-09-12T12:24:44.616160+0000","last_peered":"2024-09-12T12:24:44.616160+0000","last_clean":"2024-09-12T12:24:44.616160+0000","last_became_active":"2024-09-12T12:24:43.655523+0000","last_became_peered":"2024-09-12T12:24:43.655523+0000","last_unstale":"2024-09-12T12:24:44.616160+0000","last_undegraded":"2024-09-12T12:24:44.616160+0000","last_fullsized":"2024-09-12T12:24:44.616160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:45.830898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673541+0000","last_change":"2024-09-12T12:24:43.665791+0000","last_active":"2024-09-12T12:24:43.673541+0000","last_peered":"2024-09-12T12:24:43.673541+0000","last_clean":"2024-09-12T12:24:43.673541+0000","last_became_active":"2024-09-12T12:24:43.663819+0000","last_became_peered":"2024-09-12T12:24:43.663819+0000","last_unstale":"2024-09-12T12:24:43.673541+0000","last_undegraded":"2024-09-12T12:24:43.673541+0000","last_fullsized":"2024-09-12T12:24:43.673541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:34.993627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703214+0000","last_change":"2024-09-12T12:24:43.651132+0000","last_active":"2024-09-12T12:24:45.703214+0000","last_peered":"2024-09-12T12:24:45.703214+0000","last_clean":"2024-09-12T12:24:45.703214+0000","last_became_active":"2024-09-12T12:24:43.650439+0000","last_became_peered":"2024-09-12T12:24:43.650439+0000","last_unstale":"2024-09-12T12:24:45.703214+0000","last_undegraded":"2024-09-12T12:24:45.703214+0000","last_fullsized":"2024-09-12T12:24:45.703214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:44.191710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621336+0000","last_change":"2024-09-12T12:24:43.650634+0000","last_active":"2024-09-12T12:24:44.621336+0000","last_peered":"2024-09-12T12:24:44.621336+0000","last_clean":"2024-09-12T12:24:44.621336+0000","last_became_active":"2024-09-12T12:24:43.649447+0000","last_became_peered":"2024-09-12T12:24:43.649447+0000","last_unstale":"2024-09-12T12:24:44.621336+0000","last_undegraded":"2024-09-12T12:24:44.621336+0000","last_fullsized":"2024-09-12T12:24:44.621336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:56.893292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621208+0000","last_change":"2024-09-12T12:24:43.651354+0000","last_active":"2024-09-12T12:24:44.621208+0000","last_peered":"2024-09-12T12:24:44.621208+0000","last_clean":"2024-09-12T12:24:44.621208+0000","last_became_active":"2024-09-12T12:24:43.650570+0000","last_became_peered":"2024-09-12T12:24:43.650570+0000","last_unstale":"2024-09-12T12:24:44.621208+0000","last_undegraded":"2024-09-12T12:24:44.621208+0000","last_fullsized":"2024-09-12T12:24:44.621208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622450+0000","last_change":"2024-09-12T12:24:42.622450+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622450+0000","last_undegraded":"2024-09-12T12:24:42.622450+0000","last_fullsized":"2024-09-12T12:24:42.622450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904799+0000","last_change":"2024-09-12T12:24:43.652939+0000","last_active":"2024-09-12T12:24:44.904799+0000","last_peered":"2024-09-12T12:24:44.904799+0000","last_clean":"2024-09-12T12:24:44.904799+0000","last_became_active":"2024-09-12T12:24:43.651482+0000","last_became_peered":"2024-09-12T12:24:43.651482+0000","last_unstale":"2024-09-12T12:24:44.904799+0000","last_undegraded":"2024-09-12T12:24:44.904799+0000","last_fullsized":"2024-09-12T12:24:44.904799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702479+0000","last_change":"2024-09-12T12:24:43.637408+0000","last_active":"2024-09-12T12:24:45.702479+0000","last_peered":"2024-09-12T12:24:45.702479+0000","last_clean":"2024-09-12T12:24:45.702479+0000","last_became_active":"2024-09-12T12:24:43.636447+0000","last_became_peered":"2024-09-12T12:24:43.636447+0000","last_unstale":"2024-09-12T12:24:45.702479+0000","last_undegraded":"2024-09-12T12:24:45.702479+0000","last_fullsized":"2024-09-12T12:24:45.702479+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621413+0000","last_change":"2024-09-12T12:24:43.649963+0000","last_active":"2024-09-12T12:24:44.621413+0000","last_peered":"2024-09-12T12:24:44.621413+0000","last_clean":"2024-09-12T12:24:44.621413+0000","last_became_active":"2024-09-12T12:24:43.649206+0000","last_became_peered":"2024-09-12T12:24:43.649206+0000","last_unstale":"2024-09-12T12:24:44.621413+0000","last_undegraded":"2024-09-12T12:24:44.621413+0000","last_fullsized":"2024-09-12T12:24:44.621413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:36:44.276769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904389+0000","last_change":"2024-09-12T12:24:43.661406+0000","last_active":"2024-09-12T12:24:44.904389+0000","last_peered":"2024-09-12T12:24:44.904389+0000","last_clean":"2024-09-12T12:24:44.904389+0000","last_became_active":"2024-09-12T12:24:43.660623+0000","last_became_peered":"2024-09-12T12:24:43.660623+0000","last_unstale":"2024-09-12T12:24:44.904389+0000","last_undegraded":"2024-09-12T12:24:44.904389+0000","last_fullsized":"2024-09-12T12:24:44.904389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:16.413303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.671577+0000","last_change":"2024-09-12T12:24:43.653070+0000","last_active":"2024-09-12T12:24:43.671577+0000","last_peered":"2024-09-12T12:24:43.671577+0000","last_clean":"2024-09-12T12:24:43.671577+0000","last_became_active":"2024-09-12T12:24:43.651753+0000","last_became_peered":"2024-09-12T12:24:43.651753+0000","last_unstale":"2024-09-12T12:24:43.671577+0000","last_undegraded":"2024-09-12T12:24:43.671577+0000","last_fullsized":"2024-09-12T12:24:43.671577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:55.597998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683623+0000","last_change":"2024-09-12T12:24:43.622364+0000","last_active":"2024-09-12T12:24:44.683623+0000","last_peered":"2024-09-12T12:24:44.683623+0000","last_clean":"2024-09-12T12:24:44.683623+0000","last_became_active":"2024-09-12T12:24:43.620763+0000","last_became_peered":"2024-09-12T12:24:43.620763+0000","last_unstale":"2024-09-12T12:24:44.683623+0000","last_undegraded":"2024-09-12T12:24:44.683623+0000","last_fullsized":"2024-09-12T12:24:44.683623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:42.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673880+0000","last_change":"2024-09-12T12:24:43.670224+0000","last_active":"2024-09-12T12:24:43.673880+0000","last_peered":"2024-09-12T12:24:43.673880+0000","last_clean":"2024-09-12T12:24:43.673880+0000","last_became_active":"2024-09-12T12:24:43.669622+0000","last_became_peered":"2024-09-12T12:24:43.669622+0000","last_unstale":"2024-09-12T12:24:43.673880+0000","last_undegraded":"2024-09-12T12:24:43.673880+0000","last_fullsized":"2024-09-12T12:24:43.673880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:52:53.453156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621285+0000","last_change":"2024-09-12T12:24:43.658919+0000","last_active":"2024-09-12T12:24:44.621285+0000","last_peered":"2024-09-12T12:24:44.621285+0000","last_clean":"2024-09-12T12:24:44.621285+0000","last_became_active":"2024-09-12T12:24:43.658279+0000","last_became_peered":"2024-09-12T12:24:43.658279+0000","last_unstale":"2024-09-12T12:24:44.621285+0000","last_undegraded":"2024-09-12T12:24:44.621285+0000","last_fullsized":"2024-09-12T12:24:44.621285+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:40.554937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626138+0000","last_change":"2024-09-12T12:24:42.626138+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626138+0000","last_undegraded":"2024-09-12T12:24:42.626138+0000","last_fullsized":"2024-09-12T12:24:42.626138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673828+0000","last_change":"2024-09-12T12:24:43.666474+0000","last_active":"2024-09-12T12:24:43.673828+0000","last_peered":"2024-09-12T12:24:43.673828+0000","last_clean":"2024-09-12T12:24:43.673828+0000","last_became_active":"2024-09-12T12:24:43.665798+0000","last_became_peered":"2024-09-12T12:24:43.665798+0000","last_unstale":"2024-09-12T12:24:43.673828+0000","last_undegraded":"2024-09-12T12:24:43.673828+0000","last_fullsized":"2024-09-12T12:24:43.673828+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:28.095510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683840+0000","last_change":"2024-09-12T12:24:43.647026+0000","last_active":"2024-09-12T12:24:44.683840+0000","last_peered":"2024-09-12T12:24:44.683840+0000","last_clean":"2024-09-12T12:24:44.683840+0000","last_became_active":"2024-09-12T12:24:43.646064+0000","last_became_peered":"2024-09-12T12:24:43.646064+0000","last_unstale":"2024-09-12T12:24:44.683840+0000","last_undegraded":"2024-09-12T12:24:44.683840+0000","last_fullsized":"2024-09-12T12:24:44.683840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:38.167692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904536+0000","last_change":"2024-09-12T12:24:43.666544+0000","last_active":"2024-09-12T12:24:44.904536+0000","last_peered":"2024-09-12T12:24:44.904536+0000","last_clean":"2024-09-12T12:24:44.904536+0000","last_became_active":"2024-09-12T12:24:43.665661+0000","last_became_peered":"2024-09-12T12:24:43.665661+0000","last_unstale":"2024-09-12T12:24:44.904536+0000","last_undegraded":"2024-09-12T12:24:44.904536+0000","last_fullsized":"2024-09-12T12:24:44.904536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:22.604865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684069+0000","last_change":"2024-09-12T12:24:43.645911+0000","last_active":"2024-09-12T12:24:44.684069+0000","last_peered":"2024-09-12T12:24:44.684069+0000","last_clean":"2024-09-12T12:24:44.684069+0000","last_became_active":"2024-09-12T12:24:43.645127+0000","last_became_peered":"2024-09-12T12:24:43.645127+0000","last_unstale":"2024-09-12T12:24:44.684069+0000","last_undegraded":"2024-09-12T12:24:44.684069+0000","last_fullsized":"2024-09-12T12:24:44.684069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:01.666655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631303+0000","last_change":"2024-09-12T12:24:42.631303+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631303+0000","last_undegraded":"2024-09-12T12:24:42.631303+0000","last_fullsized":"2024-09-12T12:24:42.631303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622827+0000","last_change":"2024-09-12T12:24:42.622827+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622827+0000","last_undegraded":"2024-09-12T12:24:42.622827+0000","last_fullsized":"2024-09-12T12:24:42.622827+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621737+0000","last_change":"2024-09-12T12:24:43.646502+0000","last_active":"2024-09-12T12:24:44.621737+0000","last_peered":"2024-09-12T12:24:44.621737+0000","last_clean":"2024-09-12T12:24:44.621737+0000","last_became_active":"2024-09-12T12:24:43.645053+0000","last_became_peered":"2024-09-12T12:24:43.645053+0000","last_unstale":"2024-09-12T12:24:44.621737+0000","last_undegraded":"2024-09-12T12:24:44.621737+0000","last_fullsized":"2024-09-12T12:24:44.621737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:51:43.421188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703778+0000","last_change":"2024-09-12T12:24:43.645720+0000","last_active":"2024-09-12T12:24:45.703778+0000","last_peered":"2024-09-12T12:24:45.703778+0000","last_clean":"2024-09-12T12:24:45.703778+0000","last_became_active":"2024-09-12T12:24:43.644960+0000","last_became_peered":"2024-09-12T12:24:43.644960+0000","last_unstale":"2024-09-12T12:24:45.703778+0000","last_undegraded":"2024-09-12T12:24:45.703778+0000","last_fullsized":"2024-09-12T12:24:45.703778+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:07:07.741088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627753+0000","last_change":"2024-09-12T12:24:42.627753+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627753+0000","last_undegraded":"2024-09-12T12:24:42.627753+0000","last_fullsized":"2024-09-12T12:24:42.627753+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622169+0000","last_change":"2024-09-12T12:24:43.661393+0000","last_active":"2024-09-12T12:24:44.622169+0000","last_peered":"2024-09-12T12:24:44.622169+0000","last_clean":"2024-09-12T12:24:44.622169+0000","last_became_active":"2024-09-12T12:24:43.660705+0000","last_became_peered":"2024-09-12T12:24:43.660705+0000","last_unstale":"2024-09-12T12:24:44.622169+0000","last_undegraded":"2024-09-12T12:24:44.622169+0000","last_fullsized":"2024-09-12T12:24:44.622169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:34:20.041489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703551+0000","last_change":"2024-09-12T12:24:43.659047+0000","last_active":"2024-09-12T12:24:45.703551+0000","last_peered":"2024-09-12T12:24:45.703551+0000","last_clean":"2024-09-12T12:24:45.703551+0000","last_became_active":"2024-09-12T12:24:43.658354+0000","last_became_peered":"2024-09-12T12:24:43.658354+0000","last_unstale":"2024-09-12T12:24:45.703551+0000","last_undegraded":"2024-09-12T12:24:45.703551+0000","last_fullsized":"2024-09-12T12:24:45.703551+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:22.035037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627728+0000","last_change":"2024-09-12T12:24:42.627728+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627728+0000","last_undegraded":"2024-09-12T12:24:42.627728+0000","last_fullsized":"2024-09-12T12:24:42.627728+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684067+0000","last_change":"2024-09-12T12:24:43.645157+0000","last_active":"2024-09-12T12:24:44.684067+0000","last_peered":"2024-09-12T12:24:44.684067+0000","last_clean":"2024-09-12T12:24:44.684067+0000","last_became_active":"2024-09-12T12:24:43.644341+0000","last_became_peered":"2024-09-12T12:24:43.644341+0000","last_unstale":"2024-09-12T12:24:44.684067+0000","last_undegraded":"2024-09-12T12:24:44.684067+0000","last_fullsized":"2024-09-12T12:24:44.684067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:39.841147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673604+0000","last_change":"2024-09-12T12:24:43.664785+0000","last_active":"2024-09-12T12:24:43.673604+0000","last_peered":"2024-09-12T12:24:43.673604+0000","last_clean":"2024-09-12T12:24:43.673604+0000","last_became_active":"2024-09-12T12:24:43.663968+0000","last_became_peered":"2024-09-12T12:24:43.663968+0000","last_unstale":"2024-09-12T12:24:43.673604+0000","last_undegraded":"2024-09-12T12:24:43.673604+0000","last_fullsized":"2024-09-12T12:24:43.673604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:01.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615264+0000","last_change":"2024-09-12T12:24:43.661779+0000","last_active":"2024-09-12T12:24:44.615264+0000","last_peered":"2024-09-12T12:24:44.615264+0000","last_clean":"2024-09-12T12:24:44.615264+0000","last_became_active":"2024-09-12T12:24:43.660239+0000","last_became_peered":"2024-09-12T12:24:43.660239+0000","last_unstale":"2024-09-12T12:24:44.615264+0000","last_undegraded":"2024-09-12T12:24:44.615264+0000","last_fullsized":"2024-09-12T12:24:44.615264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:55.818309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683905+0000","last_change":"2024-09-12T12:24:43.648961+0000","last_active":"2024-09-12T12:24:44.683905+0000","last_peered":"2024-09-12T12:24:44.683905+0000","last_clean":"2024-09-12T12:24:44.683905+0000","last_became_active":"2024-09-12T12:24:43.648231+0000","last_became_peered":"2024-09-12T12:24:43.648231+0000","last_unstale":"2024-09-12T12:24:44.683905+0000","last_undegraded":"2024-09-12T12:24:44.683905+0000","last_fullsized":"2024-09-12T12:24:44.683905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:04.030268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646506+0000","last_change":"2024-09-12T12:24:42.641566+0000","last_active":"2024-09-12T12:24:42.646506+0000","last_peered":"2024-09-12T12:24:42.646506+0000","last_clean":"2024-09-12T12:24:42.646506+0000","last_became_active":"2024-09-12T12:24:42.641294+0000","last_became_peered":"2024-09-12T12:24:42.641294+0000","last_unstale":"2024-09-12T12:24:42.646506+0000","last_undegraded":"2024-09-12T12:24:42.646506+0000","last_fullsized":"2024-09-12T12:24:42.646506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:28.668303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673591+0000","last_change":"2024-09-12T12:24:43.665125+0000","last_active":"2024-09-12T12:24:43.673591+0000","last_peered":"2024-09-12T12:24:43.673591+0000","last_clean":"2024-09-12T12:24:43.673591+0000","last_became_active":"2024-09-12T12:24:43.664317+0000","last_became_peered":"2024-09-12T12:24:43.664317+0000","last_unstale":"2024-09-12T12:24:43.673591+0000","last_undegraded":"2024-09-12T12:24:43.673591+0000","last_fullsized":"2024-09-12T12:24:43.673591+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:46.448481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621763+0000","last_change":"2024-09-12T12:24:42.653011+0000","last_active":"2024-09-12T12:24:44.621763+0000","last_peered":"2024-09-12T12:24:44.621763+0000","last_clean":"2024-09-12T12:24:44.621763+0000","last_became_active":"2024-09-12T12:24:42.652145+0000","last_became_peered":"2024-09-12T12:24:42.652145+0000","last_unstale":"2024-09-12T12:24:44.621763+0000","last_undegraded":"2024-09-12T12:24:44.621763+0000","last_fullsized":"2024-09-12T12:24:44.621763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:16.133515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615814+0000","last_change":"2024-09-12T12:24:43.651221+0000","last_active":"2024-09-12T12:24:44.615814+0000","last_peered":"2024-09-12T12:24:44.615814+0000","last_clean":"2024-09-12T12:24:44.615814+0000","last_became_active":"2024-09-12T12:24:43.650519+0000","last_became_peered":"2024-09-12T12:24:43.650519+0000","last_unstale":"2024-09-12T12:24:44.615814+0000","last_undegraded":"2024-09-12T12:24:44.615814+0000","last_fullsized":"2024-09-12T12:24:44.615814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:03.987247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.649894+0000","last_change":"2024-09-12T12:24:42.646442+0000","last_active":"2024-09-12T12:24:42.649894+0000","last_peered":"2024-09-12T12:24:42.649894+0000","last_clean":"2024-09-12T12:24:42.649894+0000","last_became_active":"2024-09-12T12:24:42.643894+0000","last_became_peered":"2024-09-12T12:24:42.643894+0000","last_unstale":"2024-09-12T12:24:42.649894+0000","last_undegraded":"2024-09-12T12:24:42.649894+0000","last_fullsized":"2024-09-12T12:24:42.649894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:23:30.708412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.905160+0000","last_change":"2024-09-12T12:24:43.646657+0000","last_active":"2024-09-12T12:24:44.905160+0000","last_peered":"2024-09-12T12:24:44.905160+0000","last_clean":"2024-09-12T12:24:44.905160+0000","last_became_active":"2024-09-12T12:24:43.645851+0000","last_became_peered":"2024-09-12T12:24:43.645851+0000","last_unstale":"2024-09-12T12:24:44.905160+0000","last_undegraded":"2024-09-12T12:24:44.905160+0000","last_fullsized":"2024-09-12T12:24:44.905160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646215+0000","last_change":"2024-09-12T12:24:42.637641+0000","last_active":"2024-09-12T12:24:42.646215+0000","last_peered":"2024-09-12T12:24:42.646215+0000","last_clean":"2024-09-12T12:24:42.646215+0000","last_became_active":"2024-09-12T12:24:42.637338+0000","last_became_peered":"2024-09-12T12:24:42.637338+0000","last_unstale":"2024-09-12T12:24:42.646215+0000","last_undegraded":"2024-09-12T12:24:42.646215+0000","last_fullsized":"2024-09-12T12:24:42.646215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:15.407617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703339+0000","last_change":"2024-09-12T12:24:43.644547+0000","last_active":"2024-09-12T12:24:45.703339+0000","last_peered":"2024-09-12T12:24:45.703339+0000","last_clean":"2024-09-12T12:24:45.703339+0000","last_became_active":"2024-09-12T12:24:43.643836+0000","last_became_peered":"2024-09-12T12:24:43.643836+0000","last_unstale":"2024-09-12T12:24:45.703339+0000","last_undegraded":"2024-09-12T12:24:45.703339+0000","last_fullsized":"2024-09-12T12:24:45.703339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:31.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616266+0000","last_change":"2024-09-12T12:24:42.634480+0000","last_active":"2024-09-12T12:24:44.616266+0000","last_peered":"2024-09-12T12:24:44.616266+0000","last_clean":"2024-09-12T12:24:44.616266+0000","last_became_active":"2024-09-12T12:24:42.633367+0000","last_became_peered":"2024-09-12T12:24:42.633367+0000","last_unstale":"2024-09-12T12:24:44.616266+0000","last_undegraded":"2024-09-12T12:24:44.616266+0000","last_fullsized":"2024-09-12T12:24:44.616266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:19.812219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621856+0000","last_change":"2024-09-12T12:24:43.641831+0000","last_active":"2024-09-12T12:24:44.621856+0000","last_peered":"2024-09-12T12:24:44.621856+0000","last_clean":"2024-09-12T12:24:44.621856+0000","last_became_active":"2024-09-12T12:24:43.641063+0000","last_became_peered":"2024-09-12T12:24:43.641063+0000","last_unstale":"2024-09-12T12:24:44.621856+0000","last_undegraded":"2024-09-12T12:24:44.621856+0000","last_fullsized":"2024-09-12T12:24:44.621856+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:46.151100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646203+0000","last_change":"2024-09-12T12:24:42.637461+0000","last_active":"2024-09-12T12:24:42.646203+0000","last_peered":"2024-09-12T12:24:42.646203+0000","last_clean":"2024-09-12T12:24:42.646203+0000","last_became_active":"2024-09-12T12:24:42.637031+0000","last_became_peered":"2024-09-12T12:24:42.637031+0000","last_unstale":"2024-09-12T12:24:42.646203+0000","last_undegraded":"2024-09-12T12:24:42.646203+0000","last_fullsized":"2024-09-12T12:24:42.646203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:34:47.172517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673878+0000","last_change":"2024-09-12T12:24:43.670586+0000","last_active":"2024-09-12T12:24:43.673878+0000","last_peered":"2024-09-12T12:24:43.673878+0000","last_clean":"2024-09-12T12:24:43.673878+0000","last_became_active":"2024-09-12T12:24:43.669772+0000","last_became_peered":"2024-09-12T12:24:43.669772+0000","last_unstale":"2024-09-12T12:24:43.673878+0000","last_undegraded":"2024-09-12T12:24:43.673878+0000","last_fullsized":"2024-09-12T12:24:43.673878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:43.570019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702529+0000","last_change":"2024-09-12T12:24:42.648919+0000","last_active":"2024-09-12T12:24:45.702529+0000","last_peered":"2024-09-12T12:24:45.702529+0000","last_clean":"2024-09-12T12:24:45.702529+0000","last_became_active":"2024-09-12T12:24:42.648039+0000","last_became_peered":"2024-09-12T12:24:42.648039+0000","last_unstale":"2024-09-12T12:24:45.702529+0000","last_undegraded":"2024-09-12T12:24:45.702529+0000","last_fullsized":"2024-09-12T12:24:45.702529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:51.735892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903858+0000","last_change":"2024-09-12T12:24:43.650075+0000","last_active":"2024-09-12T12:24:44.903858+0000","last_peered":"2024-09-12T12:24:44.903858+0000","last_clean":"2024-09-12T12:24:44.903858+0000","last_became_active":"2024-09-12T12:24:43.649215+0000","last_became_peered":"2024-09-12T12:24:43.649215+0000","last_unstale":"2024-09-12T12:24:44.903858+0000","last_undegraded":"2024-09-12T12:24:44.903858+0000","last_fullsized":"2024-09-12T12:24:44.903858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:30.006601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615299+0000","last_change":"2024-09-12T12:24:42.625151+0000","last_active":"2024-09-12T12:24:44.615299+0000","last_peered":"2024-09-12T12:24:44.615299+0000","last_clean":"2024-09-12T12:24:44.615299+0000","last_became_active":"2024-09-12T12:24:42.624745+0000","last_became_peered":"2024-09-12T12:24:42.624745+0000","last_unstale":"2024-09-12T12:24:44.615299+0000","last_undegraded":"2024-09-12T12:24:44.615299+0000","last_fullsized":"2024-09-12T12:24:44.615299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:21.399432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627022+0000","last_change":"2024-09-12T12:24:42.627022+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627022+0000","last_undegraded":"2024-09-12T12:24:42.627022+0000","last_fullsized":"2024-09-12T12:24:42.627022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616802+0000","last_change":"2024-09-12T12:24:42.637921+0000","last_active":"2024-09-12T12:24:44.616802+0000","last_peered":"2024-09-12T12:24:44.616802+0000","last_clean":"2024-09-12T12:24:44.616802+0000","last_became_active":"2024-09-12T12:24:42.637229+0000","last_became_peered":"2024-09-12T12:24:42.637229+0000","last_unstale":"2024-09-12T12:24:44.616802+0000","last_undegraded":"2024-09-12T12:24:44.616802+0000","last_fullsized":"2024-09-12T12:24:44.616802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:59.256685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904941+0000","last_change":"2024-09-12T12:24:43.658926+0000","last_active":"2024-09-12T12:24:44.904941+0000","last_peered":"2024-09-12T12:24:44.904941+0000","last_clean":"2024-09-12T12:24:44.904941+0000","last_became_active":"2024-09-12T12:24:43.658247+0000","last_became_peered":"2024-09-12T12:24:43.658247+0000","last_unstale":"2024-09-12T12:24:44.904941+0000","last_undegraded":"2024-09-12T12:24:44.904941+0000","last_fullsized":"2024-09-12T12:24:44.904941+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.635866+0000","last_change":"2024-09-12T12:24:42.641455+0000","last_active":"2024-09-12T12:24:43.635866+0000","last_peered":"2024-09-12T12:24:43.635866+0000","last_clean":"2024-09-12T12:24:43.635866+0000","last_became_active":"2024-09-12T12:24:42.641145+0000","last_became_peered":"2024-09-12T12:24:42.641145+0000","last_unstale":"2024-09-12T12:24:43.635866+0000","last_undegraded":"2024-09-12T12:24:43.635866+0000","last_fullsized":"2024-09-12T12:24:43.635866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:58.912932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703939+0000","last_change":"2024-09-12T12:24:43.647661+0000","last_active":"2024-09-12T12:24:45.703939+0000","last_peered":"2024-09-12T12:24:45.703939+0000","last_clean":"2024-09-12T12:24:45.703939+0000","last_became_active":"2024-09-12T12:24:43.646874+0000","last_became_peered":"2024-09-12T12:24:43.646874+0000","last_unstale":"2024-09-12T12:24:45.703939+0000","last_undegraded":"2024-09-12T12:24:45.703939+0000","last_fullsized":"2024-09-12T12:24:45.703939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622442+0000","last_change":"2024-09-12T12:24:42.652504+0000","last_active":"2024-09-12T12:24:44.622442+0000","last_peered":"2024-09-12T12:24:44.622442+0000","last_clean":"2024-09-12T12:24:44.622442+0000","last_became_active":"2024-09-12T12:24:42.651709+0000","last_became_peered":"2024-09-12T12:24:42.651709+0000","last_unstale":"2024-09-12T12:24:44.622442+0000","last_undegraded":"2024-09-12T12:24:44.622442+0000","last_fullsized":"2024-09-12T12:24:44.622442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:44.399415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622185+0000","last_change":"2024-09-12T12:24:43.659018+0000","last_active":"2024-09-12T12:24:44.622185+0000","last_peered":"2024-09-12T12:24:44.622185+0000","last_clean":"2024-09-12T12:24:44.622185+0000","last_became_active":"2024-09-12T12:24:43.658369+0000","last_became_peered":"2024-09-12T12:24:43.658369+0000","last_unstale":"2024-09-12T12:24:44.622185+0000","last_undegraded":"2024-09-12T12:24:44.622185+0000","last_fullsized":"2024-09-12T12:24:44.622185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:26.274835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"92'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904658+0000","last_change":"2024-09-12T12:24:42.643454+0000","last_active":"2024-09-12T12:24:44.904658+0000","last_peered":"2024-09-12T12:24:44.904658+0000","last_clean":"2024-09-12T12:24:44.904658+0000","last_became_active":"2024-09-12T12:24:42.642649+0000","last_became_peered":"2024-09-12T12:24:42.642649+0000","last_unstale":"2024-09-12T12:24:44.904658+0000","last_undegraded":"2024-09-12T12:24:44.904658+0000","last_fullsized":"2024-09-12T12:24:44.904658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:30.300204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.666620+0000","last_change":"2024-09-12T12:24:43.647560+0000","last_active":"2024-09-12T12:24:43.666620+0000","last_peered":"2024-09-12T12:24:43.666620+0000","last_clean":"2024-09-12T12:24:43.666620+0000","last_became_active":"2024-09-12T12:24:43.646824+0000","last_became_peered":"2024-09-12T12:24:43.646824+0000","last_unstale":"2024-09-12T12:24:43.666620+0000","last_undegraded":"2024-09-12T12:24:43.666620+0000","last_fullsized":"2024-09-12T12:24:43.666620+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:35:20.924656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684213+0000","last_change":"2024-09-12T12:24:42.637238+0000","last_active":"2024-09-12T12:24:44.684213+0000","last_peered":"2024-09-12T12:24:44.684213+0000","last_clean":"2024-09-12T12:24:44.684213+0000","last_became_active":"2024-09-12T12:24:42.636898+0000","last_became_peered":"2024-09-12T12:24:42.636898+0000","last_unstale":"2024-09-12T12:24:44.684213+0000","last_undegraded":"2024-09-12T12:24:44.684213+0000","last_fullsized":"2024-09-12T12:24:44.684213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:06.749490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616014+0000","last_change":"2024-09-12T12:24:43.657062+0000","last_active":"2024-09-12T12:24:44.616014+0000","last_peered":"2024-09-12T12:24:44.616014+0000","last_clean":"2024-09-12T12:24:44.616014+0000","last_became_active":"2024-09-12T12:24:43.654265+0000","last_became_peered":"2024-09-12T12:24:43.654265+0000","last_unstale":"2024-09-12T12:24:44.616014+0000","last_undegraded":"2024-09-12T12:24:44.616014+0000","last_fullsized":"2024-09-12T12:24:44.616014+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:06.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.685093+0000","last_change":"2024-09-12T12:24:42.627572+0000","last_active":"2024-09-12T12:24:44.685093+0000","last_peered":"2024-09-12T12:24:44.685093+0000","last_clean":"2024-09-12T12:24:44.685093+0000","last_became_active":"2024-09-12T12:24:42.627257+0000","last_became_peered":"2024-09-12T12:24:42.627257+0000","last_unstale":"2024-09-12T12:24:44.685093+0000","last_undegraded":"2024-09-12T12:24:44.685093+0000","last_fullsized":"2024-09-12T12:24:44.685093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:04.397782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903797+0000","last_change":"2024-09-12T12:24:43.658133+0000","last_active":"2024-09-12T12:24:44.903797+0000","last_peered":"2024-09-12T12:24:44.903797+0000","last_clean":"2024-09-12T12:24:44.903797+0000","last_became_active":"2024-09-12T12:24:43.657012+0000","last_became_peered":"2024-09-12T12:24:43.657012+0000","last_unstale":"2024-09-12T12:24:44.903797+0000","last_undegraded":"2024-09-12T12:24:44.903797+0000","last_fullsized":"2024-09-12T12:24:44.903797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:17.137749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903854+0000","last_change":"2024-09-12T12:24:42.643511+0000","last_active":"2024-09-12T12:24:44.903854+0000","last_peered":"2024-09-12T12:24:44.903854+0000","last_clean":"2024-09-12T12:24:44.903854+0000","last_became_active":"2024-09-12T12:24:42.642768+0000","last_became_peered":"2024-09-12T12:24:42.642768+0000","last_unstale":"2024-09-12T12:24:44.903854+0000","last_undegraded":"2024-09-12T12:24:44.903854+0000","last_fullsized":"2024-09-12T12:24:44.903854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:23.588413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683454+0000","last_change":"2024-09-12T12:24:43.636996+0000","last_active":"2024-09-12T12:24:44.683454+0000","last_peered":"2024-09-12T12:24:44.683454+0000","last_clean":"2024-09-12T12:24:44.683454+0000","last_became_active":"2024-09-12T12:24:43.636232+0000","last_became_peered":"2024-09-12T12:24:43.636232+0000","last_unstale":"2024-09-12T12:24:44.683454+0000","last_undegraded":"2024-09-12T12:24:44.683454+0000","last_fullsized":"2024-09-12T12:24:44.683454+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:56:16.348629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615379+0000","last_change":"2024-09-12T12:24:42.642474+0000","last_active":"2024-09-12T12:24:44.615379+0000","last_peered":"2024-09-12T12:24:44.615379+0000","last_clean":"2024-09-12T12:24:44.615379+0000","last_became_active":"2024-09-12T12:24:42.642172+0000","last_became_peered":"2024-09-12T12:24:42.642172+0000","last_unstale":"2024-09-12T12:24:44.615379+0000","last_undegraded":"2024-09-12T12:24:44.615379+0000","last_fullsized":"2024-09-12T12:24:44.615379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:39.580983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703705+0000","last_change":"2024-09-12T12:24:43.639166+0000","last_active":"2024-09-12T12:24:45.703705+0000","last_peered":"2024-09-12T12:24:45.703705+0000","last_clean":"2024-09-12T12:24:45.703705+0000","last_became_active":"2024-09-12T12:24:43.638391+0000","last_became_peered":"2024-09-12T12:24:43.638391+0000","last_unstale":"2024-09-12T12:24:45.703705+0000","last_undegraded":"2024-09-12T12:24:45.703705+0000","last_fullsized":"2024-09-12T12:24:45.703705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:30:37.318619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904436+0000","last_change":"2024-09-12T12:24:42.640318+0000","last_active":"2024-09-12T12:24:44.904436+0000","last_peered":"2024-09-12T12:24:44.904436+0000","last_clean":"2024-09-12T12:24:44.904436+0000","last_became_active":"2024-09-12T12:24:42.639593+0000","last_became_peered":"2024-09-12T12:24:42.639593+0000","last_unstale":"2024-09-12T12:24:44.904436+0000","last_undegraded":"2024-09-12T12:24:44.904436+0000","last_fullsized":"2024-09-12T12:24:44.904436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:18.210648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702964+0000","last_change":"2024-09-12T12:24:43.653241+0000","last_active":"2024-09-12T12:24:45.702964+0000","last_peered":"2024-09-12T12:24:45.702964+0000","last_clean":"2024-09-12T12:24:45.702964+0000","last_became_active":"2024-09-12T12:24:43.652511+0000","last_became_peered":"2024-09-12T12:24:43.652511+0000","last_unstale":"2024-09-12T12:24:45.702964+0000","last_undegraded":"2024-09-12T12:24:45.702964+0000","last_fullsized":"2024-09-12T12:24:45.702964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:45.460852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702785+0000","last_change":"2024-09-12T12:24:42.637205+0000","last_active":"2024-09-12T12:24:45.702785+0000","last_peered":"2024-09-12T12:24:45.702785+0000","last_clean":"2024-09-12T12:24:45.702785+0000","last_became_active":"2024-09-12T12:24:42.636321+0000","last_became_peered":"2024-09-12T12:24:42.636321+0000","last_unstale":"2024-09-12T12:24:45.702785+0000","last_undegraded":"2024-09-12T12:24:45.702785+0000","last_fullsized":"2024-09-12T12:24:45.702785+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622470+0000","last_change":"2024-09-12T12:24:43.649792+0000","last_active":"2024-09-12T12:24:44.622470+0000","last_peered":"2024-09-12T12:24:44.622470+0000","last_clean":"2024-09-12T12:24:44.622470+0000","last_became_active":"2024-09-12T12:24:43.648913+0000","last_became_peered":"2024-09-12T12:24:43.648913+0000","last_unstale":"2024-09-12T12:24:44.622470+0000","last_undegraded":"2024-09-12T12:24:44.622470+0000","last_fullsized":"2024-09-12T12:24:44.622470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:51:34.860957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646718+0000","last_change":"2024-09-12T12:24:42.642838+0000","last_active":"2024-09-12T12:24:42.646718+0000","last_peered":"2024-09-12T12:24:42.646718+0000","last_clean":"2024-09-12T12:24:42.646718+0000","last_became_active":"2024-09-12T12:24:42.642310+0000","last_became_peered":"2024-09-12T12:24:42.642310+0000","last_unstale":"2024-09-12T12:24:42.646718+0000","last_undegraded":"2024-09-12T12:24:42.646718+0000","last_fullsized":"2024-09-12T12:24:42.646718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:52:42.944961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684507+0000","last_change":"2024-09-12T12:24:43.652450+0000","last_active":"2024-09-12T12:24:44.684507+0000","last_peered":"2024-09-12T12:24:44.684507+0000","last_clean":"2024-09-12T12:24:44.684507+0000","last_became_active":"2024-09-12T12:24:43.651643+0000","last_became_peered":"2024-09-12T12:24:43.651643+0000","last_unstale":"2024-09-12T12:24:44.684507+0000","last_undegraded":"2024-09-12T12:24:44.684507+0000","last_fullsized":"2024-09-12T12:24:44.684507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:20.816453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646394+0000","last_change":"2024-09-12T12:24:42.637602+0000","last_active":"2024-09-12T12:24:42.646394+0000","last_peered":"2024-09-12T12:24:42.646394+0000","last_clean":"2024-09-12T12:24:42.646394+0000","last_became_active":"2024-09-12T12:24:42.637106+0000","last_became_peered":"2024-09-12T12:24:42.637106+0000","last_unstale":"2024-09-12T12:24:42.646394+0000","last_undegraded":"2024-09-12T12:24:42.646394+0000","last_fullsized":"2024-09-12T12:24:42.646394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:02.019478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684325+0000","last_change":"2024-09-12T12:24:43.652172+0000","last_active":"2024-09-12T12:24:44.684325+0000","last_peered":"2024-09-12T12:24:44.684325+0000","last_clean":"2024-09-12T12:24:44.684325+0000","last_became_active":"2024-09-12T12:24:43.651461+0000","last_became_peered":"2024-09-12T12:24:43.651461+0000","last_unstale":"2024-09-12T12:24:44.684325+0000","last_undegraded":"2024-09-12T12:24:44.684325+0000","last_fullsized":"2024-09-12T12:24:44.684325+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:24.797814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703619+0000","last_change":"2024-09-12T12:24:42.639853+0000","last_active":"2024-09-12T12:24:45.703619+0000","last_peered":"2024-09-12T12:24:45.703619+0000","last_clean":"2024-09-12T12:24:45.703619+0000","last_became_active":"2024-09-12T12:24:42.639201+0000","last_became_peered":"2024-09-12T12:24:42.639201+0000","last_unstale":"2024-09-12T12:24:45.703619+0000","last_undegraded":"2024-09-12T12:24:45.703619+0000","last_fullsized":"2024-09-12T12:24:45.703619+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:23.071254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683367+0000","last_change":"2024-09-12T12:24:43.656139+0000","last_active":"2024-09-12T12:24:44.683367+0000","last_peered":"2024-09-12T12:24:44.683367+0000","last_clean":"2024-09-12T12:24:44.683367+0000","last_became_active":"2024-09-12T12:24:43.651199+0000","last_became_peered":"2024-09-12T12:24:43.651199+0000","last_unstale":"2024-09-12T12:24:44.683367+0000","last_undegraded":"2024-09-12T12:24:44.683367+0000","last_fullsized":"2024-09-12T12:24:44.683367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:15:43.831842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702539+0000","last_change":"2024-09-12T12:24:42.647349+0000","last_active":"2024-09-12T12:24:45.702539+0000","last_peered":"2024-09-12T12:24:45.702539+0000","last_clean":"2024-09-12T12:24:45.702539+0000","last_became_active":"2024-09-12T12:24:42.646472+0000","last_became_peered":"2024-09-12T12:24:42.646472+0000","last_unstale":"2024-09-12T12:24:45.702539+0000","last_undegraded":"2024-09-12T12:24:45.702539+0000","last_fullsized":"2024-09-12T12:24:45.702539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:05.139726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703187+0000","last_change":"2024-09-12T12:24:43.637022+0000","last_active":"2024-09-12T12:24:45.703187+0000","last_peered":"2024-09-12T12:24:45.703187+0000","last_clean":"2024-09-12T12:24:45.703187+0000","last_became_active":"2024-09-12T12:24:43.636260+0000","last_became_peered":"2024-09-12T12:24:43.636260+0000","last_unstale":"2024-09-12T12:24:45.703187+0000","last_undegraded":"2024-09-12T12:24:45.703187+0000","last_fullsized":"2024-09-12T12:24:45.703187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:15.925952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646621+0000","last_change":"2024-09-12T12:24:42.637645+0000","last_active":"2024-09-12T12:24:42.646621+0000","last_peered":"2024-09-12T12:24:42.646621+0000","last_clean":"2024-09-12T12:24:42.646621+0000","last_became_active":"2024-09-12T12:24:42.637099+0000","last_became_peered":"2024-09-12T12:24:42.637099+0000","last_unstale":"2024-09-12T12:24:42.646621+0000","last_undegraded":"2024-09-12T12:24:42.646621+0000","last_fullsized":"2024-09-12T12:24:42.646621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:44.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626273+0000","last_change":"2024-09-12T12:24:42.626273+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626273+0000","last_undegraded":"2024-09-12T12:24:42.626273+0000","last_fullsized":"2024-09-12T12:24:42.626273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703333+0000","last_change":"2024-09-12T12:24:42.639106+0000","last_active":"2024-09-12T12:24:45.703333+0000","last_peered":"2024-09-12T12:24:45.703333+0000","last_clean":"2024-09-12T12:24:45.703333+0000","last_became_active":"2024-09-12T12:24:42.638341+0000","last_became_peered":"2024-09-12T12:24:42.638341+0000","last_unstale":"2024-09-12T12:24:45.703333+0000","last_undegraded":"2024-09-12T12:24:45.703333+0000","last_fullsized":"2024-09-12T12:24:45.703333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:44.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616629+0000","last_change":"2024-09-12T12:24:43.655680+0000","last_active":"2024-09-12T12:24:44.616629+0000","last_peered":"2024-09-12T12:24:44.616629+0000","last_clean":"2024-09-12T12:24:44.616629+0000","last_became_active":"2024-09-12T12:24:43.654815+0000","last_became_peered":"2024-09-12T12:24:43.654815+0000","last_unstale":"2024-09-12T12:24:44.616629+0000","last_undegraded":"2024-09-12T12:24:44.616629+0000","last_fullsized":"2024-09-12T12:24:44.616629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:23.355405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646503+0000","last_change":"2024-09-12T12:24:42.641239+0000","last_active":"2024-09-12T12:24:42.646503+0000","last_peered":"2024-09-12T12:24:42.646503+0000","last_clean":"2024-09-12T12:24:42.646503+0000","last_became_active":"2024-09-12T12:24:42.638459+0000","last_became_peered":"2024-09-12T12:24:42.638459+0000","last_unstale":"2024-09-12T12:24:42.646503+0000","last_undegraded":"2024-09-12T12:24:42.646503+0000","last_fullsized":"2024-09-12T12:24:42.646503+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:28.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616100+0000","last_change":"2024-09-12T12:24:43.650775+0000","last_active":"2024-09-12T12:24:44.616100+0000","last_peered":"2024-09-12T12:24:44.616100+0000","last_clean":"2024-09-12T12:24:44.616100+0000","last_became_active":"2024-09-12T12:24:43.650001+0000","last_became_peered":"2024-09-12T12:24:43.650001+0000","last_unstale":"2024-09-12T12:24:44.616100+0000","last_undegraded":"2024-09-12T12:24:44.616100+0000","last_fullsized":"2024-09-12T12:24:44.616100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:24.075487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.652548+0000","last_change":"2024-09-12T12:24:42.647803+0000","last_active":"2024-09-12T12:24:42.652548+0000","last_peered":"2024-09-12T12:24:42.652548+0000","last_clean":"2024-09-12T12:24:42.652548+0000","last_became_active":"2024-09-12T12:24:42.647329+0000","last_became_peered":"2024-09-12T12:24:42.647329+0000","last_unstale":"2024-09-12T12:24:42.652548+0000","last_undegraded":"2024-09-12T12:24:42.652548+0000","last_fullsized":"2024-09-12T12:24:42.652548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:02:03.882345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702515+0000","last_change":"2024-09-12T12:24:43.648046+0000","last_active":"2024-09-12T12:24:45.702515+0000","last_peered":"2024-09-12T12:24:45.702515+0000","last_clean":"2024-09-12T12:24:45.702515+0000","last_became_active":"2024-09-12T12:24:43.647327+0000","last_became_peered":"2024-09-12T12:24:43.647327+0000","last_unstale":"2024-09-12T12:24:45.702515+0000","last_undegraded":"2024-09-12T12:24:45.702515+0000","last_fullsized":"2024-09-12T12:24:45.702515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.704161+0000","last_change":"2024-09-12T12:24:42.644303+0000","last_active":"2024-09-12T12:24:45.704161+0000","last_peered":"2024-09-12T12:24:45.704161+0000","last_clean":"2024-09-12T12:24:45.704161+0000","last_became_active":"2024-09-12T12:24:42.643559+0000","last_became_peered":"2024-09-12T12:24:42.643559+0000","last_unstale":"2024-09-12T12:24:45.704161+0000","last_undegraded":"2024-09-12T12:24:45.704161+0000","last_fullsized":"2024-09-12T12:24:45.704161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:52.789860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684880+0000","last_change":"2024-09-12T12:24:43.637804+0000","last_active":"2024-09-12T12:24:44.684880+0000","last_peered":"2024-09-12T12:24:44.684880+0000","last_clean":"2024-09-12T12:24:44.684880+0000","last_became_active":"2024-09-12T12:24:43.636765+0000","last_became_peered":"2024-09-12T12:24:43.636765+0000","last_unstale":"2024-09-12T12:24:44.684880+0000","last_undegraded":"2024-09-12T12:24:44.684880+0000","last_fullsized":"2024-09-12T12:24:44.684880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:22.403665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646546+0000","last_change":"2024-09-12T12:24:42.642157+0000","last_active":"2024-09-12T12:24:42.646546+0000","last_peered":"2024-09-12T12:24:42.646546+0000","last_clean":"2024-09-12T12:24:42.646546+0000","last_became_active":"2024-09-12T12:24:42.637757+0000","last_became_peered":"2024-09-12T12:24:42.637757+0000","last_unstale":"2024-09-12T12:24:42.646546+0000","last_undegraded":"2024-09-12T12:24:42.646546+0000","last_fullsized":"2024-09-12T12:24:42.646546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:53.864687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904303+0000","last_change":"2024-09-12T12:24:43.643084+0000","last_active":"2024-09-12T12:24:44.904303+0000","last_peered":"2024-09-12T12:24:44.904303+0000","last_clean":"2024-09-12T12:24:44.904303+0000","last_became_active":"2024-09-12T12:24:43.642362+0000","last_became_peered":"2024-09-12T12:24:43.642362+0000","last_unstale":"2024-09-12T12:24:44.904303+0000","last_undegraded":"2024-09-12T12:24:44.904303+0000","last_fullsized":"2024-09-12T12:24:44.904303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:45.403648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702332+0000","last_change":"2024-09-12T12:24:42.636916+0000","last_active":"2024-09-12T12:24:45.702332+0000","last_peered":"2024-09-12T12:24:45.702332+0000","last_clean":"2024-09-12T12:24:45.702332+0000","last_became_active":"2024-09-12T12:24:42.634574+0000","last_became_peered":"2024-09-12T12:24:42.634574+0000","last_unstale":"2024-09-12T12:24:45.702332+0000","last_undegraded":"2024-09-12T12:24:45.702332+0000","last_fullsized":"2024-09-12T12:24:45.702332+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:33.717766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683795+0000","last_change":"2024-09-12T12:24:43.644818+0000","last_active":"2024-09-12T12:24:44.683795+0000","last_peered":"2024-09-12T12:24:44.683795+0000","last_clean":"2024-09-12T12:24:44.683795+0000","last_became_active":"2024-09-12T12:24:43.644041+0000","last_became_peered":"2024-09-12T12:24:43.644041+0000","last_unstale":"2024-09-12T12:24:44.683795+0000","last_undegraded":"2024-09-12T12:24:44.683795+0000","last_fullsized":"2024-09-12T12:24:44.683795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:58.101698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.641004+0000","last_change":"2024-09-12T12:24:42.625186+0000","last_active":"2024-09-12T12:24:43.641004+0000","last_peered":"2024-09-12T12:24:43.641004+0000","last_clean":"2024-09-12T12:24:43.641004+0000","last_became_active":"2024-09-12T12:24:42.624773+0000","last_became_peered":"2024-09-12T12:24:42.624773+0000","last_unstale":"2024-09-12T12:24:43.641004+0000","last_undegraded":"2024-09-12T12:24:43.641004+0000","last_fullsized":"2024-09-12T12:24:43.641004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:29.271497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702272+0000","last_change":"2024-09-12T12:24:43.647973+0000","last_active":"2024-09-12T12:24:45.702272+0000","last_peered":"2024-09-12T12:24:45.702272+0000","last_clean":"2024-09-12T12:24:45.702272+0000","last_became_active":"2024-09-12T12:24:43.647185+0000","last_became_peered":"2024-09-12T12:24:43.647185+0000","last_unstale":"2024-09-12T12:24:45.702272+0000","last_undegraded":"2024-09-12T12:24:45.702272+0000","last_fullsized":"2024-09-12T12:24:45.702272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683462+0000","last_change":"2024-09-12T12:24:42.638683+0000","last_active":"2024-09-12T12:24:44.683462+0000","last_peered":"2024-09-12T12:24:44.683462+0000","last_clean":"2024-09-12T12:24:44.683462+0000","last_became_active":"2024-09-12T12:24:42.638385+0000","last_became_peered":"2024-09-12T12:24:42.638385+0000","last_unstale":"2024-09-12T12:24:44.683462+0000","last_undegraded":"2024-09-12T12:24:44.683462+0000","last_fullsized":"2024-09-12T12:24:44.683462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.197995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684036+0000","last_change":"2024-09-12T12:24:43.647413+0000","last_active":"2024-09-12T12:24:44.684036+0000","last_peered":"2024-09-12T12:24:44.684036+0000","last_clean":"2024-09-12T12:24:44.684036+0000","last_became_active":"2024-09-12T12:24:43.646570+0000","last_became_peered":"2024-09-12T12:24:43.646570+0000","last_unstale":"2024-09-12T12:24:44.684036+0000","last_undegraded":"2024-09-12T12:24:44.684036+0000","last_fullsized":"2024-09-12T12:24:44.684036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:53:39.696999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621547+0000","last_change":"2024-09-12T12:24:42.649739+0000","last_active":"2024-09-12T12:24:44.621547+0000","last_peered":"2024-09-12T12:24:44.621547+0000","last_clean":"2024-09-12T12:24:44.621547+0000","last_became_active":"2024-09-12T12:24:42.648926+0000","last_became_peered":"2024-09-12T12:24:42.648926+0000","last_unstale":"2024-09-12T12:24:44.621547+0000","last_undegraded":"2024-09-12T12:24:44.621547+0000","last_fullsized":"2024-09-12T12:24:44.621547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:29.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":302,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621655+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:24:44.621655+0000","last_peered":"2024-09-12T12:24:44.621655+0000","last_clean":"2024-09-12T12:24:44.621655+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:24:44.621655+0000","last_undegraded":"2024-09-12T12:24:44.621655+0000","last_fullsized":"2024-09-12T12:24:44.621655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1122304,"data_stored":476139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1748992,"data_stored":1079694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":489284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":490950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":489248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":489214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542207,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1060232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:48.461 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3e is not active+clean 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.631858+0000', 'last_change': '2024-09-12T12:24:42.631858+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.631858+0000', 'last_undegraded': '2024-09-12T12:24:42.631858+0000', 'last_fullsized': '2024-09-12T12:24:42.631858+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:PG 15.38 is not active+clean 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.622450+0000', 'last_change': '2024-09-12T12:24:42.622450+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.622450+0000', 'last_undegraded': '2024-09-12T12:24:42.622450+0000', 'last_fullsized': '2024-09-12T12:24:42.622450+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2f is not active+clean 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.626138+0000', 'last_change': '2024-09-12T12:24:42.626138+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.626138+0000', 'last_undegraded': '2024-09-12T12:24:42.626138+0000', 'last_fullsized': '2024-09-12T12:24:42.626138+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2a is not active+clean 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.631303+0000', 'last_change': '2024-09-12T12:24:42.631303+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.631303+0000', 'last_undegraded': '2024-09-12T12:24:42.631303+0000', 'last_fullsized': '2024-09-12T12:24:42.631303+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:PG 15.29 is not active+clean 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.622827+0000', 'last_change': '2024-09-12T12:24:42.622827+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.622827+0000', 'last_undegraded': '2024-09-12T12:24:42.622827+0000', 'last_fullsized': '2024-09-12T12:24:42.622827+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.462 INFO:tasks.ceph.ceph_manager.ceph:PG 15.26 is not active+clean 2024-09-12T12:24:48.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.627753+0000', 'last_change': '2024-09-12T12:24:42.627753+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.627753+0000', 'last_undegraded': '2024-09-12T12:24:42.627753+0000', 'last_fullsized': '2024-09-12T12:24:42.627753+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.463 INFO:tasks.ceph.ceph_manager.ceph:PG 15.23 is not active+clean 2024-09-12T12:24:48.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.627728+0000', 'last_change': '2024-09-12T12:24:42.627728+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.627728+0000', 'last_undegraded': '2024-09-12T12:24:42.627728+0000', 'last_fullsized': '2024-09-12T12:24:42.627728+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.463 INFO:tasks.ceph.ceph_manager.ceph:PG 15.16 is not active+clean 2024-09-12T12:24:48.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.627022+0000', 'last_change': '2024-09-12T12:24:42.627022+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.627022+0000', 'last_undegraded': '2024-09-12T12:24:42.627022+0000', 'last_fullsized': '2024-09-12T12:24:42.627022+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.463 INFO:tasks.ceph.ceph_manager.ceph:PG 15.6 is not active+clean 2024-09-12T12:24:48.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 91, 'state': 'creating+peering', 'last_fresh': '2024-09-12T12:24:42.626273+0000', 'last_change': '2024-09-12T12:24:42.626273+0000', 'last_active': '2024-09-12T12:24:42.594580+0000', 'last_peered': '2024-09-12T12:24:42.594580+0000', 'last_clean': '2024-09-12T12:24:42.594580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-12T12:24:42.626273+0000', 'last_undegraded': '2024-09-12T12:24:42.626273+0000', 'last_fullsized': '2024-09-12T12:24:42.626273+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 91, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'last_clean_scrub_stamp': '2024-09-12T12:24:42.594580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-12T12:24:48.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-12T12:24:48.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.615+0000 7f401a823640 1 Processor -- start 2024-09-12T12:24:48.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.615+0000 7f401a823640 1 -- start start 2024-09-12T12:24:48.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f401a823640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 0x7f40140d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f401a823640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f40140b7740 0x7f40140b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f401a823640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40140d30b0 con 0x7f40140d1570 2024-09-12T12:24:48.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f401a823640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f40140d3cc0 con 0x7f40140b7740 2024-09-12T12:24:48.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f401a823640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40140d3390 con 0x7f40140d1190 2024-09-12T12:24:48.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f40137fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f40140d1570 0x7f40140cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35262/0 (socket says 172.21.15.3:35262) 2024-09-12T12:24:48.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f40137fe640 1 -- 172.21.15.3:0/3449493140 learned_addr learned my addr 172.21.15.3:0/3449493140 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 0x7f40140d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4018dc0640 1 --2- 172.21.15.3:0/3449493140 >> v2:172.21.15.142:3300/0 conn(0x7f40140b7740 0x7f40140b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1739100181 0 0) 0x7f40140d30b0 con 0x7f40140d1570 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ffc003530 con 0x7f40140d1570 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 519742672 0 0) 0x7f3ffc003530 con 0x7f40140d1570 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 msgr2=0x7f40140d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 0x7f40140d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 >> v2:172.21.15.142:3300/0 conn(0x7f40140b7740 msgr2=0x7f40140b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 --2- 172.21.15.3:0/3449493140 >> v2:172.21.15.142:3300/0 conn(0x7f40140b7740 0x7f40140b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40140d3670 con 0x7f40140d1570 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f4008001680 con 0x7f40140d1570 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 >> v1:172.21.15.3:6789/0 conn(0x7f40140d1570 legacy=0x7f40140cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ffc0039c0 0x7f3ffc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ffc004810 0x7f3ffc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 0x7f3ffc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40140d3cc0 con 0x7f3ffc0039c0 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40140d3390 con 0x7f3ffc004810 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3ffc00ba80 con 0x7f40140d1190 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ffc0039c0 0x7f3ffc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f40137fe640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ffc004810 0x7f3ffc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 -- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 msgr2=0x7f3ffc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4018dc0640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 0x7f3ffc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 0x7f3ffc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 -- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ffc004810 msgr2=0x7f3ffc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:48.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ffc004810 0x7f3ffc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 -- 172.21.15.3:0/3449493140 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc003530 con 0x7f3ffc0039c0 2024-09-12T12:24:48.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f40137fe640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ffc004810 0x7f3ffc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:48.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4018dc0640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40140d1190 0x7f3ffc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:48.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4013fff640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ffc0039c0 0x7f3ffc003de0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4004009fc0 tx=0x7f400401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df2020adf29d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:48.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040b2070 con 0x7f3ffc0039c0 2024-09-12T12:24:48.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40040a5bb0 con 0x7f3ffc0039c0 2024-09-12T12:24:48.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.619+0000 7f4012ffd640 1 -- 172.21.15.3:0/3449493140 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004009c10 con 0x7f3ffc0039c0 2024-09-12T12:24:48.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- 172.21.15.3:0/3449493140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ffc0039c0 msgr2=0x7f3ffc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 --2- 172.21.15.3:0/3449493140 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ffc0039c0 0x7f3ffc003de0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4004009fc0 tx=0x7f400401cc80 comp rx=0 tx=0).stop 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4018dc0640 1 -- 172.21.15.3:0/3449493140 reap_dead start 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- 172.21.15.3:0/3449493140 shutdown_connections 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- 172.21.15.3:0/3449493140 >> 172.21.15.3:0/3449493140 conn(0x7f4014067a00 msgr2=0x7f4014067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- 172.21.15.3:0/3449493140 shutdown_connections 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- 172.21.15.3:0/3449493140 wait complete. 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 Processor -- start 2024-09-12T12:24:48.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- start start 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40141f7370 0x7f40141603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40141e7740 0x7f40141f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40141f8030 0x7f40141f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40140d2cd0 con 0x7f40141f7370 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40140d19d0 con 0x7f40141e7740 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40140d2e50 con 0x7f40141f8030 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40141f7370 0x7f40141603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40141f7370 0x7f40141603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56738/0 (socket says 172.21.15.3:56738) 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 -- 172.21.15.3:0/316964975 learned_addr learned my addr 172.21.15.3:0/316964975 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:48.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f40137fe640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40141e7740 0x7f40141f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4018dc0640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40141f8030 0x7f40141f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 -- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40141f8030 msgr2=0x7f40141f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40141f8030 0x7f40141f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 -- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40141e7740 msgr2=0x7f40141f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40141e7740 0x7f40141f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 -- 172.21.15.3:0/316964975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4014160ae0 con 0x7f40141f7370 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4013fff640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40141f7370 0x7f40141603d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f400401cab0 tx=0x7f3ffc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4010ff9640 1 -- 172.21.15.3:0/316964975 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040be070 con 0x7f40141f7370 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4010ff9640 1 -- 172.21.15.3:0/316964975 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40040b3040 con 0x7f40141f7370 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4014160dc0 con 0x7f40141f7370 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f4010ff9640 1 -- 172.21.15.3:0/316964975 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040b2050 con 0x7f40141f7370 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.623+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4014161380 con 0x7f40141f7370 2024-09-12T12:24:48.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.627+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401415e860 con 0x7f40141f7370 2024-09-12T12:24:48.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.627+0000 7f4010ff9640 1 -- 172.21.15.3:0/316964975 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40040c1070 con 0x7f40141f7370 2024-09-12T12:24:48.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.627+0000 7f4010ff9640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40000678f0 0x7f4000069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:48.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.627+0000 7f4010ff9640 1 -- 172.21.15.3:0/316964975 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f4004120540 con 0x7f40141f7370 2024-09-12T12:24:48.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.627+0000 7f40137fe640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40000678f0 0x7f4000069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:48.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.627+0000 7f40137fe640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40000678f0 0x7f4000069db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4014161870 tx=0x7f4008005150 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:48.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.631+0000 7f4010ff9640 1 -- 172.21.15.3:0/316964975 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4004016990 con 0x7f40141f7370 2024-09-12T12:24:48.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.855+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f40141f8410 con 0x7f40141f7370 2024-09-12T12:24:48.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.855+0000 7f4010ff9640 1 -- 172.21.15.3:0/316964975 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1560 (secure 0 0 0) 0x7f40040f0a30 con 0x7f40141f7370 2024-09-12T12:24:48.861 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:48.861 INFO:teuthology.orchestra.run.smithi003.stdout:{"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":323,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726143578,"num_in_osds":8,"osd_in_since":1726143567,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":88},{"state_name":"creating+peering","count":9}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592196,"bytes_used":265879552,"bytes_avail":772828233728,"bytes_total":773094113280,"inactive_pgs_ratio":0.092783503234386444,"write_bytes_sec":2217,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":62,"btime":"2024-09-12T12:24:46:664139+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":6738}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-12T12:24:31.752975+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40000678f0 msgr2=0x7f4000069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40000678f0 0x7f4000069db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4014161870 tx=0x7f4008005150 comp rx=0 tx=0).stop 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40141f7370 msgr2=0x7f40141603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40141f7370 0x7f40141603d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f400401cab0 tx=0x7f3ffc005840 comp rx=0 tx=0).stop 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 shutdown_connections 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40000678f0 0x7f4000069db0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40141f8030 0x7f40141f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40141e7740 0x7f40141f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 --2- 172.21.15.3:0/316964975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40141f7370 0x7f40141603d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:48.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 >> 172.21.15.3:0/316964975 conn(0x7f4014067a00 msgr2=0x7f40140b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:48.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 shutdown_connections 2024-09-12T12:24:48.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:48.859+0000 7f401a823640 1 -- 172.21.15.3:0/316964975 wait complete. 2024-09-12T12:24:48.880 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T12:24:48.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:49.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b1618b640 1 Processor -- start 2024-09-12T12:24:49.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b1618b640 1 -- start start 2024-09-12T12:24:49.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b1618b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b100ae250 0x7f8b100ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:49.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b1618b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b1015b620 0x7f8b1015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:49.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b1618b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b100ad0c0 con 0x7f8b100d05b0 2024-09-12T12:24:49.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b1618b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f8b100ad3a0 con 0x7f8b100ae250 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b1618b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8b100ad680 con 0x7f8b1015b620 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b0effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b100ae250 0x7f8b100ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b0effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f8b100ae250 0x7f8b100ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56754/0 (socket says 172.21.15.3:56754) 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b0effd640 1 -- 172.21.15.3:0/1920577150 learned_addr learned my addr 172.21.15.3:0/1920577150 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b0effd640 1 -- 172.21.15.3:0/1920577150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b1015b620 msgr2=0x7f8b1015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b0effd640 1 --2- 172.21.15.3:0/1920577150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b1015b620 0x7f8b1015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b0effd640 1 -- 172.21.15.3:0/1920577150 >> v1:172.21.15.3:6789/0 conn(0x7f8b100d05b0 legacy=0x7f8b100d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:49.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.003+0000 7f8b0effd640 1 -- 172.21.15.3:0/1920577150 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b100ad960 con 0x7f8b100ae250 2024-09-12T12:24:49.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b0effd640 1 --2- 172.21.15.3:0/1920577150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b100ae6e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8b040098a0 tx=0x7f8b0401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f0176d91642962 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:49.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b0e7fc640 1 -- 172.21.15.3:0/1920577150 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040ab640 con 0x7f8b100ae250 2024-09-12T12:24:49.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b0e7fc640 1 -- 172.21.15.3:0/1920577150 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b040b0070 con 0x7f8b100ae250 2024-09-12T12:24:49.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b0e7fc640 1 -- 172.21.15.3:0/1920577150 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040aba00 con 0x7f8b100ae250 2024-09-12T12:24:49.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 -- 172.21.15.3:0/1920577150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 msgr2=0x7f8b100ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 --2- 172.21.15.3:0/1920577150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b100ae6e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8b040098a0 tx=0x7f8b0401c510 comp rx=0 tx=0).stop 2024-09-12T12:24:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 -- 172.21.15.3:0/1920577150 shutdown_connections 2024-09-12T12:24:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 --2- 172.21.15.3:0/1920577150 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b1015b620 0x7f8b1015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 --2- 172.21.15.3:0/1920577150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b100ae6e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 -- 172.21.15.3:0/1920577150 >> 172.21.15.3:0/1920577150 conn(0x7f8b10067e00 msgr2=0x7f8b10068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 -- 172.21.15.3:0/1920577150 shutdown_connections 2024-09-12T12:24:49.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 -- 172.21.15.3:0/1920577150 wait complete. 2024-09-12T12:24:49.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 Processor -- start 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.007+0000 7f8b1618b640 1 -- start start 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b10158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b100d05b0 0x7f8b101507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b1015b620 0x7f8b100b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1015c9b0 con 0x7f8b100ae250 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1015c830 con 0x7f8b1015b620 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f8b10167cb0 con 0x7f8b100d05b0 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b10158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b10158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56766/0 (socket says 172.21.15.3:56766) 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 -- 172.21.15.3:0/587201337 learned_addr learned my addr 172.21.15.3:0/587201337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0effd640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b100d05b0 0x7f8b101507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:49.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0ffff640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b1015b620 0x7f8b100b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 -- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b100d05b0 msgr2=0x7f8b101507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b100d05b0 0x7f8b101507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 -- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b1015b620 msgr2=0x7f8b100b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b1015b620 0x7f8b100b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 -- 172.21.15.3:0/587201337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b100afdf0 con 0x7f8b100ae250 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0effd640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b100d05b0 0x7f8b101507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0ffff640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b1015b620 0x7f8b100b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b0f7fe640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b10158c20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c420 tx=0x7f8b0000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8af7fff640 1 -- 172.21.15.3:0/587201337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000052d0 con 0x7f8b100ae250 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8af7fff640 1 -- 172.21.15.3:0/587201337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b00005470 con 0x7f8b100ae250 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b100cba60 con 0x7f8b100ae250 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8af7fff640 1 -- 172.21.15.3:0/587201337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00007260 con 0x7f8b100ae250 2024-09-12T12:24:49.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b100b08d0 con 0x7f8b100ae250 2024-09-12T12:24:49.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b100ae6e0 con 0x7f8b100ae250 2024-09-12T12:24:49.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8af7fff640 1 -- 172.21.15.3:0/587201337 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8b00007400 con 0x7f8b100ae250 2024-09-12T12:24:49.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8af7fff640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8afc0678c0 0x7f8afc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:49.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.011+0000 7f8af7fff640 1 -- 172.21.15.3:0/587201337 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f8b000851d0 con 0x7f8b100ae250 2024-09-12T12:24:49.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.015+0000 7f8b0effd640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8afc0678c0 0x7f8afc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:49.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.015+0000 7f8b0effd640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8afc0678c0 0x7f8afc069d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8b040af000 tx=0x7f8b04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:49.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.015+0000 7f8af7fff640 1 -- 172.21.15.3:0/587201337 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8b00055d10 con 0x7f8b100ae250 2024-09-12T12:24:49.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.175+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b100b02d0 con 0x7f8afc0678c0 2024-09-12T12:24:49.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.191+0000 7f8af7fff640 1 -- 172.21.15.3:0/587201337 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251323 (secure 0 0 0) 0x7f8b100b02d0 con 0x7f8afc0678c0 2024-09-12T12:24:49.198 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:49.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8afc0678c0 msgr2=0x7f8afc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8afc0678c0 0x7f8afc069d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8b040af000 tx=0x7f8b04008040 comp rx=0 tx=0).stop 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 msgr2=0x7f8b10158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b10158c20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c420 tx=0x7f8b0000c8e0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 shutdown_connections 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f8afc0678c0 0x7f8afc069d80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b1015b620 0x7f8b100b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8b100d05b0 0x7f8b101507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 --2- 172.21.15.3:0/587201337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8b100ae250 0x7f8b10158c20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.195+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 >> 172.21.15.3:0/587201337 conn(0x7f8b10067e00 msgr2=0x7f8b100681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.199+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 shutdown_connections 2024-09-12T12:24:49.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:49.199+0000 7f8b1618b640 1 -- 172.21.15.3:0/587201337 wait complete. 2024-09-12T12:24:49.216 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-12T12:24:47.757681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":192,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259648,"kb_used_data":10104,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715072,"statfs":{"total":773094113280,"available":772828233728,"internally_reserved":0,"allocated":10346496,"data_stored":5064627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001901"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616311+0000","last_change":"2024-09-12T12:24:43.638576+0000","last_active":"2024-09-12T12:24:44.616311+0000","last_peered":"2024-09-12T12:24:44.616311+0000","last_clean":"2024-09-12T12:24:44.616311+0000","last_became_active":"2024-09-12T12:24:43.637883+0000","last_became_peered":"2024-09-12T12:24:43.637883+0000","last_unstale":"2024-09-12T12:24:44.616311+0000","last_undegraded":"2024-09-12T12:24:44.616311+0000","last_fullsized":"2024-09-12T12:24:44.616311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:31.746764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631858+0000","last_change":"2024-09-12T12:24:42.631858+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631858+0000","last_undegraded":"2024-09-12T12:24:42.631858+0000","last_fullsized":"2024-09-12T12:24:42.631858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616160+0000","last_change":"2024-09-12T12:24:43.656461+0000","last_active":"2024-09-12T12:24:44.616160+0000","last_peered":"2024-09-12T12:24:44.616160+0000","last_clean":"2024-09-12T12:24:44.616160+0000","last_became_active":"2024-09-12T12:24:43.655523+0000","last_became_peered":"2024-09-12T12:24:43.655523+0000","last_unstale":"2024-09-12T12:24:44.616160+0000","last_undegraded":"2024-09-12T12:24:44.616160+0000","last_fullsized":"2024-09-12T12:24:44.616160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:45.830898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673541+0000","last_change":"2024-09-12T12:24:43.665791+0000","last_active":"2024-09-12T12:24:43.673541+0000","last_peered":"2024-09-12T12:24:43.673541+0000","last_clean":"2024-09-12T12:24:43.673541+0000","last_became_active":"2024-09-12T12:24:43.663819+0000","last_became_peered":"2024-09-12T12:24:43.663819+0000","last_unstale":"2024-09-12T12:24:43.673541+0000","last_undegraded":"2024-09-12T12:24:43.673541+0000","last_fullsized":"2024-09-12T12:24:43.673541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:03:34.993627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703214+0000","last_change":"2024-09-12T12:24:43.651132+0000","last_active":"2024-09-12T12:24:45.703214+0000","last_peered":"2024-09-12T12:24:45.703214+0000","last_clean":"2024-09-12T12:24:45.703214+0000","last_became_active":"2024-09-12T12:24:43.650439+0000","last_became_peered":"2024-09-12T12:24:43.650439+0000","last_unstale":"2024-09-12T12:24:45.703214+0000","last_undegraded":"2024-09-12T12:24:45.703214+0000","last_fullsized":"2024-09-12T12:24:45.703214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:44.191710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621336+0000","last_change":"2024-09-12T12:24:43.650634+0000","last_active":"2024-09-12T12:24:44.621336+0000","last_peered":"2024-09-12T12:24:44.621336+0000","last_clean":"2024-09-12T12:24:44.621336+0000","last_became_active":"2024-09-12T12:24:43.649447+0000","last_became_peered":"2024-09-12T12:24:43.649447+0000","last_unstale":"2024-09-12T12:24:44.621336+0000","last_undegraded":"2024-09-12T12:24:44.621336+0000","last_fullsized":"2024-09-12T12:24:44.621336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:56.893292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621208+0000","last_change":"2024-09-12T12:24:43.651354+0000","last_active":"2024-09-12T12:24:44.621208+0000","last_peered":"2024-09-12T12:24:44.621208+0000","last_clean":"2024-09-12T12:24:44.621208+0000","last_became_active":"2024-09-12T12:24:43.650570+0000","last_became_peered":"2024-09-12T12:24:43.650570+0000","last_unstale":"2024-09-12T12:24:44.621208+0000","last_undegraded":"2024-09-12T12:24:44.621208+0000","last_fullsized":"2024-09-12T12:24:44.621208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622450+0000","last_change":"2024-09-12T12:24:42.622450+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622450+0000","last_undegraded":"2024-09-12T12:24:42.622450+0000","last_fullsized":"2024-09-12T12:24:42.622450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904799+0000","last_change":"2024-09-12T12:24:43.652939+0000","last_active":"2024-09-12T12:24:44.904799+0000","last_peered":"2024-09-12T12:24:44.904799+0000","last_clean":"2024-09-12T12:24:44.904799+0000","last_became_active":"2024-09-12T12:24:43.651482+0000","last_became_peered":"2024-09-12T12:24:43.651482+0000","last_unstale":"2024-09-12T12:24:44.904799+0000","last_undegraded":"2024-09-12T12:24:44.904799+0000","last_fullsized":"2024-09-12T12:24:44.904799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702479+0000","last_change":"2024-09-12T12:24:43.637408+0000","last_active":"2024-09-12T12:24:45.702479+0000","last_peered":"2024-09-12T12:24:45.702479+0000","last_clean":"2024-09-12T12:24:45.702479+0000","last_became_active":"2024-09-12T12:24:43.636447+0000","last_became_peered":"2024-09-12T12:24:43.636447+0000","last_unstale":"2024-09-12T12:24:45.702479+0000","last_undegraded":"2024-09-12T12:24:45.702479+0000","last_fullsized":"2024-09-12T12:24:45.702479+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621413+0000","last_change":"2024-09-12T12:24:43.649963+0000","last_active":"2024-09-12T12:24:44.621413+0000","last_peered":"2024-09-12T12:24:44.621413+0000","last_clean":"2024-09-12T12:24:44.621413+0000","last_became_active":"2024-09-12T12:24:43.649206+0000","last_became_peered":"2024-09-12T12:24:43.649206+0000","last_unstale":"2024-09-12T12:24:44.621413+0000","last_undegraded":"2024-09-12T12:24:44.621413+0000","last_fullsized":"2024-09-12T12:24:44.621413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:36:44.276769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904389+0000","last_change":"2024-09-12T12:24:43.661406+0000","last_active":"2024-09-12T12:24:44.904389+0000","last_peered":"2024-09-12T12:24:44.904389+0000","last_clean":"2024-09-12T12:24:44.904389+0000","last_became_active":"2024-09-12T12:24:43.660623+0000","last_became_peered":"2024-09-12T12:24:43.660623+0000","last_unstale":"2024-09-12T12:24:44.904389+0000","last_undegraded":"2024-09-12T12:24:44.904389+0000","last_fullsized":"2024-09-12T12:24:44.904389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:16.413303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.671577+0000","last_change":"2024-09-12T12:24:43.653070+0000","last_active":"2024-09-12T12:24:43.671577+0000","last_peered":"2024-09-12T12:24:43.671577+0000","last_clean":"2024-09-12T12:24:43.671577+0000","last_became_active":"2024-09-12T12:24:43.651753+0000","last_became_peered":"2024-09-12T12:24:43.651753+0000","last_unstale":"2024-09-12T12:24:43.671577+0000","last_undegraded":"2024-09-12T12:24:43.671577+0000","last_fullsized":"2024-09-12T12:24:43.671577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:55.597998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683623+0000","last_change":"2024-09-12T12:24:43.622364+0000","last_active":"2024-09-12T12:24:44.683623+0000","last_peered":"2024-09-12T12:24:44.683623+0000","last_clean":"2024-09-12T12:24:44.683623+0000","last_became_active":"2024-09-12T12:24:43.620763+0000","last_became_peered":"2024-09-12T12:24:43.620763+0000","last_unstale":"2024-09-12T12:24:44.683623+0000","last_undegraded":"2024-09-12T12:24:44.683623+0000","last_fullsized":"2024-09-12T12:24:44.683623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:42.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673880+0000","last_change":"2024-09-12T12:24:43.670224+0000","last_active":"2024-09-12T12:24:43.673880+0000","last_peered":"2024-09-12T12:24:43.673880+0000","last_clean":"2024-09-12T12:24:43.673880+0000","last_became_active":"2024-09-12T12:24:43.669622+0000","last_became_peered":"2024-09-12T12:24:43.669622+0000","last_unstale":"2024-09-12T12:24:43.673880+0000","last_undegraded":"2024-09-12T12:24:43.673880+0000","last_fullsized":"2024-09-12T12:24:43.673880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:52:53.453156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621285+0000","last_change":"2024-09-12T12:24:43.658919+0000","last_active":"2024-09-12T12:24:44.621285+0000","last_peered":"2024-09-12T12:24:44.621285+0000","last_clean":"2024-09-12T12:24:44.621285+0000","last_became_active":"2024-09-12T12:24:43.658279+0000","last_became_peered":"2024-09-12T12:24:43.658279+0000","last_unstale":"2024-09-12T12:24:44.621285+0000","last_undegraded":"2024-09-12T12:24:44.621285+0000","last_fullsized":"2024-09-12T12:24:44.621285+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:40.554937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626138+0000","last_change":"2024-09-12T12:24:42.626138+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626138+0000","last_undegraded":"2024-09-12T12:24:42.626138+0000","last_fullsized":"2024-09-12T12:24:42.626138+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673828+0000","last_change":"2024-09-12T12:24:43.666474+0000","last_active":"2024-09-12T12:24:43.673828+0000","last_peered":"2024-09-12T12:24:43.673828+0000","last_clean":"2024-09-12T12:24:43.673828+0000","last_became_active":"2024-09-12T12:24:43.665798+0000","last_became_peered":"2024-09-12T12:24:43.665798+0000","last_unstale":"2024-09-12T12:24:43.673828+0000","last_undegraded":"2024-09-12T12:24:43.673828+0000","last_fullsized":"2024-09-12T12:24:43.673828+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:28.095510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683840+0000","last_change":"2024-09-12T12:24:43.647026+0000","last_active":"2024-09-12T12:24:44.683840+0000","last_peered":"2024-09-12T12:24:44.683840+0000","last_clean":"2024-09-12T12:24:44.683840+0000","last_became_active":"2024-09-12T12:24:43.646064+0000","last_became_peered":"2024-09-12T12:24:43.646064+0000","last_unstale":"2024-09-12T12:24:44.683840+0000","last_undegraded":"2024-09-12T12:24:44.683840+0000","last_fullsized":"2024-09-12T12:24:44.683840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:38.167692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904536+0000","last_change":"2024-09-12T12:24:43.666544+0000","last_active":"2024-09-12T12:24:44.904536+0000","last_peered":"2024-09-12T12:24:44.904536+0000","last_clean":"2024-09-12T12:24:44.904536+0000","last_became_active":"2024-09-12T12:24:43.665661+0000","last_became_peered":"2024-09-12T12:24:43.665661+0000","last_unstale":"2024-09-12T12:24:44.904536+0000","last_undegraded":"2024-09-12T12:24:44.904536+0000","last_fullsized":"2024-09-12T12:24:44.904536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:22.604865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684069+0000","last_change":"2024-09-12T12:24:43.645911+0000","last_active":"2024-09-12T12:24:44.684069+0000","last_peered":"2024-09-12T12:24:44.684069+0000","last_clean":"2024-09-12T12:24:44.684069+0000","last_became_active":"2024-09-12T12:24:43.645127+0000","last_became_peered":"2024-09-12T12:24:43.645127+0000","last_unstale":"2024-09-12T12:24:44.684069+0000","last_undegraded":"2024-09-12T12:24:44.684069+0000","last_fullsized":"2024-09-12T12:24:44.684069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:01.666655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.631303+0000","last_change":"2024-09-12T12:24:42.631303+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.631303+0000","last_undegraded":"2024-09-12T12:24:42.631303+0000","last_fullsized":"2024-09-12T12:24:42.631303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.622827+0000","last_change":"2024-09-12T12:24:42.622827+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.622827+0000","last_undegraded":"2024-09-12T12:24:42.622827+0000","last_fullsized":"2024-09-12T12:24:42.622827+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621737+0000","last_change":"2024-09-12T12:24:43.646502+0000","last_active":"2024-09-12T12:24:44.621737+0000","last_peered":"2024-09-12T12:24:44.621737+0000","last_clean":"2024-09-12T12:24:44.621737+0000","last_became_active":"2024-09-12T12:24:43.645053+0000","last_became_peered":"2024-09-12T12:24:43.645053+0000","last_unstale":"2024-09-12T12:24:44.621737+0000","last_undegraded":"2024-09-12T12:24:44.621737+0000","last_fullsized":"2024-09-12T12:24:44.621737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:51:43.421188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703778+0000","last_change":"2024-09-12T12:24:43.645720+0000","last_active":"2024-09-12T12:24:45.703778+0000","last_peered":"2024-09-12T12:24:45.703778+0000","last_clean":"2024-09-12T12:24:45.703778+0000","last_became_active":"2024-09-12T12:24:43.644960+0000","last_became_peered":"2024-09-12T12:24:43.644960+0000","last_unstale":"2024-09-12T12:24:45.703778+0000","last_undegraded":"2024-09-12T12:24:45.703778+0000","last_fullsized":"2024-09-12T12:24:45.703778+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:07:07.741088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627753+0000","last_change":"2024-09-12T12:24:42.627753+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627753+0000","last_undegraded":"2024-09-12T12:24:42.627753+0000","last_fullsized":"2024-09-12T12:24:42.627753+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622169+0000","last_change":"2024-09-12T12:24:43.661393+0000","last_active":"2024-09-12T12:24:44.622169+0000","last_peered":"2024-09-12T12:24:44.622169+0000","last_clean":"2024-09-12T12:24:44.622169+0000","last_became_active":"2024-09-12T12:24:43.660705+0000","last_became_peered":"2024-09-12T12:24:43.660705+0000","last_unstale":"2024-09-12T12:24:44.622169+0000","last_undegraded":"2024-09-12T12:24:44.622169+0000","last_fullsized":"2024-09-12T12:24:44.622169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:34:20.041489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703551+0000","last_change":"2024-09-12T12:24:43.659047+0000","last_active":"2024-09-12T12:24:45.703551+0000","last_peered":"2024-09-12T12:24:45.703551+0000","last_clean":"2024-09-12T12:24:45.703551+0000","last_became_active":"2024-09-12T12:24:43.658354+0000","last_became_peered":"2024-09-12T12:24:43.658354+0000","last_unstale":"2024-09-12T12:24:45.703551+0000","last_undegraded":"2024-09-12T12:24:45.703551+0000","last_fullsized":"2024-09-12T12:24:45.703551+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:22.035037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627728+0000","last_change":"2024-09-12T12:24:42.627728+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627728+0000","last_undegraded":"2024-09-12T12:24:42.627728+0000","last_fullsized":"2024-09-12T12:24:42.627728+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684067+0000","last_change":"2024-09-12T12:24:43.645157+0000","last_active":"2024-09-12T12:24:44.684067+0000","last_peered":"2024-09-12T12:24:44.684067+0000","last_clean":"2024-09-12T12:24:44.684067+0000","last_became_active":"2024-09-12T12:24:43.644341+0000","last_became_peered":"2024-09-12T12:24:43.644341+0000","last_unstale":"2024-09-12T12:24:44.684067+0000","last_undegraded":"2024-09-12T12:24:44.684067+0000","last_fullsized":"2024-09-12T12:24:44.684067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:39.841147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673604+0000","last_change":"2024-09-12T12:24:43.664785+0000","last_active":"2024-09-12T12:24:43.673604+0000","last_peered":"2024-09-12T12:24:43.673604+0000","last_clean":"2024-09-12T12:24:43.673604+0000","last_became_active":"2024-09-12T12:24:43.663968+0000","last_became_peered":"2024-09-12T12:24:43.663968+0000","last_unstale":"2024-09-12T12:24:43.673604+0000","last_undegraded":"2024-09-12T12:24:43.673604+0000","last_fullsized":"2024-09-12T12:24:43.673604+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:01.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615264+0000","last_change":"2024-09-12T12:24:43.661779+0000","last_active":"2024-09-12T12:24:44.615264+0000","last_peered":"2024-09-12T12:24:44.615264+0000","last_clean":"2024-09-12T12:24:44.615264+0000","last_became_active":"2024-09-12T12:24:43.660239+0000","last_became_peered":"2024-09-12T12:24:43.660239+0000","last_unstale":"2024-09-12T12:24:44.615264+0000","last_undegraded":"2024-09-12T12:24:44.615264+0000","last_fullsized":"2024-09-12T12:24:44.615264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:55.818309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683905+0000","last_change":"2024-09-12T12:24:43.648961+0000","last_active":"2024-09-12T12:24:44.683905+0000","last_peered":"2024-09-12T12:24:44.683905+0000","last_clean":"2024-09-12T12:24:44.683905+0000","last_became_active":"2024-09-12T12:24:43.648231+0000","last_became_peered":"2024-09-12T12:24:43.648231+0000","last_unstale":"2024-09-12T12:24:44.683905+0000","last_undegraded":"2024-09-12T12:24:44.683905+0000","last_fullsized":"2024-09-12T12:24:44.683905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:04.030268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646506+0000","last_change":"2024-09-12T12:24:42.641566+0000","last_active":"2024-09-12T12:24:42.646506+0000","last_peered":"2024-09-12T12:24:42.646506+0000","last_clean":"2024-09-12T12:24:42.646506+0000","last_became_active":"2024-09-12T12:24:42.641294+0000","last_became_peered":"2024-09-12T12:24:42.641294+0000","last_unstale":"2024-09-12T12:24:42.646506+0000","last_undegraded":"2024-09-12T12:24:42.646506+0000","last_fullsized":"2024-09-12T12:24:42.646506+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:28.668303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673591+0000","last_change":"2024-09-12T12:24:43.665125+0000","last_active":"2024-09-12T12:24:43.673591+0000","last_peered":"2024-09-12T12:24:43.673591+0000","last_clean":"2024-09-12T12:24:43.673591+0000","last_became_active":"2024-09-12T12:24:43.664317+0000","last_became_peered":"2024-09-12T12:24:43.664317+0000","last_unstale":"2024-09-12T12:24:43.673591+0000","last_undegraded":"2024-09-12T12:24:43.673591+0000","last_fullsized":"2024-09-12T12:24:43.673591+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:46.448481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621763+0000","last_change":"2024-09-12T12:24:42.653011+0000","last_active":"2024-09-12T12:24:44.621763+0000","last_peered":"2024-09-12T12:24:44.621763+0000","last_clean":"2024-09-12T12:24:44.621763+0000","last_became_active":"2024-09-12T12:24:42.652145+0000","last_became_peered":"2024-09-12T12:24:42.652145+0000","last_unstale":"2024-09-12T12:24:44.621763+0000","last_undegraded":"2024-09-12T12:24:44.621763+0000","last_fullsized":"2024-09-12T12:24:44.621763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:16.133515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615814+0000","last_change":"2024-09-12T12:24:43.651221+0000","last_active":"2024-09-12T12:24:44.615814+0000","last_peered":"2024-09-12T12:24:44.615814+0000","last_clean":"2024-09-12T12:24:44.615814+0000","last_became_active":"2024-09-12T12:24:43.650519+0000","last_became_peered":"2024-09-12T12:24:43.650519+0000","last_unstale":"2024-09-12T12:24:44.615814+0000","last_undegraded":"2024-09-12T12:24:44.615814+0000","last_fullsized":"2024-09-12T12:24:44.615814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:03.987247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.649894+0000","last_change":"2024-09-12T12:24:42.646442+0000","last_active":"2024-09-12T12:24:42.649894+0000","last_peered":"2024-09-12T12:24:42.649894+0000","last_clean":"2024-09-12T12:24:42.649894+0000","last_became_active":"2024-09-12T12:24:42.643894+0000","last_became_peered":"2024-09-12T12:24:42.643894+0000","last_unstale":"2024-09-12T12:24:42.649894+0000","last_undegraded":"2024-09-12T12:24:42.649894+0000","last_fullsized":"2024-09-12T12:24:42.649894+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:23:30.708412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.905160+0000","last_change":"2024-09-12T12:24:43.646657+0000","last_active":"2024-09-12T12:24:44.905160+0000","last_peered":"2024-09-12T12:24:44.905160+0000","last_clean":"2024-09-12T12:24:44.905160+0000","last_became_active":"2024-09-12T12:24:43.645851+0000","last_became_peered":"2024-09-12T12:24:43.645851+0000","last_unstale":"2024-09-12T12:24:44.905160+0000","last_undegraded":"2024-09-12T12:24:44.905160+0000","last_fullsized":"2024-09-12T12:24:44.905160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646215+0000","last_change":"2024-09-12T12:24:42.637641+0000","last_active":"2024-09-12T12:24:42.646215+0000","last_peered":"2024-09-12T12:24:42.646215+0000","last_clean":"2024-09-12T12:24:42.646215+0000","last_became_active":"2024-09-12T12:24:42.637338+0000","last_became_peered":"2024-09-12T12:24:42.637338+0000","last_unstale":"2024-09-12T12:24:42.646215+0000","last_undegraded":"2024-09-12T12:24:42.646215+0000","last_fullsized":"2024-09-12T12:24:42.646215+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:15.407617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703339+0000","last_change":"2024-09-12T12:24:43.644547+0000","last_active":"2024-09-12T12:24:45.703339+0000","last_peered":"2024-09-12T12:24:45.703339+0000","last_clean":"2024-09-12T12:24:45.703339+0000","last_became_active":"2024-09-12T12:24:43.643836+0000","last_became_peered":"2024-09-12T12:24:43.643836+0000","last_unstale":"2024-09-12T12:24:45.703339+0000","last_undegraded":"2024-09-12T12:24:45.703339+0000","last_fullsized":"2024-09-12T12:24:45.703339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:31.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616266+0000","last_change":"2024-09-12T12:24:42.634480+0000","last_active":"2024-09-12T12:24:44.616266+0000","last_peered":"2024-09-12T12:24:44.616266+0000","last_clean":"2024-09-12T12:24:44.616266+0000","last_became_active":"2024-09-12T12:24:42.633367+0000","last_became_peered":"2024-09-12T12:24:42.633367+0000","last_unstale":"2024-09-12T12:24:44.616266+0000","last_undegraded":"2024-09-12T12:24:44.616266+0000","last_fullsized":"2024-09-12T12:24:44.616266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:19.812219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621856+0000","last_change":"2024-09-12T12:24:43.641831+0000","last_active":"2024-09-12T12:24:44.621856+0000","last_peered":"2024-09-12T12:24:44.621856+0000","last_clean":"2024-09-12T12:24:44.621856+0000","last_became_active":"2024-09-12T12:24:43.641063+0000","last_became_peered":"2024-09-12T12:24:43.641063+0000","last_unstale":"2024-09-12T12:24:44.621856+0000","last_undegraded":"2024-09-12T12:24:44.621856+0000","last_fullsized":"2024-09-12T12:24:44.621856+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:46.151100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646203+0000","last_change":"2024-09-12T12:24:42.637461+0000","last_active":"2024-09-12T12:24:42.646203+0000","last_peered":"2024-09-12T12:24:42.646203+0000","last_clean":"2024-09-12T12:24:42.646203+0000","last_became_active":"2024-09-12T12:24:42.637031+0000","last_became_peered":"2024-09-12T12:24:42.637031+0000","last_unstale":"2024-09-12T12:24:42.646203+0000","last_undegraded":"2024-09-12T12:24:42.646203+0000","last_fullsized":"2024-09-12T12:24:42.646203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:34:47.172517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.673878+0000","last_change":"2024-09-12T12:24:43.670586+0000","last_active":"2024-09-12T12:24:43.673878+0000","last_peered":"2024-09-12T12:24:43.673878+0000","last_clean":"2024-09-12T12:24:43.673878+0000","last_became_active":"2024-09-12T12:24:43.669772+0000","last_became_peered":"2024-09-12T12:24:43.669772+0000","last_unstale":"2024-09-12T12:24:43.673878+0000","last_undegraded":"2024-09-12T12:24:43.673878+0000","last_fullsized":"2024-09-12T12:24:43.673878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:43.570019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702529+0000","last_change":"2024-09-12T12:24:42.648919+0000","last_active":"2024-09-12T12:24:45.702529+0000","last_peered":"2024-09-12T12:24:45.702529+0000","last_clean":"2024-09-12T12:24:45.702529+0000","last_became_active":"2024-09-12T12:24:42.648039+0000","last_became_peered":"2024-09-12T12:24:42.648039+0000","last_unstale":"2024-09-12T12:24:45.702529+0000","last_undegraded":"2024-09-12T12:24:45.702529+0000","last_fullsized":"2024-09-12T12:24:45.702529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:51.735892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903858+0000","last_change":"2024-09-12T12:24:43.650075+0000","last_active":"2024-09-12T12:24:44.903858+0000","last_peered":"2024-09-12T12:24:44.903858+0000","last_clean":"2024-09-12T12:24:44.903858+0000","last_became_active":"2024-09-12T12:24:43.649215+0000","last_became_peered":"2024-09-12T12:24:43.649215+0000","last_unstale":"2024-09-12T12:24:44.903858+0000","last_undegraded":"2024-09-12T12:24:44.903858+0000","last_fullsized":"2024-09-12T12:24:44.903858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:30.006601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615299+0000","last_change":"2024-09-12T12:24:42.625151+0000","last_active":"2024-09-12T12:24:44.615299+0000","last_peered":"2024-09-12T12:24:44.615299+0000","last_clean":"2024-09-12T12:24:44.615299+0000","last_became_active":"2024-09-12T12:24:42.624745+0000","last_became_peered":"2024-09-12T12:24:42.624745+0000","last_unstale":"2024-09-12T12:24:44.615299+0000","last_undegraded":"2024-09-12T12:24:44.615299+0000","last_fullsized":"2024-09-12T12:24:44.615299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:21.399432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.627022+0000","last_change":"2024-09-12T12:24:42.627022+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.627022+0000","last_undegraded":"2024-09-12T12:24:42.627022+0000","last_fullsized":"2024-09-12T12:24:42.627022+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616802+0000","last_change":"2024-09-12T12:24:42.637921+0000","last_active":"2024-09-12T12:24:44.616802+0000","last_peered":"2024-09-12T12:24:44.616802+0000","last_clean":"2024-09-12T12:24:44.616802+0000","last_became_active":"2024-09-12T12:24:42.637229+0000","last_became_peered":"2024-09-12T12:24:42.637229+0000","last_unstale":"2024-09-12T12:24:44.616802+0000","last_undegraded":"2024-09-12T12:24:44.616802+0000","last_fullsized":"2024-09-12T12:24:44.616802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:59.256685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904941+0000","last_change":"2024-09-12T12:24:43.658926+0000","last_active":"2024-09-12T12:24:44.904941+0000","last_peered":"2024-09-12T12:24:44.904941+0000","last_clean":"2024-09-12T12:24:44.904941+0000","last_became_active":"2024-09-12T12:24:43.658247+0000","last_became_peered":"2024-09-12T12:24:43.658247+0000","last_unstale":"2024-09-12T12:24:44.904941+0000","last_undegraded":"2024-09-12T12:24:44.904941+0000","last_fullsized":"2024-09-12T12:24:44.904941+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.635866+0000","last_change":"2024-09-12T12:24:42.641455+0000","last_active":"2024-09-12T12:24:43.635866+0000","last_peered":"2024-09-12T12:24:43.635866+0000","last_clean":"2024-09-12T12:24:43.635866+0000","last_became_active":"2024-09-12T12:24:42.641145+0000","last_became_peered":"2024-09-12T12:24:42.641145+0000","last_unstale":"2024-09-12T12:24:43.635866+0000","last_undegraded":"2024-09-12T12:24:43.635866+0000","last_fullsized":"2024-09-12T12:24:43.635866+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:58.912932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703939+0000","last_change":"2024-09-12T12:24:43.647661+0000","last_active":"2024-09-12T12:24:45.703939+0000","last_peered":"2024-09-12T12:24:45.703939+0000","last_clean":"2024-09-12T12:24:45.703939+0000","last_became_active":"2024-09-12T12:24:43.646874+0000","last_became_peered":"2024-09-12T12:24:43.646874+0000","last_unstale":"2024-09-12T12:24:45.703939+0000","last_undegraded":"2024-09-12T12:24:45.703939+0000","last_fullsized":"2024-09-12T12:24:45.703939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622442+0000","last_change":"2024-09-12T12:24:42.652504+0000","last_active":"2024-09-12T12:24:44.622442+0000","last_peered":"2024-09-12T12:24:44.622442+0000","last_clean":"2024-09-12T12:24:44.622442+0000","last_became_active":"2024-09-12T12:24:42.651709+0000","last_became_peered":"2024-09-12T12:24:42.651709+0000","last_unstale":"2024-09-12T12:24:44.622442+0000","last_undegraded":"2024-09-12T12:24:44.622442+0000","last_fullsized":"2024-09-12T12:24:44.622442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:44.399415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622185+0000","last_change":"2024-09-12T12:24:43.659018+0000","last_active":"2024-09-12T12:24:44.622185+0000","last_peered":"2024-09-12T12:24:44.622185+0000","last_clean":"2024-09-12T12:24:44.622185+0000","last_became_active":"2024-09-12T12:24:43.658369+0000","last_became_peered":"2024-09-12T12:24:43.658369+0000","last_unstale":"2024-09-12T12:24:44.622185+0000","last_undegraded":"2024-09-12T12:24:44.622185+0000","last_fullsized":"2024-09-12T12:24:44.622185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:26.274835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"92'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904658+0000","last_change":"2024-09-12T12:24:42.643454+0000","last_active":"2024-09-12T12:24:44.904658+0000","last_peered":"2024-09-12T12:24:44.904658+0000","last_clean":"2024-09-12T12:24:44.904658+0000","last_became_active":"2024-09-12T12:24:42.642649+0000","last_became_peered":"2024-09-12T12:24:42.642649+0000","last_unstale":"2024-09-12T12:24:44.904658+0000","last_undegraded":"2024-09-12T12:24:44.904658+0000","last_fullsized":"2024-09-12T12:24:44.904658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:30.300204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.666620+0000","last_change":"2024-09-12T12:24:43.647560+0000","last_active":"2024-09-12T12:24:43.666620+0000","last_peered":"2024-09-12T12:24:43.666620+0000","last_clean":"2024-09-12T12:24:43.666620+0000","last_became_active":"2024-09-12T12:24:43.646824+0000","last_became_peered":"2024-09-12T12:24:43.646824+0000","last_unstale":"2024-09-12T12:24:43.666620+0000","last_undegraded":"2024-09-12T12:24:43.666620+0000","last_fullsized":"2024-09-12T12:24:43.666620+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:35:20.924656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684213+0000","last_change":"2024-09-12T12:24:42.637238+0000","last_active":"2024-09-12T12:24:44.684213+0000","last_peered":"2024-09-12T12:24:44.684213+0000","last_clean":"2024-09-12T12:24:44.684213+0000","last_became_active":"2024-09-12T12:24:42.636898+0000","last_became_peered":"2024-09-12T12:24:42.636898+0000","last_unstale":"2024-09-12T12:24:44.684213+0000","last_undegraded":"2024-09-12T12:24:44.684213+0000","last_fullsized":"2024-09-12T12:24:44.684213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:06.749490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616014+0000","last_change":"2024-09-12T12:24:43.657062+0000","last_active":"2024-09-12T12:24:44.616014+0000","last_peered":"2024-09-12T12:24:44.616014+0000","last_clean":"2024-09-12T12:24:44.616014+0000","last_became_active":"2024-09-12T12:24:43.654265+0000","last_became_peered":"2024-09-12T12:24:43.654265+0000","last_unstale":"2024-09-12T12:24:44.616014+0000","last_undegraded":"2024-09-12T12:24:44.616014+0000","last_fullsized":"2024-09-12T12:24:44.616014+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:06.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.685093+0000","last_change":"2024-09-12T12:24:42.627572+0000","last_active":"2024-09-12T12:24:44.685093+0000","last_peered":"2024-09-12T12:24:44.685093+0000","last_clean":"2024-09-12T12:24:44.685093+0000","last_became_active":"2024-09-12T12:24:42.627257+0000","last_became_peered":"2024-09-12T12:24:42.627257+0000","last_unstale":"2024-09-12T12:24:44.685093+0000","last_undegraded":"2024-09-12T12:24:44.685093+0000","last_fullsized":"2024-09-12T12:24:44.685093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:04.397782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903797+0000","last_change":"2024-09-12T12:24:43.658133+0000","last_active":"2024-09-12T12:24:44.903797+0000","last_peered":"2024-09-12T12:24:44.903797+0000","last_clean":"2024-09-12T12:24:44.903797+0000","last_became_active":"2024-09-12T12:24:43.657012+0000","last_became_peered":"2024-09-12T12:24:43.657012+0000","last_unstale":"2024-09-12T12:24:44.903797+0000","last_undegraded":"2024-09-12T12:24:44.903797+0000","last_fullsized":"2024-09-12T12:24:44.903797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:17.137749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903854+0000","last_change":"2024-09-12T12:24:42.643511+0000","last_active":"2024-09-12T12:24:44.903854+0000","last_peered":"2024-09-12T12:24:44.903854+0000","last_clean":"2024-09-12T12:24:44.903854+0000","last_became_active":"2024-09-12T12:24:42.642768+0000","last_became_peered":"2024-09-12T12:24:42.642768+0000","last_unstale":"2024-09-12T12:24:44.903854+0000","last_undegraded":"2024-09-12T12:24:44.903854+0000","last_fullsized":"2024-09-12T12:24:44.903854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:23.588413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683454+0000","last_change":"2024-09-12T12:24:43.636996+0000","last_active":"2024-09-12T12:24:44.683454+0000","last_peered":"2024-09-12T12:24:44.683454+0000","last_clean":"2024-09-12T12:24:44.683454+0000","last_became_active":"2024-09-12T12:24:43.636232+0000","last_became_peered":"2024-09-12T12:24:43.636232+0000","last_unstale":"2024-09-12T12:24:44.683454+0000","last_undegraded":"2024-09-12T12:24:44.683454+0000","last_fullsized":"2024-09-12T12:24:44.683454+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:56:16.348629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615379+0000","last_change":"2024-09-12T12:24:42.642474+0000","last_active":"2024-09-12T12:24:44.615379+0000","last_peered":"2024-09-12T12:24:44.615379+0000","last_clean":"2024-09-12T12:24:44.615379+0000","last_became_active":"2024-09-12T12:24:42.642172+0000","last_became_peered":"2024-09-12T12:24:42.642172+0000","last_unstale":"2024-09-12T12:24:44.615379+0000","last_undegraded":"2024-09-12T12:24:44.615379+0000","last_fullsized":"2024-09-12T12:24:44.615379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:39.580983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703705+0000","last_change":"2024-09-12T12:24:43.639166+0000","last_active":"2024-09-12T12:24:45.703705+0000","last_peered":"2024-09-12T12:24:45.703705+0000","last_clean":"2024-09-12T12:24:45.703705+0000","last_became_active":"2024-09-12T12:24:43.638391+0000","last_became_peered":"2024-09-12T12:24:43.638391+0000","last_unstale":"2024-09-12T12:24:45.703705+0000","last_undegraded":"2024-09-12T12:24:45.703705+0000","last_fullsized":"2024-09-12T12:24:45.703705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:30:37.318619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904436+0000","last_change":"2024-09-12T12:24:42.640318+0000","last_active":"2024-09-12T12:24:44.904436+0000","last_peered":"2024-09-12T12:24:44.904436+0000","last_clean":"2024-09-12T12:24:44.904436+0000","last_became_active":"2024-09-12T12:24:42.639593+0000","last_became_peered":"2024-09-12T12:24:42.639593+0000","last_unstale":"2024-09-12T12:24:44.904436+0000","last_undegraded":"2024-09-12T12:24:44.904436+0000","last_fullsized":"2024-09-12T12:24:44.904436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:18.210648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702964+0000","last_change":"2024-09-12T12:24:43.653241+0000","last_active":"2024-09-12T12:24:45.702964+0000","last_peered":"2024-09-12T12:24:45.702964+0000","last_clean":"2024-09-12T12:24:45.702964+0000","last_became_active":"2024-09-12T12:24:43.652511+0000","last_became_peered":"2024-09-12T12:24:43.652511+0000","last_unstale":"2024-09-12T12:24:45.702964+0000","last_undegraded":"2024-09-12T12:24:45.702964+0000","last_fullsized":"2024-09-12T12:24:45.702964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:45.460852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702785+0000","last_change":"2024-09-12T12:24:42.637205+0000","last_active":"2024-09-12T12:24:45.702785+0000","last_peered":"2024-09-12T12:24:45.702785+0000","last_clean":"2024-09-12T12:24:45.702785+0000","last_became_active":"2024-09-12T12:24:42.636321+0000","last_became_peered":"2024-09-12T12:24:42.636321+0000","last_unstale":"2024-09-12T12:24:45.702785+0000","last_undegraded":"2024-09-12T12:24:45.702785+0000","last_fullsized":"2024-09-12T12:24:45.702785+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622470+0000","last_change":"2024-09-12T12:24:43.649792+0000","last_active":"2024-09-12T12:24:44.622470+0000","last_peered":"2024-09-12T12:24:44.622470+0000","last_clean":"2024-09-12T12:24:44.622470+0000","last_became_active":"2024-09-12T12:24:43.648913+0000","last_became_peered":"2024-09-12T12:24:43.648913+0000","last_unstale":"2024-09-12T12:24:44.622470+0000","last_undegraded":"2024-09-12T12:24:44.622470+0000","last_fullsized":"2024-09-12T12:24:44.622470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:51:34.860957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646718+0000","last_change":"2024-09-12T12:24:42.642838+0000","last_active":"2024-09-12T12:24:42.646718+0000","last_peered":"2024-09-12T12:24:42.646718+0000","last_clean":"2024-09-12T12:24:42.646718+0000","last_became_active":"2024-09-12T12:24:42.642310+0000","last_became_peered":"2024-09-12T12:24:42.642310+0000","last_unstale":"2024-09-12T12:24:42.646718+0000","last_undegraded":"2024-09-12T12:24:42.646718+0000","last_fullsized":"2024-09-12T12:24:42.646718+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:52:42.944961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684507+0000","last_change":"2024-09-12T12:24:43.652450+0000","last_active":"2024-09-12T12:24:44.684507+0000","last_peered":"2024-09-12T12:24:44.684507+0000","last_clean":"2024-09-12T12:24:44.684507+0000","last_became_active":"2024-09-12T12:24:43.651643+0000","last_became_peered":"2024-09-12T12:24:43.651643+0000","last_unstale":"2024-09-12T12:24:44.684507+0000","last_undegraded":"2024-09-12T12:24:44.684507+0000","last_fullsized":"2024-09-12T12:24:44.684507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:20.816453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646394+0000","last_change":"2024-09-12T12:24:42.637602+0000","last_active":"2024-09-12T12:24:42.646394+0000","last_peered":"2024-09-12T12:24:42.646394+0000","last_clean":"2024-09-12T12:24:42.646394+0000","last_became_active":"2024-09-12T12:24:42.637106+0000","last_became_peered":"2024-09-12T12:24:42.637106+0000","last_unstale":"2024-09-12T12:24:42.646394+0000","last_undegraded":"2024-09-12T12:24:42.646394+0000","last_fullsized":"2024-09-12T12:24:42.646394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:39:02.019478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684325+0000","last_change":"2024-09-12T12:24:43.652172+0000","last_active":"2024-09-12T12:24:44.684325+0000","last_peered":"2024-09-12T12:24:44.684325+0000","last_clean":"2024-09-12T12:24:44.684325+0000","last_became_active":"2024-09-12T12:24:43.651461+0000","last_became_peered":"2024-09-12T12:24:43.651461+0000","last_unstale":"2024-09-12T12:24:44.684325+0000","last_undegraded":"2024-09-12T12:24:44.684325+0000","last_fullsized":"2024-09-12T12:24:44.684325+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:24.797814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703619+0000","last_change":"2024-09-12T12:24:42.639853+0000","last_active":"2024-09-12T12:24:45.703619+0000","last_peered":"2024-09-12T12:24:45.703619+0000","last_clean":"2024-09-12T12:24:45.703619+0000","last_became_active":"2024-09-12T12:24:42.639201+0000","last_became_peered":"2024-09-12T12:24:42.639201+0000","last_unstale":"2024-09-12T12:24:45.703619+0000","last_undegraded":"2024-09-12T12:24:45.703619+0000","last_fullsized":"2024-09-12T12:24:45.703619+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:23.071254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683367+0000","last_change":"2024-09-12T12:24:43.656139+0000","last_active":"2024-09-12T12:24:44.683367+0000","last_peered":"2024-09-12T12:24:44.683367+0000","last_clean":"2024-09-12T12:24:44.683367+0000","last_became_active":"2024-09-12T12:24:43.651199+0000","last_became_peered":"2024-09-12T12:24:43.651199+0000","last_unstale":"2024-09-12T12:24:44.683367+0000","last_undegraded":"2024-09-12T12:24:44.683367+0000","last_fullsized":"2024-09-12T12:24:44.683367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:15:43.831842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702539+0000","last_change":"2024-09-12T12:24:42.647349+0000","last_active":"2024-09-12T12:24:45.702539+0000","last_peered":"2024-09-12T12:24:45.702539+0000","last_clean":"2024-09-12T12:24:45.702539+0000","last_became_active":"2024-09-12T12:24:42.646472+0000","last_became_peered":"2024-09-12T12:24:42.646472+0000","last_unstale":"2024-09-12T12:24:45.702539+0000","last_undegraded":"2024-09-12T12:24:45.702539+0000","last_fullsized":"2024-09-12T12:24:45.702539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:05.139726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703187+0000","last_change":"2024-09-12T12:24:43.637022+0000","last_active":"2024-09-12T12:24:45.703187+0000","last_peered":"2024-09-12T12:24:45.703187+0000","last_clean":"2024-09-12T12:24:45.703187+0000","last_became_active":"2024-09-12T12:24:43.636260+0000","last_became_peered":"2024-09-12T12:24:43.636260+0000","last_unstale":"2024-09-12T12:24:45.703187+0000","last_undegraded":"2024-09-12T12:24:45.703187+0000","last_fullsized":"2024-09-12T12:24:45.703187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:15.925952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646621+0000","last_change":"2024-09-12T12:24:42.637645+0000","last_active":"2024-09-12T12:24:42.646621+0000","last_peered":"2024-09-12T12:24:42.646621+0000","last_clean":"2024-09-12T12:24:42.646621+0000","last_became_active":"2024-09-12T12:24:42.637099+0000","last_became_peered":"2024-09-12T12:24:42.637099+0000","last_unstale":"2024-09-12T12:24:42.646621+0000","last_undegraded":"2024-09-12T12:24:42.646621+0000","last_fullsized":"2024-09-12T12:24:42.646621+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:44.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":91,"state":"creating+peering","last_fresh":"2024-09-12T12:24:42.626273+0000","last_change":"2024-09-12T12:24:42.626273+0000","last_active":"2024-09-12T12:24:42.594580+0000","last_peered":"2024-09-12T12:24:42.594580+0000","last_clean":"2024-09-12T12:24:42.594580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-12T12:24:42.626273+0000","last_undegraded":"2024-09-12T12:24:42.626273+0000","last_fullsized":"2024-09-12T12:24:42.626273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703333+0000","last_change":"2024-09-12T12:24:42.639106+0000","last_active":"2024-09-12T12:24:45.703333+0000","last_peered":"2024-09-12T12:24:45.703333+0000","last_clean":"2024-09-12T12:24:45.703333+0000","last_became_active":"2024-09-12T12:24:42.638341+0000","last_became_peered":"2024-09-12T12:24:42.638341+0000","last_unstale":"2024-09-12T12:24:45.703333+0000","last_undegraded":"2024-09-12T12:24:45.703333+0000","last_fullsized":"2024-09-12T12:24:45.703333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:44.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616629+0000","last_change":"2024-09-12T12:24:43.655680+0000","last_active":"2024-09-12T12:24:44.616629+0000","last_peered":"2024-09-12T12:24:44.616629+0000","last_clean":"2024-09-12T12:24:44.616629+0000","last_became_active":"2024-09-12T12:24:43.654815+0000","last_became_peered":"2024-09-12T12:24:43.654815+0000","last_unstale":"2024-09-12T12:24:44.616629+0000","last_undegraded":"2024-09-12T12:24:44.616629+0000","last_fullsized":"2024-09-12T12:24:44.616629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:23.355405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646503+0000","last_change":"2024-09-12T12:24:42.641239+0000","last_active":"2024-09-12T12:24:42.646503+0000","last_peered":"2024-09-12T12:24:42.646503+0000","last_clean":"2024-09-12T12:24:42.646503+0000","last_became_active":"2024-09-12T12:24:42.638459+0000","last_became_peered":"2024-09-12T12:24:42.638459+0000","last_unstale":"2024-09-12T12:24:42.646503+0000","last_undegraded":"2024-09-12T12:24:42.646503+0000","last_fullsized":"2024-09-12T12:24:42.646503+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:28.435282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616100+0000","last_change":"2024-09-12T12:24:43.650775+0000","last_active":"2024-09-12T12:24:44.616100+0000","last_peered":"2024-09-12T12:24:44.616100+0000","last_clean":"2024-09-12T12:24:44.616100+0000","last_became_active":"2024-09-12T12:24:43.650001+0000","last_became_peered":"2024-09-12T12:24:43.650001+0000","last_unstale":"2024-09-12T12:24:44.616100+0000","last_undegraded":"2024-09-12T12:24:44.616100+0000","last_fullsized":"2024-09-12T12:24:44.616100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:24.075487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.652548+0000","last_change":"2024-09-12T12:24:42.647803+0000","last_active":"2024-09-12T12:24:42.652548+0000","last_peered":"2024-09-12T12:24:42.652548+0000","last_clean":"2024-09-12T12:24:42.652548+0000","last_became_active":"2024-09-12T12:24:42.647329+0000","last_became_peered":"2024-09-12T12:24:42.647329+0000","last_unstale":"2024-09-12T12:24:42.652548+0000","last_undegraded":"2024-09-12T12:24:42.652548+0000","last_fullsized":"2024-09-12T12:24:42.652548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:02:03.882345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702515+0000","last_change":"2024-09-12T12:24:43.648046+0000","last_active":"2024-09-12T12:24:45.702515+0000","last_peered":"2024-09-12T12:24:45.702515+0000","last_clean":"2024-09-12T12:24:45.702515+0000","last_became_active":"2024-09-12T12:24:43.647327+0000","last_became_peered":"2024-09-12T12:24:43.647327+0000","last_unstale":"2024-09-12T12:24:45.702515+0000","last_undegraded":"2024-09-12T12:24:45.702515+0000","last_fullsized":"2024-09-12T12:24:45.702515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.704161+0000","last_change":"2024-09-12T12:24:42.644303+0000","last_active":"2024-09-12T12:24:45.704161+0000","last_peered":"2024-09-12T12:24:45.704161+0000","last_clean":"2024-09-12T12:24:45.704161+0000","last_became_active":"2024-09-12T12:24:42.643559+0000","last_became_peered":"2024-09-12T12:24:42.643559+0000","last_unstale":"2024-09-12T12:24:45.704161+0000","last_undegraded":"2024-09-12T12:24:45.704161+0000","last_fullsized":"2024-09-12T12:24:45.704161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:52.789860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684880+0000","last_change":"2024-09-12T12:24:43.637804+0000","last_active":"2024-09-12T12:24:44.684880+0000","last_peered":"2024-09-12T12:24:44.684880+0000","last_clean":"2024-09-12T12:24:44.684880+0000","last_became_active":"2024-09-12T12:24:43.636765+0000","last_became_peered":"2024-09-12T12:24:43.636765+0000","last_unstale":"2024-09-12T12:24:44.684880+0000","last_undegraded":"2024-09-12T12:24:44.684880+0000","last_fullsized":"2024-09-12T12:24:44.684880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:22.403665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-12T12:24:42.646546+0000","last_change":"2024-09-12T12:24:42.642157+0000","last_active":"2024-09-12T12:24:42.646546+0000","last_peered":"2024-09-12T12:24:42.646546+0000","last_clean":"2024-09-12T12:24:42.646546+0000","last_became_active":"2024-09-12T12:24:42.637757+0000","last_became_peered":"2024-09-12T12:24:42.637757+0000","last_unstale":"2024-09-12T12:24:42.646546+0000","last_undegraded":"2024-09-12T12:24:42.646546+0000","last_fullsized":"2024-09-12T12:24:42.646546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:35:53.864687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904303+0000","last_change":"2024-09-12T12:24:43.643084+0000","last_active":"2024-09-12T12:24:44.904303+0000","last_peered":"2024-09-12T12:24:44.904303+0000","last_clean":"2024-09-12T12:24:44.904303+0000","last_became_active":"2024-09-12T12:24:43.642362+0000","last_became_peered":"2024-09-12T12:24:43.642362+0000","last_unstale":"2024-09-12T12:24:44.904303+0000","last_undegraded":"2024-09-12T12:24:44.904303+0000","last_fullsized":"2024-09-12T12:24:44.904303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:45.403648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702332+0000","last_change":"2024-09-12T12:24:42.636916+0000","last_active":"2024-09-12T12:24:45.702332+0000","last_peered":"2024-09-12T12:24:45.702332+0000","last_clean":"2024-09-12T12:24:45.702332+0000","last_became_active":"2024-09-12T12:24:42.634574+0000","last_became_peered":"2024-09-12T12:24:42.634574+0000","last_unstale":"2024-09-12T12:24:45.702332+0000","last_undegraded":"2024-09-12T12:24:45.702332+0000","last_fullsized":"2024-09-12T12:24:45.702332+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:33.717766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683795+0000","last_change":"2024-09-12T12:24:43.644818+0000","last_active":"2024-09-12T12:24:44.683795+0000","last_peered":"2024-09-12T12:24:44.683795+0000","last_clean":"2024-09-12T12:24:44.683795+0000","last_became_active":"2024-09-12T12:24:43.644041+0000","last_became_peered":"2024-09-12T12:24:43.644041+0000","last_unstale":"2024-09-12T12:24:44.683795+0000","last_undegraded":"2024-09-12T12:24:44.683795+0000","last_fullsized":"2024-09-12T12:24:44.683795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:58.101698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-12T12:24:43.641004+0000","last_change":"2024-09-12T12:24:42.625186+0000","last_active":"2024-09-12T12:24:43.641004+0000","last_peered":"2024-09-12T12:24:43.641004+0000","last_clean":"2024-09-12T12:24:43.641004+0000","last_became_active":"2024-09-12T12:24:42.624773+0000","last_became_peered":"2024-09-12T12:24:42.624773+0000","last_unstale":"2024-09-12T12:24:43.641004+0000","last_undegraded":"2024-09-12T12:24:43.641004+0000","last_fullsized":"2024-09-12T12:24:43.641004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:29.271497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702272+0000","last_change":"2024-09-12T12:24:43.647973+0000","last_active":"2024-09-12T12:24:45.702272+0000","last_peered":"2024-09-12T12:24:45.702272+0000","last_clean":"2024-09-12T12:24:45.702272+0000","last_became_active":"2024-09-12T12:24:43.647185+0000","last_became_peered":"2024-09-12T12:24:43.647185+0000","last_unstale":"2024-09-12T12:24:45.702272+0000","last_undegraded":"2024-09-12T12:24:45.702272+0000","last_fullsized":"2024-09-12T12:24:45.702272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683462+0000","last_change":"2024-09-12T12:24:42.638683+0000","last_active":"2024-09-12T12:24:44.683462+0000","last_peered":"2024-09-12T12:24:44.683462+0000","last_clean":"2024-09-12T12:24:44.683462+0000","last_became_active":"2024-09-12T12:24:42.638385+0000","last_became_peered":"2024-09-12T12:24:42.638385+0000","last_unstale":"2024-09-12T12:24:44.683462+0000","last_undegraded":"2024-09-12T12:24:44.683462+0000","last_fullsized":"2024-09-12T12:24:44.683462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.197995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684036+0000","last_change":"2024-09-12T12:24:43.647413+0000","last_active":"2024-09-12T12:24:44.684036+0000","last_peered":"2024-09-12T12:24:44.684036+0000","last_clean":"2024-09-12T12:24:44.684036+0000","last_became_active":"2024-09-12T12:24:43.646570+0000","last_became_peered":"2024-09-12T12:24:43.646570+0000","last_unstale":"2024-09-12T12:24:44.684036+0000","last_undegraded":"2024-09-12T12:24:44.684036+0000","last_fullsized":"2024-09-12T12:24:44.684036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:53:39.696999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621547+0000","last_change":"2024-09-12T12:24:42.649739+0000","last_active":"2024-09-12T12:24:44.621547+0000","last_peered":"2024-09-12T12:24:44.621547+0000","last_clean":"2024-09-12T12:24:44.621547+0000","last_became_active":"2024-09-12T12:24:42.648926+0000","last_became_peered":"2024-09-12T12:24:42.648926+0000","last_unstale":"2024-09-12T12:24:44.621547+0000","last_undegraded":"2024-09-12T12:24:44.621547+0000","last_fullsized":"2024-09-12T12:24:44.621547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:29.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":302,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621655+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:24:44.621655+0000","last_peered":"2024-09-12T12:24:44.621655+0000","last_clean":"2024-09-12T12:24:44.621655+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:24:44.621655+0000","last_undegraded":"2024-09-12T12:24:44.621655+0000","last_fullsized":"2024-09-12T12:24:44.621655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509505,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1122304,"data_stored":476139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1748992,"data_stored":1079694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":489284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":490950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":489248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":489214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542207,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1703936,"data_stored":1060232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:49.238 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-12T12:24:49.235+0000 7f3b7da3e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:24:52.221 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:24:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.355+0000 7fa96f4e0640 1 Processor -- start 2024-09-12T12:24:52.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.355+0000 7fa96f4e0640 1 -- start start 2024-09-12T12:24:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.355+0000 7fa96f4e0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9680d05b0 0x7fa9680d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.355+0000 7fa96f4e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.355+0000 7fa96f4e0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9680b42f0 0x7fa9680b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa9680d0ed0 con 0x7fa9680d05b0 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9680ad0c0 con 0x7fa9680b42f0 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9680ad3a0 con 0x7fa9680ae250 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9680d05b0 0x7fa9680d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9680d05b0 0x7fa9680d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56776/0 (socket says 172.21.15.3:56776) 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 -- 172.21.15.3:0/3982009612 learned_addr learned my addr 172.21.15.3:0/3982009612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96da7d640 1 --2- 172.21.15.3:0/3982009612 >> v2:172.21.15.142:3300/0 conn(0x7fa9680b42f0 0x7fa9680b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96ca7b640 1 --2- 172.21.15.3:0/3982009612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 -- 172.21.15.3:0/3982009612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 msgr2=0x7fa9680ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:52.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 --2- 172.21.15.3:0/3982009612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 -- 172.21.15.3:0/3982009612 >> v2:172.21.15.142:3300/0 conn(0x7fa9680b42f0 msgr2=0x7fa9680b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 --2- 172.21.15.3:0/3982009612 >> v2:172.21.15.142:3300/0 conn(0x7fa9680b42f0 0x7fa9680b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 -- 172.21.15.3:0/3982009612 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9680ad680 con 0x7fa9680d05b0 2024-09-12T12:24:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96da7d640 1 --2- 172.21.15.3:0/3982009612 >> v2:172.21.15.142:3300/0 conn(0x7fa9680b42f0 0x7fa9680b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:52.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96d27c640 1 --2- 172.21.15.3:0/3982009612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680d05b0 0x7fa9680d0990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa9680d1140 tx=0x7fa96401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92177a613e6ba615 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa95ffff640 1 -- 172.21.15.3:0/3982009612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9640b1070 con 0x7fa9680d05b0 2024-09-12T12:24:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa95ffff640 1 -- 172.21.15.3:0/3982009612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9640ac410 con 0x7fa9680d05b0 2024-09-12T12:24:52.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa95ffff640 1 -- 172.21.15.3:0/3982009612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9640ac7e0 con 0x7fa9680d05b0 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3982009612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680d05b0 msgr2=0x7fa9680d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3982009612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680d05b0 0x7fa9680d0990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa9680d1140 tx=0x7fa96401c4f0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3982009612 shutdown_connections 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3982009612 >> v2:172.21.15.142:3300/0 conn(0x7fa9680b42f0 0x7fa9680b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3982009612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3982009612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680d05b0 0x7fa9680d0990 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3982009612 >> 172.21.15.3:0/3982009612 conn(0x7fa968067e00 msgr2=0x7fa968068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3982009612 shutdown_connections 2024-09-12T12:24:52.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.359+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3982009612 wait complete. 2024-09-12T12:24:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 Processor -- start 2024-09-12T12:24:52.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 -- start start 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680b42f0 0x7fa96806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9680d05b0 0x7fa96806e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9680aed70 con 0x7fa9680b42f0 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9680aebf0 con 0x7fa9680d05b0 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9681559b0 con 0x7fa9680ae250 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680b42f0 0x7fa96806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680b42f0 0x7fa96806e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56778/0 (socket says 172.21.15.3:56778) 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 -- 172.21.15.3:0/3370995276 learned_addr learned my addr 172.21.15.3:0/3370995276 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96da7d640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9680d05b0 0x7fa96806e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96d27c640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 -- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 msgr2=0x7fa9680cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 -- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9680d05b0 msgr2=0x7fa96806e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9680d05b0 0x7fa96806e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 -- 172.21.15.3:0/3370995276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9680cb540 con 0x7fa9680b42f0 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96d27c640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96ca7b640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680b42f0 0x7fa96806e4a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa95800d800 tx=0x7fa95800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa95dffb640 1 -- 172.21.15.3:0/3370995276 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958018020 con 0x7fa9680b42f0 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa95dffb640 1 -- 172.21.15.3:0/3370995276 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9580096a0 con 0x7fa9680b42f0 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9680cb820 con 0x7fa9680b42f0 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa95dffb640 1 -- 172.21.15.3:0/3370995276 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958009a50 con 0x7fa9680b42f0 2024-09-12T12:24:52.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9680cbde0 con 0x7fa9680b42f0 2024-09-12T12:24:52.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.363+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96815c830 con 0x7fa9680b42f0 2024-09-12T12:24:52.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.367+0000 7fa95dffb640 1 -- 172.21.15.3:0/3370995276 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa95801b020 con 0x7fa9680b42f0 2024-09-12T12:24:52.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.367+0000 7fa95dffb640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678c0 0x7fa94c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.367+0000 7fa95dffb640 1 -- 172.21.15.3:0/3370995276 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fa958086560 con 0x7fa9680b42f0 2024-09-12T12:24:52.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.367+0000 7fa96d27c640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678c0 0x7fa94c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.367+0000 7fa96d27c640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678c0 0x7fa94c069d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa9640ae000 tx=0x7fa964008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:52.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.371+0000 7fa95dffb640 1 -- 172.21.15.3:0/3370995276 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa958022020 con 0x7fa9680b42f0 2024-09-12T12:24:52.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.531+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9680af210 con 0x7fa94c0678c0 2024-09-12T12:24:52.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.547+0000 7fa95dffb640 1 -- 172.21.15.3:0/3370995276 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252086 (secure 0 0 0) 0x7fa9680af210 con 0x7fa94c0678c0 2024-09-12T12:24:52.554 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:52.556 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:24:52.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678c0 msgr2=0x7fa94c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678c0 0x7fa94c069d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa9640ae000 tx=0x7fa964008040 comp rx=0 tx=0).stop 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680b42f0 msgr2=0x7fa96806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680b42f0 0x7fa96806e4a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fa95800d800 tx=0x7fa95800dcc0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 shutdown_connections 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa94c0678c0 0x7fa94c069d80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9680d05b0 0x7fa96806e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9680b42f0 0x7fa96806e4a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 --2- 172.21.15.3:0/3370995276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680ae250 0x7fa9680cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.551+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 >> 172.21.15.3:0/3370995276 conn(0x7fa968067e00 msgr2=0x7fa968072220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.555+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 shutdown_connections 2024-09-12T12:24:52.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.555+0000 7fa96f4e0640 1 -- 172.21.15.3:0/3370995276 wait complete. 2024-09-12T12:24:52.574 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-12T12:24:51.759453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":189,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259724,"kb_used_data":10184,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714996,"statfs":{"total":773094113280,"available":772828155904,"internally_reserved":0,"allocated":10428416,"data_stored":5099268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.147038"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616311+0000","last_change":"2024-09-12T12:24:43.638576+0000","last_active":"2024-09-12T12:24:44.616311+0000","last_peered":"2024-09-12T12:24:44.616311+0000","last_clean":"2024-09-12T12:24:44.616311+0000","last_became_active":"2024-09-12T12:24:43.637883+0000","last_became_peered":"2024-09-12T12:24:43.637883+0000","last_unstale":"2024-09-12T12:24:44.616311+0000","last_undegraded":"2024-09-12T12:24:44.616311+0000","last_fullsized":"2024-09-12T12:24:44.616311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:31.746764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679422+0000","last_change":"2024-09-12T12:24:43.662465+0000","last_active":"2024-09-12T12:24:44.679422+0000","last_peered":"2024-09-12T12:24:44.679422+0000","last_clean":"2024-09-12T12:24:44.679422+0000","last_became_active":"2024-09-12T12:24:43.661643+0000","last_became_peered":"2024-09-12T12:24:43.661643+0000","last_unstale":"2024-09-12T12:24:44.679422+0000","last_undegraded":"2024-09-12T12:24:44.679422+0000","last_fullsized":"2024-09-12T12:24:44.679422+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:22:30.267978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616160+0000","last_change":"2024-09-12T12:24:43.656461+0000","last_active":"2024-09-12T12:24:44.616160+0000","last_peered":"2024-09-12T12:24:44.616160+0000","last_clean":"2024-09-12T12:24:44.616160+0000","last_became_active":"2024-09-12T12:24:43.655523+0000","last_became_peered":"2024-09-12T12:24:43.655523+0000","last_unstale":"2024-09-12T12:24:44.616160+0000","last_undegraded":"2024-09-12T12:24:44.616160+0000","last_fullsized":"2024-09-12T12:24:44.616160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:45.830898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699288+0000","last_change":"2024-09-12T12:24:43.665791+0000","last_active":"2024-09-12T12:24:45.699288+0000","last_peered":"2024-09-12T12:24:45.699288+0000","last_clean":"2024-09-12T12:24:45.699288+0000","last_became_active":"2024-09-12T12:24:43.663819+0000","last_became_peered":"2024-09-12T12:24:43.663819+0000","last_unstale":"2024-09-12T12:24:45.699288+0000","last_undegraded":"2024-09-12T12:24:45.699288+0000","last_fullsized":"2024-09-12T12:24:45.699288+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:45:38.925842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703214+0000","last_change":"2024-09-12T12:24:43.651132+0000","last_active":"2024-09-12T12:24:45.703214+0000","last_peered":"2024-09-12T12:24:45.703214+0000","last_clean":"2024-09-12T12:24:45.703214+0000","last_became_active":"2024-09-12T12:24:43.650439+0000","last_became_peered":"2024-09-12T12:24:43.650439+0000","last_unstale":"2024-09-12T12:24:45.703214+0000","last_undegraded":"2024-09-12T12:24:45.703214+0000","last_fullsized":"2024-09-12T12:24:45.703214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:44.191710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621336+0000","last_change":"2024-09-12T12:24:43.650634+0000","last_active":"2024-09-12T12:24:44.621336+0000","last_peered":"2024-09-12T12:24:44.621336+0000","last_clean":"2024-09-12T12:24:44.621336+0000","last_became_active":"2024-09-12T12:24:43.649447+0000","last_became_peered":"2024-09-12T12:24:43.649447+0000","last_unstale":"2024-09-12T12:24:44.621336+0000","last_undegraded":"2024-09-12T12:24:44.621336+0000","last_fullsized":"2024-09-12T12:24:44.621336+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:56.893292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621208+0000","last_change":"2024-09-12T12:24:43.651354+0000","last_active":"2024-09-12T12:24:44.621208+0000","last_peered":"2024-09-12T12:24:44.621208+0000","last_clean":"2024-09-12T12:24:44.621208+0000","last_became_active":"2024-09-12T12:24:43.650570+0000","last_became_peered":"2024-09-12T12:24:43.650570+0000","last_unstale":"2024-09-12T12:24:44.621208+0000","last_undegraded":"2024-09-12T12:24:44.621208+0000","last_fullsized":"2024-09-12T12:24:44.621208+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.678878+0000","last_change":"2024-09-12T12:24:43.653808+0000","last_active":"2024-09-12T12:24:44.678878+0000","last_peered":"2024-09-12T12:24:44.678878+0000","last_clean":"2024-09-12T12:24:44.678878+0000","last_became_active":"2024-09-12T12:24:43.653034+0000","last_became_peered":"2024-09-12T12:24:43.653034+0000","last_unstale":"2024-09-12T12:24:44.678878+0000","last_undegraded":"2024-09-12T12:24:44.678878+0000","last_fullsized":"2024-09-12T12:24:44.678878+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:31.527341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904799+0000","last_change":"2024-09-12T12:24:43.652939+0000","last_active":"2024-09-12T12:24:44.904799+0000","last_peered":"2024-09-12T12:24:44.904799+0000","last_clean":"2024-09-12T12:24:44.904799+0000","last_became_active":"2024-09-12T12:24:43.651482+0000","last_became_peered":"2024-09-12T12:24:43.651482+0000","last_unstale":"2024-09-12T12:24:44.904799+0000","last_undegraded":"2024-09-12T12:24:44.904799+0000","last_fullsized":"2024-09-12T12:24:44.904799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702479+0000","last_change":"2024-09-12T12:24:43.637408+0000","last_active":"2024-09-12T12:24:45.702479+0000","last_peered":"2024-09-12T12:24:45.702479+0000","last_clean":"2024-09-12T12:24:45.702479+0000","last_became_active":"2024-09-12T12:24:43.636447+0000","last_became_peered":"2024-09-12T12:24:43.636447+0000","last_unstale":"2024-09-12T12:24:45.702479+0000","last_undegraded":"2024-09-12T12:24:45.702479+0000","last_fullsized":"2024-09-12T12:24:45.702479+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621413+0000","last_change":"2024-09-12T12:24:43.649963+0000","last_active":"2024-09-12T12:24:44.621413+0000","last_peered":"2024-09-12T12:24:44.621413+0000","last_clean":"2024-09-12T12:24:44.621413+0000","last_became_active":"2024-09-12T12:24:43.649206+0000","last_became_peered":"2024-09-12T12:24:43.649206+0000","last_unstale":"2024-09-12T12:24:44.621413+0000","last_undegraded":"2024-09-12T12:24:44.621413+0000","last_fullsized":"2024-09-12T12:24:44.621413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:36:44.276769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904389+0000","last_change":"2024-09-12T12:24:43.661406+0000","last_active":"2024-09-12T12:24:44.904389+0000","last_peered":"2024-09-12T12:24:44.904389+0000","last_clean":"2024-09-12T12:24:44.904389+0000","last_became_active":"2024-09-12T12:24:43.660623+0000","last_became_peered":"2024-09-12T12:24:43.660623+0000","last_unstale":"2024-09-12T12:24:44.904389+0000","last_undegraded":"2024-09-12T12:24:44.904389+0000","last_fullsized":"2024-09-12T12:24:44.904389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:10:16.413303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699353+0000","last_change":"2024-09-12T12:24:43.653070+0000","last_active":"2024-09-12T12:24:45.699353+0000","last_peered":"2024-09-12T12:24:45.699353+0000","last_clean":"2024-09-12T12:24:45.699353+0000","last_became_active":"2024-09-12T12:24:43.651753+0000","last_became_peered":"2024-09-12T12:24:43.651753+0000","last_unstale":"2024-09-12T12:24:45.699353+0000","last_undegraded":"2024-09-12T12:24:45.699353+0000","last_fullsized":"2024-09-12T12:24:45.699353+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:43:31.527341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683623+0000","last_change":"2024-09-12T12:24:43.622364+0000","last_active":"2024-09-12T12:24:44.683623+0000","last_peered":"2024-09-12T12:24:44.683623+0000","last_clean":"2024-09-12T12:24:44.683623+0000","last_became_active":"2024-09-12T12:24:43.620763+0000","last_became_peered":"2024-09-12T12:24:43.620763+0000","last_unstale":"2024-09-12T12:24:44.683623+0000","last_undegraded":"2024-09-12T12:24:44.683623+0000","last_fullsized":"2024-09-12T12:24:44.683623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:20:42.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.698939+0000","last_change":"2024-09-12T12:24:43.670224+0000","last_active":"2024-09-12T12:24:45.698939+0000","last_peered":"2024-09-12T12:24:45.698939+0000","last_clean":"2024-09-12T12:24:45.698939+0000","last_became_active":"2024-09-12T12:24:43.669622+0000","last_became_peered":"2024-09-12T12:24:43.669622+0000","last_unstale":"2024-09-12T12:24:45.698939+0000","last_undegraded":"2024-09-12T12:24:45.698939+0000","last_fullsized":"2024-09-12T12:24:45.698939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:40:19.063616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621285+0000","last_change":"2024-09-12T12:24:43.658919+0000","last_active":"2024-09-12T12:24:44.621285+0000","last_peered":"2024-09-12T12:24:44.621285+0000","last_clean":"2024-09-12T12:24:44.621285+0000","last_became_active":"2024-09-12T12:24:43.658279+0000","last_became_peered":"2024-09-12T12:24:43.658279+0000","last_unstale":"2024-09-12T12:24:44.621285+0000","last_undegraded":"2024-09-12T12:24:44.621285+0000","last_fullsized":"2024-09-12T12:24:44.621285+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:40:40.554937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679355+0000","last_change":"2024-09-12T12:24:43.659364+0000","last_active":"2024-09-12T12:24:44.679355+0000","last_peered":"2024-09-12T12:24:44.679355+0000","last_clean":"2024-09-12T12:24:44.679355+0000","last_became_active":"2024-09-12T12:24:43.658346+0000","last_became_peered":"2024-09-12T12:24:43.658346+0000","last_unstale":"2024-09-12T12:24:44.679355+0000","last_undegraded":"2024-09-12T12:24:44.679355+0000","last_fullsized":"2024-09-12T12:24:44.679355+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:24:30.174633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699132+0000","last_change":"2024-09-12T12:24:43.666474+0000","last_active":"2024-09-12T12:24:45.699132+0000","last_peered":"2024-09-12T12:24:45.699132+0000","last_clean":"2024-09-12T12:24:45.699132+0000","last_became_active":"2024-09-12T12:24:43.665798+0000","last_became_peered":"2024-09-12T12:24:43.665798+0000","last_unstale":"2024-09-12T12:24:45.699132+0000","last_undegraded":"2024-09-12T12:24:45.699132+0000","last_fullsized":"2024-09-12T12:24:45.699132+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:50:21.813734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683840+0000","last_change":"2024-09-12T12:24:43.647026+0000","last_active":"2024-09-12T12:24:44.683840+0000","last_peered":"2024-09-12T12:24:44.683840+0000","last_clean":"2024-09-12T12:24:44.683840+0000","last_became_active":"2024-09-12T12:24:43.646064+0000","last_became_peered":"2024-09-12T12:24:43.646064+0000","last_unstale":"2024-09-12T12:24:44.683840+0000","last_undegraded":"2024-09-12T12:24:44.683840+0000","last_fullsized":"2024-09-12T12:24:44.683840+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:38.167692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904536+0000","last_change":"2024-09-12T12:24:43.666544+0000","last_active":"2024-09-12T12:24:44.904536+0000","last_peered":"2024-09-12T12:24:44.904536+0000","last_clean":"2024-09-12T12:24:44.904536+0000","last_became_active":"2024-09-12T12:24:43.665661+0000","last_became_peered":"2024-09-12T12:24:43.665661+0000","last_unstale":"2024-09-12T12:24:44.904536+0000","last_undegraded":"2024-09-12T12:24:44.904536+0000","last_fullsized":"2024-09-12T12:24:44.904536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:03:22.604865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684069+0000","last_change":"2024-09-12T12:24:43.645911+0000","last_active":"2024-09-12T12:24:44.684069+0000","last_peered":"2024-09-12T12:24:44.684069+0000","last_clean":"2024-09-12T12:24:44.684069+0000","last_became_active":"2024-09-12T12:24:43.645127+0000","last_became_peered":"2024-09-12T12:24:43.645127+0000","last_unstale":"2024-09-12T12:24:44.684069+0000","last_undegraded":"2024-09-12T12:24:44.684069+0000","last_fullsized":"2024-09-12T12:24:44.684069+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:55:01.666655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679507+0000","last_change":"2024-09-12T12:24:43.659549+0000","last_active":"2024-09-12T12:24:44.679507+0000","last_peered":"2024-09-12T12:24:44.679507+0000","last_clean":"2024-09-12T12:24:44.679507+0000","last_became_active":"2024-09-12T12:24:43.658549+0000","last_became_peered":"2024-09-12T12:24:43.658549+0000","last_unstale":"2024-09-12T12:24:44.679507+0000","last_undegraded":"2024-09-12T12:24:44.679507+0000","last_fullsized":"2024-09-12T12:24:44.679507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:32.892286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.678936+0000","last_change":"2024-09-12T12:24:43.661284+0000","last_active":"2024-09-12T12:24:44.678936+0000","last_peered":"2024-09-12T12:24:44.678936+0000","last_clean":"2024-09-12T12:24:44.678936+0000","last_became_active":"2024-09-12T12:24:43.660500+0000","last_became_peered":"2024-09-12T12:24:43.660500+0000","last_unstale":"2024-09-12T12:24:44.678936+0000","last_undegraded":"2024-09-12T12:24:44.678936+0000","last_fullsized":"2024-09-12T12:24:44.678936+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:56.450729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621737+0000","last_change":"2024-09-12T12:24:43.646502+0000","last_active":"2024-09-12T12:24:44.621737+0000","last_peered":"2024-09-12T12:24:44.621737+0000","last_clean":"2024-09-12T12:24:44.621737+0000","last_became_active":"2024-09-12T12:24:43.645053+0000","last_became_peered":"2024-09-12T12:24:43.645053+0000","last_unstale":"2024-09-12T12:24:44.621737+0000","last_undegraded":"2024-09-12T12:24:44.621737+0000","last_fullsized":"2024-09-12T12:24:44.621737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:51:43.421188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703778+0000","last_change":"2024-09-12T12:24:43.645720+0000","last_active":"2024-09-12T12:24:45.703778+0000","last_peered":"2024-09-12T12:24:45.703778+0000","last_clean":"2024-09-12T12:24:45.703778+0000","last_became_active":"2024-09-12T12:24:43.644960+0000","last_became_peered":"2024-09-12T12:24:43.644960+0000","last_unstale":"2024-09-12T12:24:45.703778+0000","last_undegraded":"2024-09-12T12:24:45.703778+0000","last_fullsized":"2024-09-12T12:24:45.703778+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:07:07.741088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679816+0000","last_change":"2024-09-12T12:24:43.659332+0000","last_active":"2024-09-12T12:24:44.679816+0000","last_peered":"2024-09-12T12:24:44.679816+0000","last_clean":"2024-09-12T12:24:44.679816+0000","last_became_active":"2024-09-12T12:24:43.657467+0000","last_became_peered":"2024-09-12T12:24:43.657467+0000","last_unstale":"2024-09-12T12:24:44.679816+0000","last_undegraded":"2024-09-12T12:24:44.679816+0000","last_fullsized":"2024-09-12T12:24:44.679816+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:17.345934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622169+0000","last_change":"2024-09-12T12:24:43.661393+0000","last_active":"2024-09-12T12:24:44.622169+0000","last_peered":"2024-09-12T12:24:44.622169+0000","last_clean":"2024-09-12T12:24:44.622169+0000","last_became_active":"2024-09-12T12:24:43.660705+0000","last_became_peered":"2024-09-12T12:24:43.660705+0000","last_unstale":"2024-09-12T12:24:44.622169+0000","last_undegraded":"2024-09-12T12:24:44.622169+0000","last_fullsized":"2024-09-12T12:24:44.622169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:34:20.041489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703551+0000","last_change":"2024-09-12T12:24:43.659047+0000","last_active":"2024-09-12T12:24:45.703551+0000","last_peered":"2024-09-12T12:24:45.703551+0000","last_clean":"2024-09-12T12:24:45.703551+0000","last_became_active":"2024-09-12T12:24:43.658354+0000","last_became_peered":"2024-09-12T12:24:43.658354+0000","last_unstale":"2024-09-12T12:24:45.703551+0000","last_undegraded":"2024-09-12T12:24:45.703551+0000","last_fullsized":"2024-09-12T12:24:45.703551+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:22.035037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679389+0000","last_change":"2024-09-12T12:24:43.652975+0000","last_active":"2024-09-12T12:24:44.679389+0000","last_peered":"2024-09-12T12:24:44.679389+0000","last_clean":"2024-09-12T12:24:44.679389+0000","last_became_active":"2024-09-12T12:24:43.652191+0000","last_became_peered":"2024-09-12T12:24:43.652191+0000","last_unstale":"2024-09-12T12:24:44.679389+0000","last_undegraded":"2024-09-12T12:24:44.679389+0000","last_fullsized":"2024-09-12T12:24:44.679389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:57:51.956452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684067+0000","last_change":"2024-09-12T12:24:43.645157+0000","last_active":"2024-09-12T12:24:44.684067+0000","last_peered":"2024-09-12T12:24:44.684067+0000","last_clean":"2024-09-12T12:24:44.684067+0000","last_became_active":"2024-09-12T12:24:43.644341+0000","last_became_peered":"2024-09-12T12:24:43.644341+0000","last_unstale":"2024-09-12T12:24:44.684067+0000","last_undegraded":"2024-09-12T12:24:44.684067+0000","last_fullsized":"2024-09-12T12:24:44.684067+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:35:39.841147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699477+0000","last_change":"2024-09-12T12:24:43.664785+0000","last_active":"2024-09-12T12:24:45.699477+0000","last_peered":"2024-09-12T12:24:45.699477+0000","last_clean":"2024-09-12T12:24:45.699477+0000","last_became_active":"2024-09-12T12:24:43.663968+0000","last_became_peered":"2024-09-12T12:24:43.663968+0000","last_unstale":"2024-09-12T12:24:45.699477+0000","last_undegraded":"2024-09-12T12:24:45.699477+0000","last_fullsized":"2024-09-12T12:24:45.699477+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:55:56.450729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615264+0000","last_change":"2024-09-12T12:24:43.661779+0000","last_active":"2024-09-12T12:24:44.615264+0000","last_peered":"2024-09-12T12:24:44.615264+0000","last_clean":"2024-09-12T12:24:44.615264+0000","last_became_active":"2024-09-12T12:24:43.660239+0000","last_became_peered":"2024-09-12T12:24:43.660239+0000","last_unstale":"2024-09-12T12:24:44.615264+0000","last_undegraded":"2024-09-12T12:24:44.615264+0000","last_fullsized":"2024-09-12T12:24:44.615264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:10:55.818309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683905+0000","last_change":"2024-09-12T12:24:43.648961+0000","last_active":"2024-09-12T12:24:44.683905+0000","last_peered":"2024-09-12T12:24:44.683905+0000","last_clean":"2024-09-12T12:24:44.683905+0000","last_became_active":"2024-09-12T12:24:43.648231+0000","last_became_peered":"2024-09-12T12:24:43.648231+0000","last_unstale":"2024-09-12T12:24:44.683905+0000","last_undegraded":"2024-09-12T12:24:44.683905+0000","last_fullsized":"2024-09-12T12:24:44.683905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:04.030268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"92'4","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679475+0000","last_change":"2024-09-12T12:24:42.641566+0000","last_active":"2024-09-12T12:24:44.679475+0000","last_peered":"2024-09-12T12:24:44.679475+0000","last_clean":"2024-09-12T12:24:44.679475+0000","last_became_active":"2024-09-12T12:24:42.641294+0000","last_became_peered":"2024-09-12T12:24:42.641294+0000","last_unstale":"2024-09-12T12:24:44.679475+0000","last_undegraded":"2024-09-12T12:24:44.679475+0000","last_fullsized":"2024-09-12T12:24:44.679475+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:50.721973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699488+0000","last_change":"2024-09-12T12:24:43.665125+0000","last_active":"2024-09-12T12:24:45.699488+0000","last_peered":"2024-09-12T12:24:45.699488+0000","last_clean":"2024-09-12T12:24:45.699488+0000","last_became_active":"2024-09-12T12:24:43.664317+0000","last_became_peered":"2024-09-12T12:24:43.664317+0000","last_unstale":"2024-09-12T12:24:45.699488+0000","last_undegraded":"2024-09-12T12:24:45.699488+0000","last_fullsized":"2024-09-12T12:24:45.699488+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:24:30.174633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621763+0000","last_change":"2024-09-12T12:24:42.653011+0000","last_active":"2024-09-12T12:24:44.621763+0000","last_peered":"2024-09-12T12:24:44.621763+0000","last_clean":"2024-09-12T12:24:44.621763+0000","last_became_active":"2024-09-12T12:24:42.652145+0000","last_became_peered":"2024-09-12T12:24:42.652145+0000","last_unstale":"2024-09-12T12:24:44.621763+0000","last_undegraded":"2024-09-12T12:24:44.621763+0000","last_fullsized":"2024-09-12T12:24:44.621763+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:16.133515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615814+0000","last_change":"2024-09-12T12:24:43.651221+0000","last_active":"2024-09-12T12:24:44.615814+0000","last_peered":"2024-09-12T12:24:44.615814+0000","last_clean":"2024-09-12T12:24:44.615814+0000","last_became_active":"2024-09-12T12:24:43.650519+0000","last_became_peered":"2024-09-12T12:24:43.650519+0000","last_unstale":"2024-09-12T12:24:44.615814+0000","last_undegraded":"2024-09-12T12:24:44.615814+0000","last_fullsized":"2024-09-12T12:24:44.615814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:44:03.987247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679318+0000","last_change":"2024-09-12T12:24:42.646442+0000","last_active":"2024-09-12T12:24:44.679318+0000","last_peered":"2024-09-12T12:24:44.679318+0000","last_clean":"2024-09-12T12:24:44.679318+0000","last_became_active":"2024-09-12T12:24:42.643894+0000","last_became_peered":"2024-09-12T12:24:42.643894+0000","last_unstale":"2024-09-12T12:24:44.679318+0000","last_undegraded":"2024-09-12T12:24:44.679318+0000","last_fullsized":"2024-09-12T12:24:44.679318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:19:46.753562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.905160+0000","last_change":"2024-09-12T12:24:43.646657+0000","last_active":"2024-09-12T12:24:44.905160+0000","last_peered":"2024-09-12T12:24:44.905160+0000","last_clean":"2024-09-12T12:24:44.905160+0000","last_became_active":"2024-09-12T12:24:43.645851+0000","last_became_peered":"2024-09-12T12:24:43.645851+0000","last_unstale":"2024-09-12T12:24:44.905160+0000","last_undegraded":"2024-09-12T12:24:44.905160+0000","last_fullsized":"2024-09-12T12:24:44.905160+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679064+0000","last_change":"2024-09-12T12:24:42.637641+0000","last_active":"2024-09-12T12:24:44.679064+0000","last_peered":"2024-09-12T12:24:44.679064+0000","last_clean":"2024-09-12T12:24:44.679064+0000","last_became_active":"2024-09-12T12:24:42.637338+0000","last_became_peered":"2024-09-12T12:24:42.637338+0000","last_unstale":"2024-09-12T12:24:44.679064+0000","last_undegraded":"2024-09-12T12:24:44.679064+0000","last_fullsized":"2024-09-12T12:24:44.679064+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:45.444247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703339+0000","last_change":"2024-09-12T12:24:43.644547+0000","last_active":"2024-09-12T12:24:45.703339+0000","last_peered":"2024-09-12T12:24:45.703339+0000","last_clean":"2024-09-12T12:24:45.703339+0000","last_became_active":"2024-09-12T12:24:43.643836+0000","last_became_peered":"2024-09-12T12:24:43.643836+0000","last_unstale":"2024-09-12T12:24:45.703339+0000","last_undegraded":"2024-09-12T12:24:45.703339+0000","last_fullsized":"2024-09-12T12:24:45.703339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:42:31.852409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616266+0000","last_change":"2024-09-12T12:24:42.634480+0000","last_active":"2024-09-12T12:24:44.616266+0000","last_peered":"2024-09-12T12:24:44.616266+0000","last_clean":"2024-09-12T12:24:44.616266+0000","last_became_active":"2024-09-12T12:24:42.633367+0000","last_became_peered":"2024-09-12T12:24:42.633367+0000","last_unstale":"2024-09-12T12:24:44.616266+0000","last_undegraded":"2024-09-12T12:24:44.616266+0000","last_fullsized":"2024-09-12T12:24:44.616266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:19.812219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621856+0000","last_change":"2024-09-12T12:24:43.641831+0000","last_active":"2024-09-12T12:24:44.621856+0000","last_peered":"2024-09-12T12:24:44.621856+0000","last_clean":"2024-09-12T12:24:44.621856+0000","last_became_active":"2024-09-12T12:24:43.641063+0000","last_became_peered":"2024-09-12T12:24:43.641063+0000","last_unstale":"2024-09-12T12:24:44.621856+0000","last_undegraded":"2024-09-12T12:24:44.621856+0000","last_fullsized":"2024-09-12T12:24:44.621856+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:37:46.151100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.678973+0000","last_change":"2024-09-12T12:24:42.637461+0000","last_active":"2024-09-12T12:24:44.678973+0000","last_peered":"2024-09-12T12:24:44.678973+0000","last_clean":"2024-09-12T12:24:44.678973+0000","last_became_active":"2024-09-12T12:24:42.637031+0000","last_became_peered":"2024-09-12T12:24:42.637031+0000","last_unstale":"2024-09-12T12:24:44.678973+0000","last_undegraded":"2024-09-12T12:24:44.678973+0000","last_fullsized":"2024-09-12T12:24:44.678973+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:29.271497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.698887+0000","last_change":"2024-09-12T12:24:43.670586+0000","last_active":"2024-09-12T12:24:45.698887+0000","last_peered":"2024-09-12T12:24:45.698887+0000","last_clean":"2024-09-12T12:24:45.698887+0000","last_became_active":"2024-09-12T12:24:43.669772+0000","last_became_peered":"2024-09-12T12:24:43.669772+0000","last_unstale":"2024-09-12T12:24:45.698887+0000","last_undegraded":"2024-09-12T12:24:45.698887+0000","last_fullsized":"2024-09-12T12:24:45.698887+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:43.298312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702529+0000","last_change":"2024-09-12T12:24:42.648919+0000","last_active":"2024-09-12T12:24:45.702529+0000","last_peered":"2024-09-12T12:24:45.702529+0000","last_clean":"2024-09-12T12:24:45.702529+0000","last_became_active":"2024-09-12T12:24:42.648039+0000","last_became_peered":"2024-09-12T12:24:42.648039+0000","last_unstale":"2024-09-12T12:24:45.702529+0000","last_undegraded":"2024-09-12T12:24:45.702529+0000","last_fullsized":"2024-09-12T12:24:45.702529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:47:51.735892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903858+0000","last_change":"2024-09-12T12:24:43.650075+0000","last_active":"2024-09-12T12:24:44.903858+0000","last_peered":"2024-09-12T12:24:44.903858+0000","last_clean":"2024-09-12T12:24:44.903858+0000","last_became_active":"2024-09-12T12:24:43.649215+0000","last_became_peered":"2024-09-12T12:24:43.649215+0000","last_unstale":"2024-09-12T12:24:44.903858+0000","last_undegraded":"2024-09-12T12:24:44.903858+0000","last_fullsized":"2024-09-12T12:24:44.903858+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:30.006601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615299+0000","last_change":"2024-09-12T12:24:42.625151+0000","last_active":"2024-09-12T12:24:44.615299+0000","last_peered":"2024-09-12T12:24:44.615299+0000","last_clean":"2024-09-12T12:24:44.615299+0000","last_became_active":"2024-09-12T12:24:42.624745+0000","last_became_peered":"2024-09-12T12:24:42.624745+0000","last_unstale":"2024-09-12T12:24:44.615299+0000","last_undegraded":"2024-09-12T12:24:44.615299+0000","last_fullsized":"2024-09-12T12:24:44.615299+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:21.399432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679775+0000","last_change":"2024-09-12T12:24:43.648315+0000","last_active":"2024-09-12T12:24:44.679775+0000","last_peered":"2024-09-12T12:24:44.679775+0000","last_clean":"2024-09-12T12:24:44.679775+0000","last_became_active":"2024-09-12T12:24:43.647523+0000","last_became_peered":"2024-09-12T12:24:43.647523+0000","last_unstale":"2024-09-12T12:24:44.679775+0000","last_undegraded":"2024-09-12T12:24:44.679775+0000","last_fullsized":"2024-09-12T12:24:44.679775+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:57:38.801387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616802+0000","last_change":"2024-09-12T12:24:42.637921+0000","last_active":"2024-09-12T12:24:44.616802+0000","last_peered":"2024-09-12T12:24:44.616802+0000","last_clean":"2024-09-12T12:24:44.616802+0000","last_became_active":"2024-09-12T12:24:42.637229+0000","last_became_peered":"2024-09-12T12:24:42.637229+0000","last_unstale":"2024-09-12T12:24:44.616802+0000","last_undegraded":"2024-09-12T12:24:44.616802+0000","last_fullsized":"2024-09-12T12:24:44.616802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:42:59.256685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904941+0000","last_change":"2024-09-12T12:24:43.658926+0000","last_active":"2024-09-12T12:24:44.904941+0000","last_peered":"2024-09-12T12:24:44.904941+0000","last_clean":"2024-09-12T12:24:44.904941+0000","last_became_active":"2024-09-12T12:24:43.658247+0000","last_became_peered":"2024-09-12T12:24:43.658247+0000","last_unstale":"2024-09-12T12:24:44.904941+0000","last_undegraded":"2024-09-12T12:24:44.904941+0000","last_fullsized":"2024-09-12T12:24:44.904941+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:36:38.496292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699135+0000","last_change":"2024-09-12T12:24:42.641455+0000","last_active":"2024-09-12T12:24:45.699135+0000","last_peered":"2024-09-12T12:24:45.699135+0000","last_clean":"2024-09-12T12:24:45.699135+0000","last_became_active":"2024-09-12T12:24:42.641145+0000","last_became_peered":"2024-09-12T12:24:42.641145+0000","last_unstale":"2024-09-12T12:24:45.699135+0000","last_undegraded":"2024-09-12T12:24:45.699135+0000","last_fullsized":"2024-09-12T12:24:45.699135+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:58.912932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703939+0000","last_change":"2024-09-12T12:24:43.647661+0000","last_active":"2024-09-12T12:24:45.703939+0000","last_peered":"2024-09-12T12:24:45.703939+0000","last_clean":"2024-09-12T12:24:45.703939+0000","last_became_active":"2024-09-12T12:24:43.646874+0000","last_became_peered":"2024-09-12T12:24:43.646874+0000","last_unstale":"2024-09-12T12:24:45.703939+0000","last_undegraded":"2024-09-12T12:24:45.703939+0000","last_fullsized":"2024-09-12T12:24:45.703939+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:54:03.515944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"92'3","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622442+0000","last_change":"2024-09-12T12:24:42.652504+0000","last_active":"2024-09-12T12:24:44.622442+0000","last_peered":"2024-09-12T12:24:44.622442+0000","last_clean":"2024-09-12T12:24:44.622442+0000","last_became_active":"2024-09-12T12:24:42.651709+0000","last_became_peered":"2024-09-12T12:24:42.651709+0000","last_unstale":"2024-09-12T12:24:44.622442+0000","last_undegraded":"2024-09-12T12:24:44.622442+0000","last_fullsized":"2024-09-12T12:24:44.622442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:44.399415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622185+0000","last_change":"2024-09-12T12:24:43.659018+0000","last_active":"2024-09-12T12:24:44.622185+0000","last_peered":"2024-09-12T12:24:44.622185+0000","last_clean":"2024-09-12T12:24:44.622185+0000","last_became_active":"2024-09-12T12:24:43.658369+0000","last_became_peered":"2024-09-12T12:24:43.658369+0000","last_unstale":"2024-09-12T12:24:44.622185+0000","last_undegraded":"2024-09-12T12:24:44.622185+0000","last_fullsized":"2024-09-12T12:24:44.622185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:10:26.274835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"92'6","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904658+0000","last_change":"2024-09-12T12:24:42.643454+0000","last_active":"2024-09-12T12:24:44.904658+0000","last_peered":"2024-09-12T12:24:44.904658+0000","last_clean":"2024-09-12T12:24:44.904658+0000","last_became_active":"2024-09-12T12:24:42.642649+0000","last_became_peered":"2024-09-12T12:24:42.642649+0000","last_unstale":"2024-09-12T12:24:44.904658+0000","last_undegraded":"2024-09-12T12:24:44.904658+0000","last_fullsized":"2024-09-12T12:24:44.904658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:38:30.300204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699355+0000","last_change":"2024-09-12T12:24:43.647560+0000","last_active":"2024-09-12T12:24:45.699355+0000","last_peered":"2024-09-12T12:24:45.699355+0000","last_clean":"2024-09-12T12:24:45.699355+0000","last_became_active":"2024-09-12T12:24:43.646824+0000","last_became_peered":"2024-09-12T12:24:43.646824+0000","last_unstale":"2024-09-12T12:24:45.699355+0000","last_undegraded":"2024-09-12T12:24:45.699355+0000","last_fullsized":"2024-09-12T12:24:45.699355+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:14:24.592484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684213+0000","last_change":"2024-09-12T12:24:42.637238+0000","last_active":"2024-09-12T12:24:44.684213+0000","last_peered":"2024-09-12T12:24:44.684213+0000","last_clean":"2024-09-12T12:24:44.684213+0000","last_became_active":"2024-09-12T12:24:42.636898+0000","last_became_peered":"2024-09-12T12:24:42.636898+0000","last_unstale":"2024-09-12T12:24:44.684213+0000","last_undegraded":"2024-09-12T12:24:44.684213+0000","last_fullsized":"2024-09-12T12:24:44.684213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:16:06.749490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616014+0000","last_change":"2024-09-12T12:24:43.657062+0000","last_active":"2024-09-12T12:24:44.616014+0000","last_peered":"2024-09-12T12:24:44.616014+0000","last_clean":"2024-09-12T12:24:44.616014+0000","last_became_active":"2024-09-12T12:24:43.654265+0000","last_became_peered":"2024-09-12T12:24:43.654265+0000","last_unstale":"2024-09-12T12:24:44.616014+0000","last_undegraded":"2024-09-12T12:24:44.616014+0000","last_fullsized":"2024-09-12T12:24:44.616014+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:06.143959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.685093+0000","last_change":"2024-09-12T12:24:42.627572+0000","last_active":"2024-09-12T12:24:44.685093+0000","last_peered":"2024-09-12T12:24:44.685093+0000","last_clean":"2024-09-12T12:24:44.685093+0000","last_became_active":"2024-09-12T12:24:42.627257+0000","last_became_peered":"2024-09-12T12:24:42.627257+0000","last_unstale":"2024-09-12T12:24:44.685093+0000","last_undegraded":"2024-09-12T12:24:44.685093+0000","last_fullsized":"2024-09-12T12:24:44.685093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:04.397782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903797+0000","last_change":"2024-09-12T12:24:43.658133+0000","last_active":"2024-09-12T12:24:44.903797+0000","last_peered":"2024-09-12T12:24:44.903797+0000","last_clean":"2024-09-12T12:24:44.903797+0000","last_became_active":"2024-09-12T12:24:43.657012+0000","last_became_peered":"2024-09-12T12:24:43.657012+0000","last_unstale":"2024-09-12T12:24:44.903797+0000","last_undegraded":"2024-09-12T12:24:44.903797+0000","last_fullsized":"2024-09-12T12:24:44.903797+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:30:17.137749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.903854+0000","last_change":"2024-09-12T12:24:42.643511+0000","last_active":"2024-09-12T12:24:44.903854+0000","last_peered":"2024-09-12T12:24:44.903854+0000","last_clean":"2024-09-12T12:24:44.903854+0000","last_became_active":"2024-09-12T12:24:42.642768+0000","last_became_peered":"2024-09-12T12:24:42.642768+0000","last_unstale":"2024-09-12T12:24:44.903854+0000","last_undegraded":"2024-09-12T12:24:44.903854+0000","last_fullsized":"2024-09-12T12:24:44.903854+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:23.588413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683454+0000","last_change":"2024-09-12T12:24:43.636996+0000","last_active":"2024-09-12T12:24:44.683454+0000","last_peered":"2024-09-12T12:24:44.683454+0000","last_clean":"2024-09-12T12:24:44.683454+0000","last_became_active":"2024-09-12T12:24:43.636232+0000","last_became_peered":"2024-09-12T12:24:43.636232+0000","last_unstale":"2024-09-12T12:24:44.683454+0000","last_undegraded":"2024-09-12T12:24:44.683454+0000","last_fullsized":"2024-09-12T12:24:44.683454+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:56:16.348629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"92'8","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.615379+0000","last_change":"2024-09-12T12:24:42.642474+0000","last_active":"2024-09-12T12:24:44.615379+0000","last_peered":"2024-09-12T12:24:44.615379+0000","last_clean":"2024-09-12T12:24:44.615379+0000","last_became_active":"2024-09-12T12:24:42.642172+0000","last_became_peered":"2024-09-12T12:24:42.642172+0000","last_unstale":"2024-09-12T12:24:44.615379+0000","last_undegraded":"2024-09-12T12:24:44.615379+0000","last_fullsized":"2024-09-12T12:24:44.615379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:39.580983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703705+0000","last_change":"2024-09-12T12:24:43.639166+0000","last_active":"2024-09-12T12:24:45.703705+0000","last_peered":"2024-09-12T12:24:45.703705+0000","last_clean":"2024-09-12T12:24:45.703705+0000","last_became_active":"2024-09-12T12:24:43.638391+0000","last_became_peered":"2024-09-12T12:24:43.638391+0000","last_unstale":"2024-09-12T12:24:45.703705+0000","last_undegraded":"2024-09-12T12:24:45.703705+0000","last_fullsized":"2024-09-12T12:24:45.703705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:30:37.318619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904436+0000","last_change":"2024-09-12T12:24:42.640318+0000","last_active":"2024-09-12T12:24:44.904436+0000","last_peered":"2024-09-12T12:24:44.904436+0000","last_clean":"2024-09-12T12:24:44.904436+0000","last_became_active":"2024-09-12T12:24:42.639593+0000","last_became_peered":"2024-09-12T12:24:42.639593+0000","last_unstale":"2024-09-12T12:24:44.904436+0000","last_undegraded":"2024-09-12T12:24:44.904436+0000","last_fullsized":"2024-09-12T12:24:44.904436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:07:18.210648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702964+0000","last_change":"2024-09-12T12:24:43.653241+0000","last_active":"2024-09-12T12:24:45.702964+0000","last_peered":"2024-09-12T12:24:45.702964+0000","last_clean":"2024-09-12T12:24:45.702964+0000","last_became_active":"2024-09-12T12:24:43.652511+0000","last_became_peered":"2024-09-12T12:24:43.652511+0000","last_unstale":"2024-09-12T12:24:45.702964+0000","last_undegraded":"2024-09-12T12:24:45.702964+0000","last_fullsized":"2024-09-12T12:24:45.702964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:35:45.460852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702785+0000","last_change":"2024-09-12T12:24:42.637205+0000","last_active":"2024-09-12T12:24:45.702785+0000","last_peered":"2024-09-12T12:24:45.702785+0000","last_clean":"2024-09-12T12:24:45.702785+0000","last_became_active":"2024-09-12T12:24:42.636321+0000","last_became_peered":"2024-09-12T12:24:42.636321+0000","last_unstale":"2024-09-12T12:24:45.702785+0000","last_undegraded":"2024-09-12T12:24:45.702785+0000","last_fullsized":"2024-09-12T12:24:45.702785+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.622470+0000","last_change":"2024-09-12T12:24:43.649792+0000","last_active":"2024-09-12T12:24:44.622470+0000","last_peered":"2024-09-12T12:24:44.622470+0000","last_clean":"2024-09-12T12:24:44.622470+0000","last_became_active":"2024-09-12T12:24:43.648913+0000","last_became_peered":"2024-09-12T12:24:43.648913+0000","last_unstale":"2024-09-12T12:24:44.622470+0000","last_undegraded":"2024-09-12T12:24:44.622470+0000","last_fullsized":"2024-09-12T12:24:44.622470+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:51:34.860957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679177+0000","last_change":"2024-09-12T12:24:42.642838+0000","last_active":"2024-09-12T12:24:44.679177+0000","last_peered":"2024-09-12T12:24:44.679177+0000","last_clean":"2024-09-12T12:24:44.679177+0000","last_became_active":"2024-09-12T12:24:42.642310+0000","last_became_peered":"2024-09-12T12:24:42.642310+0000","last_unstale":"2024-09-12T12:24:44.679177+0000","last_undegraded":"2024-09-12T12:24:44.679177+0000","last_fullsized":"2024-09-12T12:24:44.679177+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:50:00.362189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684507+0000","last_change":"2024-09-12T12:24:43.652450+0000","last_active":"2024-09-12T12:24:44.684507+0000","last_peered":"2024-09-12T12:24:44.684507+0000","last_clean":"2024-09-12T12:24:44.684507+0000","last_became_active":"2024-09-12T12:24:43.651643+0000","last_became_peered":"2024-09-12T12:24:43.651643+0000","last_unstale":"2024-09-12T12:24:44.684507+0000","last_undegraded":"2024-09-12T12:24:44.684507+0000","last_fullsized":"2024-09-12T12:24:44.684507+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:20.816453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679500+0000","last_change":"2024-09-12T12:24:42.637602+0000","last_active":"2024-09-12T12:24:44.679500+0000","last_peered":"2024-09-12T12:24:44.679500+0000","last_clean":"2024-09-12T12:24:44.679500+0000","last_became_active":"2024-09-12T12:24:42.637106+0000","last_became_peered":"2024-09-12T12:24:42.637106+0000","last_unstale":"2024-09-12T12:24:44.679500+0000","last_undegraded":"2024-09-12T12:24:44.679500+0000","last_fullsized":"2024-09-12T12:24:44.679500+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:35:19.920423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684325+0000","last_change":"2024-09-12T12:24:43.652172+0000","last_active":"2024-09-12T12:24:44.684325+0000","last_peered":"2024-09-12T12:24:44.684325+0000","last_clean":"2024-09-12T12:24:44.684325+0000","last_became_active":"2024-09-12T12:24:43.651461+0000","last_became_peered":"2024-09-12T12:24:43.651461+0000","last_unstale":"2024-09-12T12:24:44.684325+0000","last_undegraded":"2024-09-12T12:24:44.684325+0000","last_fullsized":"2024-09-12T12:24:44.684325+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:47:24.797814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703619+0000","last_change":"2024-09-12T12:24:42.639853+0000","last_active":"2024-09-12T12:24:45.703619+0000","last_peered":"2024-09-12T12:24:45.703619+0000","last_clean":"2024-09-12T12:24:45.703619+0000","last_became_active":"2024-09-12T12:24:42.639201+0000","last_became_peered":"2024-09-12T12:24:42.639201+0000","last_unstale":"2024-09-12T12:24:45.703619+0000","last_undegraded":"2024-09-12T12:24:45.703619+0000","last_fullsized":"2024-09-12T12:24:45.703619+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:23.071254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683367+0000","last_change":"2024-09-12T12:24:43.656139+0000","last_active":"2024-09-12T12:24:44.683367+0000","last_peered":"2024-09-12T12:24:44.683367+0000","last_clean":"2024-09-12T12:24:44.683367+0000","last_became_active":"2024-09-12T12:24:43.651199+0000","last_became_peered":"2024-09-12T12:24:43.651199+0000","last_unstale":"2024-09-12T12:24:44.683367+0000","last_undegraded":"2024-09-12T12:24:44.683367+0000","last_fullsized":"2024-09-12T12:24:44.683367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:15:43.831842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702539+0000","last_change":"2024-09-12T12:24:42.647349+0000","last_active":"2024-09-12T12:24:45.702539+0000","last_peered":"2024-09-12T12:24:45.702539+0000","last_clean":"2024-09-12T12:24:45.702539+0000","last_became_active":"2024-09-12T12:24:42.646472+0000","last_became_peered":"2024-09-12T12:24:42.646472+0000","last_unstale":"2024-09-12T12:24:45.702539+0000","last_undegraded":"2024-09-12T12:24:45.702539+0000","last_fullsized":"2024-09-12T12:24:45.702539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:05.139726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703187+0000","last_change":"2024-09-12T12:24:43.637022+0000","last_active":"2024-09-12T12:24:45.703187+0000","last_peered":"2024-09-12T12:24:45.703187+0000","last_clean":"2024-09-12T12:24:45.703187+0000","last_became_active":"2024-09-12T12:24:43.636260+0000","last_became_peered":"2024-09-12T12:24:43.636260+0000","last_unstale":"2024-09-12T12:24:45.703187+0000","last_undegraded":"2024-09-12T12:24:45.703187+0000","last_fullsized":"2024-09-12T12:24:45.703187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:11:15.925952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679011+0000","last_change":"2024-09-12T12:24:42.637645+0000","last_active":"2024-09-12T12:24:44.679011+0000","last_peered":"2024-09-12T12:24:44.679011+0000","last_clean":"2024-09-12T12:24:44.679011+0000","last_became_active":"2024-09-12T12:24:42.637099+0000","last_became_peered":"2024-09-12T12:24:42.637099+0000","last_unstale":"2024-09-12T12:24:44.679011+0000","last_undegraded":"2024-09-12T12:24:44.679011+0000","last_fullsized":"2024-09-12T12:24:44.679011+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:48:06.299303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.680093+0000","last_change":"2024-09-12T12:24:43.661085+0000","last_active":"2024-09-12T12:24:44.680093+0000","last_peered":"2024-09-12T12:24:44.680093+0000","last_clean":"2024-09-12T12:24:44.680093+0000","last_became_active":"2024-09-12T12:24:43.660358+0000","last_became_peered":"2024-09-12T12:24:43.660358+0000","last_unstale":"2024-09-12T12:24:44.680093+0000","last_undegraded":"2024-09-12T12:24:44.680093+0000","last_fullsized":"2024-09-12T12:24:44.680093+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:45.166625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"92'2","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.703333+0000","last_change":"2024-09-12T12:24:42.639106+0000","last_active":"2024-09-12T12:24:45.703333+0000","last_peered":"2024-09-12T12:24:45.703333+0000","last_clean":"2024-09-12T12:24:45.703333+0000","last_became_active":"2024-09-12T12:24:42.638341+0000","last_became_peered":"2024-09-12T12:24:42.638341+0000","last_unstale":"2024-09-12T12:24:45.703333+0000","last_undegraded":"2024-09-12T12:24:45.703333+0000","last_fullsized":"2024-09-12T12:24:45.703333+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:44.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616629+0000","last_change":"2024-09-12T12:24:43.655680+0000","last_active":"2024-09-12T12:24:44.616629+0000","last_peered":"2024-09-12T12:24:44.616629+0000","last_clean":"2024-09-12T12:24:44.616629+0000","last_became_active":"2024-09-12T12:24:43.654815+0000","last_became_peered":"2024-09-12T12:24:43.654815+0000","last_unstale":"2024-09-12T12:24:44.616629+0000","last_undegraded":"2024-09-12T12:24:44.616629+0000","last_fullsized":"2024-09-12T12:24:44.616629+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:07:23.355405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"93'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.680347+0000","last_change":"2024-09-12T12:24:42.641239+0000","last_active":"2024-09-12T12:24:44.680347+0000","last_peered":"2024-09-12T12:24:44.680347+0000","last_clean":"2024-09-12T12:24:44.680347+0000","last_became_active":"2024-09-12T12:24:42.638459+0000","last_became_peered":"2024-09-12T12:24:42.638459+0000","last_unstale":"2024-09-12T12:24:44.680347+0000","last_undegraded":"2024-09-12T12:24:44.680347+0000","last_fullsized":"2024-09-12T12:24:44.680347+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:54.593765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.616100+0000","last_change":"2024-09-12T12:24:43.650775+0000","last_active":"2024-09-12T12:24:44.616100+0000","last_peered":"2024-09-12T12:24:44.616100+0000","last_clean":"2024-09-12T12:24:44.616100+0000","last_became_active":"2024-09-12T12:24:43.650001+0000","last_became_peered":"2024-09-12T12:24:43.650001+0000","last_unstale":"2024-09-12T12:24:44.616100+0000","last_undegraded":"2024-09-12T12:24:44.616100+0000","last_fullsized":"2024-09-12T12:24:44.616100+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:02:24.075487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679452+0000","last_change":"2024-09-12T12:24:42.647803+0000","last_active":"2024-09-12T12:24:44.679452+0000","last_peered":"2024-09-12T12:24:44.679452+0000","last_clean":"2024-09-12T12:24:44.679452+0000","last_became_active":"2024-09-12T12:24:42.647329+0000","last_became_peered":"2024-09-12T12:24:42.647329+0000","last_unstale":"2024-09-12T12:24:44.679452+0000","last_undegraded":"2024-09-12T12:24:44.679452+0000","last_fullsized":"2024-09-12T12:24:44.679452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:08:08.076923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702515+0000","last_change":"2024-09-12T12:24:43.648046+0000","last_active":"2024-09-12T12:24:45.702515+0000","last_peered":"2024-09-12T12:24:45.702515+0000","last_clean":"2024-09-12T12:24:45.702515+0000","last_became_active":"2024-09-12T12:24:43.647327+0000","last_became_peered":"2024-09-12T12:24:43.647327+0000","last_unstale":"2024-09-12T12:24:45.702515+0000","last_undegraded":"2024-09-12T12:24:45.702515+0000","last_fullsized":"2024-09-12T12:24:45.702515+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:30:08.811308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.704161+0000","last_change":"2024-09-12T12:24:42.644303+0000","last_active":"2024-09-12T12:24:45.704161+0000","last_peered":"2024-09-12T12:24:45.704161+0000","last_clean":"2024-09-12T12:24:45.704161+0000","last_became_active":"2024-09-12T12:24:42.643559+0000","last_became_peered":"2024-09-12T12:24:42.643559+0000","last_unstale":"2024-09-12T12:24:45.704161+0000","last_undegraded":"2024-09-12T12:24:45.704161+0000","last_fullsized":"2024-09-12T12:24:45.704161+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:01:52.789860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684880+0000","last_change":"2024-09-12T12:24:43.637804+0000","last_active":"2024-09-12T12:24:44.684880+0000","last_peered":"2024-09-12T12:24:44.684880+0000","last_clean":"2024-09-12T12:24:44.684880+0000","last_became_active":"2024-09-12T12:24:43.636765+0000","last_became_peered":"2024-09-12T12:24:43.636765+0000","last_unstale":"2024-09-12T12:24:44.684880+0000","last_undegraded":"2024-09-12T12:24:44.684880+0000","last_fullsized":"2024-09-12T12:24:44.684880+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:33:22.403665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.679636+0000","last_change":"2024-09-12T12:24:42.642157+0000","last_active":"2024-09-12T12:24:44.679636+0000","last_peered":"2024-09-12T12:24:44.679636+0000","last_clean":"2024-09-12T12:24:44.679636+0000","last_became_active":"2024-09-12T12:24:42.637757+0000","last_became_peered":"2024-09-12T12:24:42.637757+0000","last_unstale":"2024-09-12T12:24:44.679636+0000","last_undegraded":"2024-09-12T12:24:44.679636+0000","last_fullsized":"2024-09-12T12:24:44.679636+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:13:58.912932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.904303+0000","last_change":"2024-09-12T12:24:43.643084+0000","last_active":"2024-09-12T12:24:44.904303+0000","last_peered":"2024-09-12T12:24:44.904303+0000","last_clean":"2024-09-12T12:24:44.904303+0000","last_became_active":"2024-09-12T12:24:43.642362+0000","last_became_peered":"2024-09-12T12:24:43.642362+0000","last_unstale":"2024-09-12T12:24:44.904303+0000","last_undegraded":"2024-09-12T12:24:44.904303+0000","last_fullsized":"2024-09-12T12:24:44.904303+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:57:45.403648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702332+0000","last_change":"2024-09-12T12:24:42.636916+0000","last_active":"2024-09-12T12:24:45.702332+0000","last_peered":"2024-09-12T12:24:45.702332+0000","last_clean":"2024-09-12T12:24:45.702332+0000","last_became_active":"2024-09-12T12:24:42.634574+0000","last_became_peered":"2024-09-12T12:24:42.634574+0000","last_unstale":"2024-09-12T12:24:45.702332+0000","last_undegraded":"2024-09-12T12:24:45.702332+0000","last_fullsized":"2024-09-12T12:24:45.702332+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:31:33.717766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683795+0000","last_change":"2024-09-12T12:24:43.644818+0000","last_active":"2024-09-12T12:24:44.683795+0000","last_peered":"2024-09-12T12:24:44.683795+0000","last_clean":"2024-09-12T12:24:44.683795+0000","last_became_active":"2024-09-12T12:24:43.644041+0000","last_became_peered":"2024-09-12T12:24:43.644041+0000","last_unstale":"2024-09-12T12:24:44.683795+0000","last_undegraded":"2024-09-12T12:24:44.683795+0000","last_fullsized":"2024-09-12T12:24:44.683795+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:58.101698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.699438+0000","last_change":"2024-09-12T12:24:42.625186+0000","last_active":"2024-09-12T12:24:45.699438+0000","last_peered":"2024-09-12T12:24:45.699438+0000","last_clean":"2024-09-12T12:24:45.699438+0000","last_became_active":"2024-09-12T12:24:42.624773+0000","last_became_peered":"2024-09-12T12:24:42.624773+0000","last_unstale":"2024-09-12T12:24:45.699438+0000","last_undegraded":"2024-09-12T12:24:45.699438+0000","last_fullsized":"2024-09-12T12:24:45.699438+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:54:29.271497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:45.702272+0000","last_change":"2024-09-12T12:24:43.647973+0000","last_active":"2024-09-12T12:24:45.702272+0000","last_peered":"2024-09-12T12:24:45.702272+0000","last_clean":"2024-09-12T12:24:45.702272+0000","last_became_active":"2024-09-12T12:24:43.647185+0000","last_became_peered":"2024-09-12T12:24:43.647185+0000","last_unstale":"2024-09-12T12:24:45.702272+0000","last_undegraded":"2024-09-12T12:24:45.702272+0000","last_fullsized":"2024-09-12T12:24:45.702272+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:39:26.170622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.683462+0000","last_change":"2024-09-12T12:24:42.638683+0000","last_active":"2024-09-12T12:24:44.683462+0000","last_peered":"2024-09-12T12:24:44.683462+0000","last_clean":"2024-09-12T12:24:44.683462+0000","last_became_active":"2024-09-12T12:24:42.638385+0000","last_became_peered":"2024-09-12T12:24:42.638385+0000","last_unstale":"2024-09-12T12:24:44.683462+0000","last_undegraded":"2024-09-12T12:24:44.683462+0000","last_fullsized":"2024-09-12T12:24:44.683462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:09:36.197995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.684036+0000","last_change":"2024-09-12T12:24:43.647413+0000","last_active":"2024-09-12T12:24:44.684036+0000","last_peered":"2024-09-12T12:24:44.684036+0000","last_clean":"2024-09-12T12:24:44.684036+0000","last_became_active":"2024-09-12T12:24:43.646570+0000","last_became_peered":"2024-09-12T12:24:43.646570+0000","last_unstale":"2024-09-12T12:24:44.684036+0000","last_undegraded":"2024-09-12T12:24:44.684036+0000","last_fullsized":"2024-09-12T12:24:44.684036+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:42.594580+0000","last_clean_scrub_stamp":"2024-09-12T12:24:42.594580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:53:39.696999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"92'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621547+0000","last_change":"2024-09-12T12:24:42.649739+0000","last_active":"2024-09-12T12:24:44.621547+0000","last_peered":"2024-09-12T12:24:44.621547+0000","last_clean":"2024-09-12T12:24:44.621547+0000","last_became_active":"2024-09-12T12:24:42.648926+0000","last_became_peered":"2024-09-12T12:24:42.648926+0000","last_unstale":"2024-09-12T12:24:44.621547+0000","last_undegraded":"2024-09-12T12:24:44.621547+0000","last_fullsized":"2024-09-12T12:24:44.621547+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:24:41.590347+0000","last_clean_scrub_stamp":"2024-09-12T12:24:41.590347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:32:29.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":302,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-12T12:24:44.621655+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:24:44.621655+0000","last_peered":"2024-09-12T12:24:44.621655+0000","last_clean":"2024-09-12T12:24:44.621655+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:24:44.621655+0000","last_undegraded":"2024-09-12T12:24:44.621655+0000","last_fullsized":"2024-09-12T12:24:44.621655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509506,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":490848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1748992,"data_stored":1079694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542210,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1150976,"data_stored":489284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1167360,"data_stored":490950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1150976,"data_stored":489248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":489214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542208,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1744896,"data_stored":1080164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:24:52.578 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:24:52.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-12T12:24:52.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b90d3640 1 Processor -- start 2024-09-12T12:24:52.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b90d3640 1 -- start start 2024-09-12T12:24:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b90d3640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe8b415e750 0x7fe8b40d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b90d3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b40cf880 0x7fe8b4168400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b90d3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b40d10d0 con 0x7fe8b40d40c0 2024-09-12T12:24:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b90d3640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b40d13b0 con 0x7fe8b415e750 2024-09-12T12:24:52.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b90d3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b40d1690 con 0x7fe8b40cf880 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b2d9d640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe8b415e750 0x7fe8b40d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b359e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b40cf880 0x7fe8b4168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b259c640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fe8b40d40c0 0x7fe8b40cf170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:41318/0 (socket says 172.21.15.3:41318) 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.747+0000 7fe8b259c640 1 -- 172.21.15.3:0/1971818912 learned_addr learned my addr 172.21.15.3:0/1971818912 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b2d9d640 1 -- 172.21.15.3:0/1971818912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b40cf880 msgr2=0x7fe8b4168400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b2d9d640 1 --2- 172.21.15.3:0/1971818912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b40cf880 0x7fe8b4168400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b2d9d640 1 -- 172.21.15.3:0/1971818912 >> v1:172.21.15.142:6789/0 conn(0x7fe8b40d40c0 legacy=0x7fe8b40cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b2d9d640 1 -- 172.21.15.3:0/1971818912 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b40d1970 con 0x7fe8b415e750 2024-09-12T12:24:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b2d9d640 1 --2- 172.21.15.3:0/1971818912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b415e750 0x7fe8b40d3b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40097c0 tx=0x7fe8a401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0c073c392e24a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b1d9b640 1 -- 172.21.15.3:0/1971818912 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40b5070 con 0x7fe8b415e750 2024-09-12T12:24:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b1d9b640 1 -- 172.21.15.3:0/1971818912 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a40a8990 con 0x7fe8b415e750 2024-09-12T12:24:52.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b1d9b640 1 -- 172.21.15.3:0/1971818912 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a40a8d60 con 0x7fe8b415e750 2024-09-12T12:24:52.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 -- 172.21.15.3:0/1971818912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b415e750 msgr2=0x7fe8b40d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/1971818912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b415e750 0x7fe8b40d3b80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40097c0 tx=0x7fe8a401ec90 comp rx=0 tx=0).stop 2024-09-12T12:24:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 -- 172.21.15.3:0/1971818912 shutdown_connections 2024-09-12T12:24:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/1971818912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b40cf880 0x7fe8b4168400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/1971818912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b415e750 0x7fe8b40d3b80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 -- 172.21.15.3:0/1971818912 >> 172.21.15.3:0/1971818912 conn(0x7fe8b4067a00 msgr2=0x7fe8b4067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 -- 172.21.15.3:0/1971818912 shutdown_connections 2024-09-12T12:24:52.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.751+0000 7fe8b90d3640 1 -- 172.21.15.3:0/1971818912 wait complete. 2024-09-12T12:24:52.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 Processor -- start 2024-09-12T12:24:52.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 -- start start 2024-09-12T12:24:52.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b40cf880 0x7fe8b415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8b40d40c0 0x7fe8b40ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b415e750 0x7fe8b4161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b415eb30 con 0x7fe8b40d40c0 2024-09-12T12:24:52.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b40b6e10 con 0x7fe8b40cf880 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b415ecb0 con 0x7fe8b415e750 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8b40d40c0 0x7fe8b40ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8b40d40c0 0x7fe8b40ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56788/0 (socket says 172.21.15.3:56788) 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 -- 172.21.15.3:0/48925301 learned_addr learned my addr 172.21.15.3:0/48925301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b2d9d640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b40cf880 0x7fe8b415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b359e640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b415e750 0x7fe8b4161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 -- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b415e750 msgr2=0x7fe8b4161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b415e750 0x7fe8b4161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 -- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b40cf880 msgr2=0x7fe8b415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b40cf880 0x7fe8b415b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 -- 172.21.15.3:0/48925301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b4156a30 con 0x7fe8b40d40c0 2024-09-12T12:24:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b259c640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8b40d40c0 0x7fe8b40ca040 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c450 tx=0x7fe8a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe89b7fe640 1 -- 172.21.15.3:0/48925301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8017070 con 0x7fe8b40d40c0 2024-09-12T12:24:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe89b7fe640 1 -- 172.21.15.3:0/48925301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a8005cf0 con 0x7fe8b40d40c0 2024-09-12T12:24:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b4156d10 con 0x7fe8b40d40c0 2024-09-12T12:24:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe89b7fe640 1 -- 172.21.15.3:0/48925301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8007290 con 0x7fe8b40d40c0 2024-09-12T12:24:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b41572d0 con 0x7fe8b40d40c0 2024-09-12T12:24:52.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b40d3b80 con 0x7fe8b40d40c0 2024-09-12T12:24:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.755+0000 7fe89b7fe640 1 -- 172.21.15.3:0/48925301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe8a801a070 con 0x7fe8b40d40c0 2024-09-12T12:24:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.759+0000 7fe89b7fe640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe8a00678f0 0x7fe8a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.759+0000 7fe89b7fe640 1 -- 172.21.15.3:0/48925301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fe8a8084fd0 con 0x7fe8b40d40c0 2024-09-12T12:24:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.759+0000 7fe8b2d9d640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe8a00678f0 0x7fe8a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:52.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.759+0000 7fe8b2d9d640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe8a00678f0 0x7fe8a0069db0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40b2000 tx=0x7fe8a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:52.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.763+0000 7fe89b7fe640 1 -- 172.21.15.3:0/48925301 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe8a8055b10 con 0x7fe8b40d40c0 2024-09-12T12:24:52.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.987+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe8b40d2700 con 0x7fe8b40d40c0 2024-09-12T12:24:52.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.991+0000 7fe89b7fe640 1 -- 172.21.15.3:0/48925301 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7fe8a8055410 con 0x7fe8b40d40c0 2024-09-12T12:24:52.997 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T12:24:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAo3OJmbVzGNBAAbFcOnfX8K8/5UIeZT5KlhA== 2024-09-12T12:24:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: caps mds = "allow" 2024-09-12T12:24:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: caps mon = "allow r" 2024-09-12T12:24:52.997 INFO:teuthology.orchestra.run.smithi003.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-12T12:24:52.997 INFO:teuthology.orchestra.run.smithi003.stderr:updated caps for client.0 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe8a00678f0 msgr2=0x7fe8a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe8a00678f0 0x7fe8a0069db0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40b2000 tx=0x7fe8a4008040 comp rx=0 tx=0).stop 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8b40d40c0 msgr2=0x7fe8b40ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8b40d40c0 0x7fe8b40ca040 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c450 tx=0x7fe8a800c910 comp rx=0 tx=0).stop 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 shutdown_connections 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe8a00678f0 0x7fe8a0069db0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe8b415e750 0x7fe8b4161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8b40d40c0 0x7fe8b40ca040 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:52.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 --2- 172.21.15.3:0/48925301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b40cf880 0x7fe8b415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 >> 172.21.15.3:0/48925301 conn(0x7fe8b4067a00 msgr2=0x7fe8b40ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 shutdown_connections 2024-09-12T12:24:53.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:52.995+0000 7fe8b90d3640 1 -- 172.21.15.3:0/48925301 wait complete. 2024-09-12T12:24:53.015 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:24:53.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f208a888640 1 Processor -- start 2024-09-12T12:24:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f208a888640 1 -- start start 2024-09-12T12:24:53.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f208a888640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2084073cb0 0x7f208415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f208a888640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20840b5000 con 0x7f20840ae880 2024-09-12T12:24:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f208a888640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20840b3530 con 0x7f20840b30e0 2024-09-12T12:24:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f208a888640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2084074090 con 0x7f2084073cb0 2024-09-12T12:24:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2083fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2084073cb0 0x7f208415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2083fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2084073cb0 0x7f208415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56794/0 (socket says 172.21.15.3:56794) 2024-09-12T12:24:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2088e25640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f20840ae880 0x7f20840d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35282/0 (socket says 172.21.15.3:35282) 2024-09-12T12:24:53.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2083fff640 1 -- 172.21.15.3:0/4129446301 learned_addr learned my addr 172.21.15.3:0/4129446301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2083fff640 1 -- 172.21.15.3:0/4129446301 >> v1:172.21.15.142:6789/0 conn(0x7f20840b30e0 legacy=0x7f20840adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2083fff640 1 -- 172.21.15.3:0/4129446301 >> v1:172.21.15.3:6789/0 conn(0x7f20840ae880 legacy=0x7f20840d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2082ffd640 1 -- 172.21.15.3:0/4129446301 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 986252275 0 0) 0x7f20840b5000 con 0x7f20840ae880 2024-09-12T12:24:53.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.135+0000 7f2083fff640 1 -- 172.21.15.3:0/4129446301 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20840b37c0 con 0x7f2084073cb0 2024-09-12T12:24:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 --2- 172.21.15.3:0/4129446301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208415c710 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2078009870 tx=0x7f207801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1012a2b22edea5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2082ffd640 1 -- 172.21.15.3:0/4129446301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20780ab610 con 0x7f2084073cb0 2024-09-12T12:24:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2082ffd640 1 -- 172.21.15.3:0/4129446301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20780b0070 con 0x7f2084073cb0 2024-09-12T12:24:53.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2082ffd640 1 -- 172.21.15.3:0/4129446301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20780ab9d0 con 0x7f2084073cb0 2024-09-12T12:24:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- 172.21.15.3:0/4129446301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 msgr2=0x7f208415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 --2- 172.21.15.3:0/4129446301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208415c710 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2078009870 tx=0x7f207801c520 comp rx=0 tx=0).stop 2024-09-12T12:24:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- 172.21.15.3:0/4129446301 shutdown_connections 2024-09-12T12:24:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 --2- 172.21.15.3:0/4129446301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208415c710 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- 172.21.15.3:0/4129446301 >> 172.21.15.3:0/4129446301 conn(0x7f2084067e00 msgr2=0x7f2084068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- 172.21.15.3:0/4129446301 shutdown_connections 2024-09-12T12:24:53.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- 172.21.15.3:0/4129446301 wait complete. 2024-09-12T12:24:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 Processor -- start 2024-09-12T12:24:53.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- start start 2024-09-12T12:24:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208414b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20840ae880 0x7f20840b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840b30e0 0x7f20840ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f20840ae4a0 con 0x7f2084073cb0 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20840ae320 con 0x7f20840ae880 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f20840d0cc0 con 0x7f20840b30e0 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208414b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208414b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56796/0 (socket says 172.21.15.3:56796) 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 -- 172.21.15.3:0/902186506 learned_addr learned my addr 172.21.15.3:0/902186506 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f20837fe640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20840ae880 0x7f20840b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2088e25640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840b30e0 0x7f20840ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 -- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840b30e0 msgr2=0x7f20840ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840b30e0 0x7f20840ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 -- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20840ae880 msgr2=0x7f20840b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20840ae880 0x7f20840b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 -- 172.21.15.3:0/902186506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20840b1880 con 0x7f2084073cb0 2024-09-12T12:24:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2083fff640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208414b780 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f20780ad040 tx=0x7f207801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2080ff9640 1 -- 172.21.15.3:0/902186506 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20780b4800 con 0x7f2084073cb0 2024-09-12T12:24:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2080ff9640 1 -- 172.21.15.3:0/902186506 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20780b49c0 con 0x7f2084073cb0 2024-09-12T12:24:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20840b1b60 con 0x7f2084073cb0 2024-09-12T12:24:53.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.139+0000 7f2080ff9640 1 -- 172.21.15.3:0/902186506 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20780b4db0 con 0x7f2084073cb0 2024-09-12T12:24:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.143+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2084152fc0 con 0x7f2084073cb0 2024-09-12T12:24:53.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.143+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f208415c710 con 0x7f2084073cb0 2024-09-12T12:24:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.143+0000 7f2080ff9640 1 -- 172.21.15.3:0/902186506 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f20780b3380 con 0x7f2084073cb0 2024-09-12T12:24:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.143+0000 7f2080ff9640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20700678c0 0x7f2070069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.143+0000 7f2080ff9640 1 -- 172.21.15.3:0/902186506 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f2078120250 con 0x7f2084073cb0 2024-09-12T12:24:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.143+0000 7f20837fe640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20700678c0 0x7f2070069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.143+0000 7f20837fe640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20700678c0 0x7f2070069d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f20841598d0 tx=0x7f2074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.147+0000 7f2080ff9640 1 -- 172.21.15.3:0/902186506 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f20780a8050 con 0x7f2084073cb0 2024-09-12T12:24:53.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.339+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20840733e0 con 0x7f2084073cb0 2024-09-12T12:24:53.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.339+0000 7f2080ff9640 1 -- 172.21.15.3:0/902186506 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5216 (secure 0 0 0) 0x7f2078016110 con 0x7f2084073cb0 2024-09-12T12:24:53.343 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:24:53.343 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":62,"btime":"2024-09-12T12:24:46:664139+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5992,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/95103479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":95103479},{"type":"v1","addr":"172.21.15.142:6835","nonce":95103479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5995,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/599679496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":599679496},{"type":"v1","addr":"172.21.15.142:6837","nonce":599679496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6735,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6834/1759200537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1759200537},{"type":"v1","addr":"172.21.15.3:6834","nonce":1759200537}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:24:46.035035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6738},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6738":{"gid":6738,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6835/2492687290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6833","nonce":2492687290},{"type":"v1","addr":"172.21.15.3:6835","nonce":2492687290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6738,"qdb_cluster":[6738]},"id":7}]} 2024-09-12T12:24:53.343 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 62 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.339+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20700678c0 msgr2=0x7f2070069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.339+0000 7f208a888640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20700678c0 0x7f2070069d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f20841598d0 tx=0x7f2074008040 comp rx=0 tx=0).stop 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.339+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 msgr2=0x7f208414b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.339+0000 7f208a888640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208414b780 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f20780ad040 tx=0x7f207801ad60 comp rx=0 tx=0).stop 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 shutdown_connections 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f20700678c0 0x7f2070069d80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20840b30e0 0x7f20840ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20840ae880 0x7f20840b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 --2- 172.21.15.3:0/902186506 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2084073cb0 0x7f208414b780 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 >> 172.21.15.3:0/902186506 conn(0x7f2084067e00 msgr2=0x7f20840b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 shutdown_connections 2024-09-12T12:24:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.343+0000 7f208a888640 1 -- 172.21.15.3:0/902186506 wait complete. 2024-09-12T12:24:53.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:24:42.989579+0000', 'modified': '2024-09-12T12:24:46.035035+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6738}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6738': {'gid': 6738, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6835/2492687290', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6833', 'nonce': 2492687290}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 2492687290}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6738, 'qdb_cluster': [6738]} 2024-09-12T12:24:53.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:24:53.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T12:24:53.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea5ba1c640 1 Processor -- start 2024-09-12T12:24:53.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea5ba1c640 1 -- start start 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea5ba1c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea540ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea5ba1c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea5415b620 0x7fea5415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea5ba1c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea540ad0c0 con 0x7fea540d05b0 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea5ba1c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fea540ad3a0 con 0x7fea5415b620 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea5ba1c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea540ad680 con 0x7fea540ae250 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea59fb9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea5415b620 0x7fea5415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea59fb9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fea5415b620 0x7fea5415bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56798/0 (socket says 172.21.15.3:56798) 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea59fb9640 1 -- 172.21.15.3:0/1834534006 learned_addr learned my addr 172.21.15.3:0/1834534006 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea58fb7640 1 --2- 172.21.15.3:0/1834534006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea540ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea59fb9640 1 -- 172.21.15.3:0/1834534006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 msgr2=0x7fea540ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea59fb9640 1 --2- 172.21.15.3:0/1834534006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea540ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea59fb9640 1 -- 172.21.15.3:0/1834534006 >> v1:172.21.15.142:6789/0 conn(0x7fea540d05b0 legacy=0x7fea540d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.899+0000 7fea59fb9640 1 -- 172.21.15.3:0/1834534006 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea540ad960 con 0x7fea5415b620 2024-09-12T12:24:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 --2- 172.21.15.3:0/1834534006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea5415bab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009e50 tx=0x7fea4c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473cdf7dbdf72272 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea47fff640 1 -- 172.21.15.3:0/1834534006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0a58f0 con 0x7fea5415b620 2024-09-12T12:24:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea47fff640 1 -- 172.21.15.3:0/1834534006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c0a5a90 con 0x7fea5415b620 2024-09-12T12:24:53.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea47fff640 1 -- 172.21.15.3:0/1834534006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0a5e60 con 0x7fea5415b620 2024-09-12T12:24:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- 172.21.15.3:0/1834534006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 msgr2=0x7fea5415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/1834534006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea5415bab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009e50 tx=0x7fea4c01ca00 comp rx=0 tx=0).stop 2024-09-12T12:24:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- 172.21.15.3:0/1834534006 shutdown_connections 2024-09-12T12:24:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/1834534006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea5415bab0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/1834534006 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea540ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- 172.21.15.3:0/1834534006 >> 172.21.15.3:0/1834534006 conn(0x7fea54067e00 msgr2=0x7fea54068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- 172.21.15.3:0/1834534006 shutdown_connections 2024-09-12T12:24:53.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- 172.21.15.3:0/1834534006 wait complete. 2024-09-12T12:24:53.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 Processor -- start 2024-09-12T12:24:53.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- start start 2024-09-12T12:24:53.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea54149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea540d05b0 0x7fea540cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea540cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea5415c9b0 con 0x7fea5415b620 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea5415c830 con 0x7fea540d05b0 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea5ba1c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea54165ca0 con 0x7fea540ae250 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea540cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea540cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56808/0 (socket says 172.21.15.3:56808) 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 -- 172.21.15.3:0/2941443746 learned_addr learned my addr 172.21.15.3:0/2941443746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea58fb7640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea540d05b0 0x7fea540cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea597b8640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea54149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 -- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 msgr2=0x7fea54149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea54149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 -- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea540d05b0 msgr2=0x7fea540cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea540d05b0 0x7fea540cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea59fb9640 1 -- 172.21.15.3:0/2941443746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea540cbe80 con 0x7fea5415b620 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.903+0000 7fea58fb7640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea540d05b0 0x7fea540cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea59fb9640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea540cb740 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0af040 tx=0x7fea4c0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea45ffb640 1 -- 172.21.15.3:0/2941443746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0a58f0 con 0x7fea5415b620 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea45ffb640 1 -- 172.21.15.3:0/2941443746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4c0be070 con 0x7fea5415b620 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea540b1890 con 0x7fea5415b620 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea45ffb640 1 -- 172.21.15.3:0/2941443746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c009510 con 0x7fea5415b620 2024-09-12T12:24:53.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea540b1e50 con 0x7fea5415b620 2024-09-12T12:24:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea45ffb640 1 -- 172.21.15.3:0/2941443746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fea4c0b86b0 con 0x7fea5415b620 2024-09-12T12:24:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea45ffb640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea380678c0 0x7fea38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea45ffb640 1 -- 172.21.15.3:0/2941443746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fea4c121230 con 0x7fea5415b620 2024-09-12T12:24:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea597b8640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea380678c0 0x7fea38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea5ead8000 1 Processor -- start 2024-09-12T12:24:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea5ead8000 1 -- start start 2024-09-12T12:24:53.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea597b8640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea380678c0 0x7fea38069d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea48000f70 tx=0x7fea48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.907+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9a9290 0x55e85f9a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e85f890b60 con 0x55e85f9f2b20 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e85f7f7c70 con 0x55e85f9a9290 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e85f79ded0 con 0x55e85f9f3440 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56824/0 (socket says 172.21.15.3:56824) 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3774748675 learned_addr learned my addr 172.21.15.3:0/3774748675 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 msgr2=0x55e85f9a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9a8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9a9290 msgr2=0x55e85f9a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea59fb9640 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9a9290 0x55e85f9a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9a9290 0x55e85f9a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3774748675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85f7afe50 con 0x55e85f9f2b20 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea59fb9640 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9a9290 0x55e85f9a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9f2f00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fea48001560 tx=0x7fea48011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d549f1547f00e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea3effd640 1 -- 172.21.15.3:0/3774748675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48003600 con 0x55e85f9f2b20 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea3effd640 1 -- 172.21.15.3:0/3774748675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea480037a0 con 0x55e85f9f2b20 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea3effd640 1 -- 172.21.15.3:0/3774748675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48003b50 con 0x55e85f9f2b20 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- 172.21.15.3:0/3774748675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 msgr2=0x55e85f9f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9f2f00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fea48001560 tx=0x7fea48011000 comp rx=0 tx=0).stop 2024-09-12T12:24:53.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- 172.21.15.3:0/3774748675 shutdown_connections 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9a9290 0x55e85f9a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3774748675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9f2f00 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- 172.21.15.3:0/3774748675 >> 172.21.15.3:0/3774748675 conn(0x55e85f9af600 msgr2=0x55e85f9a7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- 172.21.15.3:0/3774748675 shutdown_connections 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- 172.21.15.3:0/3774748675 wait complete. 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- start start 2024-09-12T12:24:53.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 Processor -- start 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85f9b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e85f859b00 con 0x55e85f9f2b20 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e85f7d2360 con 0x55e85f9ae390 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e85f7cf2d0 con 0x55e85f9f3440 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 10 client.0 _ll_register_callbacks cb 0x55e85f994820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56834/0 (socket says 172.21.15.3:56834) 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3771471937 learned_addr learned my addr 172.21.15.3:0/3771471937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea59fb9640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85f9b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 msgr2=0x55e85f9b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 msgr2=0x55e85f9b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85f9b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 -- 172.21.15.3:0/3771471937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85f79e120 con 0x55e85f9f2b20 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea59fb9640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85f9b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:53.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea597b8640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9ade50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x55e85f8afac0 tx=0x7fea48013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97196089aa0803 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea33fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea33fff640 1 -- 172.21.15.3:0/3771471937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48021450 con 0x55e85f9f2b20 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea33fff640 1 -- 172.21.15.3:0/3771471937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fea48010050 con 0x55e85f9f2b20 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.911+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea480037a0 con 0x55e85f9f2b20 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea33fff640 1 -- 172.21.15.3:0/3771471937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fea48033050 con 0x55e85f9f2b20 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 10 client.6885 fetch_fsmap learned FSMap version 62 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55e85f797450 con 0x55e85f9f2b20 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea33fff640 1 -- 172.21.15.3:0/3771471937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea480037a0 con 0x55e85f9f2b20 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea33fff640 1 -- 172.21.15.3:0/3771471937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 62) ==== 3502+0+0 (secure 0 0 0) 0x7fea48036020 con 0x55e85f9f2b20 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 10 client.6885 fetch_fsmap finished waiting for FSMap version 62 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 10 client.6885 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 20 client.6885 populate_metadata read hostname 'smithi003' 2024-09-12T12:24:53.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85fa01280 0x55e85fa03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 4 client.6885 mds_command: new command op to 6738 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 --> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e85f791c00 con 0x55e85fa01280 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85fa01280 0x55e85fa03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea33fff640 10 client.6885 ms_handle_connect on v2:172.21.15.3:6833/2492687290 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85fa01280 0x55e85fa03670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81074b19078bd01f server_cookie=98969a8d2ca17d77 in_seq=0 out_seq=0 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea33fff640 1 -- 172.21.15.3:0/3771471937 <== mds.0 v2:172.21.15.3:6833/2492687290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e85f791c00 con 0x55e85fa01280 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea33fff640 10 client.6885 handle_command_reply: tid=0 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 client.6885 shutdown 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 msgr2=0x55e85f9ade50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9ade50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x55e85f8afac0 tx=0x7fea48013000 comp rx=0 tx=0).stop 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 shutdown_connections 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85fa01280 0x55e85fa03670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85f9b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9f3440 0x55e85f9b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3771471937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85f9ade50 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 >> 172.21.15.3:0/3771471937 conn(0x55e85f9a9290 msgr2=0x55e85f9a7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea33fff640 0 client.6885 ms_handle_reset on v2:172.21.15.3:6833/2492687290 2024-09-12T12:24:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 shutdown_connections 2024-09-12T12:24:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 1 -- 172.21.15.3:0/3771471937 wait complete. 2024-09-12T12:24:53.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.915+0000 7fea5ead8000 20 client.6885 trim_cache size 0 max 16384 2024-09-12T12:24:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 Processor -- start 2024-09-12T12:24:53.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- start start 2024-09-12T12:24:53.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85f9f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9ae390 0x55e85f9f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9f2b20 0x55e85fa03290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea54202730 con 0x55e85f9ae390 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fea54201ae0 con 0x55e85f9f2b20 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fea542020e0 con 0x55e85f9a9290 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9ae390 0x55e85f9f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9ae390 0x55e85f9f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56842/0 (socket says 172.21.15.3:56842) 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 -- 172.21.15.3:0/3425666816 learned_addr learned my addr 172.21.15.3:0/3425666816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea59fb9640 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9f2b20 0x55e85fa03290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea597b8640 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85f9f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 -- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 msgr2=0x55e85f9f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85f9f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 -- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9f2b20 msgr2=0x55e85fa03290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9f2b20 0x55e85fa03290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 -- 172.21.15.3:0/3425666816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85f77efd0 con 0x55e85f9ae390 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea59fb9640 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9f2b20 0x55e85fa03290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea58fb7640 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9ae390 0x55e85f9f21d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fea50000c10 tx=0x7fea50016430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f6ce5a9cf1a264 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea3e7fc640 1 -- 172.21.15.3:0/3425666816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50016b90 con 0x55e85f9ae390 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea3e7fc640 1 -- 172.21.15.3:0/3425666816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea50016d30 con 0x55e85f9ae390 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea3e7fc640 1 -- 172.21.15.3:0/3425666816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50004260 con 0x55e85f9ae390 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- 172.21.15.3:0/3425666816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9ae390 msgr2=0x55e85f9f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9ae390 0x55e85f9f21d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fea50000c10 tx=0x7fea50016430 comp rx=0 tx=0).stop 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- 172.21.15.3:0/3425666816 shutdown_connections 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9f2b20 0x55e85fa03290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9ae390 0x55e85f9f21d0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 --2- 172.21.15.3:0/3425666816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85f9f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- 172.21.15.3:0/3425666816 >> 172.21.15.3:0/3425666816 conn(0x55e85f9fe270 msgr2=0x55e85f9ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- 172.21.15.3:0/3425666816 shutdown_connections 2024-09-12T12:24:53.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- 172.21.15.3:0/3425666816 wait complete. 2024-09-12T12:24:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 -- start start 2024-09-12T12:24:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.935+0000 7fea5ead8000 1 Processor -- start 2024-09-12T12:24:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85fa06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85fa21960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85fa21ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e85f7f6f80 con 0x55e85f9f2b20 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e85f7afca0 con 0x55e85f9ae390 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55e85f7fc2a0 con 0x55e85f9a9290 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 10 client.0 _ll_register_callbacks cb 0x7fea48001430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85fa21ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85fa21ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56850/0 (socket says 172.21.15.3:56850) 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 -- 172.21.15.3:0/2462795357 learned_addr learned my addr 172.21.15.3:0/2462795357 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea597b8640 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85fa06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 -- 172.21.15.3:0/2462795357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 msgr2=0x55e85fa06120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85fa06120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 -- 172.21.15.3:0/2462795357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 msgr2=0x55e85fa21960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:53.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85fa21960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 -- 172.21.15.3:0/2462795357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85f87a750 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea59fb9640 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85fa21ea0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fea4c01a630 tx=0x7fea4c0f1290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d89553ef0529338 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 1 -- 172.21.15.3:0/2462795357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c11d030 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 1 -- 172.21.15.3:0/2462795357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fea4c11f2c0 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea50016d30 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 1 -- 172.21.15.3:0/2462795357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fea4c0f1690 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 10 client.6891 fetch_fsmap learned FSMap version 62 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55e85f785a30 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 1 -- 172.21.15.3:0/2462795357 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea50016d30 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 1 -- 172.21.15.3:0/2462795357 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 62) ==== 3502+0+0 (secure 0 0 0) 0x7fea4c0b53a0 con 0x55e85f9f2b20 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 10 client.6891 fetch_fsmap finished waiting for FSMap version 62 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 10 client.6891 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:24:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 20 client.6891 populate_metadata read hostname 'smithi003' 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85f9f3440 0x55e85fa223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 4 client.6891 mds_command: new command op to 6738 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 --> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] -- command(tid 0: {"prefix": "status"}) -- 0x55e85f8053a0 con 0x55e85f9f3440 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea597b8640 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85f9f3440 0x55e85fa223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea597b8640 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85f9f3440 0x55e85fa223e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327ef1fa8e85d901 server_cookie=98755a2f9f1509ce in_seq=0 out_seq=0 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 10 client.6891 ms_handle_connect on v2:172.21.15.3:6833/2492687290 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 1 -- 172.21.15.3:0/2462795357 <== mds.0 v2:172.21.15.3:6833/2492687290 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55e85f8053a0 con 0x55e85f9f3440 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 10 client.6891 handle_command_reply: tid=0 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 client.6891 shutdown 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 msgr2=0x55e85fa21ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85fa21ea0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fea4c01a630 tx=0x7fea4c0f1290 comp rx=0 tx=0).stop 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 shutdown_connections 2024-09-12T12:24:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:6833/2492687290,v1:172.21.15.3:6835/2492687290] conn(0x55e85f9f3440 0x55e85fa223e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e85f9f2b20 0x55e85fa21ea0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e85f9ae390 0x55e85fa21960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea33fff640 0 client.6891 ms_handle_reset on v2:172.21.15.3:6833/2492687290 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 --2- 172.21.15.3:0/2462795357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55e85f9a9290 0x55e85fa06120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 >> 172.21.15.3:0/2462795357 conn(0x55e85f9fe270 msgr2=0x55e85f9f0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 shutdown_connections 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 1 -- 172.21.15.3:0/2462795357 wait complete. 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ead8000 20 client.6891 trim_cache size 0 max 16384 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:24:53.949 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6738, 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.272213121, 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 62, 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 93, 2024-09-12T12:24:53.950 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 93, 2024-09-12T12:24:53.951 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.672535008000001 2024-09-12T12:24:53.951 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:24:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea380678c0 msgr2=0x7fea38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea380678c0 0x7fea38069d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea48000f70 tx=0x7fea48009040 comp rx=0 tx=0).stop 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 msgr2=0x7fea540cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea540cb740 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0af040 tx=0x7fea4c0197a0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 shutdown_connections 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fea380678c0 0x7fea38069d80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea5415b620 0x7fea540cb740 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fea540d05b0 0x7fea540cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 --2- 172.21.15.3:0/2941443746 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fea540ae250 0x7fea54149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 >> 172.21.15.3:0/2941443746 conn(0x7fea54067e00 msgr2=0x7fea5414b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 shutdown_connections 2024-09-12T12:24:53.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:53.939+0000 7fea5ba1c640 1 -- 172.21.15.3:0/2941443746 wait complete. 2024-09-12T12:24:53.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:24:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242fc96640 1 Processor -- start 2024-09-12T12:24:54.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242fc96640 1 -- start start 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242fc96640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f24280cb870 0x7f24280cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242fc96640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24280cc190 con 0x7f24280b0200 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242fc96640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242815f970 con 0x7f242806fa80 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242fc96640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f242806de90 con 0x7f24280cb870 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242e233640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f24280b0200 0x7f242806eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35296/0 (socket says 172.21.15.3:35296) 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242e233640 1 -- 172.21.15.3:0/3258537135 learned_addr learned my addr 172.21.15.3:0/3258537135 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242da32640 1 --2- 172.21.15.3:0/3258537135 >> v2:172.21.15.142:3300/0 conn(0x7f24280cb870 0x7f24280cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 237361737 0 0) 0x7f24280cc190 con 0x7f24280b0200 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2414003530 con 0x7f24280b0200 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 544819022 0 0) 0x7f2414003530 con 0x7f24280b0200 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 >> v2:172.21.15.142:3300/0 conn(0x7f24280cb870 msgr2=0x7f24280cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 --2- 172.21.15.3:0/3258537135 >> v2:172.21.15.142:3300/0 conn(0x7f24280cb870 0x7f24280cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 >> v1:172.21.15.142:6789/0 conn(0x7f242806fa80 legacy=0x7f242806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242815fc50 con 0x7f24280b0200 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f2420001a00 con 0x7f24280b0200 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 >> v1:172.21.15.3:6789/0 conn(0x7f24280b0200 legacy=0x7f242806eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140039c0 0x7f2414003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f2414004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140051d0 0x7f241400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f242806de90 con 0x7f2414004810 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f241400c170 con 0x7f24140039c0 2024-09-12T12:24:54.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f241400c370 con 0x7f24140051d0 2024-09-12T12:24:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242d231640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f2414004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242d231640 1 -- 172.21.15.3:0/3258537135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140051d0 msgr2=0x7f241400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242d231640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24140051d0 0x7f241400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242d231640 1 -- 172.21.15.3:0/3258537135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140039c0 msgr2=0x7f2414003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242d231640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24140039c0 0x7f2414003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242e233640 1 -- 172.21.15.3:0/3258537135 reap_dead start 2024-09-12T12:24:54.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.071+0000 7f242d231640 1 -- 172.21.15.3:0/3258537135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2414003530 con 0x7f2414004810 2024-09-12T12:24:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242d231640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f2414004c90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2418001780 tx=0x7f241801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1ea434fae5626c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180013a0 con 0x7f2414004810 2024-09-12T12:24:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24180052f0 con 0x7f2414004810 2024-09-12T12:24:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242ca30640 1 -- 172.21.15.3:0/3258537135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418003840 con 0x7f2414004810 2024-09-12T12:24:54.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 -- 172.21.15.3:0/3258537135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 msgr2=0x7f2414004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f2414004c90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2418001780 tx=0x7f241801c400 comp rx=0 tx=0).stop 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 -- 172.21.15.3:0/3258537135 shutdown_connections 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 --2- 172.21.15.3:0/3258537135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f2414004c90 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 -- 172.21.15.3:0/3258537135 >> 172.21.15.3:0/3258537135 conn(0x7f24280d1220 msgr2=0x7f24280d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 -- 172.21.15.3:0/3258537135 shutdown_connections 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 -- 172.21.15.3:0/3258537135 wait complete. 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 Processor -- start 2024-09-12T12:24:54.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.075+0000 7f242fc96640 1 -- start start 2024-09-12T12:24:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f24280c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24281fbba0 0x7f24281faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2428073470 0x7f24281fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24280b0ad0 con 0x7f2414004810 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f242806f0d0 con 0x7f2428073470 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f24280c4570 con 0x7f24281fbba0 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f24280c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f24280c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56856/0 (socket says 172.21.15.3:56856) 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 -- 172.21.15.3:0/1199316136 learned_addr learned my addr 172.21.15.3:0/1199316136 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242e233640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2428073470 0x7f24281fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242d231640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24281fbba0 0x7f24281faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 -- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24281fbba0 msgr2=0x7f24281faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24281fbba0 0x7f24281faf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 -- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2428073470 msgr2=0x7f24281fb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2428073470 0x7f24281fb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 -- 172.21.15.3:0/1199316136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242809bde0 con 0x7f2414004810 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242da32640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f24280c4030 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f242400e7a0 tx=0x7f242400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f241e7fc640 1 -- 172.21.15.3:0/1199316136 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424017070 con 0x7f2414004810 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f241e7fc640 1 -- 172.21.15.3:0/1199316136 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242400c8c0 con 0x7f2414004810 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f241e7fc640 1 -- 172.21.15.3:0/1199316136 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242400cc90 con 0x7f2414004810 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2428072990 con 0x7f2414004810 2024-09-12T12:24:54.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242809ba70 con 0x7f2414004810 2024-09-12T12:24:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24280cc060 con 0x7f2414004810 2024-09-12T12:24:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f241e7fc640 1 -- 172.21.15.3:0/1199316136 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f242401a070 con 0x7f2414004810 2024-09-12T12:24:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f241e7fc640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f240c0678f0 0x7f240c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.079+0000 7f241e7fc640 1 -- 172.21.15.3:0/1199316136 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f2424085eb0 con 0x7f2414004810 2024-09-12T12:24:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.083+0000 7f242d231640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f240c0678f0 0x7f240c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.083+0000 7f242d231640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f240c0678f0 0x7f240c069db0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2418001750 tx=0x7f24180011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.087+0000 7f241e7fc640 1 -- 172.21.15.3:0/1199316136 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f24240569f0 con 0x7f2414004810 2024-09-12T12:24:54.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24280c46f0 con 0x7f2414004810 2024-09-12T12:24:54.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f241e7fc640 1 -- 172.21.15.3:0/1199316136 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f24240562f0 con 0x7f2414004810 2024-09-12T12:24:54.282 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:24:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f240c0678f0 msgr2=0x7f240c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f240c0678f0 0x7f240c069db0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2418001750 tx=0x7f24180011a0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 msgr2=0x7f24280c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f24280c4030 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f242400e7a0 tx=0x7f242400ec60 comp rx=0 tx=0).stop 2024-09-12T12:24:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 shutdown_connections 2024-09-12T12:24:54.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f240c0678f0 0x7f240c069db0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2428073470 0x7f24281fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f24281fbba0 0x7f24281faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 --2- 172.21.15.3:0/1199316136 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2414004810 0x7f24280c4030 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 >> 172.21.15.3:0/1199316136 conn(0x7f24280d1220 msgr2=0x7f242815ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 shutdown_connections 2024-09-12T12:24:54.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54.279+0000 7f242fc96640 1 -- 172.21.15.3:0/1199316136 wait complete. 2024-09-12T12:24:54.299 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:24:54.299 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:24:54.299 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:24:54.300 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:24:54.300 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:54.300 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:24:54.300 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:24:54.300 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:24:54.300 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:24:54.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:54.300 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:24:54.306 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:24:54.306 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:24:54.306 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:24:54.306 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42192sec preferred_lft 42192sec 2024-09-12T12:24:54.307 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:24:54.308 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:54.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:54.310 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:24:54.359 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:24:54.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:54.361 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:24:54.410 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:24:54.411 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:24:54.411 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:24:54.411 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:24:54.411 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:24:54.411 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:24:54.411 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:24:54.412 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:24:54.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:24:54.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:24:54.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:24:54.412 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:24:54.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:24:54.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:54.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:54.532 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:24:54.538 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:24:54.538 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:24:54.538 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:24:54.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:54.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:24:54.592 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:24:54.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:54.592 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:54.637 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:24:54.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:54.638 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:54.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:24:54.743 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:54.792 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:24:54.792 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:24:54.792 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:24:54.792 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:24:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:54.794 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:54.795 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:54.795 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:54.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:54.796 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:24:54.842 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:24:54.842 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:24:54.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:24:54.903 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:24:54.951 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:24:54.952 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:24:54.953 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:24:54.954 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:54.954 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:54.954 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:54.954 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:54.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:54.955 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:24:54.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:24:54.967+0000 7efdd6932580 -1 init, newargv = 0x5654a6602180 newargc=15 2024-09-12T12:24:54.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[44502]: starting ceph client 2024-09-12T12:24:54.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[44502]: starting fuse 2024-09-12T12:24:56.078 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:24:56.095 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:24:56.144 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:24:56.145 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:24:56.146 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:56.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:56.147 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:24:56.194 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:24:56.194 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:24:56.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:24:56.195 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:24:56.196 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:24:56.321 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.44502.asok 2024-09-12T12:24:56.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:56.333 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44502.asok 2024-09-12T12:24:56.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:56.334 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44502.asok status 2024-09-12T12:24:56.495 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:24:56.495 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:24:56.495 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:24:56.495 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "44502", 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6903, 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:24:56.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6903 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3634001155 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:24:56.497 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3634001155 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.6903 192.168.0.1:0/3634001155", 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3634001155", 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 62, 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 93, 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:24:56.498 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:24:56.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:56.509 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:24:56.509 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:56.516 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:24:56.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:56.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:56.517 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:24:56.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:24:56.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d85d34640 1 Processor -- start 2024-09-12T12:24:56.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d85d34640 1 -- start start 2024-09-12T12:24:56.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d85d34640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d80074c10 0x7f6d800b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:56.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d85d34640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6d800d0180 0x7f6d8006d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:56.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d85d34640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d800b5640 con 0x7f6d80162a80 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d85d34640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6d80162e60 con 0x7f6d80074c10 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d85d34640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6d800b4490 con 0x7f6d800d0180 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d80074c10 0x7f6d800b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6d80074c10 0x7f6d800b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56866/0 (socket says 172.21.15.3:56866) 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6d80162a80 0x7f6d80072c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35302/0 (socket says 172.21.15.3:35302) 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7effd640 1 -- 172.21.15.3:0/1363531324 learned_addr learned my addr 172.21.15.3:0/1363531324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7ffff640 1 --2- 172.21.15.3:0/1363531324 >> v2:172.21.15.142:3300/0 conn(0x7f6d800d0180 0x7f6d8006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7effd640 1 -- 172.21.15.3:0/1363531324 >> v2:172.21.15.142:3300/0 conn(0x7f6d800d0180 msgr2=0x7f6d8006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7effd640 1 --2- 172.21.15.3:0/1363531324 >> v2:172.21.15.142:3300/0 conn(0x7f6d800d0180 0x7f6d8006d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7effd640 1 -- 172.21.15.3:0/1363531324 >> v1:172.21.15.3:6789/0 conn(0x7f6d80162a80 legacy=0x7f6d80072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:56.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7e7fc640 1 -- 172.21.15.3:0/1363531324 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271233451 0 0) 0x7f6d800b5640 con 0x7f6d80162a80 2024-09-12T12:24:56.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.739+0000 7f6d7effd640 1 -- 172.21.15.3:0/1363531324 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d800b5920 con 0x7f6d80074c10 2024-09-12T12:24:56.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d7effd640 1 --2- 172.21.15.3:0/1363531324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d800b4bf0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009870 tx=0x7f6d6801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d7715a43487e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:56.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d7e7fc640 1 -- 172.21.15.3:0/1363531324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d680aa650 con 0x7f6d80074c10 2024-09-12T12:24:56.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d7e7fc640 1 -- 172.21.15.3:0/1363531324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d680af070 con 0x7f6d80074c10 2024-09-12T12:24:56.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d7e7fc640 1 -- 172.21.15.3:0/1363531324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d680aaa10 con 0x7f6d80074c10 2024-09-12T12:24:56.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 -- 172.21.15.3:0/1363531324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 msgr2=0x7f6d800b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:56.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 --2- 172.21.15.3:0/1363531324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d800b4bf0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009870 tx=0x7f6d6801c4f0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 -- 172.21.15.3:0/1363531324 shutdown_connections 2024-09-12T12:24:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 --2- 172.21.15.3:0/1363531324 >> v2:172.21.15.142:3300/0 conn(0x7f6d800d0180 0x7f6d8006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 --2- 172.21.15.3:0/1363531324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d800b4bf0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 -- 172.21.15.3:0/1363531324 >> 172.21.15.3:0/1363531324 conn(0x7f6d800b27b0 msgr2=0x7f6d800b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 -- 172.21.15.3:0/1363531324 shutdown_connections 2024-09-12T12:24:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.743+0000 7f6d85d34640 1 -- 172.21.15.3:0/1363531324 wait complete. 2024-09-12T12:24:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 Processor -- start 2024-09-12T12:24:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 -- start start 2024-09-12T12:24:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d80150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:56.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d800d0180 0x7f6d80151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d80162a80 0x7f6d80151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8016fa70 con 0x7f6d80074c10 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8006e2c0 con 0x7f6d80162a80 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8016fbf0 con 0x7f6d800d0180 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d80150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d80150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56868/0 (socket says 172.21.15.3:56868) 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 -- 172.21.15.3:0/2324396999 learned_addr learned my addr 172.21.15.3:0/2324396999 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 -- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d800d0180 msgr2=0x7f6d80151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d800d0180 0x7f6d80151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 -- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d80162a80 msgr2=0x7f6d80151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d80162a80 0x7f6d80151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 -- 172.21.15.3:0/2324396999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d80151dc0 con 0x7f6d80074c10 2024-09-12T12:24:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d7f7fe640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d80150e00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c7c0 tx=0x7f6d7000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d77fff640 1 -- 172.21.15.3:0/2324396999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70018070 con 0x7f6d80074c10 2024-09-12T12:24:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d77fff640 1 -- 172.21.15.3:0/2324396999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7000a640 con 0x7f6d80074c10 2024-09-12T12:24:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d800aa720 con 0x7f6d80074c10 2024-09-12T12:24:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d77fff640 1 -- 172.21.15.3:0/2324396999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7000aa30 con 0x7f6d80074c10 2024-09-12T12:24:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d80156840 con 0x7f6d80074c10 2024-09-12T12:24:56.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d800b4bf0 con 0x7f6d80074c10 2024-09-12T12:24:56.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d77fff640 1 -- 172.21.15.3:0/2324396999 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d7001b070 con 0x7f6d80074c10 2024-09-12T12:24:56.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d77fff640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d600678c0 0x7f6d60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:56.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.747+0000 7f6d77fff640 1 -- 172.21.15.3:0/2324396999 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f6d700855b0 con 0x7f6d80074c10 2024-09-12T12:24:56.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.751+0000 7f6d7effd640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d600678c0 0x7f6d60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:56.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.751+0000 7f6d7effd640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d600678c0 0x7f6d60069d80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6d680ae000 tx=0x7f6d68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:56.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.751+0000 7f6d77fff640 1 -- 172.21.15.3:0/2324396999 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6d700561d0 con 0x7f6d80074c10 2024-09-12T12:24:56.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.951+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d800b3510 con 0x7f6d80074c10 2024-09-12T12:24:56.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d77fff640 1 -- 172.21.15.3:0/2324396999 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f6d70055ad0 con 0x7f6d80074c10 2024-09-12T12:24:56.957 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d600678c0 msgr2=0x7f6d60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d600678c0 0x7f6d60069d80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6d680ae000 tx=0x7f6d68008040 comp rx=0 tx=0).stop 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 msgr2=0x7f6d80150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d80150e00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000c7c0 tx=0x7f6d7000cc80 comp rx=0 tx=0).stop 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 shutdown_connections 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6d600678c0 0x7f6d60069d80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d80162a80 0x7f6d80151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6d800d0180 0x7f6d80151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 --2- 172.21.15.3:0/2324396999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d80074c10 0x7f6d80150e00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:56.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 >> 172.21.15.3:0/2324396999 conn(0x7f6d800b27b0 msgr2=0x7f6d800b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:56.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 shutdown_connections 2024-09-12T12:24:56.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:56.955+0000 7f6d85d34640 1 -- 172.21.15.3:0/2324396999 wait complete. 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:24:56.977 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:24:56.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:56.978 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:24:56.985 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:24:56.985 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:24:56.985 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:24:56.986 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.986 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:24:56.986 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.986 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:56.986 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:56.986 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:24:56.987 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:56.987 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:56.987 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:24:56.987 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:24:56.987 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:56.987 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:24:56.988 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:56.988 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:24:56.988 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42187sec preferred_lft 42187sec 2024-09-12T12:24:56.988 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:24:56.988 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.988 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:24:56.989 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:56.989 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:24:56.989 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.989 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:24:56.989 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:24:56.990 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:24:56.990 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.990 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:24:56.990 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.990 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:56.990 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:56.991 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:24:56.991 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.991 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:24:56.991 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:56.991 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:24:56.991 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:24:56.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:56.992 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:24:57.045 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:24:57.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:57.048 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:24:57.100 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:24:57.100 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:24:57.100 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:24:57.101 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:24:57.101 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:24:57.101 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:24:57.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:24:57.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:24:57.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:24:57.102 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:24:57.188 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:24:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:24:57.216 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:24:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:57.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:57.224 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:24:57.229 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:24:57.230 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:24:57.230 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:24:57.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:57.230 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:24:57.281 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:24:57.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:57.282 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:57.327 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:24:57.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:24:57.327 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:57.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:24:57.433 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:57.481 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:24:57.481 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:24:57.481 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:24:57.482 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:24:57.483 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:24:57.484 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:24:57.484 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:24:57.484 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:57.484 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:57.484 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:57.484 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:57.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:57.485 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:24:57.531 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:24:57.531 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:24:57.576 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:24:57.590 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:57.639 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:24:57.639 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:24:57.640 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:24:57.641 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:24:57.642 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:57.642 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:57.642 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:57.642 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:57.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:57.643 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:24:57.655 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:24:57.655+0000 7fd6e9f64580 -1 init, newargv = 0x55c97c313130 newargc=15 2024-09-12T12:24:57.655 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36579]: starting ceph client 2024-09-12T12:24:57.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36579]: starting fuse 2024-09-12T12:24:58.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:24:58.777 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:24:58.829 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:24:58.830 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:24:58.831 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:24:58.832 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:24:58.832 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:24:58.832 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:58.832 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:24:58.832 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:58.832 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:24:58.832 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:58.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:58.833 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:24:58.879 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:24:58.879 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:58.880 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:24:58.881 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:24:59.004 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.36579.asok 2024-09-12T12:24:59.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:24:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:24:59.016 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36579.asok 2024-09-12T12:24:59.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:59.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36579.asok status 2024-09-12T12:24:59.172 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "36579", 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 6121, 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:24:59.173 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 6121 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1118237767 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:24:59.174 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 1118237767 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.6121 192.168.0.1:0/1118237767", 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/1118237767", 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 62, 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 93, 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:24:59.175 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:24:59.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:59.186 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:24:59.186 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:59.192 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:24:59.192 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:59.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:24:59.193 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:24:59.246 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:59.246 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:59.253 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:24:59.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.431+0000 7fccd0d4c640 1 Processor -- start 2024-09-12T12:24:59.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.431+0000 7fccd0d4c640 1 -- start start 2024-09-12T12:24:59.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccd0d4c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcccc0af410 0x7fcccc15ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccd0d4c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcccc15ef90 0x7fcccc0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccd0d4c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcccc15f430 con 0x7fcccc15c000 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccd0d4c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcccc15f9f0 con 0x7fcccc0af410 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccd0d4c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fcccc15f710 con 0x7fcccc15ef90 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccc9d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcccc0af410 0x7fcccc15ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccca59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcccc15c000 0x7fcccc0b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55366/0 (socket says 172.21.15.3:55366) 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccca59c640 1 -- 172.21.15.3:0/1359510959 learned_addr learned my addr 172.21.15.3:0/1359510959 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccc9d9b640 1 -- 172.21.15.3:0/1359510959 >> v2:172.21.15.142:3300/0 conn(0x7fcccc15ef90 msgr2=0x7fcccc0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fcccad9d640 1 --2- 172.21.15.3:0/1359510959 >> v2:172.21.15.142:3300/0 conn(0x7fcccc15ef90 0x7fcccc0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccc9d9b640 1 --2- 172.21.15.3:0/1359510959 >> v2:172.21.15.142:3300/0 conn(0x7fcccc15ef90 0x7fcccc0d05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccc959a640 1 -- 172.21.15.3:0/1359510959 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138336097 0 0) 0x7fcccc15f430 con 0x7fcccc15c000 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccc9d9b640 1 -- 172.21.15.3:0/1359510959 >> v1:172.21.15.3:6789/0 conn(0x7fcccc15c000 legacy=0x7fcccc0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccc9d9b640 1 -- 172.21.15.3:0/1359510959 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc0b3210 con 0x7fcccc0af410 2024-09-12T12:24:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.435+0000 7fccc9d9b640 1 --2- 172.21.15.3:0/1359510959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc0af410 0x7fcccc15ea50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009d90 tx=0x7fccb401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b3705c6e5e792f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccc959a640 1 -- 172.21.15.3:0/1359510959 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb40b2070 con 0x7fcccc0af410 2024-09-12T12:24:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccc959a640 1 -- 172.21.15.3:0/1359510959 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb40a5840 con 0x7fcccc0af410 2024-09-12T12:24:59.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccc959a640 1 -- 172.21.15.3:0/1359510959 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb40a5c10 con 0x7fcccc0af410 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- 172.21.15.3:0/1359510959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc0af410 msgr2=0x7fcccc15ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/1359510959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc0af410 0x7fcccc15ea50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009d90 tx=0x7fccb401c870 comp rx=0 tx=0).stop 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- 172.21.15.3:0/1359510959 shutdown_connections 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/1359510959 >> v2:172.21.15.142:3300/0 conn(0x7fcccc15ef90 0x7fcccc0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/1359510959 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc0af410 0x7fcccc15ea50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- 172.21.15.3:0/1359510959 >> 172.21.15.3:0/1359510959 conn(0x7fcccc067e00 msgr2=0x7fcccc068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- 172.21.15.3:0/1359510959 shutdown_connections 2024-09-12T12:24:59.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- 172.21.15.3:0/1359510959 wait complete. 2024-09-12T12:24:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 Processor -- start 2024-09-12T12:24:59.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- start start 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcccc0af410 0x7fcccc14f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcccc15c000 0x7fcccc14e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc15ef90 0x7fcccc14da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc06dc00 con 0x7fcccc15ef90 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc16b090 con 0x7fcccc15c000 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcccc14f3e0 con 0x7fcccc0af410 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc15ef90 0x7fcccc14da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc15ef90 0x7fcccc14da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34106/0 (socket says 172.21.15.3:34106) 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 -- 172.21.15.3:0/912351981 learned_addr learned my addr 172.21.15.3:0/912351981 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccc9d9b640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcccc15c000 0x7fcccc14e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccca59c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcccc0af410 0x7fcccc14f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 -- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcccc0af410 msgr2=0x7fcccc14f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcccc0af410 0x7fcccc14f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 -- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcccc15c000 msgr2=0x7fcccc14e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcccc15c000 0x7fcccc14e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 -- 172.21.15.3:0/912351981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc159300 con 0x7fcccc15ef90 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fcccad9d640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc15ef90 0x7fcccc14da30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c5c0 tx=0x7fccc400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccc2ffd640 1 -- 172.21.15.3:0/912351981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4015070 con 0x7fcccc15ef90 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccc2ffd640 1 -- 172.21.15.3:0/912351981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc4004220 con 0x7fcccc15ef90 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.439+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcccc14e7d0 con 0x7fcccc15ef90 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccc2ffd640 1 -- 172.21.15.3:0/912351981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4003620 con 0x7fcccc15ef90 2024-09-12T12:24:59.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcccc14ed90 con 0x7fcccc15ef90 2024-09-12T12:24:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcccc15ea50 con 0x7fcccc15ef90 2024-09-12T12:24:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccc2ffd640 1 -- 172.21.15.3:0/912351981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fccc40037c0 con 0x7fcccc15ef90 2024-09-12T12:24:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccc2ffd640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccac0678c0 0x7fccac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccc2ffd640 1 -- 172.21.15.3:0/912351981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fccc4085f40 con 0x7fcccc15ef90 2024-09-12T12:24:59.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccca59c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccac0678c0 0x7fccac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.443+0000 7fccca59c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccac0678c0 0x7fccac069d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccbc000f70 tx=0x7fccbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:59.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.447+0000 7fccc2ffd640 1 -- 172.21.15.3:0/912351981 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fccc4056a70 con 0x7fcccc15ef90 2024-09-12T12:24:59.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.631+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcccc073cb0 con 0x7fcccc15ef90 2024-09-12T12:24:59.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.631+0000 7fccc2ffd640 1 -- 172.21.15.3:0/912351981 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7fccc4056370 con 0x7fcccc15ef90 2024-09-12T12:24:59.634 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:24:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.631+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccac0678c0 msgr2=0x7fccac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.631+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccac0678c0 0x7fccac069d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccbc000f70 tx=0x7fccbc009040 comp rx=0 tx=0).stop 2024-09-12T12:24:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc15ef90 msgr2=0x7fcccc14da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc15ef90 0x7fcccc14da30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fccc400c5c0 tx=0x7fccc400ca80 comp rx=0 tx=0).stop 2024-09-12T12:24:59.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 shutdown_connections 2024-09-12T12:24:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fccac0678c0 0x7fccac069d80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcccc15ef90 0x7fcccc14da30 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcccc15c000 0x7fcccc14e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 --2- 172.21.15.3:0/912351981 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcccc0af410 0x7fcccc14f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 >> 172.21.15.3:0/912351981 conn(0x7fcccc067e00 msgr2=0x7fcccc154580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 shutdown_connections 2024-09-12T12:24:59.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.635+0000 7fccd0d4c640 1 -- 172.21.15.3:0/912351981 wait complete. 2024-09-12T12:24:59.656 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:24:59.656 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:24:59.662 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:24:59.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca7562640 1 Processor -- start 2024-09-12T12:24:59.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca7562640 1 -- start start 2024-09-12T12:24:59.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca7562640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca00b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca7562640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faca00ae880 0x7faca00aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca7562640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca00b5000 con 0x7faca0073cb0 2024-09-12T12:24:59.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca7562640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7faca0074090 con 0x7faca00ae880 2024-09-12T12:24:59.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca7562640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faca00b37c0 con 0x7faca015c710 2024-09-12T12:24:59.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca4afd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca00b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca5aff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faca00ae880 0x7faca00aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca4afd640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca00b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:57572/0 (socket says 172.21.15.3:57572) 2024-09-12T12:24:59.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca4afd640 1 -- 172.21.15.3:0/300785816 learned_addr learned my addr 172.21.15.3:0/300785816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:59.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca5aff640 1 --2- 172.21.15.3:0/300785816 >> v2:172.21.15.142:3300/0 conn(0x7faca00ae880 0x7faca00aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:59120/0 (socket says 172.21.15.3:59120) 2024-09-12T12:24:59.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca5aff640 1 -- 172.21.15.3:0/300785816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 msgr2=0x7faca00b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:59.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca5aff640 1 --2- 172.21.15.3:0/300785816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca00b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca5aff640 1 -- 172.21.15.3:0/300785816 >> v1:172.21.15.142:6789/0 conn(0x7faca0073cb0 legacy=0x7faca00adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.831+0000 7faca5aff640 1 -- 172.21.15.3:0/300785816 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00b3aa0 con 0x7faca00ae880 2024-09-12T12:24:59.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca5aff640 1 --2- 172.21.15.3:0/300785816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca00aecd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac98009d90 tx=0x7fac9801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bf4122d5a1343 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7fac93fff640 1 -- 172.21.15.3:0/300785816 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac980b2070 con 0x7faca00ae880 2024-09-12T12:24:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7fac93fff640 1 -- 172.21.15.3:0/300785816 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac980a5830 con 0x7faca00ae880 2024-09-12T12:24:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7fac93fff640 1 -- 172.21.15.3:0/300785816 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac980a5c00 con 0x7faca00ae880 2024-09-12T12:24:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 -- 172.21.15.3:0/300785816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 msgr2=0x7faca00aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 --2- 172.21.15.3:0/300785816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca00aecd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac98009d90 tx=0x7fac9801c970 comp rx=0 tx=0).stop 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 -- 172.21.15.3:0/300785816 shutdown_connections 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 --2- 172.21.15.3:0/300785816 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca00b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 --2- 172.21.15.3:0/300785816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca00aecd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 -- 172.21.15.3:0/300785816 >> 172.21.15.3:0/300785816 conn(0x7faca0067e00 msgr2=0x7faca0068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 -- 172.21.15.3:0/300785816 shutdown_connections 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.835+0000 7faca7562640 1 -- 172.21.15.3:0/300785816 wait complete. 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 Processor -- start 2024-09-12T12:24:59.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 -- start start 2024-09-12T12:24:59.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faca0073cb0 0x7faca0152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca0150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faca0168f40 con 0x7faca0073cb0 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faca0168dc0 con 0x7faca00ae880 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7faca01690c0 con 0x7faca015c710 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faca0073cb0 0x7faca0152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faca0073cb0 0x7faca0152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34120/0 (socket says 172.21.15.3:34120) 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 -- 172.21.15.3:0/4030277159 learned_addr learned my addr 172.21.15.3:0/4030277159 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca4afd640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca5aff640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca0150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 -- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 msgr2=0x7faca0150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca0150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 -- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 msgr2=0x7faca014e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:59.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca014e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 -- 172.21.15.3:0/4030277159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00aff40 con 0x7faca0073cb0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca4afd640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca52fe640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faca0073cb0 0x7faca0152ab0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c450 tx=0x7fac9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7fac91ffb640 1 -- 172.21.15.3:0/4030277159 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94016020 con 0x7faca0073cb0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7fac91ffb640 1 -- 172.21.15.3:0/4030277159 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac940052d0 con 0x7faca0073cb0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca0157c30 con 0x7faca0073cb0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7fac91ffb640 1 -- 172.21.15.3:0/4030277159 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94007260 con 0x7faca0073cb0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca01581f0 con 0x7faca0073cb0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca00aecd0 con 0x7faca0073cb0 2024-09-12T12:24:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.839+0000 7fac91ffb640 1 -- 172.21.15.3:0/4030277159 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac94007400 con 0x7faca0073cb0 2024-09-12T12:24:59.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.843+0000 7fac91ffb640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac840678c0 0x7fac84069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:59.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.843+0000 7fac91ffb640 1 -- 172.21.15.3:0/4030277159 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fac94085530 con 0x7faca0073cb0 2024-09-12T12:24:59.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.843+0000 7faca4afd640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac840678c0 0x7fac84069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:59.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.843+0000 7faca4afd640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac840678c0 0x7fac84069d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faca014ee90 tx=0x7fac9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:59.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:24:59.847+0000 7fac91ffb640 1 -- 172.21.15.3:0/4030277159 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fac94020020 con 0x7faca0073cb0 2024-09-12T12:25:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.023+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faca015a5e0 con 0x7faca0073cb0 2024-09-12T12:25:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.023+0000 7fac91ffb640 1 -- 172.21.15.3:0/4030277159 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7fac94005480 con 0x7faca0073cb0 2024-09-12T12:25:00.027 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:25:00.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac840678c0 msgr2=0x7fac84069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:00.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac840678c0 0x7fac84069d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faca014ee90 tx=0x7fac9c008040 comp rx=0 tx=0).stop 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faca0073cb0 msgr2=0x7faca0152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faca0073cb0 0x7faca0152ab0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fac9400c450 tx=0x7fac9400c910 comp rx=0 tx=0).stop 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 shutdown_connections 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fac840678c0 0x7fac84069d80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faca015c710 0x7faca0150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faca00ae880 0x7faca014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 --2- 172.21.15.3:0/4030277159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faca0073cb0 0x7faca0152ab0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 >> 172.21.15.3:0/4030277159 conn(0x7faca0067e00 msgr2=0x7faca00af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 shutdown_connections 2024-09-12T12:25:00.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:00.027+0000 7faca7562640 1 -- 172.21.15.3:0/4030277159 wait complete. 2024-09-12T12:25:00.047 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:25:00.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:00.048 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:00.058 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[44502]: fuse finished with error 0 and tester_r 0 2024-09-12T12:25:00.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:06.162 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:25:06.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:06.162 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:25:06.170 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:06.170 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:06.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:25:06.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.343+0000 7fe52f455640 1 Processor -- start 2024-09-12T12:25:06.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.343+0000 7fe52f455640 1 -- start start 2024-09-12T12:25:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52f455640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe5280d1190 0x7fe5280d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52f455640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5280d1570 0x7fe5280cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52f455640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280cf950 0x7fe5280b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52f455640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe5280d3cc0 con 0x7fe5280d1570 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52f455640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe5280d30b0 con 0x7fe5280d1190 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52f455640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5280d3390 con 0x7fe5280cf950 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5280d1570 0x7fe5280cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe5280d1570 0x7fe5280cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34134/0 (socket says 172.21.15.3:34134) 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 -- 172.21.15.3:0/2212250694 learned_addr learned my addr 172.21.15.3:0/2212250694 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52d1f1640 1 --2- 172.21.15.3:0/2212250694 >> v2:172.21.15.142:3300/0 conn(0x7fe5280d1190 0x7fe5280d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 -- 172.21.15.3:0/2212250694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280cf950 msgr2=0x7fe5280b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52d9f2640 1 --2- 172.21.15.3:0/2212250694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280cf950 0x7fe5280b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 --2- 172.21.15.3:0/2212250694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280cf950 0x7fe5280b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 -- 172.21.15.3:0/2212250694 >> v2:172.21.15.142:3300/0 conn(0x7fe5280d1190 msgr2=0x7fe5280d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 --2- 172.21.15.3:0/2212250694 >> v2:172.21.15.142:3300/0 conn(0x7fe5280d1190 0x7fe5280d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 -- 172.21.15.3:0/2212250694 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5280d3670 con 0x7fe5280d1570 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52d9f2640 1 --2- 172.21.15.3:0/2212250694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280cf950 0x7fe5280b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:06.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52d1f1640 1 --2- 172.21.15.3:0/2212250694 >> v2:172.21.15.142:3300/0 conn(0x7fe5280d1190 0x7fe5280d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:06.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe52c9f0640 1 --2- 172.21.15.3:0/2212250694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe5280cf410 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe52806d9e0 tx=0x7fe51801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb854b49ea29484 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:06.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe51ffff640 1 -- 172.21.15.3:0/2212250694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180b5070 con 0x7fe5280d1570 2024-09-12T12:25:06.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe51ffff640 1 -- 172.21.15.3:0/2212250694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5180a89d0 con 0x7fe5280d1570 2024-09-12T12:25:06.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.347+0000 7fe51ffff640 1 -- 172.21.15.3:0/2212250694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5180a8da0 con 0x7fe5280d1570 2024-09-12T12:25:06.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- 172.21.15.3:0/2212250694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 msgr2=0x7fe5280cf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 --2- 172.21.15.3:0/2212250694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe5280cf410 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe52806d9e0 tx=0x7fe51801ed10 comp rx=0 tx=0).stop 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- 172.21.15.3:0/2212250694 shutdown_connections 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 --2- 172.21.15.3:0/2212250694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280cf950 0x7fe5280b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 --2- 172.21.15.3:0/2212250694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe5280cf410 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 --2- 172.21.15.3:0/2212250694 >> v2:172.21.15.142:3300/0 conn(0x7fe5280d1190 0x7fe5280d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- 172.21.15.3:0/2212250694 >> 172.21.15.3:0/2212250694 conn(0x7fe528067a00 msgr2=0x7fe528067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- 172.21.15.3:0/2212250694 shutdown_connections 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- 172.21.15.3:0/2212250694 wait complete. 2024-09-12T12:25:06.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 Processor -- start 2024-09-12T12:25:06.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- start start 2024-09-12T12:25:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5280cf950 0x7fe5281f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280d1190 0x7fe5280ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe52815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280d2cd0 con 0x7fe5280d1570 2024-09-12T12:25:06.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280d1a00 con 0x7fe5280cf950 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe5280d2e50 con 0x7fe5280d1190 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe52815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe52815ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34136/0 (socket says 172.21.15.3:34136) 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 -- 172.21.15.3:0/2096295074 learned_addr learned my addr 172.21.15.3:0/2096295074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 -- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280d1190 msgr2=0x7fe5280ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52c9f0640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280d1190 0x7fe5280ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d1f1640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5280cf950 0x7fe5281f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280d1190 0x7fe5280ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 -- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5280cf950 msgr2=0x7fe5281f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5280cf950 0x7fe5281f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 -- 172.21.15.3:0/2096295074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5281618a0 con 0x7fe5280d1570 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52d9f2640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe52815ea70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe52000cef0 tx=0x7fe52000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:06.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe51dffb640 1 -- 172.21.15.3:0/2096295074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52001a070 con 0x7fe5280d1570 2024-09-12T12:25:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe51dffb640 1 -- 172.21.15.3:0/2096295074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52000d840 con 0x7fe5280d1570 2024-09-12T12:25:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe51dffb640 1 -- 172.21.15.3:0/2096295074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52000dc30 con 0x7fe5280d1570 2024-09-12T12:25:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5280c9b80 con 0x7fe5280d1570 2024-09-12T12:25:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.351+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5280ca140 con 0x7fe5280d1570 2024-09-12T12:25:06.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.355+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe52815e860 con 0x7fe5280d1570 2024-09-12T12:25:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.355+0000 7fe51dffb640 1 -- 172.21.15.3:0/2096295074 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe52001d070 con 0x7fe5280d1570 2024-09-12T12:25:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.355+0000 7fe51dffb640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe50c0678c0 0x7fe50c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.355+0000 7fe51dffb640 1 -- 172.21.15.3:0/2096295074 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fe5200864c0 con 0x7fe5280d1570 2024-09-12T12:25:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.355+0000 7fe52d1f1640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe50c0678c0 0x7fe50c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.355+0000 7fe52d1f1640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe50c0678c0 0x7fe50c069d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:06.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.359+0000 7fe51dffb640 1 -- 172.21.15.3:0/2096295074 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe520057000 con 0x7fe5280d1570 2024-09-12T12:25:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.539+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe5280ba4c0 con 0x7fe5280d1570 2024-09-12T12:25:06.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.539+0000 7fe51dffb640 1 -- 172.21.15.3:0/2096295074 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7fe520056900 con 0x7fe5280d1570 2024-09-12T12:25:06.545 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:25:06.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe50c0678c0 msgr2=0x7fe50c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe50c0678c0 0x7fe50c069d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).stop 2024-09-12T12:25:06.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 msgr2=0x7fe52815ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe52815ea70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe52000cef0 tx=0x7fe52000a710 comp rx=0 tx=0).stop 2024-09-12T12:25:06.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 shutdown_connections 2024-09-12T12:25:06.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe50c0678c0 0x7fe50c069d80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe5280d1570 0x7fe52815ea70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe5280d1190 0x7fe5280ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 --2- 172.21.15.3:0/2096295074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5280cf950 0x7fe5281f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 >> 172.21.15.3:0/2096295074 conn(0x7fe528067a00 msgr2=0x7fe5280d4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 shutdown_connections 2024-09-12T12:25:06.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.543+0000 7fe52f455640 1 -- 172.21.15.3:0/2096295074 wait complete. 2024-09-12T12:25:06.565 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:06.565 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:06.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:25:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2592da9640 1 Processor -- start 2024-09-12T12:25:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2592da9640 1 -- start start 2024-09-12T12:25:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2592da9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f258c160070 0x7f258c1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2592da9640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c0d0000 con 0x7f258c0d2f70 2024-09-12T12:25:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2592da9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c0d02e0 con 0x7f258c15c470 2024-09-12T12:25:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2592da9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f258c1609e0 con 0x7f258c160070 2024-09-12T12:25:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2591346640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f258c160070 0x7f258c1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f258bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f258c0d2f70 0x7f258c0cd920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:55376/0 (socket says 172.21.15.3:55376) 2024-09-12T12:25:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f258bfff640 1 -- 172.21.15.3:0/2987278864 learned_addr learned my addr 172.21.15.3:0/2987278864 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:06.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2591346640 1 -- 172.21.15.3:0/2987278864 >> v1:172.21.15.142:6789/0 conn(0x7f258c15c470 legacy=0x7f258c0d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:06.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2591346640 1 -- 172.21.15.3:0/2987278864 >> v1:172.21.15.3:6789/0 conn(0x7f258c0d2f70 legacy=0x7f258c0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.699+0000 7f2591346640 1 -- 172.21.15.3:0/2987278864 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c0d05c0 con 0x7f258c160070 2024-09-12T12:25:06.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2591346640 1 --2- 172.21.15.3:0/2987278864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c160070 0x7f258c1604a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f257c0097c0 tx=0x7f257c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f141b489dc64977f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f258b7fe640 1 -- 172.21.15.3:0/2987278864 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0b0070 con 0x7f258c160070 2024-09-12T12:25:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f258b7fe640 1 -- 172.21.15.3:0/2987278864 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0ab600 con 0x7f258c160070 2024-09-12T12:25:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f258b7fe640 1 -- 172.21.15.3:0/2987278864 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0ab9d0 con 0x7f258c160070 2024-09-12T12:25:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- 172.21.15.3:0/2987278864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c160070 msgr2=0x7f258c1604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 --2- 172.21.15.3:0/2987278864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c160070 0x7f258c1604a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f257c0097c0 tx=0x7f257c01c460 comp rx=0 tx=0).stop 2024-09-12T12:25:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- 172.21.15.3:0/2987278864 shutdown_connections 2024-09-12T12:25:06.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 --2- 172.21.15.3:0/2987278864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c160070 0x7f258c1604a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- 172.21.15.3:0/2987278864 >> 172.21.15.3:0/2987278864 conn(0x7f258c0d2300 msgr2=0x7f258c0d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- 172.21.15.3:0/2987278864 shutdown_connections 2024-09-12T12:25:06.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- 172.21.15.3:0/2987278864 wait complete. 2024-09-12T12:25:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 Processor -- start 2024-09-12T12:25:06.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- start start 2024-09-12T12:25:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f258c0d2f70 0x7f258c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c15c470 0x7f258c13b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258c160070 0x7f258c0b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0b5d30 con 0x7f258c15c470 2024-09-12T12:25:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0b5bb0 con 0x7f258c0d2f70 2024-09-12T12:25:06.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.703+0000 7f2592da9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f258c16a3c0 con 0x7f258c160070 2024-09-12T12:25:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c15c470 0x7f258c13b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c15c470 0x7f258c13b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34152/0 (socket says 172.21.15.3:34152) 2024-09-12T12:25:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 -- 172.21.15.3:0/541846372 learned_addr learned my addr 172.21.15.3:0/541846372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f2590b45640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f258c0d2f70 0x7f258c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f2591346640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258c160070 0x7f258c0b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 -- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258c160070 msgr2=0x7f258c0b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258c160070 0x7f258c0b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 -- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f258c0d2f70 msgr2=0x7f258c152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f258c0d2f70 0x7f258c152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 -- 172.21.15.3:0/541846372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c13c3d0 con 0x7f258c15c470 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f2590b45640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f258c0d2f70 0x7f258c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f258bfff640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c15c470 0x7f258c13b9b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f257400c450 tx=0x7f257400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f25897fa640 1 -- 172.21.15.3:0/541846372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574017070 con 0x7f258c15c470 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f25897fa640 1 -- 172.21.15.3:0/541846372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2574005cf0 con 0x7f258c15c470 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f25897fa640 1 -- 172.21.15.3:0/541846372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25740072f0 con 0x7f258c15c470 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c13c6b0 con 0x7f258c15c470 2024-09-12T12:25:06.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1528b0 con 0x7f258c15c470 2024-09-12T12:25:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c1604a0 con 0x7f258c15c470 2024-09-12T12:25:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f25897fa640 1 -- 172.21.15.3:0/541846372 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f257401a070 con 0x7f258c15c470 2024-09-12T12:25:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f25897fa640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f256c0678f0 0x7f256c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f25897fa640 1 -- 172.21.15.3:0/541846372 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f2574084f90 con 0x7f258c15c470 2024-09-12T12:25:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.707+0000 7f2590b45640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f256c0678f0 0x7f256c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.711+0000 7f2590b45640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f256c0678f0 0x7f256c069db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f25840045c0 tx=0x7f2584009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:06.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.711+0000 7f25897fa640 1 -- 172.21.15.3:0/541846372 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2574055480 con 0x7f258c15c470 2024-09-12T12:25:06.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.887+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f258c0b8280 con 0x7f258c15c470 2024-09-12T12:25:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f25897fa640 1 -- 172.21.15.3:0/541846372 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f2574059780 con 0x7f258c15c470 2024-09-12T12:25:06.893 INFO:teuthology.orchestra.run.smithi003.stderr:listed 0 entries 2024-09-12T12:25:06.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f256c0678f0 msgr2=0x7f256c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f256c0678f0 0x7f256c069db0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f25840045c0 tx=0x7f2584009040 comp rx=0 tx=0).stop 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c15c470 msgr2=0x7f258c13b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c15c470 0x7f258c13b9b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f257400c450 tx=0x7f257400c910 comp rx=0 tx=0).stop 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 shutdown_connections 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f256c0678f0 0x7f256c069db0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f258c160070 0x7f258c0b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f258c15c470 0x7f258c13b9b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 --2- 172.21.15.3:0/541846372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f258c0d2f70 0x7f258c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 >> 172.21.15.3:0/541846372 conn(0x7f258c0d2300 msgr2=0x7f258c0b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 shutdown_connections 2024-09-12T12:25:06.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:06.891+0000 7f2592da9640 1 -- 172.21.15.3:0/541846372 wait complete. 2024-09-12T12:25:06.913 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:25:06.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:06.914 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:06.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36579]: fuse finished with error 0 and tester_r 0 2024-09-12T12:25:06.928 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:13.030 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:25:13.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:13.030 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:25:13.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:25:13.041 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:25:13.085 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T12:25:13.085 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-12T12:25:13.085 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.086 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:25:13.087 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-12T12:25:13.088 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-12T12:25:13.089 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-12T12:25:13.090 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:25:13.091 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-12T12:25:13.092 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.093 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-12T12:25:13.094 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-12T12:25:13.095 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-12T12:25:13.096 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-12T12:25:13.097 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:25:13.097 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:25:13.097 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:25:13.097 INFO:teuthology.orchestra.run.smithi003.stdout: mds_sessionmap_keys_per_op = 1 2024-09-12T12:25:13.097 INFO:teuthology.orchestra.run.smithi003.stdout:[client.badguy] 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.114 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.115 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.116 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-09-12T12:25:13.117 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.118 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-09-12T12:25:13.119 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-09-12T12:25:13.120 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:25:13.121 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:25:13.122 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on pool no app = False 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:25:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:25:13.124 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout: mds_sessionmap_keys_per_op = 1 2024-09-12T12:25:13.125 INFO:teuthology.orchestra.run.smithi142.stdout:[client.badguy] 2024-09-12T12:25:13.127 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:25:13.127 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:13.127 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:25:13.127 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:13.127 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:25:13.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:13.128 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:25:13.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:13.153 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:25:13.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-09-12T12:25:13.154 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:25:13.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-09-12T12:25:13.182 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:25:13.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-09-12T12:25:13.183 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:25:13.184 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-09-12T12:25:13.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1fdd9640 1 Processor -- start 2024-09-12T12:25:13.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1fdd9640 1 -- start start 2024-09-12T12:25:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1fdd9640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a18158db0 0x7f3a18162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1fdd9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a180626e0 0x7f3a18062b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1fdd9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a180b0f30 con 0x7f3a180b4590 2024-09-12T12:25:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1fdd9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3a180b0980 con 0x7f3a180626e0 2024-09-12T12:25:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1fdd9640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f3a180b0c50 con 0x7f3a18158db0 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1e376640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a180626e0 0x7f3a18062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1e376640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3a180626e0 0x7f3a18062b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55512/0 (socket says 172.21.15.3:55512) 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1e376640 1 -- 172.21.15.3:0/3193013322 learned_addr learned my addr 172.21.15.3:0/3193013322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1db75640 1 --2- 172.21.15.3:0/3193013322 >> v2:172.21.15.142:3300/0 conn(0x7f3a18158db0 0x7f3a18162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1e376640 1 -- 172.21.15.3:0/3193013322 >> v2:172.21.15.142:3300/0 conn(0x7f3a18158db0 msgr2=0x7f3a18162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1e376640 1 --2- 172.21.15.3:0/3193013322 >> v2:172.21.15.142:3300/0 conn(0x7f3a18158db0 0x7f3a18162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1e376640 1 -- 172.21.15.3:0/3193013322 >> v1:172.21.15.142:6789/0 conn(0x7f3a180b4590 legacy=0x7f3a180b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7f3a1e376640 1 -- 172.21.15.3:0/3193013322 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a18063040 con 0x7f3a180626e0 2024-09-12T12:25:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7fd760c38640 1 Processor -- start 2024-09-12T12:25:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7fd760c38640 1 -- start start 2024-09-12T12:25:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7fd2bea69640 1 Processor -- start 2024-09-12T12:25:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.315+0000 7fd2bea69640 1 -- start start 2024-09-12T12:25:13.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1e376640 1 --2- 172.21.15.3:0/3193013322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a18062b00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3a180d2480 tx=0x7f3a1001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96585a4e2bc2abf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd760c38640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd75c15b3f0 0x7fd75c0d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd760c38640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd75c071860 con 0x7fd75c06df20 2024-09-12T12:25:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd760c38640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd75c06d9c0 con 0x7fd75c06eb80 2024-09-12T12:25:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd760c38640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd75c071b00 con 0x7fd75c15b3f0 2024-09-12T12:25:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1cb73640 1 -- 172.21.15.3:0/3193013322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a100a5840 con 0x7f3a180626e0 2024-09-12T12:25:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1cb73640 1 -- 172.21.15.3:0/3193013322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a100a59e0 con 0x7f3a180626e0 2024-09-12T12:25:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1cb73640 1 -- 172.21.15.3:0/3193013322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a100a5db0 con 0x7f3a180626e0 2024-09-12T12:25:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd75a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd75c15b3f0 0x7fd75c0d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd75a59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd75c15b3f0 0x7fd75c0d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55524/0 (socket says 172.21.15.3:55524) 2024-09-12T12:25:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd75a59c640 1 -- 172.21.15.3:0/3444569740 learned_addr learned my addr 172.21.15.3:0/3444569740 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2bea69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2b80b0780 0x7fd2b80b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2bea69640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b80b19c0 con 0x7fd2b8062df0 2024-09-12T12:25:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2bea69640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b80b1ca0 con 0x7fd2b815e6c0 2024-09-12T12:25:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2bea69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b80b1770 con 0x7fd2b80b0780 2024-09-12T12:25:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd2b8062df0 0x7fd2b80ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:43034/0 (socket says 172.21.15.3:43034) 2024-09-12T12:25:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 -- 172.21.15.3:0/2889655573 learned_addr learned my addr 172.21.15.3:0/2889655573 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd75a59c640 1 -- 172.21.15.3:0/3444569740 >> v1:172.21.15.142:6789/0 conn(0x7fd75c06eb80 legacy=0x7fd75c072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd75959a640 1 -- 172.21.15.3:0/3444569740 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052295866 0 0) 0x7fd75c071860 con 0x7fd75c06df20 2024-09-12T12:25:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd75a59c640 1 -- 172.21.15.3:0/3444569740 >> v1:172.21.15.3:6789/0 conn(0x7fd75c06df20 legacy=0x7fd75c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd75a59c640 1 -- 172.21.15.3:0/3444569740 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c071de0 con 0x7fd75c15b3f0 2024-09-12T12:25:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699758308 0 0) 0x7fd2b80b19c0 con 0x7fd2b8062df0 2024-09-12T12:25:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2a8003530 con 0x7fd2b8062df0 2024-09-12T12:25:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 -- 172.21.15.3:0/3193013322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 msgr2=0x7f3a18062b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 --2- 172.21.15.3:0/3193013322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a18062b00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3a180d2480 tx=0x7f3a1001c940 comp rx=0 tx=0).stop 2024-09-12T12:25:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779370433 0 0) 0x7fd2b80b1ca0 con 0x7fd2b815e6c0 2024-09-12T12:25:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2b80b19c0 con 0x7fd2b815e6c0 2024-09-12T12:25:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 -- 172.21.15.3:0/3193013322 shutdown_connections 2024-09-12T12:25:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 --2- 172.21.15.3:0/3193013322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a18062b00 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 --2- 172.21.15.3:0/3193013322 >> v2:172.21.15.142:3300/0 conn(0x7f3a18158db0 0x7f3a18162f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 -- 172.21.15.3:0/3193013322 >> 172.21.15.3:0/3193013322 conn(0x7f3a1806bc70 msgr2=0x7f3a1806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4125893779 0 0) 0x7fd2a8003530 con 0x7fd2b8062df0 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2b80b0780 msgr2=0x7fd2b80b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2b80b0780 0x7fd2b80b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 >> v1:172.21.15.142:6789/0 conn(0x7fd2b815e6c0 legacy=0x7fd2b80626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b80b1f80 con 0x7fd2b8062df0 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fd2a4001680 con 0x7fd2b8062df0 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 >> v1:172.21.15.3:6789/0 conn(0x7fd2b8062df0 legacy=0x7fd2b80ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8003b80 0x7fd2a8003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2a80052f0 0x7fd2a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b80b1770 con 0x7fd2a8004810 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a800c1c0 con 0x7fd2a8003b80 2024-09-12T12:25:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a800c3c0 con 0x7fd2a80052f0 2024-09-12T12:25:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 -- 172.21.15.3:0/3193013322 shutdown_connections 2024-09-12T12:25:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7f3a1fdd9640 1 -- 172.21.15.3:0/3193013322 wait complete. 2024-09-12T12:25:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2bc805640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8003b80 0x7fd2a8003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 -- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2a80052f0 msgr2=0x7fd2a800b9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T12:25:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2a80052f0 0x7fd2a800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 -- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8003b80 msgr2=0x7fd2a8003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8003b80 0x7fd2a8003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.319+0000 7fd2b7fff640 1 -- 172.21.15.3:0/2889655573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a8003530 con 0x7fd2a8004810 2024-09-12T12:25:13.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 Processor -- start 2024-09-12T12:25:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 -- start start 2024-09-12T12:25:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd75a59c640 1 --2- 172.21.15.3:0/3444569740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c15b3f0 0x7fd75c0d1610 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009870 tx=0x7fd74c01ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e599b0dc04073d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2b7fff640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4007fd0 tx=0x7fd2a401cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872a6e79c8cffbc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4007bf0 con 0x7fd2a8004810 2024-09-12T12:25:13.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a40a9070 con 0x7fd2a8004810 2024-09-12T12:25:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2b77fe640 1 -- 172.21.15.3:0/2889655573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4005d20 con 0x7fd2a8004810 2024-09-12T12:25:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd75959a640 1 -- 172.21.15.3:0/3444569740 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c0b5070 con 0x7fd75c15b3f0 2024-09-12T12:25:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd75959a640 1 -- 172.21.15.3:0/3444569740 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74c0a8a50 con 0x7fd75c15b3f0 2024-09-12T12:25:13.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd75959a640 1 -- 172.21.15.3:0/3444569740 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c0a8e20 con 0x7fd75c15b3f0 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a181580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a180b4590 0x7f3a181611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a18158db0 0x7f3a1814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1815f520 con 0x7f3a180626e0 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18159190 con 0x7f3a18158db0 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1815f6a0 con 0x7f3a180b4590 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a181580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a181580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55548/0 (socket says 172.21.15.3:55548) 2024-09-12T12:25:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 -- 172.21.15.3:0/3074407897 learned_addr learned my addr 172.21.15.3:0/3074407897 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1e376640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a18158db0 0x7f3a1814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1d374640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a180b4590 0x7f3a181611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2bea69640 1 -- 172.21.15.3:0/2889655573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a8004810 msgr2=0x7fd2a8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2bea69640 1 --2- 172.21.15.3:0/2889655573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2a8004810 0x7fd2a8004c90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4007fd0 tx=0x7fd2a401cb60 comp rx=0 tx=0).stop 2024-09-12T12:25:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2bd006640 1 -- 172.21.15.3:0/2889655573 reap_dead start 2024-09-12T12:25:13.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 -- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a180b4590 msgr2=0x7f3a181611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a180b4590 0x7f3a181611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 -- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a18158db0 msgr2=0x7f3a1814def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a18158db0 0x7f3a1814def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2bea69640 1 -- 172.21.15.3:0/2889655573 shutdown_connections 2024-09-12T12:25:13.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2bea69640 1 -- 172.21.15.3:0/2889655573 >> 172.21.15.3:0/2889655573 conn(0x7fd2b80733e0 msgr2=0x7fd2b80737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:13.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 -- 172.21.15.3:0/3074407897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a181585f0 con 0x7f3a180626e0 2024-09-12T12:25:13.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2bea69640 1 -- 172.21.15.3:0/2889655573 shutdown_connections 2024-09-12T12:25:13.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1db75640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a181580b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400d620 tx=0x7f3a1400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a0e7fc640 1 -- 172.21.15.3:0/3074407897 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a140125f0 con 0x7f3a180626e0 2024-09-12T12:25:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a0e7fc640 1 -- 172.21.15.3:0/3074407897 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a14019070 con 0x7f3a180626e0 2024-09-12T12:25:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 -- 172.21.15.3:0/3074407897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a18151810 con 0x7f3a180626e0 2024-09-12T12:25:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a0e7fc640 1 -- 172.21.15.3:0/3074407897 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a140129b0 con 0x7f3a180626e0 2024-09-12T12:25:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7f3a1fdd9640 1 -- 172.21.15.3:0/3074407897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a180b05c0 con 0x7f3a180626e0 2024-09-12T12:25:13.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd760c38640 1 -- 172.21.15.3:0/3444569740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c15b3f0 msgr2=0x7fd75c0d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd760c38640 1 --2- 172.21.15.3:0/3444569740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c15b3f0 0x7fd75c0d1610 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009870 tx=0x7fd74c01ed70 comp rx=0 tx=0).stop 2024-09-12T12:25:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd760c38640 1 -- 172.21.15.3:0/3444569740 shutdown_connections 2024-09-12T12:25:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd760c38640 1 --2- 172.21.15.3:0/3444569740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c15b3f0 0x7fd75c0d1610 secure :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009870 tx=0x7fd74c01ed70 comp rx=0 tx=0).stop 2024-09-12T12:25:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd760c38640 1 -- 172.21.15.3:0/3444569740 >> 172.21.15.3:0/3444569740 conn(0x7fd75c0d0af0 msgr2=0x7fd75c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.323+0000 7fd2bea69640 1 -- 172.21.15.3:0/2889655573 wait complete. 2024-09-12T12:25:13.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f40052a0 con 0x7f3a180626e0 2024-09-12T12:25:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 Processor -- start 2024-09-12T12:25:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 -- start start 2024-09-12T12:25:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 -- 172.21.15.3:0/3444569740 shutdown_connections 2024-09-12T12:25:13.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 -- 172.21.15.3:0/3444569740 wait complete. 2024-09-12T12:25:13.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7f3a0e7fc640 1 -- 172.21.15.3:0/3074407897 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3a14012b50 con 0x7f3a180626e0 2024-09-12T12:25:13.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7f3a0e7fc640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39fc0678c0 0x7f39fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8004810 0x7fd2b8149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2b8155920 0x7fd2b8153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b8152020 0x7fd2b81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b80b2990 con 0x7fd2b8152020 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8063500 con 0x7fd2a8004810 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bea69640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b80b2b10 con 0x7fd2b8155920 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b8152020 0x7fd2b81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b8152020 0x7fd2b81524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55558/0 (socket says 172.21.15.3:55558) 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 -- 172.21.15.3:0/2104751093 learned_addr learned my addr 172.21.15.3:0/2104751093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:13.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7f3a0e7fc640 1 -- 172.21.15.3:0/3074407897 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7f3a14085fd0 con 0x7f3a180626e0 2024-09-12T12:25:13.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 Processor -- start 2024-09-12T12:25:13.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 -- start start 2024-09-12T12:25:13.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 -- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2b8155920 msgr2=0x7fd2b8153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2b8155920 0x7fd2b8153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bc805640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8004810 0x7fd2b8149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 -- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8004810 msgr2=0x7fd2b8149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8004810 0x7fd2b8149a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 -- 172.21.15.3:0/2104751093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b81547e0 con 0x7fd2b8152020 2024-09-12T12:25:13.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd2bd006640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b8152020 0x7fd2b81524b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b0b60 tx=0x7fd2b000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c06df20 0x7fd75c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd75c06eb80 0x7fd75c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c1f8dd0 0x7fd75c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c072b90 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c072a10 con 0x7fd75c06df20 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd760c38640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd75c072d10 con 0x7fd75c06eb80 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c1f8dd0 0x7fd75c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c1f8dd0 0x7fd75c1513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55562/0 (socket says 172.21.15.3:55562) 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 -- 172.21.15.3:0/1465292307 learned_addr learned my addr 172.21.15.3:0/1465292307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd759d9b640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd75c06eb80 0x7fd75c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75a59c640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c06df20 0x7fd75c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 -- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd75c06eb80 msgr2=0x7fd75c150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd75c06eb80 0x7fd75c150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 -- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c06df20 msgr2=0x7fd75c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c06df20 0x7fd75c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 -- 172.21.15.3:0/1465292307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c157100 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.327+0000 7fd75ad9d640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c1f8dd0 0x7fd75c1513a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd75400c760 tx=0x7fd75400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7f3a1d374640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39fc0678c0 0x7f39fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7f3a1d374640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39fc0678c0 0x7f39fc069d80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3a180d10b0 tx=0x7f3a08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff276ea3640 1 Processor -- start 2024-09-12T12:25:13.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff276ea3640 1 -- start start 2024-09-12T12:25:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff276ea3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff2700d4080 0x7ff270073860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff276ea3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2700b37f0 0x7ff2700b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd74affd640 1 -- 172.21.15.3:0/1465292307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754003b80 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd2b57fa640 1 -- 172.21.15.3:0/2104751093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0017070 con 0x7fd2b8152020 2024-09-12T12:25:13.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c156e20 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff276ea3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2700cf6f0 con 0x7ff27015e7f0 2024-09-12T12:25:13.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1f85d0 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff276ea3640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff2700d4510 con 0x7ff2700d4080 2024-09-12T12:25:13.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff276ea3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff2700cf9d0 con 0x7ff2700b37f0 2024-09-12T12:25:13.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff274c3f640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7ff27015e7f0 0x7ff2700d2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:36654/0 (socket says 172.21.15.3:36654) 2024-09-12T12:25:13.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff274c3f640 1 -- 172.21.15.3:0/666961653 learned_addr learned my addr 172.21.15.3:0/666961653 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:13.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff275440640 1 --2- 172.21.15.3:0/666961653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2700b37f0 0x7ff2700b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7f3a0e7fc640 1 -- 172.21.15.3:0/3074407897 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3a14056ba0 con 0x7f3a180626e0 2024-09-12T12:25:13.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff26ffff640 1 --2- 172.21.15.3:0/666961653 >> v2:172.21.15.3:3300/0 conn(0x7ff2700d4080 0x7ff270073860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff26ffff640 1 -- 172.21.15.3:0/666961653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2700b37f0 msgr2=0x7ff2700b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:13.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff26ffff640 1 --2- 172.21.15.3:0/666961653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2700b37f0 0x7ff2700b7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff26ffff640 1 -- 172.21.15.3:0/666961653 >> v1:172.21.15.142:6789/0 conn(0x7ff27015e7f0 legacy=0x7ff2700d2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7ff26ffff640 1 -- 172.21.15.3:0/666961653 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff270073340 con 0x7ff2700d4080 2024-09-12T12:25:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b8154ac0 con 0x7fd2b8152020 2024-09-12T12:25:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b814ade0 con 0x7fd2b8152020 2024-09-12T12:25:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b8154fe0 con 0x7fd2b8152020 2024-09-12T12:25:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.335+0000 7fd2b57fa640 1 -- 172.21.15.3:0/2104751093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b000a7b0 con 0x7fd2b8152020 2024-09-12T12:25:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.335+0000 7fd2b57fa640 1 -- 172.21.15.3:0/2104751093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000aba0 con 0x7fd2b8152020 2024-09-12T12:25:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.335+0000 7ff26ffff640 1 --2- 172.21.15.3:0/666961653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700d4080 0x7ff270073860 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0097c0 tx=0x7ff25c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24c75910a46e92d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.335+0000 7fd2b57fa640 1 -- 172.21.15.3:0/2104751093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd2b001a070 con 0x7fd2b8152020 2024-09-12T12:25:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.335+0000 7fd2b57fa640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd29c0678f0 0x7fd29c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd2bc805640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd29c0678f0 0x7fd29c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd74affd640 1 -- 172.21.15.3:0/1465292307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd754003d20 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75c0d1610 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.331+0000 7fd74affd640 1 -- 172.21.15.3:0/1465292307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75400a9f0 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd74affd640 1 -- 172.21.15.3:0/1465292307 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd75400ab90 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd74affd640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd73c0678f0 0x7fd73c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd74affd640 1 -- 172.21.15.3:0/1465292307 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fd754086650 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd75a59c640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd73c0678f0 0x7fd73c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd2bc805640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd29c0678f0 0x7fd29c069db0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8169f30 tx=0x7fd2ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd75a59c640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd73c0678f0 0x7fd73c069db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0b2000 tx=0x7fd74c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7ff26f7fe640 1 -- 172.21.15.3:0/666961653 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0b5070 con 0x7ff2700d4080 2024-09-12T12:25:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.339+0000 7fd2b57fa640 1 -- 172.21.15.3:0/2104751093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7fd2b0053d00 con 0x7fd2b8152020 2024-09-12T12:25:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.343+0000 7fd74affd640 1 -- 172.21.15.3:0/1465292307 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd754057170 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff26f7fe640 1 -- 172.21.15.3:0/666961653 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c0a8990 con 0x7ff2700d4080 2024-09-12T12:25:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff26f7fe640 1 -- 172.21.15.3:0/666961653 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0a8d60 con 0x7ff2700d4080 2024-09-12T12:25:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7fd2b57fa640 1 -- 172.21.15.3:0/2104751093 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd2b8154fe0 con 0x7fd2b8152020 2024-09-12T12:25:13.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- 172.21.15.3:0/666961653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700d4080 msgr2=0x7ff270073860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 --2- 172.21.15.3:0/666961653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700d4080 0x7ff270073860 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0097c0 tx=0x7ff25c01ec90 comp rx=0 tx=0).stop 2024-09-12T12:25:13.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- 172.21.15.3:0/666961653 shutdown_connections 2024-09-12T12:25:13.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 --2- 172.21.15.3:0/666961653 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff2700b37f0 0x7ff2700b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 --2- 172.21.15.3:0/666961653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700d4080 0x7ff270073860 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- 172.21.15.3:0/666961653 >> 172.21.15.3:0/666961653 conn(0x7ff2700d1120 msgr2=0x7ff2700d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- 172.21.15.3:0/666961653 shutdown_connections 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- 172.21.15.3:0/666961653 wait complete. 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 Processor -- start 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- start start 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700b37f0 0x7ff270154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff27015e7f0 0x7ff2701618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff270156300 0x7ff2700b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff2700d31b0 con 0x7ff2700b37f0 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff2700d3030 con 0x7ff270156300 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff276ea3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7ff2700b31c0 con 0x7ff27015e7f0 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff274c3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700b37f0 0x7ff270154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff274c3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700b37f0 0x7ff270154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55584/0 (socket says 172.21.15.3:55584) 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff274c3f640 1 -- 172.21.15.3:0/896702625 learned_addr learned my addr 172.21.15.3:0/896702625 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.347+0000 7ff26ffff640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff27015e7f0 0x7ff2701618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff275440640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff270156300 0x7ff2700b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff274c3f640 1 -- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff27015e7f0 msgr2=0x7ff2701618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff274c3f640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff27015e7f0 0x7ff2701618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff274c3f640 1 -- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff270156300 msgr2=0x7ff2700b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff274c3f640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff270156300 0x7ff2700b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff274c3f640 1 -- 172.21.15.3:0/896702625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff270156870 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26ffff640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff27015e7f0 0x7ff2701618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff274c3f640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700b37f0 0x7ff270154c10 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff26400c400 tx=0x7ff26400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26d7fa640 1 -- 172.21.15.3:0/896702625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264017070 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26d7fa640 1 -- 172.21.15.3:0/896702625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff264005cf0 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff270156b50 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26d7fa640 1 -- 172.21.15.3:0/896702625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640072d0 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff270157110 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff270157350 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26d7fa640 1 -- 172.21.15.3:0/896702625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff26401a070 con 0x7ff2700b37f0 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26d7fa640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2540678c0 0x7ff254069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26d7fa640 1 -- 172.21.15.3:0/896702625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5709+0+0 (secure 0 0 0) 0x7ff264084f20 con 0x7ff2700b37f0 2024-09-12T12:25:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.351+0000 7ff26ffff640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2540678c0 0x7ff254069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.355+0000 7ff26ffff640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2540678c0 0x7ff254069d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0b2000 tx=0x7ff25c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.355+0000 7ff26d7fa640 1 -- 172.21.15.3:0/896702625 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7ff264055410 con 0x7ff2700b37f0 2024-09-12T12:25:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.563+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7f39f4005860 con 0x7f3a180626e0 2024-09-12T12:25:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.567+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7fd75c074ed0 con 0x7fd75c1f8dd0 2024-09-12T12:25:13.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.579+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7fd2b80ae1d0 con 0x7fd2b8152020 2024-09-12T12:25:13.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:13.583+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7ff270073860 con 0x7ff2700b37f0 2024-09-12T12:25:14.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.055+0000 7f3a0e7fc640 1 -- 172.21.15.3:0/3074407897 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6735 v63) ==== 95+0+0 (secure 0 0 0) 0x7f3a140564a0 con 0x7f3a180626e0 2024-09-12T12:25:14.057 INFO:teuthology.orchestra.run.smithi003.stderr:failed mds gid 6735 2024-09-12T12:25:14.060 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:25:14.055+0000 7f0a5ac30640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39fc0678c0 msgr2=0x7f39fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39fc0678c0 0x7f39fc069d80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3a180d10b0 tx=0x7f3a08009040 comp rx=0 tx=0).stop 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 msgr2=0x7f3a181580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a181580b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400d620 tx=0x7f3a1400dae0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 shutdown_connections 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39fc0678c0 0x7f39fc069d80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a18158db0 0x7f3a1814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a180b4590 0x7f3a181611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 --2- 172.21.15.3:0/3074407897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3a180626e0 0x7f3a181580b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 >> 172.21.15.3:0/3074407897 conn(0x7f3a1806bc70 msgr2=0x7f3a180d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 shutdown_connections 2024-09-12T12:25:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7f39fbfff640 1 -- 172.21.15.3:0/3074407897 wait complete. 2024-09-12T12:25:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.059+0000 7fd74affd640 1 -- 172.21.15.3:0/1465292307 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6738 v64) ==== 95+0+0 (secure 0 0 0) 0x7fd754056a70 con 0x7fd75c1f8dd0 2024-09-12T12:25:14.063 INFO:teuthology.orchestra.run.smithi003.stderr:failed mds gid 6738 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd73c0678f0 msgr2=0x7fd73c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd73c0678f0 0x7fd73c069db0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0b2000 tx=0x7fd74c008040 comp rx=0 tx=0).stop 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c1f8dd0 msgr2=0x7fd75c1513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c1f8dd0 0x7fd75c1513a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fd75400c760 tx=0x7fd75400cc20 comp rx=0 tx=0).stop 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 shutdown_connections 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd73c0678f0 0x7fd73c069db0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd75c1f8dd0 0x7fd75c1513a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd75c06eb80 0x7fd75c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 --2- 172.21.15.3:0/1465292307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd75c06df20 0x7fd75c156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 >> 172.21.15.3:0/1465292307 conn(0x7fd75c0d0af0 msgr2=0x7fd75c158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 shutdown_connections 2024-09-12T12:25:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.063+0000 7fd760c38640 1 -- 172.21.15.3:0/1465292307 wait complete. 2024-09-12T12:25:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.067+0000 7fd2b57fa640 1 -- 172.21.15.3:0/2104751093 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5995 v65) ==== 95+0+0 (secure 0 0 0) 0x7fd2b0021020 con 0x7fd2b8152020 2024-09-12T12:25:14.070 INFO:teuthology.orchestra.run.smithi003.stderr:failed mds gid 5995 2024-09-12T12:25:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd29c0678f0 msgr2=0x7fd29c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd29c0678f0 0x7fd29c069db0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8169f30 tx=0x7fd2ac009040 comp rx=0 tx=0).stop 2024-09-12T12:25:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b8152020 msgr2=0x7fd2b81524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:14.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b8152020 0x7fd2b81524b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b0b60 tx=0x7fd2b000cb90 comp rx=0 tx=0).stop 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 shutdown_connections 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd29c0678f0 0x7fd29c069db0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd2b8152020 0x7fd2b81524b0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd2b8155920 0x7fd2b8153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 --2- 172.21.15.3:0/2104751093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2a8004810 0x7fd2b8149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 >> 172.21.15.3:0/2104751093 conn(0x7fd2b80733e0 msgr2=0x7fd2b80737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 shutdown_connections 2024-09-12T12:25:14.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:14.071+0000 7fd2bea69640 1 -- 172.21.15.3:0/2104751093 wait complete. 2024-09-12T12:25:14.090 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-12T12:25:14.090 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-12T12:25:14.092 INFO:tasks.ceph.mds.c:Started 2024-09-12T12:25:14.092 DEBUG:teuthology.parallel:result is None 2024-09-12T12:25:14.100 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-12T12:25:14.100 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-12T12:25:14.104 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-12T12:25:14.104 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-12T12:25:14.107 INFO:tasks.ceph.mds.b:Started 2024-09-12T12:25:14.107 DEBUG:teuthology.parallel:result is None 2024-09-12T12:25:14.149 INFO:tasks.ceph.mds.a:Started 2024-09-12T12:25:14.150 DEBUG:teuthology.parallel:result is None 2024-09-12T12:25:14.154 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:25:14.185 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-09-12T12:25:14.207 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:25:15.058 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:25:15.055+0000 7f0a5ac30640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:25:15.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.055+0000 7ff26d7fa640 1 -- 172.21.15.3:0/896702625 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5992 v66) ==== 95+0+0 (secure 0 0 0) 0x7ff264059710 con 0x7ff2700b37f0 2024-09-12T12:25:15.060 INFO:teuthology.orchestra.run.smithi003.stderr:failed mds gid 5992 2024-09-12T12:25:15.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.059+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2540678c0 msgr2=0x7ff254069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.059+0000 7ff276ea3640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2540678c0 0x7ff254069d80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0b2000 tx=0x7ff25c0aa070 comp rx=0 tx=0).stop 2024-09-12T12:25:15.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.059+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700b37f0 msgr2=0x7ff270154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.059+0000 7ff276ea3640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700b37f0 0x7ff270154c10 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff26400c400 tx=0x7ff26400c8c0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 shutdown_connections 2024-09-12T12:25:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7ff2540678c0 0x7ff254069d80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff270156300 0x7ff2700b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff27015e7f0 0x7ff2701618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 --2- 172.21.15.3:0/896702625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2700b37f0 0x7ff270154c10 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 >> 172.21.15.3:0/896702625 conn(0x7ff2700d1120 msgr2=0x7ff270155510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 shutdown_connections 2024-09-12T12:25:15.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.063+0000 7ff276ea3640 1 -- 172.21.15.3:0/896702625 wait complete. 2024-09-12T12:25:15.090 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-12T12:25:15.090 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-12T12:25:15.092 INFO:tasks.ceph.mds.d:Started 2024-09-12T12:25:15.092 DEBUG:teuthology.parallel:result is None 2024-09-12T12:25:15.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:15.168 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-09-12T12:25:15.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde83a9640 1 Processor -- start 2024-09-12T12:25:15.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde83a9640 1 -- start start 2024-09-12T12:25:15.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde83a9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcde0099920 0x7fcde006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:15.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde83a9640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde0099db0 con 0x7fcde00d04e0 2024-09-12T12:25:15.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde83a9640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde00ccb30 con 0x7fcde00b3cf0 2024-09-12T12:25:15.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde83a9640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcde00cd740 con 0x7fcde0099920 2024-09-12T12:25:15.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde6946640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcde0099920 0x7fcde006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:15.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde6946640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcde0099920 0x7fcde006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55634/0 (socket says 172.21.15.3:55634) 2024-09-12T12:25:15.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde6946640 1 -- 172.21.15.3:0/3291028774 learned_addr learned my addr 172.21.15.3:0/3291028774 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:15.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde6946640 1 -- 172.21.15.3:0/3291028774 >> v1:172.21.15.142:6789/0 conn(0x7fcde00b3cf0 legacy=0x7fcde006d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde6946640 1 -- 172.21.15.3:0/3291028774 >> v1:172.21.15.3:6789/0 conn(0x7fcde00d04e0 legacy=0x7fcde00d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.259+0000 7fcde6946640 1 -- 172.21.15.3:0/3291028774 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde00cce10 con 0x7fcde0099920 2024-09-12T12:25:15.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde6946640 1 --2- 172.21.15.3:0/3291028774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde0099920 0x7fcde006dce0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009e50 tx=0x7fcdd801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726a6912068852cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:15.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde5143640 1 -- 172.21.15.3:0/3291028774 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80a5660 con 0x7fcde0099920 2024-09-12T12:25:15.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde5143640 1 -- 172.21.15.3:0/3291028774 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd80a5800 con 0x7fcde0099920 2024-09-12T12:25:15.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde5143640 1 -- 172.21.15.3:0/3291028774 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd80a5bd0 con 0x7fcde0099920 2024-09-12T12:25:15.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 -- 172.21.15.3:0/3291028774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde0099920 msgr2=0x7fcde006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 --2- 172.21.15.3:0/3291028774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde0099920 0x7fcde006dce0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009e50 tx=0x7fcdd801c9a0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 -- 172.21.15.3:0/3291028774 shutdown_connections 2024-09-12T12:25:15.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 --2- 172.21.15.3:0/3291028774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde0099920 0x7fcde006dce0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 -- 172.21.15.3:0/3291028774 >> 172.21.15.3:0/3291028774 conn(0x7fcde00628b0 msgr2=0x7fcde0062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:15.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 -- 172.21.15.3:0/3291028774 shutdown_connections 2024-09-12T12:25:15.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 -- 172.21.15.3:0/3291028774 wait complete. 2024-09-12T12:25:15.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 Processor -- start 2024-09-12T12:25:15.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.263+0000 7fcde83a9640 1 -- start start 2024-09-12T12:25:15.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde0099920 0x7fcde0154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:15.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde00b3cf0 0x7fcde0156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:15.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde00d04e0 0x7fcde00ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:15.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0161f10 con 0x7fcde00b3cf0 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0161d90 con 0x7fcde0099920 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcde016ca30 con 0x7fcde00d04e0 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde00b3cf0 0x7fcde0156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde00b3cf0 0x7fcde0156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55646/0 (socket says 172.21.15.3:55646) 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 -- 172.21.15.3:0/1808520400 learned_addr learned my addr 172.21.15.3:0/1808520400 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde6946640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde00d04e0 0x7fcde00ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde6145640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde0099920 0x7fcde0154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 -- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde00d04e0 msgr2=0x7fcde00ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde00d04e0 0x7fcde00ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 -- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde0099920 msgr2=0x7fcde0154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde0099920 0x7fcde0154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 -- 172.21.15.3:0/1808520400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde01589f0 con 0x7fcde00b3cf0 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde5944640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde00b3cf0 0x7fcde0156dd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00c370 tx=0x7fcddc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:15.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcdd6ffd640 1 -- 172.21.15.3:0/1808520400 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0052d0 con 0x7fcde00b3cf0 2024-09-12T12:25:15.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcdd6ffd640 1 -- 172.21.15.3:0/1808520400 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcddc005470 con 0x7fcde00b3cf0 2024-09-12T12:25:15.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcdd6ffd640 1 -- 172.21.15.3:0/1808520400 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc007260 con 0x7fcde00b3cf0 2024-09-12T12:25:15.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde0158cd0 con 0x7fcde00b3cf0 2024-09-12T12:25:15.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde0159290 con 0x7fcde00b3cf0 2024-09-12T12:25:15.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde00d08c0 con 0x7fcde00b3cf0 2024-09-12T12:25:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcdd6ffd640 1 -- 172.21.15.3:0/1808520400 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcddc007400 con 0x7fcde00b3cf0 2024-09-12T12:25:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcdd6ffd640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcdc4067910 0x7fcdc4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcdd6ffd640 1 -- 172.21.15.3:0/1808520400 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7fcddc085060 con 0x7fcde00b3cf0 2024-09-12T12:25:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.267+0000 7fcde6145640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcdc4067910 0x7fcdc4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:15.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.271+0000 7fcde6145640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcdc4067910 0x7fcdc4069dd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0000f70 tx=0x7fcdd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:15.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.271+0000 7fcdd6ffd640 1 -- 172.21.15.3:0/1808520400 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fcddc055c60 con 0x7fcde00b3cf0 2024-09-12T12:25:15.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.471+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcde00b0050 con 0x7fcde00b3cf0 2024-09-12T12:25:15.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.471+0000 7fcdd6ffd640 1 -- 172.21.15.3:0/1808520400 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+4388 (secure 0 0 0) 0x7fcddc055560 con 0x7fcde00b3cf0 2024-09-12T12:25:15.476 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:15.476 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":67,"btime":"2024-09-12T12:25:15:066069+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:15.066037+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-12T12:25:15.477 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 67 2024-09-12T12:25:15.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcdc4067910 msgr2=0x7fcdc4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcdc4067910 0x7fcdc4069dd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0000f70 tx=0x7fcdd0009040 comp rx=0 tx=0).stop 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde00b3cf0 msgr2=0x7fcde0156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde00b3cf0 0x7fcde0156dd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00c370 tx=0x7fcddc00c830 comp rx=0 tx=0).stop 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 shutdown_connections 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcdc4067910 0x7fcdc4069dd0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcde00d04e0 0x7fcde00ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcde00b3cf0 0x7fcde0156dd0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 --2- 172.21.15.3:0/1808520400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcde0099920 0x7fcde0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 >> 172.21.15.3:0/1808520400 conn(0x7fcde00628b0 msgr2=0x7fcde00cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 shutdown_connections 2024-09-12T12:25:15.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:15.475+0000 7fcde83a9640 1 -- 172.21.15.3:0/1808520400 wait complete. 2024-09-12T12:25:15.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:24:42.989579+0000', 'modified': '2024-09-12T12:25:15.066037+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.3:6835/1654386752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1654386752}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1654386752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-12T12:25:15.899 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6987:up:replay 2024-09-12T12:25:16.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:17.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.031+0000 7f0fe2791640 1 Processor -- start 2024-09-12T12:25:17.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fe2791640 1 -- start start 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fe2791640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fdc161c40 0x7f0fdc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fe2791640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0fdc0d46e0 0x7f0fdc0d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fe2791640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fdc0cfcf0 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fe2791640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0fdc0d5060 con 0x7f0fdc161c40 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fe2791640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0fdc162020 con 0x7f0fdc0d46e0 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdbfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fdc161c40 0x7f0fdc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdbfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0fdc161c40 0x7f0fdc0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55652/0 (socket says 172.21.15.3:55652) 2024-09-12T12:25:17.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdbfff640 1 -- 172.21.15.3:0/713115610 learned_addr learned my addr 172.21.15.3:0/713115610 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdb7fe640 1 --2- 172.21.15.3:0/713115610 >> v2:172.21.15.142:3300/0 conn(0x7f0fdc0d46e0 0x7f0fdc0d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdbfff640 1 -- 172.21.15.3:0/713115610 >> v2:172.21.15.142:3300/0 conn(0x7f0fdc0d46e0 msgr2=0x7f0fdc0d4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdbfff640 1 --2- 172.21.15.3:0/713115610 >> v2:172.21.15.142:3300/0 conn(0x7f0fdc0d46e0 0x7f0fdc0d4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdbfff640 1 -- 172.21.15.3:0/713115610 >> v1:172.21.15.3:6789/0 conn(0x7f0fdc0b61d0 legacy=0x7f0fdc0b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:17.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.035+0000 7f0fdbfff640 1 -- 172.21.15.3:0/713115610 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc0d2a30 con 0x7f0fdc161c40 2024-09-12T12:25:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 --2- 172.21.15.3:0/713115610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc161c40 0x7f0fdc0cf7b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0d0c00 tx=0x7f0fd401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246b1bd25633295b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdaffd640 1 -- 172.21.15.3:0/713115610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40a5620 con 0x7f0fdc161c40 2024-09-12T12:25:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdaffd640 1 -- 172.21.15.3:0/713115610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd40a57c0 con 0x7f0fdc161c40 2024-09-12T12:25:17.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdaffd640 1 -- 172.21.15.3:0/713115610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40a5b90 con 0x7f0fdc161c40 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- 172.21.15.3:0/713115610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc161c40 msgr2=0x7f0fdc0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 --2- 172.21.15.3:0/713115610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc161c40 0x7f0fdc0cf7b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0d0c00 tx=0x7f0fd401c960 comp rx=0 tx=0).stop 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- 172.21.15.3:0/713115610 shutdown_connections 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 --2- 172.21.15.3:0/713115610 >> v2:172.21.15.142:3300/0 conn(0x7f0fdc0d46e0 0x7f0fdc0d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 --2- 172.21.15.3:0/713115610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc161c40 0x7f0fdc0cf7b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- 172.21.15.3:0/713115610 >> 172.21.15.3:0/713115610 conn(0x7f0fdc072690 msgr2=0x7f0fdc072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- 172.21.15.3:0/713115610 shutdown_connections 2024-09-12T12:25:17.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- 172.21.15.3:0/713115610 wait complete. 2024-09-12T12:25:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 Processor -- start 2024-09-12T12:25:17.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- start start 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc0b61d0 0x7f0fdc15cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fdc0d46e0 0x7f0fdc1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fdc161c40 0x7f0fdc162b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc0d2670 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc0d3db0 con 0x7f0fdc0d46e0 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fe2791640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0fdc0d27f0 con 0x7f0fdc161c40 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc0b61d0 0x7f0fdc15cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc0b61d0 0x7f0fdc15cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55666/0 (socket says 172.21.15.3:55666) 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 -- 172.21.15.3:0/1455742977 learned_addr learned my addr 172.21.15.3:0/1455742977 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 -- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fdc161c40 msgr2=0x7f0fdc162b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fdc161c40 0x7f0fdc162b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 -- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fdc0d46e0 msgr2=0x7f0fdc1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:17.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fdc0d46e0 0x7f0fdc1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 -- 172.21.15.3:0/1455742977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc15d1e0 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.039+0000 7f0fdbfff640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc0b61d0 0x7f0fdc15cca0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4019570 tx=0x7f0fd4019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fd8ff9640 1 -- 172.21.15.3:0/1455742977 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40bc020 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fd8ff9640 1 -- 172.21.15.3:0/1455742977 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd40b8690 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fdc1639f0 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fd8ff9640 1 -- 172.21.15.3:0/1455742977 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40b8a80 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fdc162340 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fdc163f10 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fd8ff9640 1 -- 172.21.15.3:0/1455742977 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0fd40bf020 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fd8ff9640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fbc0678c0 0x7f0fbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fd8ff9640 1 -- 172.21.15.3:0/1455742977 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f0fd4120d10 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fdb7fe640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fbc0678c0 0x7f0fbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:17.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.043+0000 7f0fdb7fe640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fbc0678c0 0x7f0fbc069d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc076180 tx=0x7f0fc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.047+0000 7f0fd8ff9640 1 -- 172.21.15.3:0/1455742977 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0fdc163f10 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.251+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fdc0d4b20 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fd8ff9640 1 -- 172.21.15.3:0/1455742977 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5216 (secure 0 0 0) 0x7f0fd40c6030 con 0x7f0fdc0b61d0 2024-09-12T12:25:17.257 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:17.258 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":69,"btime":"2024-09-12T12:25:17:083698+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:16.087729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-12T12:25:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 69 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fbc0678c0 msgr2=0x7f0fbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fbc0678c0 0x7f0fbc069d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc076180 tx=0x7f0fc4008040 comp rx=0 tx=0).stop 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc0b61d0 msgr2=0x7f0fdc15cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc0b61d0 0x7f0fdc15cca0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4019570 tx=0x7f0fd4019700 comp rx=0 tx=0).stop 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 shutdown_connections 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0fbc0678c0 0x7f0fbc069d80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fdc161c40 0x7f0fdc162b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fdc0d46e0 0x7f0fdc1589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 --2- 172.21.15.3:0/1455742977 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fdc0b61d0 0x7f0fdc15cca0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 >> 172.21.15.3:0/1455742977 conn(0x7f0fdc072690 msgr2=0x7f0fdc15eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 shutdown_connections 2024-09-12T12:25:17.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:17.255+0000 7f0fe2791640 1 -- 172.21.15.3:0/1455742977 wait complete. 2024-09-12T12:25:17.314 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:25:17.317 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:25:17.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:24:42.989579+0000', 'modified': '2024-09-12T12:25:16.087729+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.3:6835/1654386752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1654386752}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1654386752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-12T12:25:17.681 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6987:up:rejoin 2024-09-12T12:25:18.681 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:18.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f13742ea640 1 Processor -- start 2024-09-12T12:25:18.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f13742ea640 1 -- start start 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f13742ea640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f136c15e5a0 0x7f136c0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f13742ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c15e980 0x7f136c0d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f13742ea640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136c0d2d50 con 0x7f136c0d1d20 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f13742ea640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f136c0d2b00 con 0x7f136c15e5a0 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f13742ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f136c0d3030 con 0x7f136c15e980 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372086640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f136c15e5a0 0x7f136c0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372086640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f136c15e5a0 0x7f136c0cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54292/0 (socket says 172.21.15.3:54292) 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372086640 1 -- 172.21.15.3:0/4210873612 learned_addr learned my addr 172.21.15.3:0/4210873612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:18.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372887640 1 --2- 172.21.15.3:0/4210873612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c15e980 0x7f136c0d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:18.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372086640 1 -- 172.21.15.3:0/4210873612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c15e980 msgr2=0x7f136c0d3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:18.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372086640 1 --2- 172.21.15.3:0/4210873612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c15e980 0x7f136c0d3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:18.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372086640 1 -- 172.21.15.3:0/4210873612 >> v1:172.21.15.3:6789/0 conn(0x7f136c0d1d20 legacy=0x7f136c0b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:18.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.815+0000 7f1372086640 1 -- 172.21.15.3:0/4210873612 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c0d3310 con 0x7f136c15e5a0 2024-09-12T12:25:18.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f1372086640 1 --2- 172.21.15.3:0/4210873612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c0cf770 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1368009870 tx=0x7f136801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3907800052abff71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:18.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f1371084640 1 -- 172.21.15.3:0/4210873612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13680b2070 con 0x7f136c15e5a0 2024-09-12T12:25:18.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f1371084640 1 -- 172.21.15.3:0/4210873612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13680a9070 con 0x7f136c15e5a0 2024-09-12T12:25:18.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f1371084640 1 -- 172.21.15.3:0/4210873612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13680ad880 con 0x7f136c15e5a0 2024-09-12T12:25:18.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- 172.21.15.3:0/4210873612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 msgr2=0x7f136c0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 --2- 172.21.15.3:0/4210873612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c0cf770 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1368009870 tx=0x7f136801c560 comp rx=0 tx=0).stop 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- 172.21.15.3:0/4210873612 shutdown_connections 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 --2- 172.21.15.3:0/4210873612 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c15e980 0x7f136c0d3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 --2- 172.21.15.3:0/4210873612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c0cf770 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- 172.21.15.3:0/4210873612 >> 172.21.15.3:0/4210873612 conn(0x7f136c0626e0 msgr2=0x7f136c062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- 172.21.15.3:0/4210873612 shutdown_connections 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- 172.21.15.3:0/4210873612 wait complete. 2024-09-12T12:25:18.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 Processor -- start 2024-09-12T12:25:18.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- start start 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c0d1d20 0x7f136c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f136c15e980 0x7f136c156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f136c16b4c0 con 0x7f136c15e5a0 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f136c0b81b0 con 0x7f136c15e980 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.819+0000 7f13742ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f136c16b640 con 0x7f136c0d1d20 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54300/0 (socket says 172.21.15.3:54300) 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 -- 172.21.15.3:0/2637406662 learned_addr learned my addr 172.21.15.3:0/2637406662 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1372086640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c0d1d20 0x7f136c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:18.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1372887640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f136c15e980 0x7f136c156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 -- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c0d1d20 msgr2=0x7f136c151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c0d1d20 0x7f136c151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 -- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f136c15e980 msgr2=0x7f136c156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f136c15e980 0x7f136c156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 -- 172.21.15.3:0/2637406662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c0b51e0 con 0x7f136c15e5a0 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1371885640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c156950 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c370 tx=0x7f135c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1362ffd640 1 -- 172.21.15.3:0/2637406662 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0052d0 con 0x7f136c15e5a0 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1362ffd640 1 -- 172.21.15.3:0/2637406662 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c005470 con 0x7f136c15e5a0 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136c0b54c0 con 0x7f136c15e5a0 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1362ffd640 1 -- 172.21.15.3:0/2637406662 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c007260 con 0x7f136c15e5a0 2024-09-12T12:25:18.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136c0b5a80 con 0x7f136c15e5a0 2024-09-12T12:25:18.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136c069250 con 0x7f136c15e5a0 2024-09-12T12:25:18.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1362ffd640 1 -- 172.21.15.3:0/2637406662 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f135c007400 con 0x7f136c15e5a0 2024-09-12T12:25:18.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1362ffd640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13500678f0 0x7f1350069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:18.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1362ffd640 1 -- 172.21.15.3:0/2637406662 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f135c085210 con 0x7f136c15e5a0 2024-09-12T12:25:18.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.823+0000 7f1372086640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13500678f0 0x7f1350069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:18.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.827+0000 7f1372086640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13500678f0 0x7f1350069db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f13680af000 tx=0x7f1368008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:18.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:18.827+0000 7f1362ffd640 1 -- 172.21.15.3:0/2637406662 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f135c055c00 con 0x7f136c15e5a0 2024-09-12T12:25:19.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f136c0cf770 con 0x7f136c15e5a0 2024-09-12T12:25:19.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f1362ffd640 1 -- 172.21.15.3:0/2637406662 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5223 (secure 0 0 0) 0x7f135c055500 con 0x7f136c15e5a0 2024-09-12T12:25:19.035 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:19.035 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":70,"btime":"2024-09-12T12:25:18:085753+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:18.085740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":7}]} 2024-09-12T12:25:19.035 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 70 2024-09-12T12:25:19.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13500678f0 msgr2=0x7f1350069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13500678f0 0x7f1350069db0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f13680af000 tx=0x7f1368008040 comp rx=0 tx=0).stop 2024-09-12T12:25:19.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 msgr2=0x7f136c156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c156950 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c370 tx=0x7f135c00c830 comp rx=0 tx=0).stop 2024-09-12T12:25:19.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 shutdown_connections 2024-09-12T12:25:19.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f13500678f0 0x7f1350069db0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f136c15e980 0x7f136c156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c15e5a0 0x7f136c156950 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.031+0000 7f13742ea640 1 --2- 172.21.15.3:0/2637406662 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f136c0d1d20 0x7f136c151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.035+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 >> 172.21.15.3:0/2637406662 conn(0x7f136c0626e0 msgr2=0x7f136c0b7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.035+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 shutdown_connections 2024-09-12T12:25:19.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.035+0000 7f13742ea640 1 -- 172.21.15.3:0/2637406662 wait complete. 2024-09-12T12:25:19.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:24:42.989579+0000', 'modified': '2024-09-12T12:25:18.085740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.3:6835/1654386752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1654386752}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1654386752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6987, 'qdb_cluster': [6987]} 2024-09-12T12:25:19.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T12:25:19.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T12:25:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.591+0000 7fcf18dfc640 1 Processor -- start 2024-09-12T12:25:19.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.591+0000 7fcf18dfc640 1 -- start start 2024-09-12T12:25:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf18dfc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 0x7fcf140d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf18dfc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf140d2d50 con 0x7fcf140d1d20 2024-09-12T12:25:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf18dfc640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf140d3030 con 0x7fcf1415e5a0 2024-09-12T12:25:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf18dfc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf140d2b00 con 0x7fcf1415e980 2024-09-12T12:25:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf0bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fcf140d1d20 0x7fcf140b7890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49304/0 (socket says 172.21.15.3:49304) 2024-09-12T12:25:19.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf0bfff640 1 -- 172.21.15.3:0/3376911648 learned_addr learned my addr 172.21.15.3:0/3376911648 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 0x7fcf140d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3152634862 0 0) 0x7fcf140d2d50 con 0x7fcf140d1d20 2024-09-12T12:25:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf00003530 con 0x7fcf140d1d20 2024-09-12T12:25:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4109226178 0 0) 0x7fcf00003530 con 0x7fcf140d1d20 2024-09-12T12:25:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 msgr2=0x7fcf140d3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 0x7fcf140d3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 >> v1:172.21.15.142:6789/0 conn(0x7fcf1415e5a0 legacy=0x7fcf140cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf140d3310 con 0x7fcf140d1d20 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fcefc001680 con 0x7fcf140d1d20 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 >> v1:172.21.15.3:6789/0 conn(0x7fcf140d1d20 legacy=0x7fcf140b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf000039f0 0x7fcf00003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf00004810 0x7fcf00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 0x7fcf000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcf140d2b00 con 0x7fcf000039f0 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0000bab0 con 0x7fcf00004810 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0000bcb0 con 0x7fcf1415e980 2024-09-12T12:25:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf1259c640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf000039f0 0x7fcf00003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf00004810 0x7fcf00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 0x7fcf000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf1259c640 1 -- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 msgr2=0x7fcf000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf1259c640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf1415e980 0x7fcf000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf1259c640 1 -- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf00004810 msgr2=0x7fcf00004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf1259c640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf00004810 0x7fcf00004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf1259c640 1 -- 172.21.15.3:0/3376911648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf00003530 con 0x7fcf000039f0 2024-09-12T12:25:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf1259c640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf000039f0 0x7fcf00003e30 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcf04001750 tx=0x7fcf0401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a927275fee8cb33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040013a0 con 0x7fcf000039f0 2024-09-12T12:25:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.595+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf040052f0 con 0x7fcf000039f0 2024-09-12T12:25:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf11d9b640 1 -- 172.21.15.3:0/3376911648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04003840 con 0x7fcf000039f0 2024-09-12T12:25:19.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- 172.21.15.3:0/3376911648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf000039f0 msgr2=0x7fcf00003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 --2- 172.21.15.3:0/3376911648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf000039f0 0x7fcf00003e30 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcf04001750 tx=0x7fcf0401c3c0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf12d9d640 1 -- 172.21.15.3:0/3376911648 reap_dead start 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- 172.21.15.3:0/3376911648 shutdown_connections 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- 172.21.15.3:0/3376911648 >> 172.21.15.3:0/3376911648 conn(0x7fcf140626e0 msgr2=0x7fcf14062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- 172.21.15.3:0/3376911648 shutdown_connections 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- 172.21.15.3:0/3376911648 wait complete. 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 Processor -- start 2024-09-12T12:25:19.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- start start 2024-09-12T12:25:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf14151570 0x7fcf14159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf14151950 0x7fcf14159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140cb7c0 0x7fcf14156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcf140d4a50 con 0x7fcf14151950 2024-09-12T12:25:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf140d48d0 con 0x7fcf140cb7c0 2024-09-12T12:25:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf18dfc640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf140d4bd0 con 0x7fcf14151570 2024-09-12T12:25:19.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf14151950 0x7fcf14159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf14151950 0x7fcf14159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54322/0 (socket says 172.21.15.3:54322) 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 -- 172.21.15.3:0/2784703829 learned_addr learned my addr 172.21.15.3:0/2784703829 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf1259c640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf14151570 0x7fcf14159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140cb7c0 0x7fcf14156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 -- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf14151570 msgr2=0x7fcf14159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf14151570 0x7fcf14159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 -- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140cb7c0 msgr2=0x7fcf14156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140cb7c0 0x7fcf14156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.599+0000 7fcf0bfff640 1 -- 172.21.15.3:0/2784703829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf141542b0 con 0x7fcf14151950 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf14151950 0x7fcf14159740 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e7e0 tx=0x7fcefc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf0affd640 1 -- 172.21.15.3:0/2784703829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00ee90 con 0x7fcf14151950 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf0affd640 1 -- 172.21.15.3:0/2784703829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc018070 con 0x7fcf14151950 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf14154590 con 0x7fcf14151950 2024-09-12T12:25:19.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf0affd640 1 -- 172.21.15.3:0/2784703829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00cb10 con 0x7fcf14151950 2024-09-12T12:25:19.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf14157440 con 0x7fcf14151950 2024-09-12T12:25:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf0affd640 1 -- 172.21.15.3:0/2784703829 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcefc00ccb0 con 0x7fcf14151950 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf0affd640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcef4067a90 0x7fcef4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf0affd640 1 -- 172.21.15.3:0/2784703829 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7fcefc085a00 con 0x7fcf14151950 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf1259c640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcef4067a90 0x7fcef4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf1beb8000 1 Processor -- start 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf1beb8000 1 -- start start 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.603+0000 7fcf1259c640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcef4067a90 0x7fcef4069f50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcf00004790 tx=0x7fcf040011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7332b20 0x5636b7332f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b72e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72e9290 0x5636b72e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636b71d0b60 con 0x5636b7333440 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636b7137c70 con 0x5636b7332b20 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5636b70dded0 con 0x5636b72e9290 2024-09-12T12:25:19.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b72e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b72e8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54324/0 (socket says 172.21.15.3:54324) 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 -- 172.21.15.3:0/3884955798 learned_addr learned my addr 172.21.15.3:0/3884955798 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1259c640 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7332b20 0x5636b7332f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72e9290 0x5636b72e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 -- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72e9290 msgr2=0x5636b72e9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72e9290 0x5636b72e9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 -- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7332b20 msgr2=0x5636b7332f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7332b20 0x5636b7332f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 -- 172.21.15.3:0/3884955798 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b70efe50 con 0x5636b7333440 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72e9290 0x5636b72e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b72e8d50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00c4b0 tx=0x7fcefc055850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d472af3418e191b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcef3fff640 1 -- 172.21.15.3:0/3884955798 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc082070 con 0x5636b7333440 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcef3fff640 1 -- 172.21.15.3:0/3884955798 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc05e960 con 0x5636b7333440 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcef3fff640 1 -- 172.21.15.3:0/3884955798 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0524a0 con 0x5636b7333440 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- 172.21.15.3:0/3884955798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 msgr2=0x5636b72e8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b72e8d50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00c4b0 tx=0x7fcefc055850 comp rx=0 tx=0).stop 2024-09-12T12:25:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- 172.21.15.3:0/3884955798 shutdown_connections 2024-09-12T12:25:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72e9290 0x5636b72e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b72e8d50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/3884955798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7332b20 0x5636b7332f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- 172.21.15.3:0/3884955798 >> 172.21.15.3:0/3884955798 conn(0x5636b72ef600 msgr2=0x5636b72e7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- 172.21.15.3:0/3884955798 shutdown_connections 2024-09-12T12:25:19.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- 172.21.15.3:0/3884955798 wait complete. 2024-09-12T12:25:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- start start 2024-09-12T12:25:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 Processor -- start 2024-09-12T12:25:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b72eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b72f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b72ee330 0x5636b72f4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636b7199b00 con 0x5636b72ee330 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636b7112360 con 0x5636b7333440 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5636b710f2d0 con 0x5636b7332b20 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1beb8000 10 client.0 _ll_register_callbacks cb 0x5636b72d4820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b72ee330 0x5636b72f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b72ee330 0x5636b72f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54340/0 (socket says 172.21.15.3:54340) 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 -- 172.21.15.3:0/846083989 learned_addr learned my addr 172.21.15.3:0/846083989 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1259c640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b72eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b72f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 -- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 msgr2=0x5636b72eddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b72eddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 -- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 msgr2=0x5636b72f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b72f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf12d9d640 1 -- 172.21.15.3:0/846083989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b70de120 con 0x5636b72ee330 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf1259c640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b72eddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.607+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b72f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b72ee330 0x5636b72f4230 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf1415ed60 tx=0x7fcf0c004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c103fe075628597f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 1 -- 172.21.15.3:0/846083989 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c01c020 con 0x5636b72ee330 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 1 -- 172.21.15.3:0/846083989 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7fcf0c012040 con 0x5636b72ee330 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc05e960 con 0x5636b72ee330 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 1 -- 172.21.15.3:0/846083989 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c026050 con 0x5636b72ee330 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 10 client.7029 fetch_fsmap learned FSMap version 70 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5636b70d7450 con 0x5636b72ee330 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 1 -- 172.21.15.3:0/846083989 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc05e960 con 0x5636b72ee330 2024-09-12T12:25:19.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 1 -- 172.21.15.3:0/846083989 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fcf0c029020 con 0x5636b72ee330 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 10 client.7029 fetch_fsmap finished waiting for FSMap version 70 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 10 client.7029 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 20 client.7029 populate_metadata read hostname 'smithi003' 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b7341280 0x5636b7343670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 4 client.7029 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 --> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b70d1c00 con 0x5636b7341280 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1259c640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b7341280 0x5636b7343670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1259c640 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b7341280 0x5636b7343670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6a02a56200d7ae server_cookie=288e2e707addaaeb in_seq=0 out_seq=0 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 10 client.7029 ms_handle_connect on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 1 -- 172.21.15.3:0/846083989 <== mds.0 v2:172.21.15.3:6834/1654386752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636b70d1c00 con 0x5636b7341280 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 10 client.7029 handle_command_reply: tid=0 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 client.7029 shutdown 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b72ee330 msgr2=0x5636b72f4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b72ee330 0x5636b72f4230 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fcf1415ed60 tx=0x7fcf0c004780 comp rx=0 tx=0).stop 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 shutdown_connections 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b7341280 0x5636b7343670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b72ee330 0x5636b72f4230 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b72f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/846083989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b72eddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcef17fa640 0 client.7029 ms_handle_reset on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 >> 172.21.15.3:0/846083989 conn(0x5636b72e9290 msgr2=0x5636b72e7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 shutdown_connections 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 1 -- 172.21.15.3:0/846083989 wait complete. 2024-09-12T12:25:19.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.611+0000 7fcf1beb8000 20 client.7029 trim_cache size 0 max 16384 2024-09-12T12:25:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 Processor -- start 2024-09-12T12:25:19.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- start start 2024-09-12T12:25:19.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72ee330 0x5636b7331c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7332b20 0x5636b7332170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b7343670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14203a60 con 0x5636b7332b20 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14202df0 con 0x5636b7333440 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fcf14203410 con 0x5636b72ee330 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7332b20 0x5636b7332170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7332b20 0x5636b7332170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54352/0 (socket says 172.21.15.3:54352) 2024-09-12T12:25:19.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 -- 172.21.15.3:0/1767737016 learned_addr learned my addr 172.21.15.3:0/1767737016 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 -- 172.21.15.3:0/1767737016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72ee330 msgr2=0x5636b7331c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/1767737016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72ee330 0x5636b7331c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 -- 172.21.15.3:0/1767737016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 msgr2=0x5636b7343670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/1767737016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b7343670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 -- 172.21.15.3:0/1767737016 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b7333270 con 0x5636b7332b20 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/1767737016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7332b20 0x5636b7332170 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000c10 tx=0x7fcefc058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d7e636a93812a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcef37fe640 1 -- 172.21.15.3:0/1767737016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc082040 con 0x5636b7332b20 2024-09-12T12:25:19.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcef37fe640 1 -- 172.21.15.3:0/1767737016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc05e960 con 0x5636b7332b20 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcef37fe640 1 -- 172.21.15.3:0/1767737016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0520d0 con 0x5636b7332b20 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- 172.21.15.3:0/1767737016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7332b20 msgr2=0x5636b7332170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/1767737016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7332b20 0x5636b7332170 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000c10 tx=0x7fcefc058000 comp rx=0 tx=0).stop 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- 172.21.15.3:0/1767737016 shutdown_connections 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/1767737016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b7333440 0x5636b7343670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/1767737016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7332b20 0x5636b7332170 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/1767737016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b72ee330 0x5636b7331c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- 172.21.15.3:0/1767737016 >> 172.21.15.3:0/1767737016 conn(0x5636b72e9290 msgr2=0x5636b72ed5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- 172.21.15.3:0/1767737016 shutdown_connections 2024-09-12T12:25:19.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.631+0000 7fcf1beb8000 1 -- 172.21.15.3:0/1767737016 wait complete. 2024-09-12T12:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- start start 2024-09-12T12:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 Processor -- start 2024-09-12T12:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b72ee330 0x5636b7346aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b73a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b73a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5636b7136f80 con 0x5636b7333440 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5636b70efca0 con 0x5636b72ee330 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x5636b713c2a0 con 0x5636b7332b20 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 10 client.0 _ll_register_callbacks cb 0x7fcf0c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b73a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b73a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54356/0 (socket says 172.21.15.3:54356) 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 -- 172.21.15.3:0/2714313337 learned_addr learned my addr 172.21.15.3:0/2714313337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf0bfff640 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b73a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1259c640 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b72ee330 0x5636b7346aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 -- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 msgr2=0x5636b73a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b73a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 -- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b72ee330 msgr2=0x5636b7346aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b72ee330 0x5636b7346aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 -- 172.21.15.3:0/2714313337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b70befd0 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf12d9d640 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b73a6860 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c018bb0 tx=0x7fcf0c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fffe70833338b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 1 -- 172.21.15.3:0/2714313337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c02e070 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 1 -- 172.21.15.3:0/2714313337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7fcf0c0213f0 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc05e960 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 1 -- 172.21.15.3:0/2714313337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c039050 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 10 client.7035 fetch_fsmap learned FSMap version 70 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5636b71ba750 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 1 -- 172.21.15.3:0/2714313337 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcefc05e960 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 1 -- 172.21.15.3:0/2714313337 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fcf0c03c020 con 0x5636b7333440 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 10 client.7035 fetch_fsmap finished waiting for FSMap version 70 2024-09-12T12:25:19.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 10 client.7035 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 20 client.7035 populate_metadata read hostname 'smithi003' 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b73a6da0 0x5636b73a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 4 client.7035 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 --> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] -- command(tid 0: {"prefix": "status"}) -- 0x5636b71453a0 con 0x5636b73a6da0 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1259c640 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b73a6da0 0x5636b73a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1259c640 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b73a6da0 0x5636b73a2c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b2899c360943e3 server_cookie=4980f512bcb0a0d7 in_seq=0 out_seq=0 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 10 client.7035 ms_handle_connect on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 1 -- 172.21.15.3:0/2714313337 <== mds.0 v2:172.21.15.3:6834/1654386752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5636b71453a0 con 0x5636b73a6da0 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 10 client.7035 handle_command_reply: tid=0 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 client.7035 shutdown 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 msgr2=0x5636b73a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b73a6860 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c018bb0 tx=0x7fcf0c004a50 comp rx=0 tx=0).stop 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 shutdown_connections 2024-09-12T12:25:19.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x5636b73a6da0 0x5636b73a2c80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5636b7333440 0x5636b73a6860 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5636b7332b20 0x5636b73a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 --2- 172.21.15.3:0/2714313337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636b72ee330 0x5636b7346aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 >> 172.21.15.3:0/2714313337 conn(0x5636b72e9290 msgr2=0x5636b7330c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcef17fa640 0 client.7035 ms_handle_reset on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 shutdown_connections 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 1 -- 172.21.15.3:0/2714313337 wait complete. 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.635+0000 7fcf1beb8000 20 client.7035 trim_cache size 0 max 16384 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:19.647 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6987, 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 4.5663725570000002, 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 70, 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 97, 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 97, 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 5.431621936 2024-09-12T12:25:19.648 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcef4067a90 msgr2=0x7fcef4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcef4067a90 0x7fcef4069f50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcf00004790 tx=0x7fcf040011a0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf14151950 msgr2=0x7fcf14159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf14151950 0x7fcf14159740 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e7e0 tx=0x7fcefc00eca0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 shutdown_connections 2024-09-12T12:25:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fcef4067a90 0x7fcef4069f50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf140cb7c0 0x7fcf14156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcf14151950 0x7fcf14159740 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 --2- 172.21.15.3:0/2784703829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf14151570 0x7fcf14159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 >> 172.21.15.3:0/2784703829 conn(0x7fcf140626e0 msgr2=0x7fcf14062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 shutdown_connections 2024-09-12T12:25:19.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.639+0000 7fcf18dfc640 1 -- 172.21.15.3:0/2784703829 wait complete. 2024-09-12T12:25:19.658 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.771+0000 7fbbdc48e640 1 Processor -- start 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.771+0000 7fbbdc48e640 1 -- start start 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbd40b4c70 0x7fbbd406e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbbd406eba0 0x7fbbd40b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd40d00b0 con 0x7fbbd40b4c70 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fbbd40cfdd0 con 0x7fbbd406eba0 2024-09-12T12:25:19.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd406db80 con 0x7fbbd40b3df0 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbd40b4c70 0x7fbbd406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbbd40b4c70 0x7fbbd406e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54366/0 (socket says 172.21.15.3:54366) 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 -- 172.21.15.3:0/3357199787 learned_addr learned my addr 172.21.15.3:0/3357199787 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdaa2b640 1 --2- 172.21.15.3:0/3357199787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbd9a29640 1 --2- 172.21.15.3:0/3357199787 >> v2:172.21.15.142:3300/0 conn(0x7fbbd406eba0 0x7fbbd40b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 -- 172.21.15.3:0/3357199787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 msgr2=0x7fbbd40d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 --2- 172.21.15.3:0/3357199787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 -- 172.21.15.3:0/3357199787 >> v2:172.21.15.142:3300/0 conn(0x7fbbd406eba0 msgr2=0x7fbbd40b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 --2- 172.21.15.3:0/3357199787 >> v2:172.21.15.142:3300/0 conn(0x7fbbd406eba0 0x7fbbd40b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 -- 172.21.15.3:0/3357199787 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd406de60 con 0x7fbbd40b4c70 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbd9a29640 1 --2- 172.21.15.3:0/3357199787 >> v2:172.21.15.142:3300/0 conn(0x7fbbd406eba0 0x7fbbd40b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:19.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdaa2b640 1 --2- 172.21.15.3:0/3357199787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbda22a640 1 --2- 172.21.15.3:0/3357199787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd40b4c70 0x7fbbd406e660 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40afb10 tx=0x7fbbd001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b85e164b737b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbd9228640 1 -- 172.21.15.3:0/3357199787 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00b0070 con 0x7fbbd40b4c70 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbd9228640 1 -- 172.21.15.3:0/3357199787 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd00a4e50 con 0x7fbbd40b4c70 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbd9228640 1 -- 172.21.15.3:0/3357199787 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ab820 con 0x7fbbd40b4c70 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- 172.21.15.3:0/3357199787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd40b4c70 msgr2=0x7fbbd406e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/3357199787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd40b4c70 0x7fbbd406e660 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbbd40afb10 tx=0x7fbbd001c400 comp rx=0 tx=0).stop 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- 172.21.15.3:0/3357199787 shutdown_connections 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/3357199787 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/3357199787 >> v2:172.21.15.142:3300/0 conn(0x7fbbd406eba0 0x7fbbd40b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/3357199787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd40b4c70 0x7fbbd406e660 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- 172.21.15.3:0/3357199787 >> 172.21.15.3:0/3357199787 conn(0x7fbbd40628b0 msgr2=0x7fbbd4062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- 172.21.15.3:0/3357199787 shutdown_connections 2024-09-12T12:25:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.775+0000 7fbbdc48e640 1 -- 172.21.15.3:0/3357199787 wait complete. 2024-09-12T12:25:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 Processor -- start 2024-09-12T12:25:19.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 -- start start 2024-09-12T12:25:19.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd406eba0 0x7fbbd40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd40b4c70 0x7fbbd414bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd415ff00 con 0x7fbbd406eba0 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd40cfb00 con 0x7fbbd40b4c70 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd4160080 con 0x7fbbd40b3df0 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd406eba0 0x7fbbd40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd406eba0 0x7fbbd40aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54374/0 (socket says 172.21.15.3:54374) 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 -- 172.21.15.3:0/2615680791 learned_addr learned my addr 172.21.15.3:0/2615680791 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbd9a29640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdaa2b640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd40b4c70 0x7fbbd414bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 -- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 msgr2=0x7fbbd40adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 -- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd40b4c70 msgr2=0x7fbbd414bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd40b4c70 0x7fbbd414bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 -- 172.21.15.3:0/2615680791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd40af3b0 con 0x7fbbd406eba0 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbd9a29640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbda22a640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd406eba0 0x7fbbd40aedd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00ae040 tx=0x7fbbd001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbc6ffd640 1 -- 172.21.15.3:0/2615680791 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ba020 con 0x7fbbd406eba0 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbc6ffd640 1 -- 172.21.15.3:0/2615680791 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd00aa780 con 0x7fbbd406eba0 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbc6ffd640 1 -- 172.21.15.3:0/2615680791 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00b0040 con 0x7fbbd406eba0 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd40728a0 con 0x7fbbd406eba0 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd40ae1a0 con 0x7fbbd406eba0 2024-09-12T12:25:19.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.779+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd40d04e0 con 0x7fbbd406eba0 2024-09-12T12:25:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.783+0000 7fbbc6ffd640 1 -- 172.21.15.3:0/2615680791 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbd00bd020 con 0x7fbbd406eba0 2024-09-12T12:25:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.783+0000 7fbbc6ffd640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678f0 0x7fbbb8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.783+0000 7fbbc6ffd640 1 -- 172.21.15.3:0/2615680791 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7fbbd01206c0 con 0x7fbbd406eba0 2024-09-12T12:25:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.783+0000 7fbbd9a29640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678f0 0x7fbbb8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:19.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.783+0000 7fbbd9a29640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678f0 0x7fbbb8069db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbbd41f8480 tx=0x7fbbc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:19.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.787+0000 7fbbc6ffd640 1 -- 172.21.15.3:0/2615680791 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbbd00c4030 con 0x7fbbd406eba0 2024-09-12T12:25:19.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.987+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbbd406d830 con 0x7fbbd406eba0 2024-09-12T12:25:19.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.987+0000 7fbbc6ffd640 1 -- 172.21.15.3:0/2615680791 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7fbbd0016110 con 0x7fbbd406eba0 2024-09-12T12:25:19.992 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:25:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678f0 msgr2=0x7fbbb8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678f0 0x7fbbb8069db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbbd41f8480 tx=0x7fbbc8008040 comp rx=0 tx=0).stop 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd406eba0 msgr2=0x7fbbd40aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd406eba0 0x7fbbd40aedd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00ae040 tx=0x7fbbd001ac40 comp rx=0 tx=0).stop 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 shutdown_connections 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbbb80678f0 0x7fbbb8069db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbd40b4c70 0x7fbbd414bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbbd40b3df0 0x7fbbd40adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 --2- 172.21.15.3:0/2615680791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbd406eba0 0x7fbbd40aedd0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:19.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 >> 172.21.15.3:0/2615680791 conn(0x7fbbd40628b0 msgr2=0x7fbbd4156760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:19.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 shutdown_connections 2024-09-12T12:25:19.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:19.991+0000 7fbbdc48e640 1 -- 172.21.15.3:0/2615680791 wait complete. 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:25:20.009 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:25:20.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:20.010 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T12:25:20.016 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:25:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:25:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:25:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T12:25:20.016 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-12T12:25:20.017 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42166sec preferred_lft 42166sec 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 02:42:78:71:83:46 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.018 INFO:teuthology.orchestra.run.smithi003.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:25:20.019 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.020 INFO:teuthology.orchestra.run.smithi003.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:25:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:25:20.020 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:20.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:20.020 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:25:20.071 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T12:25:20.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:20.072 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T12:25:20.073 INFO:tasks.ceph.mds.a.smithi003.stderr:2024-09-12T12:25:20.067+0000 7fe4c71fb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:25:20.118 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T12:25:20.119 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:25:20.119 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-12T12:25:20.119 INFO:teuthology.orchestra.run.smithi003.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:25:20.119 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-12T12:25:20.119 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:25:20.119 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:25:20.119 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T12:25:20.119 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T12:25:20.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T12:25:20.120 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T12:25:20.120 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T12:25:20.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:25:20.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:25:20.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:20.243 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:25:20.249 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:25:20.249 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:25:20.249 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:25:20.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:20.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T12:25:20.304 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-12T12:25:20.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:25:20.304 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:20.354 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:25:20.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:25:20.355 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:20.406 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:25:20.463 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:25:20.512 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:25:20.513 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:20.514 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:20.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:20.515 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:25:20.562 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:25:20.562 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-12T12:25:20.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:25:20.623 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:25:20.670 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:25:20.671 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:20.672 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:20.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:20.673 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:25:20.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:2024-09-12T12:25:20.687+0000 7f61ff879580 -1 init, newargv = 0x556a890e22a0 newargc=15 2024-09-12T12:25:20.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[45057]: starting ceph client 2024-09-12T12:25:20.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[45057]: starting fuse 2024-09-12T12:25:21.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-09-12T12:25:21.815 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:25:21.864 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-09-12T12:25:21.864 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/run/lock 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:25:21.865 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/fs/nfsd 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/run/rpc_pipefs 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-0 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-1 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-2 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph/osd/ceph-3 2024-09-12T12:25:21.866 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-09-12T12:25:21.867 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:21.867 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:21.867 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:21.867 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:21.867 INFO:teuthology.orchestra.run.smithi003.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:21.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:21.868 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-09-12T12:25:21.914 INFO:teuthology.orchestra.run.smithi003.stdout:45 2024-09-12T12:25:21.914 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:25:21.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:25:21.914 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-09-12T12:25:21.914 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-09-12T12:25:21.914 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-09-12T12:25:21.914 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-09-12T12:25:21.914 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-09-12T12:25:21.915 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-09-12T12:25:21.916 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-09-12T12:25:21.916 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:25:21.916 DEBUG:teuthology.orchestra.run.smithi003:> 2024-09-12T12:25:21.916 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.0")) 2024-09-12T12:25:21.916 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-09-12T12:25:22.040 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.0.45057.asok 2024-09-12T12:25:22.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:25:22.053 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45057.asok 2024-09-12T12:25:22.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.054 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45057.asok status 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "45057", 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:25:22.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7047, 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 7047 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3119306952 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:22.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3119306952 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.7047 192.168.0.1:0/3119306952", 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/3119306952", 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 70, 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 97, 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-09-12T12:25:22.221 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:22.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.233 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-09-12T12:25:22.233 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:22.241 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-09-12T12:25:22.241 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:22.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:22.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-12T12:25:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42e8f3640 1 Processor -- start 2024-09-12T12:25:22.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42e8f3640 1 -- start start 2024-09-12T12:25:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42e8f3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe4280b3cf0 0x7fe428099920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42e8f3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe42806d5d0 0x7fe42806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42e8f3640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4280ccb30 con 0x7fe4280d04e0 2024-09-12T12:25:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42e8f3640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe4280cd740 con 0x7fe4280b3cf0 2024-09-12T12:25:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42e8f3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe4280cce10 con 0x7fe42806d5d0 2024-09-12T12:25:22.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe427fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe4280d04e0 0x7fe4280d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49312/0 (socket says 172.21.15.3:49312) 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe427fff640 1 -- 172.21.15.3:0/718236434 learned_addr learned my addr 172.21.15.3:0/718236434 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe4277fe640 1 --2- 172.21.15.3:0/718236434 >> v2:172.21.15.3:3300/0 conn(0x7fe4280b3cf0 0x7fe428099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe42ce90640 1 --2- 172.21.15.3:0/718236434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe42806d5d0 0x7fe42806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe426ffd640 1 -- 172.21.15.3:0/718236434 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120345003 0 0) 0x7fe4280ccb30 con 0x7fe4280d04e0 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe426ffd640 1 -- 172.21.15.3:0/718236434 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe410003530 con 0x7fe4280d04e0 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe4277fe640 1 -- 172.21.15.3:0/718236434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe42806d5d0 msgr2=0x7fe42806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe4277fe640 1 --2- 172.21.15.3:0/718236434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe42806d5d0 0x7fe42806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe4277fe640 1 -- 172.21.15.3:0/718236434 >> v1:172.21.15.3:6789/0 conn(0x7fe4280d04e0 legacy=0x7fe4280d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:22.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.467+0000 7fe4277fe640 1 -- 172.21.15.3:0/718236434 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4280cd0f0 con 0x7fe4280b3cf0 2024-09-12T12:25:22.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe4277fe640 1 --2- 172.21.15.3:0/718236434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4280b3cf0 0x7fe428099920 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0097c0 tx=0x7fe41c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cdaaf3e1f0bb1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:22.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe426ffd640 1 -- 172.21.15.3:0/718236434 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0b0070 con 0x7fe4280b3cf0 2024-09-12T12:25:22.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe426ffd640 1 -- 172.21.15.3:0/718236434 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41c0ab600 con 0x7fe4280b3cf0 2024-09-12T12:25:22.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe426ffd640 1 -- 172.21.15.3:0/718236434 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0ab9d0 con 0x7fe4280b3cf0 2024-09-12T12:25:22.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- 172.21.15.3:0/718236434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4280b3cf0 msgr2=0x7fe428099920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:22.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/718236434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4280b3cf0 0x7fe428099920 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0097c0 tx=0x7fe41c01c230 comp rx=0 tx=0).stop 2024-09-12T12:25:22.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- 172.21.15.3:0/718236434 shutdown_connections 2024-09-12T12:25:22.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/718236434 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe42806d5d0 0x7fe42806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/718236434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4280b3cf0 0x7fe428099920 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- 172.21.15.3:0/718236434 >> 172.21.15.3:0/718236434 conn(0x7fe4280628b0 msgr2=0x7fe428062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:22.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- 172.21.15.3:0/718236434 shutdown_connections 2024-09-12T12:25:22.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- 172.21.15.3:0/718236434 wait complete. 2024-09-12T12:25:22.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 Processor -- start 2024-09-12T12:25:22.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- start start 2024-09-12T12:25:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42806d5d0 0x7fe4280adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4280b3cf0 0x7fe4280afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4280d04e0 0x7fe4280b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe42816b420 con 0x7fe42806d5d0 2024-09-12T12:25:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe428161fa0 con 0x7fe4280b3cf0 2024-09-12T12:25:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe42e8f3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe42816b5a0 con 0x7fe4280d04e0 2024-09-12T12:25:22.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe427fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42806d5d0 0x7fe4280adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe427fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42806d5d0 0x7fe4280adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54398/0 (socket says 172.21.15.3:54398) 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.471+0000 7fe427fff640 1 -- 172.21.15.3:0/1190445503 learned_addr learned my addr 172.21.15.3:0/1190445503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe427fff640 1 -- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4280d04e0 msgr2=0x7fe4280b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe427fff640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4280d04e0 0x7fe4280b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe427fff640 1 -- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4280b3cf0 msgr2=0x7fe4280afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe4277fe640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4280b3cf0 0x7fe4280afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe427fff640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4280b3cf0 0x7fe4280afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe427fff640 1 -- 172.21.15.3:0/1190445503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4280ae5b0 con 0x7fe42806d5d0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe427fff640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42806d5d0 0x7fe4280adb80 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe41800c3c0 tx=0x7fe41800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe424ff9640 1 -- 172.21.15.3:0/1190445503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418017070 con 0x7fe42806d5d0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe424ff9640 1 -- 172.21.15.3:0/1190445503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe418005cf0 con 0x7fe42806d5d0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4280ae890 con 0x7fe42806d5d0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe424ff9640 1 -- 172.21.15.3:0/1190445503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4180072f0 con 0x7fe42806d5d0 2024-09-12T12:25:22.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe428153a20 con 0x7fe42806d5d0 2024-09-12T12:25:22.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42806da60 con 0x7fe42806d5d0 2024-09-12T12:25:22.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe424ff9640 1 -- 172.21.15.3:0/1190445503 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe41801a070 con 0x7fe42806d5d0 2024-09-12T12:25:22.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe424ff9640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe4140678c0 0x7fe414069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:22.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe424ff9640 1 -- 172.21.15.3:0/1190445503 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7fe418085120 con 0x7fe42806d5d0 2024-09-12T12:25:22.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe4277fe640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe4140678c0 0x7fe414069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:22.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.475+0000 7fe4277fe640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe4140678c0 0x7fe414069d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0ad000 tx=0x7fe41c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:22.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.479+0000 7fe424ff9640 1 -- 172.21.15.3:0/1190445503 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe418055bb0 con 0x7fe42806d5d0 2024-09-12T12:25:22.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.683+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe4280cae10 con 0x7fe42806d5d0 2024-09-12T12:25:22.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.683+0000 7fe424ff9640 1 -- 172.21.15.3:0/1190445503 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7fe4180554b0 con 0x7fe42806d5d0 2024-09-12T12:25:22.688 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe4140678c0 msgr2=0x7fe414069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe4140678c0 0x7fe414069d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0ad000 tx=0x7fe41c008040 comp rx=0 tx=0).stop 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42806d5d0 msgr2=0x7fe4280adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42806d5d0 0x7fe4280adb80 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe41800c3c0 tx=0x7fe41800c880 comp rx=0 tx=0).stop 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 shutdown_connections 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe4140678c0 0x7fe414069d80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe4280d04e0 0x7fe4280b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe4280b3cf0 0x7fe4280afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 --2- 172.21.15.3:0/1190445503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe42806d5d0 0x7fe4280adb80 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 >> 172.21.15.3:0/1190445503 conn(0x7fe4280628b0 msgr2=0x7fe4280cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 shutdown_connections 2024-09-12T12:25:22.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:22.687+0000 7fe42e8f3640 1 -- 172.21.15.3:0/1190445503 wait complete. 2024-09-12T12:25:22.708 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:25:22.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:25:22.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-09-12T12:25:22.708 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-12T12:25:22.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:22.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:25:22.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:25:22.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:25:22.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:25:22.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.709 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-09-12T12:25:22.716 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:25:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:25:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:25:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-09-12T12:25:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42162sec preferred_lft 42162sec 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-09-12T12:25:22.717 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 02:42:0c:a6:cd:e9 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-12T12:25:22.718 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.719 INFO:teuthology.orchestra.run.smithi142.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-12T12:25:22.719 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:22.719 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-12T12:25:22.719 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:25:22.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.719 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:25:22.776 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-09-12T12:25:22.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.778 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-09-12T12:25:22.831 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-12T12:25:22.832 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:25:22.832 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:25:22.833 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-09-12T12:25:22.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:25:22.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:25:22.833 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:25:22.833 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-09-12T12:25:22.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:25:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:25:22.952 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:25:22.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.960 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:25:22.965 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:25:22.965 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:25:22.966 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:25:22.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:22.966 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-09-12T12:25:23.022 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-12T12:25:23.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:25:23.023 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:23.075 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-12T12:25:23.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:25:23.076 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:23.122 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:25:23.181 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:25:23.228 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:25:23.229 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:23.230 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:23.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:23.231 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:25:23.279 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T12:25:23.279 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-12T12:25:23.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:25:23.338 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:25:23.387 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:25:23.388 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:23.389 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:23.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:23.390 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:25:23.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:2024-09-12T12:25:23.398+0000 7f214ec2c580 -1 init, newargv = 0x55e381364300 newargc=15 2024-09-12T12:25:23.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36730]: starting ceph client 2024-09-12T12:25:23.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36730]: starting fuse 2024-09-12T12:25:24.506 DEBUG:teuthology.orchestra.run.smithi142:> sudo modprobe fuse 2024-09-12T12:25:24.521 DEBUG:teuthology.orchestra.run.smithi142:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/sys 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/proc 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/dev 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/pts 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/run 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/ 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/security 2024-09-12T12:25:24.568 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/shm 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/run/lock 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/cgroup 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/pstore 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/bpf 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/hugepages 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/dev/mqueue 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/debug 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/tracing 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/proc/fs/nfsd 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/fs/fuse/connections 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/sys/kernel/config 2024-09-12T12:25:24.569 INFO:teuthology.orchestra.run.smithi142.stdout:/run/credentials/systemd-sysusers.service 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/run/rpc_pipefs 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/run/user/1000 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-4 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-5 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-6 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/var/lib/ceph/osd/ceph-7 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:24.570 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:24.571 INFO:teuthology.orchestra.run.smithi142.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:24.571 INFO:teuthology.orchestra.run.smithi142.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:24.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:24.572 DEBUG:teuthology.orchestra.run.smithi142:> ls /sys/fs/fuse/connections 2024-09-12T12:25:24.618 INFO:teuthology.orchestra.run.smithi142.stdout:45 2024-09-12T12:25:24.619 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-12T12:25:24.619 DEBUG:teuthology.orchestra.run.smithi142:> sudo stdin-killer -- python3 -c ' 2024-09-12T12:25:24.619 DEBUG:teuthology.orchestra.run.smithi142:> import glob 2024-09-12T12:25:24.619 DEBUG:teuthology.orchestra.run.smithi142:> import re 2024-09-12T12:25:24.619 DEBUG:teuthology.orchestra.run.smithi142:> import os 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> import subprocess 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> def _find_admin_socket(client_name): 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> files = glob.glob(asok_path) 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> # Given a non-glob path, it better be there 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> if "*" not in asok_path: 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> assert(len(files) == 1) 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> return files[0] 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:25:24.620 DEBUG:teuthology.orchestra.run.smithi142:> for f in files: 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> contents = proc_f.read() 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> if mountpoint in contents: 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> return f 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> print(_find_admin_socket("client.1")) 2024-09-12T12:25:24.621 DEBUG:teuthology.orchestra.run.smithi142:> ' 2024-09-12T12:25:24.744 INFO:teuthology.orchestra.run.smithi142.stdout:/var/run/ceph/ceph-client.1.36730.asok 2024-09-12T12:25:24.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-09-12T12:25:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:25:24.756 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36730.asok 2024-09-12T12:25:24.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:24.757 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36730.asok status 2024-09-12T12:25:24.911 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "metadata": { 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "entity_id": "1", 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "hostname": "smithi142", 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "pid": "36730", 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "root": "/" 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:25:24.912 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_count": 0, 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "dentry_pinned_count": 0, 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "id": 6217, 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "inst": { 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "name": { 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "client", 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "num": 6217 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 3830538424 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-09-12T12:25:24.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": { 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "type": "v1", 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "nonce": 3830538424 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "inst_str": "client.6217 192.168.0.1:0/3830538424", 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "addr_str": "192.168.0.1:0/3830538424", 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "inode_count": 1, 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_epoch": 70, 2024-09-12T12:25:24.914 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch": 97, 2024-09-12T12:25:24.915 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_epoch_barrier": 0, 2024-09-12T12:25:24.915 INFO:teuthology.orchestra.run.smithi142.stdout: "blocklisted": false, 2024-09-12T12:25:24.915 INFO:teuthology.orchestra.run.smithi142.stdout: "fs_name": "cephfs" 2024-09-12T12:25:24.915 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-09-12T12:25:24.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:24.926 DEBUG:teuthology.orchestra.run.smithi142:> stat --file-system '--printf=%T 2024-09-12T12:25:24.926 DEBUG:teuthology.orchestra.run.smithi142:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:24.932 INFO:teuthology.orchestra.run.smithi142.stdout:fuseblk 2024-09-12T12:25:24.932 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:24.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:24.933 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:24.989 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T12:25:25.145 INFO:teuthology.orchestra.run.smithi003.stdout:[{"id":6217,"entity":{"name":{"type":"client","num":6217},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3830538424}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":1.7279846139999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.99602467229186886,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6217 192.168.0.1:0/3830538424","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"1","hostname":"smithi142","mount_point":"/home/ubuntu/cephtest/mnt.1","pid":"36730","root":"/"}},{"id":7047,"entity":{"name":{"type":"client","num":7047},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3119306952}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":0,"uptime":4.4359604939999997,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":0.98981224717993033,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":1,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7047 192.168.0.1:0/3119306952","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc","ceph_version":"ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)","entity_id":"0","hostname":"smithi003","mount_point":"/home/ubuntu/cephtest/mnt.0","pid":"45057","root":"/"}}] 2024-09-12T12:25:25.146 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "1", "hostname": "smithi142", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "36730", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3830538424, "type": "any" }, "name": { "num": 6217, "type": "client" } }, "id": 6217, "inst": "client.6217 192.168.0.1:0/3830538424", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9960246722918689 }, "state": "open", "uptime": 1.727984614 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "0", "hostname": "smithi003", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "45057", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3119306952, "type": "any" }, "name": { "num": 7047, "type": "client" } }, "id": 7047, "inst": "client.7047 192.168.0.1:0/3119306952", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 1, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 0, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 0.9898122471799303 }, "state": "open", "uptime": 4.435960494 } ] 2024-09-12T12:25:25.146 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-12T12:25:25.307 INFO:teuthology.orchestra.run.smithi003.stdout:{"message":"","return_code":0} 2024-09-12T12:25:25.308 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-12T12:25:25.308 DEBUG:teuthology.orchestra.run.smithi003:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-09-12T12:25:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 -- start start 2024-09-12T12:25:25.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 Processor -- start 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e6809800 0x55b5e6884580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e6809400 0x55b5e6884b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e6809000 0x55b5e6885080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b5e75f0180 con 0x55b5e6809400 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b5e75f0300 con 0x55b5e6809000 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b5e75f0480 con 0x55b5e6809800 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e6809400 0x55b5e6884b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e6809400 0x55b5e6884b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54420/0 (socket says 172.21.15.3:54420) 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 -- 172.21.15.3:0/1107992561 learned_addr learned my addr 172.21.15.3:0/1107992561 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc913d30640 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e6809800 0x55b5e6884580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc914531640 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e6809000 0x55b5e6885080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 -- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e6809800 msgr2=0x55b5e6884580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e6809800 0x55b5e6884580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 -- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e6809000 msgr2=0x55b5e6885080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e6809000 0x55b5e6885080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 -- 172.21.15.3:0/1107992561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e68e4b40 con 0x55b5e6809400 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91352f640 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e6809400 0x55b5e6884b00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55b5e76809c0 tx=0x55b5e68e69c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5900fc233b8e3bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91152b640 1 -- 172.21.15.3:0/1107992561 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5e68409c0 con 0x55b5e6809400 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91152b640 1 -- 172.21.15.3:0/1107992561 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b5e6850e00 con 0x55b5e6809400 2024-09-12T12:25:25.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91152b640 1 -- 172.21.15.3:0/1107992561 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5e68404e0 con 0x55b5e6809400 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 -- 172.21.15.3:0/1107992561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e68e4d20 con 0x55b5e6809400 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 1 -- 172.21.15.3:0/1107992561 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b5e68e4f00 con 0x55b5e6809400 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 4 waiting for MDS map... 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91152b640 1 -- 172.21.15.3:0/1107992561 <== mon.0 v2:172.21.15.3:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x55b5e75d4a00 con 0x55b5e6809400 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc91152b640 1 -- 172.21.15.3:0/1107992561 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x55b5e685f180 con 0x55b5e6809400 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 4 Got MDS map 70 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 10 main: main 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.343+0000 7fc915571400 4 main: connecting to RADOS... 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 Processor -- start 2024-09-12T12:25:25.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- start start 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e75f3c00 0x55b5e6885600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e7784000 0x55b5e6885b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7784400 0x55b5e6886100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b5e75f0600 con 0x55b5e75f3c00 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b5e75f0780 con 0x55b5e7784000 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b5e75f0900 con 0x55b5e7784400 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e75f3c00 0x55b5e6885600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e75f3c00 0x55b5e6885600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54428/0 (socket says 172.21.15.3:54428) 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 -- 172.21.15.3:0/3768549811 learned_addr learned my addr 172.21.15.3:0/3768549811 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc91352f640 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7784400 0x55b5e6886100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 -- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7784400 msgr2=0x55b5e6886100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:25.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc913d30640 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e7784000 0x55b5e6885b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7784400 0x55b5e6886100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 -- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e7784000 msgr2=0x55b5e6885b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e7784000 0x55b5e6885b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 -- 172.21.15.3:0/3768549811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e68e41e0 con 0x55b5e75f3c00 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc91352f640 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7784400 0x55b5e6886100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc913d30640 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e7784000 0x55b5e6885b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e75f3c00 0x55b5e6885600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x55b5e772a870 tx=0x55b5e7734c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a028c47528535a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc90f527640 1 -- 172.21.15.3:0/3768549811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5e68409c0 con 0x55b5e75f3c00 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc90f527640 1 -- 172.21.15.3:0/3768549811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b5e6850a80 con 0x55b5e75f3c00 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc90f527640 1 -- 172.21.15.3:0/3768549811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5e68404e0 con 0x55b5e75f3c00 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- 172.21.15.3:0/3768549811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e75f3c00 msgr2=0x55b5e6885600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e75f3c00 0x55b5e6885600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x55b5e772a870 tx=0x55b5e7734c00 comp rx=0 tx=0).stop 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- 172.21.15.3:0/3768549811 shutdown_connections 2024-09-12T12:25:25.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7784400 0x55b5e6886100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e7784000 0x55b5e6885b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- 172.21.15.3:0/3768549811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e75f3c00 0x55b5e6885600 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- 172.21.15.3:0/3768549811 >> 172.21.15.3:0/3768549811 conn(0x55b5e6808c00 msgr2=0x55b5e6837800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- 172.21.15.3:0/3768549811 shutdown_connections 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- 172.21.15.3:0/3768549811 wait complete. 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 Processor -- start 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- start start 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e75f3c00 0x55b5e6887180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7785800 0x55b5e6886c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e7785400 0x55b5e6886680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b5e75f1980 con 0x55b5e7785400 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b5e75f0300 con 0x55b5e75f3c00 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc915571400 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55b5e75f0c00 con 0x55b5e7785800 2024-09-12T12:25:25.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc91352f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e7785400 0x55b5e6886680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc91352f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e7785400 0x55b5e6886680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54440/0 (socket says 172.21.15.3:54440) 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc91352f640 1 -- 172.21.15.3:0/174853942 learned_addr learned my addr 172.21.15.3:0/174853942 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc913d30640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7785800 0x55b5e6886c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.347+0000 7fc914531640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e75f3c00 0x55b5e6887180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc91352f640 1 -- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7785800 msgr2=0x55b5e6886c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc91352f640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e7785800 0x55b5e6886c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc91352f640 1 -- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e75f3c00 msgr2=0x55b5e6887180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc91352f640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e75f3c00 0x55b5e6887180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc91352f640 1 -- 172.21.15.3:0/174853942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e68e5a40 con 0x55b5e7785400 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc91352f640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e7785400 0x55b5e6886680 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x55b5e7681830 tx=0x55b5e68e6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc90dd24640 1 -- 172.21.15.3:0/174853942 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5e6840b60 con 0x55b5e7785400 2024-09-12T12:25:25.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc90dd24640 1 -- 172.21.15.3:0/174853942 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b5e6850c40 con 0x55b5e7785400 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b5e68e5c20 con 0x55b5e7785400 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc90dd24640 1 -- 172.21.15.3:0/174853942 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b5e6840340 con 0x55b5e7785400 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e68e4d20 con 0x55b5e7785400 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 4 main: resolving pool 14 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc90dd24640 1 -- 172.21.15.3:0/174853942 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x55b5e6afec00 con 0x55b5e7785400 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc90dd24640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55b5e7797800 0x55b5e6887700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc90dd24640 1 -- 172.21.15.3:0/174853942 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x55b5e6af9000 con 0x55b5e7785400 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 4 main: creating IoCtx.. 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55b5e7785000 0x55b5e6886100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55b5e7784c00 con 0x55b5e7785000 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc913d30640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55b5e7785000 0x55b5e6886100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc914531640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55b5e7797800 0x55b5e6887700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc913d30640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55b5e7785000 0x55b5e6886100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc913d30640 1 -- 172.21.15.3:0/174853942 <== osd.0 v2:172.21.15.3:6816/3370158904 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv5 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x55b5e68fe000 con 0x55b5e7785000 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- osd_op(unknown.0.0:2 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55b5e7784800 con 0x55b5e7785000 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc914531640 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55b5e7797800 0x55b5e6887700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55b5e772a870 tx=0x55b5e68e6a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc913d30640 1 -- 172.21.15.3:0/174853942 <== osd.0 v2:172.21.15.3:6816/3370158904 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=845b] v0'0 uv5 ondisk = 0) ==== 159+0+845 (crc 0 0 0) 0x55b5e68fe000 con 0x55b5e7785000 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.351+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- osd_op(unknown.0.0:3 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55b5e7784400 con 0x55b5e7785000 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc913d30640 1 -- 172.21.15.3:0/174853942 <== osd.0 v2:172.21.15.3:6816/3370158904 3 ==== osd_op_reply(3 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x55b5e68fe000 con 0x55b5e7785000 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stdout: "0": { 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stdout: "data": { 2024-09-12T12:25:25.362 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [ 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 7047, 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": { 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 7047 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "any", 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:25:25.363 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3119306952 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open", 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "num_leases": 0, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 0, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "request_load_avg": 0, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 0.0039999659999999998, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "requests_in_flight": 0, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "num_completed_requests": 0, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "num_completed_flushes": 0, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "reconnecting": false, 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "recall_caps": { 2024-09-12T12:25:25.364 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 60 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "release_caps": { 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 60 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "recall_caps_throttle": { 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 1.5 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "recall_caps_throttle2o": { 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 0.5 2024-09-12T12:25:25.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "session_cache_liveness": { 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 300 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition": { 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 30 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "last_trim_completed_requests_tid": 0, 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "last_trim_completed_flushes_tid": 0, 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "delegated_inos": [], 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": "client.7047 192.168.0.1:0/3119306952", 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "completed_requests": [], 2024-09-12T12:25:25.366 INFO:teuthology.orchestra.run.smithi003.stdout: "prealloc_inos": [], 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "client_metadata": { 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "client_features": { 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "feature_bits": "0x00000000003fffff" 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "metric_spec": { 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "metric_flags": { 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "feature_bits": "0x000000000000ffff" 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "0", 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-09-12T12:25:25.367 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "45057", 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6217, 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "entity": { 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 6217 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "any", 2024-09-12T12:25:25.368 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3830538424 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "open", 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "num_leases": 0, 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "num_caps": 0, 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "request_load_avg": 0, 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 0.0039999659999999998, 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "requests_in_flight": 0, 2024-09-12T12:25:25.369 INFO:teuthology.orchestra.run.smithi003.stdout: "num_completed_requests": 0, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "num_completed_flushes": 0, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "reconnecting": false, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "recall_caps": { 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 60 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "release_caps": { 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 60 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "recall_caps_throttle": { 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 1.5 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.370 INFO:teuthology.orchestra.run.smithi003.stdout: "recall_caps_throttle2o": { 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 0.5 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "session_cache_liveness": { 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 300 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition": { 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "value": 0, 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "halflife": 30 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "last_trim_completed_requests_tid": 0, 2024-09-12T12:25:25.371 INFO:teuthology.orchestra.run.smithi003.stdout: "last_trim_completed_flushes_tid": 0, 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "delegated_inos": [], 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": "client.6217 192.168.0.1:0/3830538424", 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "completed_requests": [], 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "prealloc_inos": [], 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "client_metadata": { 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "client_features": { 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "feature_bits": "0x00000000003fffff" 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "metric_spec": { 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "metric_flags": { 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "feature_bits": "0x000000000000ffff" 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.372 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "1", 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi142", 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "36730", 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: "result": 0 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:25.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55b5e7785000 msgr2=0x55b5e6886100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55b5e7785000 0x55b5e6886100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55b5e7797800 msgr2=0x55b5e6887700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55b5e7797800 0x55b5e6887700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55b5e772a870 tx=0x55b5e68e6a80 comp rx=0 tx=0).stop 2024-09-12T12:25:25.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e7785400 msgr2=0x55b5e6886680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 --2- 172.21.15.3:0/174853942 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e7785400 0x55b5e6886680 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x55b5e7681830 tx=0x55b5e68e6600 comp rx=0 tx=0).stop 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc913d30640 1 -- 172.21.15.3:0/174853942 reap_dead start 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 shutdown_connections 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 >> 172.21.15.3:0/174853942 conn(0x55b5e6808c00 msgr2=0x55b5e6837800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 shutdown_connections 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/174853942 wait complete. 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/1107992561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e6809400 msgr2=0x55b5e6884b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e6809400 0x55b5e6884b00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55b5e76809c0 tx=0x55b5e68e69c0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/1107992561 shutdown_connections 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5e6809000 0x55b5e6885080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5e6809400 0x55b5e6884b00 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 --2- 172.21.15.3:0/1107992561 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55b5e6809800 0x55b5e6884580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/1107992561 >> 172.21.15.3:0/1107992561 conn(0x55b5e6809c00 msgr2=0x55b5e6837000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:25.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/1107992561 shutdown_connections 2024-09-12T12:25:25.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.355+0000 7fc915571400 1 -- 172.21.15.3:0/1107992561 wait complete. 2024-09-12T12:25:25.377 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.068849, result: { "0": { "data": { "sessions": [ { "id": 7047, "entity": { "name": { "type": "client", "num": 7047 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 3119306952 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.0039999659999999998, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7047 192.168.0.1:0/3119306952", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x00000000003fffff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "entity_id": "0", "hostname": "smithi003", "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "45057", "root": "/" } }, { "id": 6217, "entity": { "name": { "type": "client", "num": 6217 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 3830538424 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.0039999659999999998, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.6217 192.168.0.1:0/3830538424", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x00000000003fffff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "entity_id": "1", "hostname": "smithi142", "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "36730", "root": "/" } } ] }, "result": 0 } } 2024-09-12T12:25:25.378 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [ { "id": 7047, "entity": { "name": { "type": "client", "num": 7047 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 3119306952 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.003999966, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.7047 192.168.0.1:0/3119306952", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x00000000003fffff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "entity_id": "0", "hostname": "smithi003", "mount_point": "/home/ubuntu/cephtest/mnt.0", "pid": "45057", "root": "/" } }, { "id": 6217, "entity": { "name": { "type": "client", "num": 6217 }, "addr": { "type": "any", "addr": "192.168.0.1:0", "nonce": 3830538424 } }, "state": "open", "num_leases": 0, "num_caps": 0, "request_load_avg": 0, "uptime": 0.003999966, "requests_in_flight": 0, "num_completed_requests": 0, "num_completed_flushes": 0, "reconnecting": false, "recall_caps": { "value": 0, "halflife": 60 }, "release_caps": { "value": 0, "halflife": 60 }, "recall_caps_throttle": { "value": 0, "halflife": 1.5 }, "recall_caps_throttle2o": { "value": 0, "halflife": 0.5 }, "session_cache_liveness": { "value": 0, "halflife": 300 }, "cap_acquisition": { "value": 0, "halflife": 30 }, "last_trim_completed_requests_tid": 0, "last_trim_completed_flushes_tid": 0, "delegated_inos": [], "inst": "client.6217 192.168.0.1:0/3830538424", "completed_requests": [], "prealloc_inos": [], "client_metadata": { "client_features": { "feature_bits": "0x00000000003fffff" }, "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", "entity_id": "1", "hostname": "smithi142", "mount_point": "/home/ubuntu/cephtest/mnt.1", "pid": "36730", "root": "/" } } ] }, "result": 0 } } 2024-09-12T12:25:25.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:25.378 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir bravo) 2024-09-12T12:25:25.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:25.429 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch bravo/file_a) 2024-09-12T12:25:25.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:25.483 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest/mnt.1 && exec touch bravo/file_b) 2024-09-12T12:25:25.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-09-12T12:25:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49d49ed640 1 Processor -- start 2024-09-12T12:25:25.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49d49ed640 1 -- start start 2024-09-12T12:25:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49d49ed640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 0x7f49d00d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49d49ed640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d00b5fa0 con 0x7f49d015fab0 2024-09-12T12:25:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49d49ed640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d00b6240 con 0x7f49d015fe90 2024-09-12T12:25:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49d49ed640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49d00b5040 con 0x7f49d00b7d30 2024-09-12T12:25:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49ce59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f49d015fab0 0x7f49d00c7670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:49318/0 (socket says 172.21.15.3:49318) 2024-09-12T12:25:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49ce59c640 1 -- 172.21.15.3:0/1433201231 learned_addr learned my addr 172.21.15.3:0/1433201231 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:25.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49ced9d640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 0x7f49d00d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3673187016 0 0) 0x7f49d00b5fa0 con 0x7f49d015fab0 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49b8003530 con 0x7f49d015fab0 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.655+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3169414165 0 0) 0x7f49b8003530 con 0x7f49d015fab0 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 msgr2=0x7f49d00d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 0x7f49d00d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 >> v1:172.21.15.142:6789/0 conn(0x7f49d015fe90 legacy=0x7f49d00c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d00b6520 con 0x7f49d015fab0 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f49c4001680 con 0x7f49d015fab0 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 >> v1:172.21.15.3:6789/0 conn(0x7f49d015fab0 legacy=0x7f49d00c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 0x7f49b80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49b8003f70 0x7f49b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49b8004d50 0x7f49b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49b800ba80 con 0x7f49b8004d50 2024-09-12T12:25:25.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49b800bc30 con 0x7f49b8003f70 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49b800be30 con 0x7f49d00b7d30 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49b8004d50 0x7f49b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ce59c640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 0x7f49b80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cdd9b640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49b8003f70 0x7f49b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 -- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 msgr2=0x7f49b80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d00b7d30 0x7f49b80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 -- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49b8003f70 msgr2=0x7f49b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49b8003f70 0x7f49b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 -- 172.21.15.3:0/1433201231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b8003530 con 0x7f49b8004d50 2024-09-12T12:25:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49b8004d50 0x7f49b80051d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009fc0 tx=0x7f49c801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a99b6a238e503f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c80b2070 con 0x7f49b8004d50 2024-09-12T12:25:25.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c80a5c90 con 0x7f49b8004d50 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49cd59a640 1 -- 172.21.15.3:0/1433201231 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8009c10 con 0x7f49b8004d50 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49d49ed640 1 -- 172.21.15.3:0/1433201231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49b8004d50 msgr2=0x7f49b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49d49ed640 1 --2- 172.21.15.3:0/1433201231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49b8004d50 0x7f49b80051d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009fc0 tx=0x7f49c801cc80 comp rx=0 tx=0).stop 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49ced9d640 1 -- 172.21.15.3:0/1433201231 reap_dead start 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49d49ed640 1 -- 172.21.15.3:0/1433201231 shutdown_connections 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49d49ed640 1 -- 172.21.15.3:0/1433201231 >> 172.21.15.3:0/1433201231 conn(0x7f49d00b41f0 msgr2=0x7f49d00b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49d49ed640 1 -- 172.21.15.3:0/1433201231 shutdown_connections 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.659+0000 7f49d49ed640 1 -- 172.21.15.3:0/1433201231 wait complete. 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 Processor -- start 2024-09-12T12:25:25.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 -- start start 2024-09-12T12:25:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49d00cbae0 0x7f49d00a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d00cbec0 0x7f49d014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d014e150 0x7f49d01f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f49d00c8490 con 0x7f49d00cbae0 2024-09-12T12:25:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49d00b6890 con 0x7f49d00cbec0 2024-09-12T12:25:25.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f49d00c8610 con 0x7f49d014e150 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49d00cbae0 0x7f49d00a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49d00cbae0 0x7f49d00a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54454/0 (socket says 172.21.15.3:54454) 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 -- 172.21.15.3:0/829876396 learned_addr learned my addr 172.21.15.3:0/829876396 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49cdd9b640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d00cbec0 0x7f49d014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ced9d640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d014e150 0x7f49d01f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 -- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d014e150 msgr2=0x7f49d01f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d014e150 0x7f49d01f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 -- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d00cbec0 msgr2=0x7f49d014dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d00cbec0 0x7f49d014dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 -- 172.21.15.3:0/829876396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d014e5d0 con 0x7f49d00cbae0 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49ce59c640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49d00cbae0 0x7f49d00a0090 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f49c4000f70 tx=0x7f49c400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49beffd640 1 -- 172.21.15.3:0/829876396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c400c8a0 con 0x7f49d00cbae0 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49beffd640 1 -- 172.21.15.3:0/829876396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c4018070 con 0x7f49d00cbae0 2024-09-12T12:25:25.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d009f2d0 con 0x7f49d00cbae0 2024-09-12T12:25:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49beffd640 1 -- 172.21.15.3:0/829876396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c400cca0 con 0x7f49d00cbae0 2024-09-12T12:25:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d009f890 con 0x7f49d00cbae0 2024-09-12T12:25:25.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.663+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d00c7e30 con 0x7f49d00cbae0 2024-09-12T12:25:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.667+0000 7f49beffd640 1 -- 172.21.15.3:0/829876396 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f49c401d460 con 0x7f49d00cbae0 2024-09-12T12:25:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.667+0000 7f49beffd640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f49b00678f0 0x7f49b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.667+0000 7f49beffd640 1 -- 172.21.15.3:0/829876396 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f49c4085cb0 con 0x7f49d00cbae0 2024-09-12T12:25:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.667+0000 7f49cdd9b640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f49b00678f0 0x7f49b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:25.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.667+0000 7f49cdd9b640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f49b00678f0 0x7f49b0069db0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f49d015b500 tx=0x7f49c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:25.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.671+0000 7f49beffd640 1 -- 172.21.15.3:0/829876396 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f49c4056890 con 0x7f49d00cbae0 2024-09-12T12:25:25.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:25.871+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f49d00d0090 con 0x7f49d00cbae0 2024-09-12T12:25:26.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.835+0000 7f49beffd640 1 -- 172.21.15.3:0/829876396 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v71) ==== 105+0+0 (secure 0 0 0) 0x7f49c4056190 con 0x7f49d00cbae0 2024-09-12T12:25:26.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f49b00678f0 msgr2=0x7f49b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f49b00678f0 0x7f49b0069db0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f49d015b500 tx=0x7f49c0009040 comp rx=0 tx=0).stop 2024-09-12T12:25:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49d00cbae0 msgr2=0x7f49d00a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49d00cbae0 0x7f49d00a0090 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f49c4000f70 tx=0x7f49c400ed50 comp rx=0 tx=0).stop 2024-09-12T12:25:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 shutdown_connections 2024-09-12T12:25:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f49b00678f0 0x7f49b0069db0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49d014e150 0x7f49d01f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49d00cbec0 0x7f49d014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 --2- 172.21.15.3:0/829876396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f49d00cbae0 0x7f49d00a0090 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 >> 172.21.15.3:0/829876396 conn(0x7f49d00b41f0 msgr2=0x7f49d00d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 shutdown_connections 2024-09-12T12:25:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:26.839+0000 7f49d49ed640 1 -- 172.21.15.3:0/829876396 wait complete. 2024-09-12T12:25:26.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:27.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed2d6a640 1 Processor -- start 2024-09-12T12:25:27.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed2d6a640 1 -- start start 2024-09-12T12:25:27.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed2d6a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ecc15b350 0x7f4ecc06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:27.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed2d6a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc0d0bf0 0x7f4ecc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:27.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed2d6a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ecc15ef40 con 0x7f4ecc06ec30 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed2d6a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4ecc15fb10 con 0x7f4ecc15b350 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed2d6a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc15f1e0 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed0b06640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ecc15b350 0x7f4ecc06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed0b06640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4ecc15b350 0x7f4ecc06e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54468/0 (socket says 172.21.15.3:54468) 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed0b06640 1 -- 172.21.15.3:0/594952465 learned_addr learned my addr 172.21.15.3:0/594952465 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed1307640 1 --2- 172.21.15.3:0/594952465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc0d0bf0 0x7f4ecc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed0b06640 1 -- 172.21.15.3:0/594952465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc0d0bf0 msgr2=0x7f4ecc0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed0b06640 1 --2- 172.21.15.3:0/594952465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc0d0bf0 0x7f4ecc0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed0b06640 1 -- 172.21.15.3:0/594952465 >> v1:172.21.15.142:6789/0 conn(0x7f4ecc06ec30 legacy=0x7f4ecc0d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:27.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.003+0000 7f4ed0b06640 1 -- 172.21.15.3:0/594952465 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc15f4c0 con 0x7f4ecc15b350 2024-09-12T12:25:27.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed0b06640 1 --2- 172.21.15.3:0/594952465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc15b350 0x7f4ecc06e6f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009e50 tx=0x7f4ebc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48f3ea4de78bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecb7fe640 1 -- 172.21.15.3:0/594952465 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0a5780 con 0x7f4ecc15b350 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecb7fe640 1 -- 172.21.15.3:0/594952465 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc0a5920 con 0x7f4ecc15b350 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecb7fe640 1 -- 172.21.15.3:0/594952465 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0a5cf0 con 0x7f4ecc15b350 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/594952465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc15b350 msgr2=0x7f4ecc06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/594952465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc15b350 0x7f4ecc06e6f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009e50 tx=0x7f4ebc01c9f0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/594952465 shutdown_connections 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/594952465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc0d0bf0 0x7f4ecc0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/594952465 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc15b350 0x7f4ecc06e6f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/594952465 >> 172.21.15.3:0/594952465 conn(0x7f4ecc0628b0 msgr2=0x7f4ecc062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:27.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/594952465 shutdown_connections 2024-09-12T12:25:27.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/594952465 wait complete. 2024-09-12T12:25:27.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 Processor -- start 2024-09-12T12:25:27.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- start start 2024-09-12T12:25:27.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc06ec30 0x7f4ecc0712b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc0d0bf0 0x7f4ecc152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc15b350 0x7f4ecc152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0cf0a0 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0cef20 con 0x7f4ecc15b350 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed2d6a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0cf220 con 0x7f4ecc06ec30 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc0d0bf0 0x7f4ecc152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc0d0bf0 0x7f4ecc152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54484/0 (socket says 172.21.15.3:54484) 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 -- 172.21.15.3:0/1541500228 learned_addr learned my addr 172.21.15.3:0/1541500228 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed0b06640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc06ec30 0x7f4ecc0712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ed1307640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc15b350 0x7f4ecc152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 -- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc06ec30 msgr2=0x7f4ecc0712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc06ec30 0x7f4ecc0712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 -- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc15b350 msgr2=0x7f4ecc152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc15b350 0x7f4ecc152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.007+0000 7f4ecbfff640 1 -- 172.21.15.3:0/1541500228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc073510 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ecbfff640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc0d0bf0 0x7f4ecc152720 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b440 tx=0x7f4ec000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ec97fa640 1 -- 172.21.15.3:0/1541500228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0015020 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ec97fa640 1 -- 172.21.15.3:0/1541500228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0011830 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc0737f0 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ec97fa640 1 -- 172.21.15.3:0/1541500228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0011c20 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc073db0 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc073fb0 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ec97fa640 1 -- 172.21.15.3:0/1541500228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ec0018020 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ec97fa640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4eb80678c0 0x7f4eb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ec97fa640 1 -- 172.21.15.3:0/1541500228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f4ec0085970 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ed0b06640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4eb80678c0 0x7f4eb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:27.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.011+0000 7f4ed0b06640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4eb80678c0 0x7f4eb8069d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0ae000 tx=0x7f4ebc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:27.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.015+0000 7f4ec97fa640 1 -- 172.21.15.3:0/1541500228 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4ecc073fb0 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.215+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ecc063170 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ec97fa640 1 -- 172.21.15.3:0/1541500228 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5239 (secure 0 0 0) 0x7f4ec001f030 con 0x7f4ecc0d0bf0 2024-09-12T12:25:27.221 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:27.222 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"btime":"2024-09-12T12:25:26:846159+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:26.846130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6987,"mds_1":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":1,"incarnation":72,"state":"up:creating","state_seq":1,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":7}]} 2024-09-12T12:25:27.222 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 72 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4eb80678c0 msgr2=0x7f4eb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4eb80678c0 0x7f4eb8069d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0ae000 tx=0x7f4ebc0a7070 comp rx=0 tx=0).stop 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc0d0bf0 msgr2=0x7f4ecc152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc0d0bf0 0x7f4ecc152720 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000b440 tx=0x7f4ec000b900 comp rx=0 tx=0).stop 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 shutdown_connections 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4eb80678c0 0x7f4eb8069d80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ecc15b350 0x7f4ecc152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4ecc0d0bf0 0x7f4ecc152720 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 --2- 172.21.15.3:0/1541500228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ecc06ec30 0x7f4ecc0712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:27.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 >> 172.21.15.3:0/1541500228 conn(0x7f4ecc0628b0 msgr2=0x7f4ecc0b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 shutdown_connections 2024-09-12T12:25:27.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:27.219+0000 7f4ed2d6a640 1 -- 172.21.15.3:0/1541500228 wait complete. 2024-09-12T12:25:27.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:24:42.989579+0000', 'modified': '2024-09-12T12:25:26.846130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6987, 'mds_1': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 1, 'incarnation': 72, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.3:6833/1847705632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6832', 'nonce': 1847705632}, {'type': 'v1', 'addr': '172.21.15.3:6833', 'nonce': 1847705632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.3:6835/1654386752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1654386752}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1654386752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6987, 'qdb_cluster': [6987]} 2024-09-12T12:25:27.645 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6972:up:creating 2024-09-12T12:25:28.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:28.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f83a1c4b640 1 Processor -- start 2024-09-12T12:25:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f83a1c4b640 1 -- start start 2024-09-12T12:25:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f83a1c4b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f839c0b3cf0 0x7f839c099920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f83a1c4b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839c0ccb30 con 0x7f839c0d04e0 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f83a1c4b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f839c0cce10 con 0x7f839c06d5d0 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f83a1c4b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f839c0cd740 con 0x7f839c0b3cf0 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f839c0d04e0 0x7f839c0d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33914/0 (socket says 172.21.15.3:33914) 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 -- 172.21.15.3:0/1035842726 learned_addr learned my addr 172.21.15.3:0/1035842726 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839affd640 1 --2- 172.21.15.3:0/1035842726 >> v2:172.21.15.142:3300/0 conn(0x7f839c0b3cf0 0x7f839c099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539591755 0 0) 0x7f839c0ccb30 con 0x7f839c0d04e0 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8384003530 con 0x7f839c0d04e0 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 288806077 0 0) 0x7f8384003530 con 0x7f839c0d04e0 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 >> v2:172.21.15.142:3300/0 conn(0x7f839c0b3cf0 msgr2=0x7f839c099920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 --2- 172.21.15.3:0/1035842726 >> v2:172.21.15.142:3300/0 conn(0x7f839c0b3cf0 0x7f839c099920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 >> v1:172.21.15.142:6789/0 conn(0x7f839c06d5d0 legacy=0x7f839c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:28.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0cd0f0 con 0x7f839c0d04e0 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f838c001680 con 0x7f839c0d04e0 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 >> v1:172.21.15.3:6789/0 conn(0x7f839c0d04e0 legacy=0x7f839c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8384003a10 0x7f8384003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8384004810 0x7f8384004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384005220 0x7f838400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0cd740 con 0x7f8384003a10 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f838400c170 con 0x7f8384004810 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f838400c370 con 0x7f8384005220 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8384003a10 0x7f8384003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839bfff640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384005220 0x7f838400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839affd640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8384004810 0x7f8384004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 -- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384005220 msgr2=0x7f838400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:28.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384005220 0x7f838400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 -- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8384004810 msgr2=0x7f8384004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8384004810 0x7f8384004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839b7fe640 1 -- 172.21.15.3:0/1035842726 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8384003530 con 0x7f8384003a10 2024-09-12T12:25:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839affd640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8384004810 0x7f8384004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:28.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.779+0000 7f839bfff640 1 -- 172.21.15.3:0/1035842726 reap_dead start 2024-09-12T12:25:28.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f839b7fe640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8384003a10 0x7f8384003e30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c160 tx=0x7f838c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f021354c6b67f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c007c70 con 0x7f8384003a10 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c0a9070 con 0x7f8384003a10 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f839a7fc640 1 -- 172.21.15.3:0/1035842726 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c005dc0 con 0x7f8384003a10 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 -- 172.21.15.3:0/1035842726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8384003a10 msgr2=0x7f8384003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8384003a10 0x7f8384003e30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c160 tx=0x7f838c01cba0 comp rx=0 tx=0).stop 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 -- 172.21.15.3:0/1035842726 shutdown_connections 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/1035842726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8384003a10 0x7f8384003e30 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 -- 172.21.15.3:0/1035842726 >> 172.21.15.3:0/1035842726 conn(0x7f839c0628b0 msgr2=0x7f839c062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 -- 172.21.15.3:0/1035842726 shutdown_connections 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 -- 172.21.15.3:0/1035842726 wait complete. 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 Processor -- start 2024-09-12T12:25:28.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.783+0000 7f83a1c4b640 1 -- start start 2024-09-12T12:25:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384003a10 0x7f839c14b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f839c154890 0x7f839c0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f839c0b2900 0x7f839c0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f839c161f10 con 0x7f839c0b2900 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f839c161d90 con 0x7f839c154890 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0d08c0 con 0x7f8384003a10 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f839c0b2900 0x7f839c0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f839c0b2900 0x7f839c0afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47650/0 (socket says 172.21.15.3:47650) 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 -- 172.21.15.3:0/930664151 learned_addr learned my addr 172.21.15.3:0/930664151 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839affd640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f839c154890 0x7f839c0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839b7fe640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384003a10 0x7f839c14b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 -- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384003a10 msgr2=0x7f839c14b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384003a10 0x7f839c14b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 -- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f839c154890 msgr2=0x7f839c0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f839c154890 0x7f839c0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 -- 172.21.15.3:0/930664151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c0ceae0 con 0x7f839c0b2900 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839affd640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f839c154890 0x7f839c0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839b7fe640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384003a10 0x7f839c14b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839bfff640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f839c0b2900 0x7f839c0afe50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f839c0ca8b0 tx=0x7f839400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f8383fff640 1 -- 172.21.15.3:0/930664151 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394003980 con 0x7f839c0b2900 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f8383fff640 1 -- 172.21.15.3:0/930664151 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8394018070 con 0x7f839c0b2900 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c0cedc0 con 0x7f839c0b2900 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f8383fff640 1 -- 172.21.15.3:0/930664151 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394003d80 con 0x7f839c0b2900 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c0cf380 con 0x7f839c0b2900 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c0b2db0 con 0x7f839c0b2900 2024-09-12T12:25:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f8383fff640 1 -- 172.21.15.3:0/930664151 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83940050d0 con 0x7f839c0b2900 2024-09-12T12:25:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f8383fff640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f83880678c0 0x7f8388069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f8383fff640 1 -- 172.21.15.3:0/930664151 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f8394085a60 con 0x7f839c0b2900 2024-09-12T12:25:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.787+0000 7f839b7fe640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f83880678c0 0x7f8388069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.791+0000 7f839b7fe640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f83880678c0 0x7f8388069d80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8384004790 tx=0x7f838c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.791+0000 7f8383fff640 1 -- 172.21.15.3:0/930664151 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f83940564f0 con 0x7f839c0b2900 2024-09-12T12:25:28.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.995+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f839c0d0bc0 con 0x7f839c0b2900 2024-09-12T12:25:29.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.995+0000 7f8383fff640 1 -- 172.21.15.3:0/930664151 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5242 (secure 0 0 0) 0x7f8394055df0 con 0x7f839c0b2900 2024-09-12T12:25:29.001 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:29.001 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":73,"btime":"2024-09-12T12:25:27:849130+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:27.849121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6987,"mds_1":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":1,"incarnation":72,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6972,6987]},"id":7}]} 2024-09-12T12:25:29.001 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 73 2024-09-12T12:25:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f83880678c0 msgr2=0x7f8388069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f83880678c0 0x7f8388069d80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8384004790 tx=0x7f838c0a7070 comp rx=0 tx=0).stop 2024-09-12T12:25:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f839c0b2900 msgr2=0x7f839c0afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f839c0b2900 0x7f839c0afe50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f839c0ca8b0 tx=0x7f839400ec10 comp rx=0 tx=0).stop 2024-09-12T12:25:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 shutdown_connections 2024-09-12T12:25:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f83880678c0 0x7f8388069d80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f839c0b2900 0x7f839c0afe50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f839c154890 0x7f839c0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 --2- 172.21.15.3:0/930664151 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8384003a10 0x7f839c14b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 >> 172.21.15.3:0/930664151 conn(0x7f839c0628b0 msgr2=0x7f839c0ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 shutdown_connections 2024-09-12T12:25:29.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:28.999+0000 7f83a1c4b640 1 -- 172.21.15.3:0/930664151 wait complete. 2024-09-12T12:25:29.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T12:24:42.989579+0000', 'modified': '2024-09-12T12:25:27.849121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 6987, 'mds_1': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 1, 'incarnation': 72, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.3:6833/1847705632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6832', 'nonce': 1847705632}, {'type': 'v1', 'addr': '172.21.15.3:6833', 'nonce': 1847705632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.3:6835/1654386752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6834', 'nonce': 1654386752}, {'type': 'v1', 'addr': '172.21.15.3:6835', 'nonce': 1654386752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6987, 'qdb_cluster': [6972, 6987]} 2024-09-12T12:25:29.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-09-12T12:25:29.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-12T12:25:29.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d36cf640 1 Processor -- start 2024-09-12T12:25:29.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d36cf640 1 -- start start 2024-09-12T12:25:29.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d36cf640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f67cc0b6820 0x7f67cc15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d36cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc160510 0x7f67cc071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d36cf640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67cc073a60 con 0x7f67cc159ee0 2024-09-12T12:25:29.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d36cf640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc0ce6a0 con 0x7f67cc0b6820 2024-09-12T12:25:29.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d36cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc160960 con 0x7f67cc160510 2024-09-12T12:25:29.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d146b640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f67cc159ee0 0x7f67cc06fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33930/0 (socket says 172.21.15.3:33930) 2024-09-12T12:25:29.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d146b640 1 -- 172.21.15.3:0/1829451488 learned_addr learned my addr 172.21.15.3:0/1829451488 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:29.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/1829451488 >> v2:172.21.15.142:3300/0 conn(0x7f67cc0b6820 0x7f67cc15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc160510 0x7f67cc071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833097834 0 0) 0x7f67cc073a60 con 0x7f67cc159ee0 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.555+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67b8003530 con 0x7f67cc159ee0 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3525214857 0 0) 0x7f67b8003530 con 0x7f67cc159ee0 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc160510 msgr2=0x7f67cc071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc160510 0x7f67cc071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 >> v2:172.21.15.142:3300/0 conn(0x7f67cc0b6820 msgr2=0x7f67cc15ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 --2- 172.21.15.3:0/1829451488 >> v2:172.21.15.142:3300/0 conn(0x7f67cc0b6820 0x7f67cc15ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc0734a0 con 0x7f67cc159ee0 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f67c8001680 con 0x7f67cc159ee0 2024-09-12T12:25:29.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 >> v1:172.21.15.3:6789/0 conn(0x7f67cc159ee0 legacy=0x7f67cc06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b80039c0 0x7f67b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc160510 0x7f67b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b8004d50 0x7f67b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc160960 con 0x7f67b8004d50 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67b800b9d0 con 0x7f67b80039c0 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67b800bbd0 con 0x7f67cc160510 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b8004d50 0x7f67b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d1c6c640 1 -- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc160510 msgr2=0x7f67b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc160510 0x7f67b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d146b640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b80039c0 0x7f67b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d1c6c640 1 -- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b80039c0 msgr2=0x7f67b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67b80039c0 0x7f67b8003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d1c6c640 1 -- 172.21.15.3:0/1829451488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b8003530 con 0x7f67b8004d50 2024-09-12T12:25:29.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b8004d50 0x7f67b80051d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009fc0 tx=0x7f67c401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c653725207a9088 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b2070 con 0x7f67b8004d50 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c40a5ab0 con 0x7f67b8004d50 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.559+0000 7f67c3fff640 1 -- 172.21.15.3:0/1829451488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4009c10 con 0x7f67b8004d50 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- 172.21.15.3:0/1829451488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b8004d50 msgr2=0x7f67b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 --2- 172.21.15.3:0/1829451488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67b8004d50 0x7f67b80051d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009fc0 tx=0x7f67c401ccc0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d1c6c640 1 -- 172.21.15.3:0/1829451488 reap_dead start 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- 172.21.15.3:0/1829451488 shutdown_connections 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- 172.21.15.3:0/1829451488 >> 172.21.15.3:0/1829451488 conn(0x7f67cc0d0ad0 msgr2=0x7f67cc0d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- 172.21.15.3:0/1829451488 shutdown_connections 2024-09-12T12:25:29.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- 172.21.15.3:0/1829451488 wait complete. 2024-09-12T12:25:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 Processor -- start 2024-09-12T12:25:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- start start 2024-09-12T12:25:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc14be70 0x7f67cc0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc14c250 0x7f67cc164430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67cc164970 0x7f67cc150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc072990 con 0x7f67cc14c250 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc072810 con 0x7f67cc164970 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc072b10 con 0x7f67cc14be70 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc14c250 0x7f67cc164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc14c250 0x7f67cc164430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47680/0 (socket says 172.21.15.3:47680) 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 -- 172.21.15.3:0/1495385493 learned_addr learned my addr 172.21.15.3:0/1495385493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67cc164970 0x7f67cc150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d146b640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc14be70 0x7f67cc0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 -- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc14be70 msgr2=0x7f67cc0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc14be70 0x7f67cc0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 -- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67cc164970 msgr2=0x7f67cc150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67cc164970 0x7f67cc150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 -- 172.21.15.3:0/1495385493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc150420 con 0x7f67cc14c250 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc14c250 0x7f67cc164430 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00e7c0 tx=0x7f67bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67c1ffb640 1 -- 172.21.15.3:0/1495385493 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc017070 con 0x7f67cc14c250 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67c1ffb640 1 -- 172.21.15.3:0/1495385493 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67bc00ee70 con 0x7f67cc14c250 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc0ceb60 con 0x7f67cc14c250 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67c1ffb640 1 -- 172.21.15.3:0/1495385493 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00cb60 con 0x7f67cc14c250 2024-09-12T12:25:29.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.563+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc14c980 con 0x7f67cc14c250 2024-09-12T12:25:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67c1ffb640 1 -- 172.21.15.3:0/1495385493 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67bc01a070 con 0x7f67cc14c250 2024-09-12T12:25:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67c1ffb640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b00678c0 0x7f67b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67c1ffb640 1 -- 172.21.15.3:0/1495385493 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f67bc085bb0 con 0x7f67cc14c250 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d146b640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b00678c0 0x7f67b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 Processor -- start 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 -- start start 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d146b640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b00678c0 0x7f67b0069d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000f70 tx=0x7f67c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 0x55acdd618150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd618690 0x55acdd5ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd5ce6a0 0x55acdd5cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55acdd4b1b60 con 0x55acdd618690 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55acdd418c70 con 0x55acdd5ce6a0 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55acdd3beed0 con 0x55acdd617d70 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd618690 0x55acdd5ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd618690 0x55acdd5ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47690/0 (socket says 172.21.15.3:47690) 2024-09-12T12:25:29.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 -- 172.21.15.3:0/3061040176 learned_addr learned my addr 172.21.15.3:0/3061040176 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d146b640 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 0x55acdd618150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd5ce6a0 0x55acdd5cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 -- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 msgr2=0x55acdd618150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 0x55acdd618150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 -- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd5ce6a0 msgr2=0x55acdd5cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd5ce6a0 0x55acdd5cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 -- 172.21.15.3:0/3061040176 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdd3d0e50 con 0x55acdd618690 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd5ce6a0 0x55acdd5cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd618690 0x55acdd5ce160 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f67bc05e5e0 tx=0x7f67bc056070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b2a481c3d26d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67b6ffd640 1 -- 172.21.15.3:0/3061040176 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc021040 con 0x55acdd618690 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67b6ffd640 1 -- 172.21.15.3:0/3061040176 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67bc00cd00 con 0x55acdd618690 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.567+0000 7f67b6ffd640 1 -- 172.21.15.3:0/3061040176 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc055380 con 0x55acdd618690 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/3061040176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd618690 msgr2=0x55acdd5ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd618690 0x55acdd5ce160 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f67bc05e5e0 tx=0x7f67bc056070 comp rx=0 tx=0).stop 2024-09-12T12:25:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/3061040176 shutdown_connections 2024-09-12T12:25:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd5ce6a0 0x55acdd5cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd618690 0x55acdd5ce160 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- 172.21.15.3:0/3061040176 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 0x55acdd618150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/3061040176 >> 172.21.15.3:0/3061040176 conn(0x55acdd5d0600 msgr2=0x55acdd5c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/3061040176 shutdown_connections 2024-09-12T12:25:29.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/3061040176 wait complete. 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- start start 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 Processor -- start 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd617d70 0x55acdd61deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd61e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd604870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55acdd47ab00 con 0x55acdd604490 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55acdd3f3360 con 0x55acdd617d70 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55acdd3f02d0 con 0x55acdd618690 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 10 client.0 _ll_register_callbacks cb 0x55acdd5b5820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd604870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd604870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47706/0 (socket says 172.21.15.3:47706) 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 -- 172.21.15.3:0/272220775 learned_addr learned my addr 172.21.15.3:0/272220775 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd61e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d146b640 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd617d70 0x55acdd61deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 -- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 msgr2=0x55acdd61e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd61e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 -- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd617d70 msgr2=0x55acdd61deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd617d70 0x55acdd61deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 -- 172.21.15.3:0/272220775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdd3bf120 con 0x55acdd604490 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd604870 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f67c401caf0 tx=0x7f67b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7252ad935af1c8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67abfff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67abfff640 1 -- 172.21.15.3:0/272220775 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b2040 con 0x55acdd604490 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67abfff640 1 -- 172.21.15.3:0/272220775 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f67c4003dc0 con 0x55acdd604490 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67bc00cd00 con 0x55acdd604490 2024-09-12T12:25:29.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67abfff640 1 -- 172.21.15.3:0/272220775 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f67c4009c10 con 0x55acdd604490 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 10 client.7098 fetch_fsmap learned FSMap version 73 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55acdd3b8450 con 0x55acdd604490 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67abfff640 1 -- 172.21.15.3:0/272220775 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67bc00cd00 con 0x55acdd604490 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67abfff640 1 -- 172.21.15.3:0/272220775 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 73) ==== 3522+0+0 (secure 0 0 0) 0x7f67cc072dc0 con 0x55acdd604490 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 10 client.7098 fetch_fsmap finished waiting for FSMap version 73 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 10 client.7098 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 20 client.7098 populate_metadata read hostname 'smithi003' 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd605280 0x55acdd605660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 4 client.7098 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 --> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acdd3b2c00 con 0x55acdd605280 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.571+0000 7f67d146b640 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd605280 0x55acdd605660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67abfff640 10 client.7098 ms_handle_connect on v2:172.21.15.3:6832/1847705632 2024-09-12T12:25:29.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d146b640 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd605280 0x55acdd605660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2b4cd402d4edf97 server_cookie=460ee14ddd99175c in_seq=0 out_seq=0 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67abfff640 1 -- 172.21.15.3:0/272220775 <== mds.1 v2:172.21.15.3:6832/1847705632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55acdd3b2c00 con 0x55acdd605280 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67abfff640 10 client.7098 handle_command_reply: tid=0 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 client.7098 shutdown 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 msgr2=0x55acdd604870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd604870 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f67c401caf0 tx=0x7f67b8005840 comp rx=0 tx=0).stop 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 shutdown_connections 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd605280 0x55acdd605660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd604870 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd61e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67abfff640 0 client.7098 ms_handle_reset on v2:172.21.15.3:6832/1847705632 2024-09-12T12:25:29.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 --2- 172.21.15.3:0/272220775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd617d70 0x55acdd61deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 >> 172.21.15.3:0/272220775 conn(0x55acdd5ce6a0 msgr2=0x55acdd5cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 shutdown_connections 2024-09-12T12:25:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 1 -- 172.21.15.3:0/272220775 wait complete. 2024-09-12T12:25:29.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.575+0000 7f67d678b000 20 client.7098 trim_cache size 0 max 16384 2024-09-12T12:25:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 Processor -- start 2024-09-12T12:25:29.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- start start 2024-09-12T12:25:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd62e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 0x55acdd62e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd618690 0x55acdd61cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc200370 con 0x55acdd604490 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc0ce860 con 0x55acdd618690 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc0cc730 con 0x55acdd617d70 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd62e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd62e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47708/0 (socket says 172.21.15.3:47708) 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 -- 172.21.15.3:0/3964262174 learned_addr learned my addr 172.21.15.3:0/3964262174 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 -- 172.21.15.3:0/3964262174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 msgr2=0x55acdd62e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 --2- 172.21.15.3:0/3964262174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 0x55acdd62e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 -- 172.21.15.3:0/3964262174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd618690 msgr2=0x55acdd61cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 --2- 172.21.15.3:0/3964262174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd618690 0x55acdd61cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 -- 172.21.15.3:0/3964262174 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdd39ffd0 con 0x55acdd604490 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d146b640 1 --2- 172.21.15.3:0/3964262174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd62e260 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000c10 tx=0x7f67c800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d31982b714204d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67b67fc640 1 -- 172.21.15.3:0/3964262174 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c80074f0 con 0x55acdd604490 2024-09-12T12:25:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67b67fc640 1 -- 172.21.15.3:0/3964262174 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c8007690 con 0x55acdd604490 2024-09-12T12:25:29.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67b67fc640 1 -- 172.21.15.3:0/3964262174 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8004ab0 con 0x55acdd604490 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- 172.21.15.3:0/3964262174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 msgr2=0x55acdd62e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- 172.21.15.3:0/3964262174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd62e260 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000c10 tx=0x7f67c800c000 comp rx=0 tx=0).stop 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- 172.21.15.3:0/3964262174 shutdown_connections 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- 172.21.15.3:0/3964262174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd618690 0x55acdd61cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- 172.21.15.3:0/3964262174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd617d70 0x55acdd62e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- 172.21.15.3:0/3964262174 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd604490 0x55acdd62e260 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- 172.21.15.3:0/3964262174 >> 172.21.15.3:0/3964262174 conn(0x55acdd5ce6a0 msgr2=0x55acdd5c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- 172.21.15.3:0/3964262174 shutdown_connections 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- 172.21.15.3:0/3964262174 wait complete. 2024-09-12T12:25:29.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- start start 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 Processor -- start 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd604490 0x55acdd68b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd617d70 0x55acdd68bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd621cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55acdd417f80 con 0x55acdd617d70 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55acdd3d0ca0 con 0x55acdd604490 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55acdd41d2a0 con 0x55acdd618690 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d678b000 10 client.0 _ll_register_callbacks cb 0x7f67c40b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d0c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd617d70 0x55acdd68bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d0c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd617d70 0x55acdd68bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47724/0 (socket says 172.21.15.3:47724) 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.595+0000 7f67d0c6a640 1 -- 172.21.15.3:0/2589887395 learned_addr learned my addr 172.21.15.3:0/2589887395 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d1c6c640 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd621cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d146b640 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd604490 0x55acdd68b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d0c6a640 1 -- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 msgr2=0x55acdd621cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd621cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d0c6a640 1 -- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd604490 msgr2=0x55acdd68b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd604490 0x55acdd68b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d0c6a640 1 -- 172.21.15.3:0/2589887395 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdd49b750 con 0x55acdd617d70 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d0c6a640 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd617d70 0x55acdd68bb20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0047c0 tx=0x7f67bc083ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adabdf1b2702857 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 1 -- 172.21.15.3:0/2589887395 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc083070 con 0x55acdd617d70 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 1 -- 172.21.15.3:0/2589887395 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f67bc055380 con 0x55acdd617d70 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c8007690 con 0x55acdd617d70 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 1 -- 172.21.15.3:0/2589887395 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f67bc021040 con 0x55acdd617d70 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 10 client.7104 fetch_fsmap learned FSMap version 73 2024-09-12T12:25:29.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55acdd3a6a30 con 0x55acdd617d70 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 1 -- 172.21.15.3:0/2589887395 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c8007690 con 0x55acdd617d70 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 1 -- 172.21.15.3:0/2589887395 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 73) ==== 3522+0+0 (secure 0 0 0) 0x7f67bc024060 con 0x55acdd617d70 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 10 client.7104 fetch_fsmap finished waiting for FSMap version 73 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 10 client.7104 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 20 client.7104 populate_metadata read hostname 'smithi003' 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd622230 0x55acdd622610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 4 client.7104 mds_command: new command op to 6972 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 --> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] -- command(tid 0: {"prefix": "status"}) -- 0x55acdd4263a0 con 0x55acdd622230 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d146b640 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd622230 0x55acdd622610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d146b640 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd622230 0x55acdd622610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd89276d38d8b41 server_cookie=f6c29738e7a5b0ba in_seq=0 out_seq=0 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 10 client.7104 ms_handle_connect on v2:172.21.15.3:6832/1847705632 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 1 -- 172.21.15.3:0/2589887395 <== mds.1 v2:172.21.15.3:6832/1847705632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+316 (crc 0 0 0) 0x55acdd4263a0 con 0x55acdd622230 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 10 client.7104 handle_command_reply: tid=0 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 client.7104 shutdown 2024-09-12T12:25:29.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd617d70 msgr2=0x55acdd68bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd617d70 0x55acdd68bb20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0047c0 tx=0x7f67bc083ef0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 shutdown_connections 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:6832/1847705632,v1:172.21.15.3:6833/1847705632] conn(0x55acdd622230 0x55acdd622610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55acdd618690 0x55acdd621cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55acdd617d70 0x55acdd68bb20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67abfff640 0 client.7104 ms_handle_reset on v2:172.21.15.3:6832/1847705632 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 --2- 172.21.15.3:0/2589887395 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acdd604490 0x55acdd68b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 >> 172.21.15.3:0/2589887395 conn(0x55acdd5ce6a0 msgr2=0x55acdd62d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 shutdown_connections 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 1 -- 172.21.15.3:0/2589887395 wait complete. 2024-09-12T12:25:29.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d678b000 20 client.7104 trim_cache size 0 max 16384 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6972, 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 2.753846684, 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 73, 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 97, 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 97, 2024-09-12T12:25:29.609 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.4472954 2024-09-12T12:25:29.610 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b00678c0 msgr2=0x7f67b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b00678c0 0x7f67b0069d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000f70 tx=0x7f67c8009040 comp rx=0 tx=0).stop 2024-09-12T12:25:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc14c250 msgr2=0x7f67cc164430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc14c250 0x7f67cc164430 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00e7c0 tx=0x7f67bc00ec80 comp rx=0 tx=0).stop 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 shutdown_connections 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f67b00678c0 0x7f67b0069d80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67cc164970 0x7f67cc150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67cc14c250 0x7f67cc164430 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 --2- 172.21.15.3:0/1495385493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67cc14be70 0x7f67cc0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 >> 172.21.15.3:0/1495385493 conn(0x7f67cc0d0ad0 msgr2=0x7f67cc0b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 shutdown_connections 2024-09-12T12:25:29.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.599+0000 7f67d36cf640 1 -- 172.21.15.3:0/1495385493 wait complete. 2024-09-12T12:25:29.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-12T12:25:29.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049d07c640 1 Processor -- start 2024-09-12T12:25:29.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049d07c640 1 -- start start 2024-09-12T12:25:29.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049d07c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04980b38b0 0x7f04980b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049d07c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f04980d04e0 0x7f04980d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049d07c640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04980d00b0 con 0x7f04980b4c70 2024-09-12T12:25:29.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049d07c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f04980d0eb0 con 0x7f04980b38b0 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049d07c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f04980cfdd0 con 0x7f04980d04e0 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04980b38b0 0x7f04980b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049659c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f04980b38b0 0x7f04980b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47726/0 (socket says 172.21.15.3:47726) 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049659c640 1 -- 172.21.15.3:0/2830493737 learned_addr learned my addr 172.21.15.3:0/2830493737 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049759e640 1 --2- 172.21.15.3:0/2830493737 >> v2:172.21.15.142:3300/0 conn(0x7f04980d04e0 0x7f04980d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049659c640 1 -- 172.21.15.3:0/2830493737 >> v2:172.21.15.142:3300/0 conn(0x7f04980d04e0 msgr2=0x7f04980d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049659c640 1 --2- 172.21.15.3:0/2830493737 >> v2:172.21.15.142:3300/0 conn(0x7f04980d04e0 0x7f04980d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049659c640 1 -- 172.21.15.3:0/2830493737 >> v1:172.21.15.142:6789/0 conn(0x7f04980b4c70 legacy=0x7f049806e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049659c640 1 -- 172.21.15.3:0/2830493737 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049806db80 con 0x7f04980b38b0 2024-09-12T12:25:29.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.735+0000 7f049759e640 1 --2- 172.21.15.3:0/2830493737 >> v2:172.21.15.142:3300/0 conn(0x7f04980d04e0 0x7f04980d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:29.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049659c640 1 --2- 172.21.15.3:0/2830493737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b38b0 0x7f04980b3d10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f049806f3c0 tx=0x7f048c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decfaa68ac3e7264 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f0495d9b640 1 -- 172.21.15.3:0/2830493737 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0b0070 con 0x7f04980b38b0 2024-09-12T12:25:29.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f0495d9b640 1 -- 172.21.15.3:0/2830493737 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0ab600 con 0x7f04980b38b0 2024-09-12T12:25:29.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f0495d9b640 1 -- 172.21.15.3:0/2830493737 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0ab9d0 con 0x7f04980b38b0 2024-09-12T12:25:29.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 -- 172.21.15.3:0/2830493737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b38b0 msgr2=0x7f04980b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 --2- 172.21.15.3:0/2830493737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b38b0 0x7f04980b3d10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f049806f3c0 tx=0x7f048c01c400 comp rx=0 tx=0).stop 2024-09-12T12:25:29.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 -- 172.21.15.3:0/2830493737 shutdown_connections 2024-09-12T12:25:29.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 --2- 172.21.15.3:0/2830493737 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b38b0 0x7f04980b3d10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 --2- 172.21.15.3:0/2830493737 >> v2:172.21.15.142:3300/0 conn(0x7f04980d04e0 0x7f04980d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 -- 172.21.15.3:0/2830493737 >> 172.21.15.3:0/2830493737 conn(0x7f04980628b0 msgr2=0x7f0498062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 -- 172.21.15.3:0/2830493737 shutdown_connections 2024-09-12T12:25:29.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 -- 172.21.15.3:0/2830493737 wait complete. 2024-09-12T12:25:29.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 Processor -- start 2024-09-12T12:25:29.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.739+0000 7f049d07c640 1 -- start start 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04980b38b0 0x7f0498156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b4c70 0x7f04980ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980d04e0 0x7f04980add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f04981692a0 con 0x7f04980b4c70 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04980cfac0 con 0x7f04980d04e0 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0498169420 con 0x7f04980b38b0 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b4c70 0x7f04980ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b4c70 0x7f04980ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47728/0 (socket says 172.21.15.3:47728) 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 -- 172.21.15.3:0/1521282995 learned_addr learned my addr 172.21.15.3:0/1521282995 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049759e640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980d04e0 0x7f04980add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f0496d9d640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04980b38b0 0x7f0498156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 -- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04980b38b0 msgr2=0x7f0498156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04980b38b0 0x7f0498156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 -- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980d04e0 msgr2=0x7f04980add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980d04e0 0x7f04980add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 -- 172.21.15.3:0/1521282995 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0498165650 con 0x7f04980b4c70 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049759e640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980d04e0 0x7f04980add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049659c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b4c70 0x7f04980ad840 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f048c0ae040 tx=0x7f048c01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f047f7fe640 1 -- 172.21.15.3:0/1521282995 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0ba020 con 0x7f04980b4c70 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f047f7fe640 1 -- 172.21.15.3:0/1521282995 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0b6640 con 0x7f04980b4c70 2024-09-12T12:25:29.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0498165930 con 0x7f04980b4c70 2024-09-12T12:25:29.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f047f7fe640 1 -- 172.21.15.3:0/1521282995 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0b0040 con 0x7f04980b4c70 2024-09-12T12:25:29.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0498166f50 con 0x7f04980b4c70 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f047f7fe640 1 -- 172.21.15.3:0/1521282995 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f048c0bd020 con 0x7f04980b4c70 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f047f7fe640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04840678c0 0x7f0484069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.743+0000 7f047f7fe640 1 -- 172.21.15.3:0/1521282995 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f048c11ffb0 con 0x7f04980b4c70 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f0496d9d640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04840678c0 0x7f0484069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 Processor -- start 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- start start 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f0496d9d640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04840678c0 0x7f0484069d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0488000f70 tx=0x7f0488009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b1b20 0x55597d0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b2440 0x55597d067d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d068290 0x55597d068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55597cf4fb60 con 0x55597d0b2440 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55597ceb6c70 con 0x55597d0b1b20 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55597ce5ced0 con 0x55597d068290 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b2440 0x55597d067d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b2440 0x55597d067d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47738/0 (socket says 172.21.15.3:47738) 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 -- 172.21.15.3:0/3895500674 learned_addr learned my addr 172.21.15.3:0/3895500674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049759e640 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d068290 0x55597d068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b1b20 0x55597d0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 -- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d068290 msgr2=0x55597d068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d068290 0x55597d068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 -- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b1b20 msgr2=0x55597d0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b1b20 0x55597d0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 -- 172.21.15.3:0/3895500674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55597ce6ee50 con 0x55597d0b2440 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f049659c640 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b2440 0x55597d067d50 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f048c11e450 tx=0x7f048c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8268a3cb3e128afd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f047cff9640 1 -- 172.21.15.3:0/3895500674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0f8f20 con 0x55597d0b2440 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f047cff9640 1 -- 172.21.15.3:0/3895500674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0ee640 con 0x55597d0b2440 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f047cff9640 1 -- 172.21.15.3:0/3895500674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0ec9b0 con 0x55597d0b2440 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- 172.21.15.3:0/3895500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b2440 msgr2=0x55597d067d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b2440 0x55597d067d50 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f048c11e450 tx=0x7f048c0191a0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- 172.21.15.3:0/3895500674 shutdown_connections 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d068290 0x55597d068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b2440 0x55597d067d50 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 --2- 172.21.15.3:0/3895500674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b1b20 0x55597d0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- 172.21.15.3:0/3895500674 >> 172.21.15.3:0/3895500674 conn(0x55597d06e600 msgr2=0x55597d066630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- 172.21.15.3:0/3895500674 shutdown_connections 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- 172.21.15.3:0/3895500674 wait complete. 2024-09-12T12:25:29.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 -- start start 2024-09-12T12:25:29.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.747+0000 7f04a0138000 1 Processor -- start 2024-09-12T12:25:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55597cf18b00 con 0x55597d06d390 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55597ce91360 con 0x55597d0b2440 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55597ce8e2d0 con 0x55597d0b1b20 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 10 client.0 _ll_register_callbacks cb 0x55597d053820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47744/0 (socket says 172.21.15.3:47744) 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 -- 172.21.15.3:0/762586839 learned_addr learned my addr 172.21.15.3:0/762586839 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0496d9d640 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 -- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 msgr2=0x55597d06ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d06ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 -- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 msgr2=0x55597d072cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d072cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 -- 172.21.15.3:0/762586839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55597ce5d120 con 0x55597d06d390 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0496d9d640 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d06ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f049759e640 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d073230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f04900079d0 tx=0x7f0490007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf0c068b03339d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 1 -- 172.21.15.3:0/762586839 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049000fc60 con 0x55597d06d390 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 1 -- 172.21.15.3:0/762586839 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f049000fe00 con 0x55597d06d390 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f048c0ee640 con 0x55597d06d390 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 1 -- 172.21.15.3:0/762586839 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f0490023050 con 0x55597d06d390 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 10 client.7116 fetch_fsmap learned FSMap version 73 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55597ce56450 con 0x55597d06d390 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 1 -- 172.21.15.3:0/762586839 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0ee640 con 0x55597d06d390 2024-09-12T12:25:29.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 1 -- 172.21.15.3:0/762586839 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 73) ==== 3522+0+0 (secure 0 0 0) 0x7f0490026020 con 0x55597d06d390 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 10 client.7116 fetch_fsmap finished waiting for FSMap version 73 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 10 client.7116 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 20 client.7116 populate_metadata read hostname 'smithi003' 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0c0280 0x55597d0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 4 client.7116 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 --> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55597ce50c00 con 0x55597d0c0280 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0496d9d640 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0c0280 0x55597d0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0496d9d640 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0c0280 0x55597d0c2670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0c21f55ae54092 server_cookie=c38de419a2c054f3 in_seq=0 out_seq=0 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 10 client.7116 ms_handle_connect on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 1 -- 172.21.15.3:0/762586839 <== mds.0 v2:172.21.15.3:6834/1654386752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55597ce50c00 con 0x55597d0c0280 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 10 client.7116 handle_command_reply: tid=0 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 client.7116 shutdown 2024-09-12T12:25:29.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 msgr2=0x55597d073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d073230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f04900079d0 tx=0x7f0490007e90 comp rx=0 tx=0).stop 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 shutdown_connections 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0c0280 0x55597d0c2670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d073230 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f0475ffb640 0 client.7116 ms_handle_reset on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 --2- 172.21.15.3:0/762586839 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d06ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 >> 172.21.15.3:0/762586839 conn(0x55597d068290 msgr2=0x55597d066d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 shutdown_connections 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 1 -- 172.21.15.3:0/762586839 wait complete. 2024-09-12T12:25:29.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.751+0000 7f04a0138000 20 client.7116 trim_cache size 0 max 16384 2024-09-12T12:25:29.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 Processor -- start 2024-09-12T12:25:29.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 -- start start 2024-09-12T12:25:29.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d0b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d0b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0498203cd0 con 0x55597d06d390 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04980c5d40 con 0x55597d0b2440 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f04982027a0 con 0x55597d0b1b20 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d0b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d0b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47750/0 (socket says 172.21.15.3:47750) 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 -- 172.21.15.3:0/3448455914 learned_addr learned my addr 172.21.15.3:0/3448455914 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f049759e640 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f049659c640 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d0b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 -- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 msgr2=0x55597d0b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d0b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 -- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 msgr2=0x55597d0c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 -- 172.21.15.3:0/3448455914 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55597ce3dfd0 con 0x55597d06d390 2024-09-12T12:25:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f049759e640 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f049659c640 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d0b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d0b0c90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x55597ceb5e90 tx=0x7f0488013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a6c909fb1788ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0477fff640 1 -- 172.21.15.3:0/3448455914 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488023670 con 0x55597d06d390 2024-09-12T12:25:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0477fff640 1 -- 172.21.15.3:0/3448455914 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0488023810 con 0x55597d06d390 2024-09-12T12:25:29.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.771+0000 7f0477fff640 1 -- 172.21.15.3:0/3448455914 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488023bc0 con 0x55597d06d390 2024-09-12T12:25:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3448455914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 msgr2=0x55597d0b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d0b0c90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x55597ceb5e90 tx=0x7f0488013000 comp rx=0 tx=0).stop 2024-09-12T12:25:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3448455914 shutdown_connections 2024-09-12T12:25:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d0b1b20 0x55597d0b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- 172.21.15.3:0/3448455914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d06d390 0x55597d0b0c90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3448455914 >> 172.21.15.3:0/3448455914 conn(0x55597d068290 msgr2=0x55597d06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3448455914 shutdown_connections 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3448455914 wait complete. 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- start start 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 Processor -- start 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d06d390 0x55597d0c5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b1b20 0x55597d0e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55597ceb5f80 con 0x55597d0b1b20 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55597ce6eca0 con 0x55597d0b2440 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55597cebb2a0 con 0x55597d06d390 2024-09-12T12:25:29.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 10 client.0 _ll_register_callbacks cb 0x7f04900096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b1b20 0x55597d0e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b1b20 0x55597d0e0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47762/0 (socket says 172.21.15.3:47762) 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 -- 172.21.15.3:0/3664727725 learned_addr learned my addr 172.21.15.3:0/3664727725 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049759e640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d06d390 0x55597d0c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 -- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d06d390 msgr2=0x55597d0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d06d390 0x55597d0c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 -- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 msgr2=0x55597d0e0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0e0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 -- 172.21.15.3:0/3664727725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55597cf39750 con 0x55597d0b1b20 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049759e640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d06d390 0x55597d0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f049659c640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b1b20 0x55597d0e0960 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f048c0f0500 tx=0x7f048c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2b1f6ac18dd36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:29.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0475ffb640 1 -- 172.21.15.3:0/3664727725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c11b070 con 0x55597d0b1b20 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0475ffb640 1 -- 172.21.15.3:0/3664727725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f048c109d00 con 0x55597d0b1b20 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0488023810 con 0x55597d0b1b20 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0475ffb640 1 -- 172.21.15.3:0/3664727725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f048c0f8f20 con 0x55597d0b1b20 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 10 client.7122 fetch_fsmap learned FSMap version 73 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55597ce44a30 con 0x55597d0b1b20 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0475ffb640 1 -- 172.21.15.3:0/3664727725 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0488023810 con 0x55597d0b1b20 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0475ffb640 1 -- 172.21.15.3:0/3664727725 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 73) ==== 3522+0+0 (secure 0 0 0) 0x7f048c0b3030 con 0x55597d0b1b20 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 10 client.7122 fetch_fsmap finished waiting for FSMap version 73 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 10 client.7122 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 20 client.7122 populate_metadata read hostname 'smithi003' 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0e13e0 0x55597d124a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:29.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 4 client.7122 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 --> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] -- command(tid 0: {"prefix": "status"}) -- 0x55597cec43a0 con 0x55597d0e13e0 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0e13e0 0x55597d124a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0475ffb640 10 client.7122 ms_handle_connect on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.775+0000 7f0496d9d640 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0e13e0 0x55597d124a60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c121ca8c1ea676 server_cookie=ee1b63d95806efbf in_seq=0 out_seq=0 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f0475ffb640 1 -- 172.21.15.3:0/3664727725 <== mds.0 v2:172.21.15.3:6834/1654386752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55597cec43a0 con 0x55597d0e13e0 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f0475ffb640 10 client.7122 handle_command_reply: tid=0 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 client.7122 shutdown 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b1b20 msgr2=0x55597d0e0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b1b20 0x55597d0e0960 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f048c0f0500 tx=0x7f048c000bd0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 shutdown_connections 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:6834/1654386752,v1:172.21.15.3:6835/1654386752] conn(0x55597d0e13e0 0x55597d124a60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55597d0b2440 0x55597d0e0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55597d0b1b20 0x55597d0e0960 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 --2- 172.21.15.3:0/3664727725 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55597d06d390 0x55597d0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f0475ffb640 0 client.7122 ms_handle_reset on v2:172.21.15.3:6834/1654386752 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 >> 172.21.15.3:0/3664727725 conn(0x55597d068290 msgr2=0x55597d0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 shutdown_connections 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 1 -- 172.21.15.3:0/3664727725 wait complete. 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f04a0138000 20 client.7122 trim_cache size 0 max 16384 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "252c40bb-cfe5-4c51-b4ad-5433711beaac", 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 6987, 2024-09-12T12:25:29.787 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 14.707017735000001, 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 73, 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 97, 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 97, 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 15.572270778 2024-09-12T12:25:29.788 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04840678c0 msgr2=0x7f0484069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04840678c0 0x7f0484069d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0488000f70 tx=0x7f0488009040 comp rx=0 tx=0).stop 2024-09-12T12:25:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b4c70 msgr2=0x7f04980ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b4c70 0x7f04980ad840 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f048c0ae040 tx=0x7f048c01ac40 comp rx=0 tx=0).stop 2024-09-12T12:25:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 shutdown_connections 2024-09-12T12:25:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f04840678c0 0x7f0484069d80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04980d04e0 0x7f04980add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04980b4c70 0x7f04980ad840 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 --2- 172.21.15.3:0/1521282995 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f04980b38b0 0x7f0498156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 >> 172.21.15.3:0/1521282995 conn(0x7f04980628b0 msgr2=0x7f04980b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 shutdown_connections 2024-09-12T12:25:29.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:29.779+0000 7f049d07c640 1 -- 172.21.15.3:0/1521282995 wait complete. 2024-09-12T12:25:29.799 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-12T12:25:29.940 INFO:teuthology.orchestra.run.smithi003.stdout:{"message":"","return_code":0} 2024-09-12T12:25:29.941 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-12T12:25:29.941 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-09-12T12:25:30.086 INFO:teuthology.orchestra.run.smithi003.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":40,"op_send_bytes":11562,"op_resend":29,"op_reply":40,"op_latency":{"avgcount":40,"sum":0.419996261,"avgtime":0.010499906},"op_inflight":0,"oplen_avg":{"avgcount":40,"sum":62},"op":40,"op_r":0,"op_w":40,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-09-12T12:25:30.086 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 40, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 40, "avgtime": 0.010499906, "sum": 0.419996261 }, "op_pg": 0, "op_r": 0, "op_reply": 40, "op_resend": 29, "op_rmw": 0, "op_send": 40, "op_send_bytes": 11562, "op_w": 40, "oplen_avg": { "avgcount": 40, "sum": 62 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-09-12T12:25:30.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json export dir /bravo 1 2024-09-12T12:25:30.231 INFO:teuthology.orchestra.run.smithi003.stdout:{"return_code":0} 2024-09-12T12:25:30.231 DEBUG:tasks.cephfs.filesystem:_json_asok output { "return_code": 0 } 2024-09-12T12:25:30.231 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-09-12T12:25:30.374 INFO:teuthology.orchestra.run.smithi003.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":40,"op_send_bytes":11562,"op_resend":29,"op_reply":40,"op_latency":{"avgcount":40,"sum":0.419996261,"avgtime":0.010499906},"op_inflight":0,"oplen_avg":{"avgcount":40,"sum":62},"op":40,"op_r":0,"op_w":40,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":2,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":15,"omap_rd":0,"omap_del":1}} 2024-09-12T12:25:30.374 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 15, "op": 40, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 40, "avgtime": 0.010499906, "sum": 0.419996261 }, "op_pg": 0, "op_r": 0, "op_reply": 40, "op_resend": 29, "op_rmw": 0, "op_send": 40, "op_send_bytes": 11562, "op_w": 40, "oplen_avg": { "avgcount": 40, "sum": 62 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-09-12T12:25:30.374 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-12T12:25:35.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-09-12T12:25:35.521 INFO:teuthology.orchestra.run.smithi003.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":42,"op_send_bytes":15177,"op_resend":29,"op_reply":42,"op_latency":{"avgcount":42,"sum":0.427996190,"avgtime":0.010190385},"op_inflight":0,"oplen_avg":{"avgcount":42,"sum":65},"op":42,"op_r":0,"op_w":42,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":3,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":17,"omap_rd":0,"omap_del":1}} 2024-09-12T12:25:35.522 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 17, "op": 42, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 42, "avgtime": 0.010190385, "sum": 0.42799619 }, "op_pg": 0, "op_r": 0, "op_reply": 42, "op_resend": 29, "op_rmw": 0, "op_send": 42, "op_send_bytes": 15177, "op_w": 42, "oplen_avg": { "avgcount": 42, "sum": 65 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-09-12T12:25:35.522 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-09-12T12:25:40.523 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump objecter 2024-09-12T12:25:40.675 INFO:teuthology.orchestra.run.smithi003.stdout:{"objecter":{"op_active":0,"op_laggy":0,"op_send":44,"op_send_bytes":15391,"op_resend":29,"op_reply":44,"op_latency":{"avgcount":44,"sum":0.431996154,"avgtime":0.009818094},"op_inflight":0,"oplen_avg":{"avgcount":44,"sum":68},"op":44,"op_r":0,"op_w":44,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":10,"osdop_read":0,"osdop_write":4,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":20,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":0,"map_full":0,"map_inc":0,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":19,"omap_rd":0,"omap_del":1}} 2024-09-12T12:25:40.675 DEBUG:tasks.cephfs.filesystem:_json_asok output { "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 0, "map_full": 0, "map_inc": 0, "omap_del": 1, "omap_rd": 0, "omap_wr": 19, "op": 44, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 44, "avgtime": 0.009818094, "sum": 0.431996154 }, "op_pg": 0, "op_r": 0, "op_reply": 44, "op_resend": 29, "op_rmw": 0, "op_send": 44, "op_send_bytes": 15391, "op_w": 44, "oplen_avg": { "avgcount": 44, "sum": 68 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 10, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 20, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 } } 2024-09-12T12:25:40.676 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-09-12T12:25:40.676 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:25:40.676 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:40.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:25:40.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.851+0000 7f40de8f4640 1 Processor -- start 2024-09-12T12:25:40.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.851+0000 7f40de8f4640 1 -- start start 2024-09-12T12:25:40.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.851+0000 7f40de8f4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40d80ceef0 0x7f40d80cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:40.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.851+0000 7f40de8f4640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f40d80ce0a0 0x7f40d806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.851+0000 7f40de8f4640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d80cf810 con 0x7f40d80d0e00 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.851+0000 7f40de8f4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f40d806e100 con 0x7f40d80ceef0 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.851+0000 7f40de8f4640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f40d806e3e0 con 0x7f40d80ce0a0 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40d80ceef0 0x7f40d80cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f40d80ceef0 0x7f40d80cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40134/0 (socket says 172.21.15.3:40134) 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 -- 172.21.15.3:0/2349870813 learned_addr learned my addr 172.21.15.3:0/2349870813 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40dce91640 1 --2- 172.21.15.3:0/2349870813 >> v2:172.21.15.142:3300/0 conn(0x7f40d80ce0a0 0x7f40d806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 -- 172.21.15.3:0/2349870813 >> v2:172.21.15.142:3300/0 conn(0x7f40d80ce0a0 msgr2=0x7f40d806eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:40.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 --2- 172.21.15.3:0/2349870813 >> v2:172.21.15.142:3300/0 conn(0x7f40d80ce0a0 0x7f40d806eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:40.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 -- 172.21.15.3:0/2349870813 >> v1:172.21.15.142:6789/0 conn(0x7f40d80d0e00 legacy=0x7f40d80cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:40.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 -- 172.21.15.3:0/2349870813 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d806e6c0 con 0x7f40d80ceef0 2024-09-12T12:25:40.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d7fff640 1 --2- 172.21.15.3:0/2349870813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ceef0 0x7f40d80cf2d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f40d81600f0 tx=0x7f40c801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac7858bd7039b97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d6ffd640 1 -- 172.21.15.3:0/2349870813 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c80b4070 con 0x7f40d80ceef0 2024-09-12T12:25:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d6ffd640 1 -- 172.21.15.3:0/2349870813 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c80a7ac0 con 0x7f40d80ceef0 2024-09-12T12:25:40.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40d6ffd640 1 -- 172.21.15.3:0/2349870813 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c80af440 con 0x7f40d80ceef0 2024-09-12T12:25:40.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 -- 172.21.15.3:0/2349870813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ceef0 msgr2=0x7f40d80cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:40.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 --2- 172.21.15.3:0/2349870813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ceef0 0x7f40d80cf2d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f40d81600f0 tx=0x7f40c801ebf0 comp rx=0 tx=0).stop 2024-09-12T12:25:40.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 -- 172.21.15.3:0/2349870813 shutdown_connections 2024-09-12T12:25:40.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 --2- 172.21.15.3:0/2349870813 >> v2:172.21.15.142:3300/0 conn(0x7f40d80ce0a0 0x7f40d806eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:40.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 --2- 172.21.15.3:0/2349870813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ceef0 0x7f40d80cf2d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:40.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 -- 172.21.15.3:0/2349870813 >> 172.21.15.3:0/2349870813 conn(0x7f40d80b2ef0 msgr2=0x7f40d80b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:40.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 -- 172.21.15.3:0/2349870813 shutdown_connections 2024-09-12T12:25:40.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.855+0000 7f40de8f4640 1 -- 172.21.15.3:0/2349870813 wait complete. 2024-09-12T12:25:40.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 Processor -- start 2024-09-12T12:25:40.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 -- start start 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ce0a0 0x7f40d809c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d80ceef0 0x7f40d80c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d80d0e00 0x7f40d809ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f40d8072b20 con 0x7f40d80ce0a0 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40d815bda0 con 0x7f40d80ceef0 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f40d815ec70 con 0x7f40d80d0e00 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ce0a0 0x7f40d809c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ce0a0 0x7f40d809c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40140/0 (socket says 172.21.15.3:40140) 2024-09-12T12:25:40.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 -- 172.21.15.3:0/3139430251 learned_addr learned my addr 172.21.15.3:0/3139430251 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40dce91640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d80d0e00 0x7f40d809ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d77fe640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d80ceef0 0x7f40d80c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 -- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d80d0e00 msgr2=0x7f40d809ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d80d0e00 0x7f40d809ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 -- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d80ceef0 msgr2=0x7f40d80c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d80ceef0 0x7f40d80c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 -- 172.21.15.3:0/3139430251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d809b4f0 con 0x7f40d80ce0a0 2024-09-12T12:25:40.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40dce91640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d80d0e00 0x7f40d809ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d77fe640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d80ceef0 0x7f40d80c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d7fff640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ce0a0 0x7f40d809c2b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f40c800b2c0 tx=0x7f40c801d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d4ff9640 1 -- 172.21.15.3:0/3139430251 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c80bf070 con 0x7f40d80ce0a0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d4ff9640 1 -- 172.21.15.3:0/3139430251 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c80ba440 con 0x7f40d80ce0a0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d809b7d0 con 0x7f40d80ce0a0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40d4ff9640 1 -- 172.21.15.3:0/3139430251 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c80b4070 con 0x7f40d80ce0a0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d809bd90 con 0x7f40d80ce0a0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.859+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d80cecb0 con 0x7f40d80ce0a0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.863+0000 7f40d4ff9640 1 -- 172.21.15.3:0/3139430251 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40c80c2070 con 0x7f40d80ce0a0 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.863+0000 7f40d4ff9640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c40678c0 0x7f40c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.863+0000 7f40d4ff9640 1 -- 172.21.15.3:0/3139430251 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f40c8120360 con 0x7f40d80ce0a0 2024-09-12T12:25:40.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.863+0000 7f40d77fe640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c40678c0 0x7f40c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:40.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.863+0000 7f40d77fe640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c40678c0 0x7f40c4069d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40d80b34a0 tx=0x7f40cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:40.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:40.867+0000 7f40d4ff9640 1 -- 172.21.15.3:0/3139430251 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f40c80187e0 con 0x7f40d80ce0a0 2024-09-12T12:25:41.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.043+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40d815fe40 con 0x7f40d80ce0a0 2024-09-12T12:25:41.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40d4ff9640 1 -- 172.21.15.3:0/3139430251 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v97) ==== 80+0+240 (secure 0 0 0) 0x7f40c80f0d60 con 0x7f40d80ce0a0 2024-09-12T12:25:41.049 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6835/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:41.049 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6834/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:41.049 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6835/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:41.050 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6833/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:41.050 INFO:teuthology.orchestra.run.smithi003.stderr:listed 4 entries 2024-09-12T12:25:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c40678c0 msgr2=0x7f40c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c40678c0 0x7f40c4069d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40d80b34a0 tx=0x7f40cc009040 comp rx=0 tx=0).stop 2024-09-12T12:25:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ce0a0 msgr2=0x7f40d809c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ce0a0 0x7f40d809c2b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f40c800b2c0 tx=0x7f40c801d430 comp rx=0 tx=0).stop 2024-09-12T12:25:41.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 shutdown_connections 2024-09-12T12:25:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f40c40678c0 0x7f40c4069d80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f40d80d0e00 0x7f40d809ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d80ceef0 0x7f40d80c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 --2- 172.21.15.3:0/3139430251 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f40d80ce0a0 0x7f40d809c2b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 >> 172.21.15.3:0/3139430251 conn(0x7f40d80b2ef0 msgr2=0x7f40d815bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 shutdown_connections 2024-09-12T12:25:41.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.047+0000 7f40de8f4640 1 -- 172.21.15.3:0/3139430251 wait complete. 2024-09-12T12:25:41.068 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:25:41.068 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:41.075 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:25:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.243+0000 7f247244d640 1 Processor -- start 2024-09-12T12:25:41.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.243+0000 7f247244d640 1 -- start start 2024-09-12T12:25:41.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f247244d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 0x7f246c0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f247244d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f246c0d1c30 con 0x7f246c15fb20 2024-09-12T12:25:41.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f247244d640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f246c0d0b90 con 0x7f246c06fd80 2024-09-12T12:25:41.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f247244d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f246c0704f0 con 0x7f246c0d0640 2024-09-12T12:25:41.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f246c15fb20 0x7f246c06f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:46762/0 (socket says 172.21.15.3:46762) 2024-09-12T12:25:41.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 -- 172.21.15.3:0/1914783444 learned_addr learned my addr 172.21.15.3:0/1914783444 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:41.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f24709ea640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 0x7f246c0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1670695175 0 0) 0x7f246c0d1c30 con 0x7f246c15fb20 2024-09-12T12:25:41.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2458003530 con 0x7f246c15fb20 2024-09-12T12:25:41.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3022502015 0 0) 0x7f2458003530 con 0x7f246c15fb20 2024-09-12T12:25:41.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 msgr2=0x7f246c0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:41.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 0x7f246c0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 >> v1:172.21.15.142:6789/0 conn(0x7f246c06fd80 legacy=0x7f246c0af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c0d0e70 con 0x7f246c15fb20 2024-09-12T12:25:41.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f245c001680 con 0x7f246c15fb20 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 >> v1:172.21.15.3:6789/0 conn(0x7f246c15fb20 legacy=0x7f246c06f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24580039f0 0x7f2458003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 0x7f2458004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458004d50 0x7f24580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f245800ba80 con 0x7f24580039f0 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f245800bc00 con 0x7f2458004d50 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f245800be00 con 0x7f246c0d0640 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24580039f0 0x7f2458003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f24709ea640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458004d50 0x7f24580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f2463fff640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 0x7f2458004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 -- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 msgr2=0x7f2458004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:41.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c0d0640 0x7f2458004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 -- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458004d50 msgr2=0x7f24580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:41.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2458004d50 0x7f24580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 -- 172.21.15.3:0/1914783444 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2458003530 con 0x7f24580039f0 2024-09-12T12:25:41.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246bfff640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24580039f0 0x7f2458003e10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f245c001ec0 tx=0x7f245c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d534062b40c2c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:41.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c007d40 con 0x7f24580039f0 2024-09-12T12:25:41.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f245c0a9070 con 0x7f24580039f0 2024-09-12T12:25:41.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.247+0000 7f246b7fe640 1 -- 172.21.15.3:0/1914783444 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c005d40 con 0x7f24580039f0 2024-09-12T12:25:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- 172.21.15.3:0/1914783444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24580039f0 msgr2=0x7f2458003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 --2- 172.21.15.3:0/1914783444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24580039f0 0x7f2458003e10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f245c001ec0 tx=0x7f245c01cc60 comp rx=0 tx=0).stop 2024-09-12T12:25:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f24709ea640 1 -- 172.21.15.3:0/1914783444 reap_dead start 2024-09-12T12:25:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- 172.21.15.3:0/1914783444 shutdown_connections 2024-09-12T12:25:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- 172.21.15.3:0/1914783444 >> 172.21.15.3:0/1914783444 conn(0x7f246c0b40c0 msgr2=0x7f246c0b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:41.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- 172.21.15.3:0/1914783444 shutdown_connections 2024-09-12T12:25:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- 172.21.15.3:0/1914783444 wait complete. 2024-09-12T12:25:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 Processor -- start 2024-09-12T12:25:41.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- start start 2024-09-12T12:25:41.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580039f0 0x7f246c151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f246c0cf830 0x7f246c152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c1534a0 0x7f246c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f246c160080 con 0x7f246c0cf830 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f246c15ff00 con 0x7f24580039f0 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f246c168310 con 0x7f246c1534a0 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f246c0cf830 0x7f246c152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f246c0cf830 0x7f246c152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40150/0 (socket says 172.21.15.3:40150) 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 -- 172.21.15.3:0/3443926900 learned_addr learned my addr 172.21.15.3:0/3443926900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 -- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c1534a0 msgr2=0x7f246c0ce980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c1534a0 0x7f246c0ce980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 -- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580039f0 msgr2=0x7f246c151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580039f0 0x7f246c151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 -- 172.21.15.3:0/3443926900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246c0cd950 con 0x7f246c0cf830 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f2463fff640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f246c0cf830 0x7f246c152f60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f245400e530 tx=0x7f245400e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f24697fa640 1 -- 172.21.15.3:0/3443926900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24540053a0 con 0x7f246c0cf830 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f24697fa640 1 -- 172.21.15.3:0/3443926900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2454018070 con 0x7f246c0cf830 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f24697fa640 1 -- 172.21.15.3:0/3443926900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454003820 con 0x7f246c0cf830 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246c151a40 con 0x7f246c0cf830 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.251+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246c0cdea0 con 0x7f246c0cf830 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.255+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f246c0af3a0 con 0x7f246c0cf830 2024-09-12T12:25:41.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.255+0000 7f24697fa640 1 -- 172.21.15.3:0/3443926900 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f24540039c0 con 0x7f246c0cf830 2024-09-12T12:25:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.255+0000 7f24697fa640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f244c0678c0 0x7f244c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.255+0000 7f24697fa640 1 -- 172.21.15.3:0/3443926900 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f24540858e0 con 0x7f246c0cf830 2024-09-12T12:25:41.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.255+0000 7f246bfff640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f244c0678c0 0x7f244c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.255+0000 7f246bfff640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f244c0678c0 0x7f244c069d80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ab000 tx=0x7f245c001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:41.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.259+0000 7f24697fa640 1 -- 172.21.15.3:0/3443926900 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2454056360 con 0x7f246c0cf830 2024-09-12T12:25:41.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.431+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f246c15f2e0 con 0x7f246c0cf830 2024-09-12T12:25:41.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f24697fa640 1 -- 172.21.15.3:0/3443926900 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v97) ==== 80+0+240 (secure 0 0 0) 0x7f2454055c60 con 0x7f246c0cf830 2024-09-12T12:25:41.438 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6835/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:41.438 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6834/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:41.438 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6835/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:41.438 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6833/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:listed 4 entries 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f244c0678c0 msgr2=0x7f244c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f244c0678c0 0x7f244c069d80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f245c0ab000 tx=0x7f245c001a50 comp rx=0 tx=0).stop 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f246c0cf830 msgr2=0x7f246c152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f246c0cf830 0x7f246c152f60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f245400e530 tx=0x7f245400e9f0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 shutdown_connections 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f244c0678c0 0x7f244c069d80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f246c1534a0 0x7f246c0ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f246c0cf830 0x7f246c152f60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 --2- 172.21.15.3:0/3443926900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24580039f0 0x7f246c151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 >> 172.21.15.3:0/3443926900 conn(0x7f246c0b40c0 msgr2=0x7f246c159a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 shutdown_connections 2024-09-12T12:25:41.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.435+0000 7f247244d640 1 -- 172.21.15.3:0/3443926900 wait complete. 2024-09-12T12:25:41.457 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-09-12T12:25:41.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:41.458 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-12T12:25:41.469 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi003.stderr:ceph-fuse[45057]: fuse finished with error 0 and tester_r 0 2024-09-12T12:25:41.471 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:41.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:25:41.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:41.510 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:25:41.556 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:41.556 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:41.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:25:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f3795c33640 1 Processor -- start 2024-09-12T12:25:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f3795c33640 1 -- start start 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f3795c33640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f379015b350 0x7f379006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f3795c33640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f37900d0bf0 0x7f37900cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f3795c33640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379015ef40 con 0x7f379006ec30 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f3795c33640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f379015fb10 con 0x7f379015b350 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f3795c33640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f379015f1e0 con 0x7f37900d0bf0 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f378f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f379015b350 0x7f379006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f378f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f379015b350 0x7f379006e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40166/0 (socket says 172.21.15.3:40166) 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f378f7fe640 1 -- 172.21.15.3:0/220089162 learned_addr learned my addr 172.21.15.3:0/220089162 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.715+0000 7f378ffff640 1 --2- 172.21.15.3:0/220089162 >> v2:172.21.15.142:3300/0 conn(0x7f37900d0bf0 0x7f37900cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378f7fe640 1 -- 172.21.15.3:0/220089162 >> v2:172.21.15.142:3300/0 conn(0x7f37900d0bf0 msgr2=0x7f37900cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378f7fe640 1 --2- 172.21.15.3:0/220089162 >> v2:172.21.15.142:3300/0 conn(0x7f37900d0bf0 0x7f37900cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378f7fe640 1 -- 172.21.15.3:0/220089162 >> v1:172.21.15.3:6789/0 conn(0x7f379006ec30 legacy=0x7f37900d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378f7fe640 1 -- 172.21.15.3:0/220089162 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379015f4c0 con 0x7f379015b350 2024-09-12T12:25:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378f7fe640 1 --2- 172.21.15.3:0/220089162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379015b350 0x7f379006e6f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f379006e650 tx=0x7f378001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d53fc9d7498d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378e7fc640 1 -- 172.21.15.3:0/220089162 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800a57d0 con 0x7f379015b350 2024-09-12T12:25:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378e7fc640 1 -- 172.21.15.3:0/220089162 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37800a5970 con 0x7f379015b350 2024-09-12T12:25:41.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f378e7fc640 1 -- 172.21.15.3:0/220089162 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800a5d40 con 0x7f379015b350 2024-09-12T12:25:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 -- 172.21.15.3:0/220089162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379015b350 msgr2=0x7f379006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 --2- 172.21.15.3:0/220089162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379015b350 0x7f379006e6f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f379006e650 tx=0x7f378001c9f0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 -- 172.21.15.3:0/220089162 shutdown_connections 2024-09-12T12:25:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 --2- 172.21.15.3:0/220089162 >> v2:172.21.15.142:3300/0 conn(0x7f37900d0bf0 0x7f37900cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 --2- 172.21.15.3:0/220089162 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379015b350 0x7f379006e6f0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 -- 172.21.15.3:0/220089162 >> 172.21.15.3:0/220089162 conn(0x7f37900628b0 msgr2=0x7f3790062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:41.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 -- 172.21.15.3:0/220089162 shutdown_connections 2024-09-12T12:25:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.719+0000 7f3795c33640 1 -- 172.21.15.3:0/220089162 wait complete. 2024-09-12T12:25:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 Processor -- start 2024-09-12T12:25:41.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 -- start start 2024-09-12T12:25:41.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379006ec30 0x7f37900c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37900d0bf0 0x7f37900c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379015b350 0x7f37900ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f37900cf060 con 0x7f37900d0bf0 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37900ceee0 con 0x7f379015b350 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f37900cf1e0 con 0x7f379006ec30 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37900d0bf0 0x7f37900c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37900d0bf0 0x7f37900c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40168/0 (socket says 172.21.15.3:40168) 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 -- 172.21.15.3:0/733957674 learned_addr learned my addr 172.21.15.3:0/733957674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:41.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378f7fe640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379006ec30 0x7f37900c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378ffff640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379015b350 0x7f37900ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 -- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379006ec30 msgr2=0x7f37900c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379006ec30 0x7f37900c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 -- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379015b350 msgr2=0x7f37900ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379015b350 0x7f37900ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 -- 172.21.15.3:0/733957674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37901509f0 con 0x7f37900d0bf0 2024-09-12T12:25:41.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378ffff640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379015b350 0x7f37900ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378f7fe640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379006ec30 0x7f37900c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f378effd640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37900d0bf0 0x7f37900c9ad0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f378400c370 tx=0x7f378400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3777fff640 1 -- 172.21.15.3:0/733957674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840052d0 con 0x7f37900d0bf0 2024-09-12T12:25:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3777fff640 1 -- 172.21.15.3:0/733957674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3784005470 con 0x7f37900d0bf0 2024-09-12T12:25:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3777fff640 1 -- 172.21.15.3:0/733957674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784007260 con 0x7f37900d0bf0 2024-09-12T12:25:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37900af410 con 0x7f37900d0bf0 2024-09-12T12:25:41.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.723+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37900af9d0 con 0x7f37900d0bf0 2024-09-12T12:25:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.727+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379015ab10 con 0x7f37900d0bf0 2024-09-12T12:25:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.727+0000 7f3777fff640 1 -- 172.21.15.3:0/733957674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3784007400 con 0x7f37900d0bf0 2024-09-12T12:25:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.727+0000 7f3777fff640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f377c0678f0 0x7f377c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.727+0000 7f3777fff640 1 -- 172.21.15.3:0/733957674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f3784085260 con 0x7f37900d0bf0 2024-09-12T12:25:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.727+0000 7f378f7fe640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f377c0678f0 0x7f377c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:41.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.727+0000 7f378f7fe640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f377c0678f0 0x7f377c069db0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37800ae000 tx=0x7f37800a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:41.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.731+0000 7f3777fff640 1 -- 172.21.15.3:0/733957674 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3784055c60 con 0x7f37900d0bf0 2024-09-12T12:25:41.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.907+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37900cdf60 con 0x7f37900d0bf0 2024-09-12T12:25:41.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3777fff640 1 -- 172.21.15.3:0/733957674 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v97) ==== 80+0+240 (secure 0 0 0) 0x7f3784085080 con 0x7f37900d0bf0 2024-09-12T12:25:41.913 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6835/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:41.913 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6834/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:41.913 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6835/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:41.913 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6833/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:41.914 INFO:teuthology.orchestra.run.smithi003.stderr:listed 4 entries 2024-09-12T12:25:41.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f377c0678f0 msgr2=0x7f377c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f377c0678f0 0x7f377c069db0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f37800ae000 tx=0x7f37800a7070 comp rx=0 tx=0).stop 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37900d0bf0 msgr2=0x7f37900c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37900d0bf0 0x7f37900c9ad0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f378400c370 tx=0x7f378400c830 comp rx=0 tx=0).stop 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 shutdown_connections 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f377c0678f0 0x7f377c069db0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379015b350 0x7f37900ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37900d0bf0 0x7f37900c9ad0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 --2- 172.21.15.3:0/733957674 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379006ec30 0x7f37900c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:41.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 >> 172.21.15.3:0/733957674 conn(0x7f37900628b0 msgr2=0x7f37900aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:41.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 shutdown_connections 2024-09-12T12:25:41.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:41.911+0000 7f3795c33640 1 -- 172.21.15.3:0/733957674 wait complete. 2024-09-12T12:25:41.933 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:41.933 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:41.940 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-12T12:25:42.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846fbf3640 1 Processor -- start 2024-09-12T12:25:42.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846fbf3640 1 -- start start 2024-09-12T12:25:42.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846fbf3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84680d04e0 0x7f8468099920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:42.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846fbf3640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84680ccb30 con 0x7f846806d5d0 2024-09-12T12:25:42.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846fbf3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84680cce10 con 0x7f84680b3cf0 2024-09-12T12:25:42.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846fbf3640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f84680cd740 con 0x7f84680d04e0 2024-09-12T12:25:42.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846d98f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84680d04e0 0x7f8468099920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:42.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846d98f640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84680d04e0 0x7f8468099920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40170/0 (socket says 172.21.15.3:40170) 2024-09-12T12:25:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846d98f640 1 -- 172.21.15.3:0/1665349102 learned_addr learned my addr 172.21.15.3:0/1665349102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846d98f640 1 -- 172.21.15.3:0/1665349102 >> v1:172.21.15.142:6789/0 conn(0x7f84680b3cf0 legacy=0x7f84680d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846d98f640 1 -- 172.21.15.3:0/1665349102 >> v1:172.21.15.3:6789/0 conn(0x7f846806d5d0 legacy=0x7f846806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:42.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.055+0000 7f846d98f640 1 -- 172.21.15.3:0/1665349102 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84680cd0f0 con 0x7f84680d04e0 2024-09-12T12:25:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846d98f640 1 --2- 172.21.15.3:0/1665349102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f8468099920 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8458009870 tx=0x7f845801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe574ad3b0b5094 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846c98d640 1 -- 172.21.15.3:0/1665349102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580b0070 con 0x7f84680d04e0 2024-09-12T12:25:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846c98d640 1 -- 172.21.15.3:0/1665349102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84580ab600 con 0x7f84680d04e0 2024-09-12T12:25:42.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846c98d640 1 -- 172.21.15.3:0/1665349102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580ab9d0 con 0x7f84680d04e0 2024-09-12T12:25:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846fbf3640 1 -- 172.21.15.3:0/1665349102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 msgr2=0x7f8468099920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1665349102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f8468099920 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8458009870 tx=0x7f845801c490 comp rx=0 tx=0).stop 2024-09-12T12:25:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846fbf3640 1 -- 172.21.15.3:0/1665349102 shutdown_connections 2024-09-12T12:25:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1665349102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f8468099920 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846fbf3640 1 -- 172.21.15.3:0/1665349102 >> 172.21.15.3:0/1665349102 conn(0x7f84680628b0 msgr2=0x7f8468062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846fbf3640 1 -- 172.21.15.3:0/1665349102 shutdown_connections 2024-09-12T12:25:42.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.059+0000 7f846fbf3640 1 -- 172.21.15.3:0/1665349102 wait complete. 2024-09-12T12:25:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 Processor -- start 2024-09-12T12:25:42.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 -- start start 2024-09-12T12:25:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846806d5d0 0x7f846814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84680b3cf0 0x7f84680b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f84680b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:42.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f846816c430 con 0x7f84680d04e0 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8468161fa0 con 0x7f846806d5d0 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f846816c5b0 con 0x7f84680b3cf0 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f84680b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f84680b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40186/0 (socket says 172.21.15.3:40186) 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 -- 172.21.15.3:0/1062321505 learned_addr learned my addr 172.21.15.3:0/1062321505 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846d98f640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846806d5d0 0x7f846814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846d18e640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84680b3cf0 0x7f84680b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:42.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 -- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84680b3cf0 msgr2=0x7f84680b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84680b3cf0 0x7f84680b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 -- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846806d5d0 msgr2=0x7f846814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846806d5d0 0x7f846814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 -- 172.21.15.3:0/1062321505 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84680b2cb0 con 0x7f84680d04e0 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846e190640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f84680b0a70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f846000c370 tx=0x7f846000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f845e7fc640 1 -- 172.21.15.3:0/1062321505 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460005220 con 0x7f84680d04e0 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f845e7fc640 1 -- 172.21.15.3:0/1062321505 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84600053c0 con 0x7f84680d04e0 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84680afed0 con 0x7f84680d04e0 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f845e7fc640 1 -- 172.21.15.3:0/1062321505 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460007270 con 0x7f84680d04e0 2024-09-12T12:25:42.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84680b24b0 con 0x7f84680d04e0 2024-09-12T12:25:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8468099920 con 0x7f84680d04e0 2024-09-12T12:25:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.063+0000 7f845e7fc640 1 -- 172.21.15.3:0/1062321505 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8460019020 con 0x7f84680d04e0 2024-09-12T12:25:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.067+0000 7f845e7fc640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f844c0678c0 0x7f844c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.067+0000 7f845e7fc640 1 -- 172.21.15.3:0/1062321505 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f8460085030 con 0x7f84680d04e0 2024-09-12T12:25:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.067+0000 7f846d98f640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f844c0678c0 0x7f844c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:42.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.067+0000 7f846d98f640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f844c0678c0 0x7f844c069d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f84580ad000 tx=0x7f84580a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:42.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.071+0000 7f845e7fc640 1 -- 172.21.15.3:0/1062321505 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f8460055ac0 con 0x7f84680d04e0 2024-09-12T12:25:42.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.247+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f846806de50 con 0x7f84680d04e0 2024-09-12T12:25:42.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f845e7fc640 1 -- 172.21.15.3:0/1062321505 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v97) ==== 80+0+240 (secure 0 0 0) 0x7f84600553c0 con 0x7f84680d04e0 2024-09-12T12:25:42.253 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6835/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:42.253 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.142:6834/95103479 2024-09-13T12:25:15.057602+0000 2024-09-12T12:25:42.254 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6835/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:42.254 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6833/2492687290 2024-09-13T12:25:14.060842+0000 2024-09-12T12:25:42.254 INFO:teuthology.orchestra.run.smithi003.stderr:listed 4 entries 2024-09-12T12:25:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f844c0678c0 msgr2=0x7f844c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f844c0678c0 0x7f844c069d80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f84580ad000 tx=0x7f84580a6000 comp rx=0 tx=0).stop 2024-09-12T12:25:42.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 msgr2=0x7f84680b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f84680b0a70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f846000c370 tx=0x7f846000c830 comp rx=0 tx=0).stop 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 shutdown_connections 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f844c0678c0 0x7f844c069d80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84680d04e0 0x7f84680b0a70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84680b3cf0 0x7f84680b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 --2- 172.21.15.3:0/1062321505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f846806d5d0 0x7f846814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 >> 172.21.15.3:0/1062321505 conn(0x7f84680628b0 msgr2=0x7f8468062c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 shutdown_connections 2024-09-12T12:25:42.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:42.251+0000 7f846fbf3640 1 -- 172.21.15.3:0/1062321505 wait complete. 2024-09-12T12:25:42.273 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi142.front.sepia.ceph.com... 2024-09-12T12:25:42.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:42.273 DEBUG:teuthology.orchestra.run.smithi142:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-12T12:25:42.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi142.stderr:ceph-fuse[36730]: fuse finished with error 0 and tester_r 0 2024-09-12T12:25:42.288 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:25:47.429 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:25:47.432 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:25:48.390 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:25:48.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:48.390 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:25:48.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-12T12:25:48.557 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T12:25:48.557 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-12T12:25:48.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-12T12:25:48.716 INFO:teuthology.orchestra.run.smithi003.stdout:{"message":"","return_code":0} 2024-09-12T12:25:48.716 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-12T12:25:48.717 DEBUG:teuthology.orchestra.run.smithi003:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show session 2024-09-12T12:25:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 -- start start 2024-09-12T12:25:48.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 Processor -- start 2024-09-12T12:25:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab68193800 0x55ab6820e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab68193400 0x55ab6820eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab68193000 0x55ab6820f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ab68f72180 con 0x55ab68193000 2024-09-12T12:25:48.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ab68f72300 con 0x55ab68193400 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ab68f72480 con 0x55ab68193800 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab68193000 0x55ab6820f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab68193000 0x55ab6820f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40936/0 (socket says 172.21.15.3:40936) 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 -- 172.21.15.3:0/4046109555 learned_addr learned my addr 172.21.15.3:0/4046109555 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab68193400 0x55ab6820eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1d96d640 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab68193800 0x55ab6820e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 -- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab68193800 msgr2=0x55ab6820e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab68193800 0x55ab6820e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 -- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab68193400 msgr2=0x55ab6820eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab68193400 0x55ab6820eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 -- 172.21.15.3:0/4046109555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68f5ab40 con 0x55ab68193000 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab68193400 0x55ab6820eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1e16e640 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab68193000 0x55ab6820f080 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x55ab69007350 tx=0x55ab682735c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2759171ff6cc952 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1b168640 1 -- 172.21.15.3:0/4046109555 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab681ca4e0 con 0x55ab68193000 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1b168640 1 -- 172.21.15.3:0/4046109555 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ab681da1c0 con 0x55ab68193000 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1b168640 1 -- 172.21.15.3:0/4046109555 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab681cab60 con 0x55ab68193000 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/4046109555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab68f5ad20 con 0x55ab68193000 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/4046109555 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ab68f5af00 con 0x55ab68193000 2024-09-12T12:25:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 4 waiting for MDS map... 2024-09-12T12:25:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1b168640 1 -- 172.21.15.3:0/4046109555 <== mon.0 v2:172.21.15.3:3300/0 4 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x55ab68f61000 con 0x55ab68193000 2024-09-12T12:25:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1b168640 1 -- 172.21.15.3:0/4046109555 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 74) ==== 3526+0+0 (secure 0 0 0) 0x55ab681e9180 con 0x55ab68193000 2024-09-12T12:25:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 4 Got MDS map 74 2024-09-12T12:25:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 10 main: main 2024-09-12T12:25:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 4 main: connecting to RADOS... 2024-09-12T12:25:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 Processor -- start 2024-09-12T12:25:48.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.751+0000 7f0d1f1ae400 1 -- start start 2024-09-12T12:25:48.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9000 0x55ab6820f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9400 0x55ab6820fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ab68f72600 con 0x55ab690b9800 2024-09-12T12:25:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ab68f72780 con 0x55ab690b9400 2024-09-12T12:25:48.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ab68f72900 con 0x55ab690b9000 2024-09-12T12:25:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40946/0 (socket says 172.21.15.3:40946) 2024-09-12T12:25:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/1011617271 learned_addr learned my addr 172.21.15.3:0/1011617271 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d96d640 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9400 0x55ab6820fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9000 0x55ab6820f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9000 msgr2=0x55ab6820f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9000 0x55ab6820f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9400 msgr2=0x55ab6820fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9400 0x55ab6820fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/1011617271 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68f5ad20 con 0x55ab690b9800 2024-09-12T12:25:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d96d640 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9400 0x55ab6820fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210100 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x55ab690cc690 tx=0x55ab690cec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6011c8fc5efeb7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:48.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d19164640 1 -- 172.21.15.3:0/1011617271 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab681ca4e0 con 0x55ab690b9800 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d19164640 1 -- 172.21.15.3:0/1011617271 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ab681dae00 con 0x55ab690b9800 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d19164640 1 -- 172.21.15.3:0/1011617271 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab681cab60 con 0x55ab690b9800 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/1011617271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 msgr2=0x55ab68210100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210100 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x55ab690cc690 tx=0x55ab690cec60 comp rx=0 tx=0).stop 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/1011617271 shutdown_connections 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210100 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9400 0x55ab6820fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/1011617271 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9000 0x55ab6820f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/1011617271 >> 172.21.15.3:0/1011617271 conn(0x55ab68192c00 msgr2=0x55ab681c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/1011617271 shutdown_connections 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/1011617271 wait complete. 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 Processor -- start 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- start start 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9000 0x55ab68211180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9400 0x55ab68210c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ab68f72900 con 0x55ab690b9800 2024-09-12T12:25:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ab68f72a80 con 0x55ab690b9000 2024-09-12T12:25:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1f1ae400 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x55ab68f72780 con 0x55ab690b9400 2024-09-12T12:25:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40960/0 (socket says 172.21.15.3:40960) 2024-09-12T12:25:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/2655170098 learned_addr learned my addr 172.21.15.3:0/2655170098 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d96d640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9400 0x55ab68210c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9000 0x55ab68211180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9400 msgr2=0x55ab68210c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab690b9400 0x55ab68210c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9000 msgr2=0x55ab68211180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab690b9000 0x55ab68211180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 -- 172.21.15.3:0/2655170098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab68f5bc20 con 0x55ab690b9800 2024-09-12T12:25:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210680 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x55ab690cd350 tx=0x55ab690cec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:48.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d17961640 1 -- 172.21.15.3:0/2655170098 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab681ca000 con 0x55ab690b9800 2024-09-12T12:25:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d17961640 1 -- 172.21.15.3:0/2655170098 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ab681dac40 con 0x55ab690b9800 2024-09-12T12:25:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.755+0000 7f0d17961640 1 -- 172.21.15.3:0/2655170098 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ab681ca4e0 con 0x55ab690b9800 2024-09-12T12:25:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab68f5be00 con 0x55ab690b9800 2024-09-12T12:25:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab68f5bc20 con 0x55ab690b9800 2024-09-12T12:25:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 4 main: resolving pool 14 2024-09-12T12:25:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d17961640 1 -- 172.21.15.3:0/2655170098 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x55ab68496c00 con 0x55ab690b9800 2024-09-12T12:25:48.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d17961640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55ab69120c00 0x55ab68211700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d17961640 1 -- 172.21.15.3:0/2655170098 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x55ab6848ea00 con 0x55ab690b9800 2024-09-12T12:25:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 4 main: creating IoCtx.. 2024-09-12T12:25:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55ab690b9c00 0x55ab68210100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55ab69121000 con 0x55ab690b9c00 2024-09-12T12:25:48.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55ab690b9c00 0x55ab68210100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1d96d640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55ab69120c00 0x55ab68211700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1d16c640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55ab690b9c00 0x55ab68210100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1d16c640 1 -- 172.21.15.3:0/2655170098 <== osd.0 v2:172.21.15.3:6816/3370158904 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv10 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x55ab68f1e000 con 0x55ab690b9c00 2024-09-12T12:25:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- osd_op(unknown.0.0:2 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x55ab69121400 con 0x55ab690b9c00 2024-09-12T12:25:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1d96d640 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55ab69120c00 0x55ab68211700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55ab681faea0 tx=0x55ab68270c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:48.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1d16c640 1 -- 172.21.15.3:0/2655170098 <== osd.0 v2:172.21.15.3:6816/3370158904 2 ==== osd_op_reply(2 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv10 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x55ab68f1e000 con 0x55ab690b9c00 2024-09-12T12:25:48.777 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:48.777 INFO:teuthology.orchestra.run.smithi003.stdout: "0": { 2024-09-12T12:25:48.777 INFO:teuthology.orchestra.run.smithi003.stdout: "data": { 2024-09-12T12:25:48.777 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions": [] 2024-09-12T12:25:48.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T12:25:48.778 INFO:teuthology.orchestra.run.smithi003.stdout: "result": 0 2024-09-12T12:25:48.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T12:25:48.778 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:48.778 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55ab690b9c00 msgr2=0x55ab68210100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x55ab690b9c00 0x55ab68210100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55ab69120c00 msgr2=0x55ab68211700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x55ab69120c00 0x55ab68211700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55ab681faea0 tx=0x55ab68270c00 comp rx=0 tx=0).stop 2024-09-12T12:25:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 msgr2=0x55ab68210680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/2655170098 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab690b9800 0x55ab68210680 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x55ab690cd350 tx=0x55ab690cec00 comp rx=0 tx=0).stop 2024-09-12T12:25:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1d96d640 1 -- 172.21.15.3:0/2655170098 reap_dead start 2024-09-12T12:25:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 shutdown_connections 2024-09-12T12:25:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 >> 172.21.15.3:0/2655170098 conn(0x55ab68192c00 msgr2=0x55ab681c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 shutdown_connections 2024-09-12T12:25:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.759+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/2655170098 wait complete. 2024-09-12T12:25:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/4046109555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab68193000 msgr2=0x55ab6820f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab68193000 0x55ab6820f080 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x55ab69007350 tx=0x55ab682735c0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/4046109555 shutdown_connections 2024-09-12T12:25:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ab68193000 0x55ab6820f080 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab68193400 0x55ab6820eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 --2- 172.21.15.3:0/4046109555 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55ab68193800 0x55ab6820e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/4046109555 >> 172.21.15.3:0/4046109555 conn(0x55ab68193c00 msgr2=0x55ab681c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/4046109555 shutdown_connections 2024-09-12T12:25:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:48.763+0000 7f0d1f1ae400 1 -- 172.21.15.3:0/4046109555 wait complete. 2024-09-12T12:25:48.785 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'session'] in time 0:00:00.067503, result: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-09-12T12:25:48.785 INFO:tasks.cephfs.test_sessionmap:SessionMap: { "0": { "data": { "sessions": [] }, "result": 0 } } 2024-09-12T12:25:48.785 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-12T12:25:48.785 DEBUG:teuthology.orchestra.run.smithi003:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:25:48.920 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-12T12:25:48.960 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-12T12:25:48.960 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:25:48.960 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:25:48.960 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:48.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:25:48.968 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:25:48.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:48.969 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:25:49.018 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-12T12:25:49.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:25:49.019 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:25:49.019 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:49.020 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:49.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:25:49.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:25:49.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:49.029 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-12T12:25:49.075 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-12T12:25:49.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:25:49.076 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:25:49.076 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:25:49.076 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:49.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:25:49.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:25:49.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:49.084 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-12T12:25:49.130 INFO:teuthology.orchestra.run.smithi003.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-12T12:25:49.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:25:49.131 INFO:tasks.cephfs.mount:Terminating background process 2024-09-12T12:25:49.131 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:49.132 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:49.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:25:49.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:25:49.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:49.138 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-12T12:25:49.187 INFO:teuthology.orchestra.run.smithi142.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-12T12:25:49.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:25:49.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:49.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 Processor -- start 2024-09-12T12:25:49.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 -- start start 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f76ac0d1d20 0x7f76ac0b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76ac15e980 0x7f76ac0b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f76ac0d2b00 con 0x7f76ac15e980 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f76ac0d2d50 con 0x7f76ac0d1d20 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76b21eb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0d3030 con 0x7f76ac15e5a0 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76ac15e980 0x7f76ac0b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f76ac15e980 0x7f76ac0b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40962/0 (socket says 172.21.15.3:40962) 2024-09-12T12:25:49.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 -- 172.21.15.3:0/1069834016 learned_addr learned my addr 172.21.15.3:0/1069834016 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76aaffd640 1 --2- 172.21.15.3:0/1069834016 >> v2:172.21.15.142:3300/0 conn(0x7f76ac0d1d20 0x7f76ac0b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76ab7fe640 1 --2- 172.21.15.3:0/1069834016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 -- 172.21.15.3:0/1069834016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 msgr2=0x7f76ac0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 --2- 172.21.15.3:0/1069834016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac0cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 -- 172.21.15.3:0/1069834016 >> v2:172.21.15.142:3300/0 conn(0x7f76ac0d1d20 msgr2=0x7f76ac0b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 --2- 172.21.15.3:0/1069834016 >> v2:172.21.15.142:3300/0 conn(0x7f76ac0d1d20 0x7f76ac0b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76abfff640 1 -- 172.21.15.3:0/1069834016 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac0d3310 con 0x7f76ac15e980 2024-09-12T12:25:49.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.323+0000 7f76aaffd640 1 --2- 172.21.15.3:0/1069834016 >> v2:172.21.15.142:3300/0 conn(0x7f76ac0d1d20 0x7f76ac0b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:49.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76abfff640 1 --2- 172.21.15.3:0/1069834016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac0b7dd0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f76ac08d750 tx=0x7f76a401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c44e837cee5458 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:49.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76aa7fc640 1 -- 172.21.15.3:0/1069834016 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40a5690 con 0x7f76ac15e980 2024-09-12T12:25:49.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76aa7fc640 1 -- 172.21.15.3:0/1069834016 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a40a5830 con 0x7f76ac15e980 2024-09-12T12:25:49.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76aa7fc640 1 -- 172.21.15.3:0/1069834016 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40a5c00 con 0x7f76ac15e980 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 -- 172.21.15.3:0/1069834016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 msgr2=0x7f76ac0b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1069834016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac0b7dd0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f76ac08d750 tx=0x7f76a401c960 comp rx=0 tx=0).stop 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 -- 172.21.15.3:0/1069834016 shutdown_connections 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1069834016 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac0b7dd0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1069834016 >> v2:172.21.15.142:3300/0 conn(0x7f76ac0d1d20 0x7f76ac0b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1069834016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 -- 172.21.15.3:0/1069834016 >> 172.21.15.3:0/1069834016 conn(0x7f76ac0626e0 msgr2=0x7f76ac062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:49.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 -- 172.21.15.3:0/1069834016 shutdown_connections 2024-09-12T12:25:49.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 -- 172.21.15.3:0/1069834016 wait complete. 2024-09-12T12:25:49.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 Processor -- start 2024-09-12T12:25:49.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.327+0000 7f76b21eb640 1 -- start start 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ac0d1d20 0x7f76ac14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0d4a10 con 0x7f76ac15e980 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac0d4890 con 0x7f76ac0d1d20 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac0d4b90 con 0x7f76ac15e5a0 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40978/0 (socket says 172.21.15.3:40978) 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 -- 172.21.15.3:0/1286946177 learned_addr learned my addr 172.21.15.3:0/1286946177 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:49.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76ab7fe640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ac0d1d20 0x7f76ac14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76aaffd640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 -- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 msgr2=0x7f76ac157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 -- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ac0d1d20 msgr2=0x7f76ac14fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ac0d1d20 0x7f76ac14fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 -- 172.21.15.3:0/1286946177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac1556c0 con 0x7f76ac15e980 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76ab7fe640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ac0d1d20 0x7f76ac14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:49.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76abfff640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac159b10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f76a40af040 tx=0x7f76a4019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:49.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f7693fff640 1 -- 172.21.15.3:0/1286946177 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40bc020 con 0x7f76ac15e980 2024-09-12T12:25:49.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f7693fff640 1 -- 172.21.15.3:0/1286946177 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a40b8670 con 0x7f76ac15e980 2024-09-12T12:25:49.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac155980 con 0x7f76ac15e980 2024-09-12T12:25:49.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f7693fff640 1 -- 172.21.15.3:0/1286946177 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4009510 con 0x7f76ac15e980 2024-09-12T12:25:49.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac155f40 con 0x7f76ac15e980 2024-09-12T12:25:49.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac069250 con 0x7f76ac15e980 2024-09-12T12:25:49.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f7693fff640 1 -- 172.21.15.3:0/1286946177 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f76a40bf020 con 0x7f76ac15e980 2024-09-12T12:25:49.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f7693fff640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f76980678f0 0x7f7698069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f7693fff640 1 -- 172.21.15.3:0/1286946177 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f76a4120bf0 con 0x7f76ac15e980 2024-09-12T12:25:49.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.331+0000 7f76ab7fe640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f76980678f0 0x7f7698069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.335+0000 7f76ab7fe640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f76980678f0 0x7f7698069db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f769c007a70 tx=0x7f769c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:49.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.339+0000 7f7693fff640 1 -- 172.21.15.3:0/1286946177 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f76a40c6030 con 0x7f76ac15e980 2024-09-12T12:25:49.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.539+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76ac15bc40 con 0x7f76ac15e980 2024-09-12T12:25:49.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.539+0000 7f7693fff640 1 -- 172.21.15.3:0/1286946177 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5243 (secure 0 0 0) 0x7f76a40165e0 con 0x7f76ac15e980 2024-09-12T12:25:49.543 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:49.543 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":74,"btime":"2024-09-12T12:25:34:876876+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:34.212723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6987,"mds_1":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":1,"incarnation":72,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987,6972]},"id":7}]} 2024-09-12T12:25:49.543 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 74 2024-09-12T12:25:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f76980678f0 msgr2=0x7f7698069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f76980678f0 0x7f7698069db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f769c007a70 tx=0x7f769c008040 comp rx=0 tx=0).stop 2024-09-12T12:25:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 msgr2=0x7f76ac159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac159b10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f76a40af040 tx=0x7f76a4019700 comp rx=0 tx=0).stop 2024-09-12T12:25:49.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 shutdown_connections 2024-09-12T12:25:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f76980678f0 0x7f7698069db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f76ac15e980 0x7f76ac159b10 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f76ac15e5a0 0x7f76ac157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 --2- 172.21.15.3:0/1286946177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76ac0d1d20 0x7f76ac14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 >> 172.21.15.3:0/1286946177 conn(0x7f76ac0626e0 msgr2=0x7f76ac15ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 shutdown_connections 2024-09-12T12:25:49.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.543+0000 7f76b21eb640 1 -- 172.21.15.3:0/1286946177 wait complete. 2024-09-12T12:25:49.564 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.679+0000 7f379ab6a640 1 Processor -- start 2024-09-12T12:25:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.679+0000 7f379ab6a640 1 -- start start 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f379ab6a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f379415e5a0 0x7f37940cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f379ab6a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f379415e980 0x7f37940d3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f379ab6a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37940d2d50 con 0x7f37940d1d20 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f379ab6a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f37940d2b00 con 0x7f379415e980 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f379ab6a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f37940d3030 con 0x7f379415e5a0 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f379415e980 0x7f37940d3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f379415e980 0x7f37940d3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40988/0 (socket says 172.21.15.3:40988) 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 -- 172.21.15.3:0/3718551614 learned_addr learned my addr 172.21.15.3:0/3718551614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3793fff640 1 --1- 172.21.15.3:0/3718551614 >> v1:172.21.15.3:6789/0 conn(0x7f37940d1d20 0x7f37940b7890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45022/0 (socket says 172.21.15.3:45022) 2024-09-12T12:25:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 -- 172.21.15.3:0/3718551614 >> v2:172.21.15.142:3300/0 conn(0x7f379415e5a0 msgr2=0x7f37940cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 --2- 172.21.15.3:0/3718551614 >> v2:172.21.15.142:3300/0 conn(0x7f379415e5a0 0x7f37940cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 -- 172.21.15.3:0/3718551614 >> v1:172.21.15.3:6789/0 conn(0x7f37940d1d20 legacy=0x7f37940b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 -- 172.21.15.3:0/3718551614 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37940d3310 con 0x7f379415e980 2024-09-12T12:25:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f3799107640 1 --2- 172.21.15.3:0/3718551614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379415e980 0x7f37940d3b10 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f379406cdd0 tx=0x7f378c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc43c38e58ae340 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f37937fe640 1 -- 172.21.15.3:0/3718551614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0b0070 con 0x7f379415e980 2024-09-12T12:25:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f37937fe640 1 -- 172.21.15.3:0/3718551614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378c0ab600 con 0x7f379415e980 2024-09-12T12:25:49.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.683+0000 7f37937fe640 1 -- 172.21.15.3:0/3718551614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0ab9d0 con 0x7f379415e980 2024-09-12T12:25:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- 172.21.15.3:0/3718551614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379415e980 msgr2=0x7f37940d3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 --2- 172.21.15.3:0/3718551614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379415e980 0x7f37940d3b10 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f379406cdd0 tx=0x7f378c01c520 comp rx=0 tx=0).stop 2024-09-12T12:25:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- 172.21.15.3:0/3718551614 shutdown_connections 2024-09-12T12:25:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 --2- 172.21.15.3:0/3718551614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f379415e980 0x7f37940d3b10 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 --2- 172.21.15.3:0/3718551614 >> v2:172.21.15.142:3300/0 conn(0x7f379415e5a0 0x7f37940cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- 172.21.15.3:0/3718551614 >> 172.21.15.3:0/3718551614 conn(0x7f37940626e0 msgr2=0x7f3794062ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:49.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- 172.21.15.3:0/3718551614 shutdown_connections 2024-09-12T12:25:49.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- 172.21.15.3:0/3718551614 wait complete. 2024-09-12T12:25:49.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 Processor -- start 2024-09-12T12:25:49.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- start start 2024-09-12T12:25:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37940d1d20 0x7f37940b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379415e5a0 0x7f37940b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379415e980 0x7f37940af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f379416b4c0 con 0x7f37940d1d20 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f37940b81b0 con 0x7f379415e5a0 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f379416b640 con 0x7f379415e980 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37940d1d20 0x7f37940b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37940d1d20 0x7f37940b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40998/0 (socket says 172.21.15.3:40998) 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 -- 172.21.15.3:0/786428903 learned_addr learned my addr 172.21.15.3:0/786428903 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 -- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379415e980 msgr2=0x7f37940af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379415e980 0x7f37940af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 -- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379415e5a0 msgr2=0x7f37940b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379415e5a0 0x7f37940b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 -- 172.21.15.3:0/786428903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37940b1d50 con 0x7f37940d1d20 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f3798906640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37940d1d20 0x7f37940b1700 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f378400ba80 tx=0x7f378400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f37917fa640 1 -- 172.21.15.3:0/786428903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784019070 con 0x7f37940d1d20 2024-09-12T12:25:49.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f37917fa640 1 -- 172.21.15.3:0/786428903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378400cc20 con 0x7f37940d1d20 2024-09-12T12:25:49.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37940b2030 con 0x7f37940d1d20 2024-09-12T12:25:49.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f37917fa640 1 -- 172.21.15.3:0/786428903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784014440 con 0x7f37940d1d20 2024-09-12T12:25:49.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.687+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37940b25f0 con 0x7f37940d1d20 2024-09-12T12:25:49.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.691+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3794069250 con 0x7f37940d1d20 2024-09-12T12:25:49.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.691+0000 7f37917fa640 1 -- 172.21.15.3:0/786428903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f378401c070 con 0x7f37940d1d20 2024-09-12T12:25:49.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.691+0000 7f37917fa640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37740678c0 0x7f3774069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:49.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.691+0000 7f37917fa640 1 -- 172.21.15.3:0/786428903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f3784085a50 con 0x7f37940d1d20 2024-09-12T12:25:49.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.691+0000 7f3793fff640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37740678c0 0x7f3774069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:49.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.691+0000 7f3793fff640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37740678c0 0x7f3774069d80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f377c000f70 tx=0x7f377c007a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:49.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.695+0000 7f37917fa640 1 -- 172.21.15.3:0/786428903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f3784055e00 con 0x7f37940d1d20 2024-09-12T12:25:49.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.891+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37940cf770 con 0x7f37940d1d20 2024-09-12T12:25:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.891+0000 7f37917fa640 1 -- 172.21.15.3:0/786428903 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5243 (secure 0 0 0) 0x7f378405a100 con 0x7f37940d1d20 2024-09-12T12:25:49.896 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:49.896 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":74,"btime":"2024-09-12T12:25:34:876876+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:34.212723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6987,"mds_1":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":1,"incarnation":72,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987,6972]},"id":7}]} 2024-09-12T12:25:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 74 2024-09-12T12:25:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37740678c0 msgr2=0x7f3774069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37740678c0 0x7f3774069d80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f377c000f70 tx=0x7f377c007a10 comp rx=0 tx=0).stop 2024-09-12T12:25:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37940d1d20 msgr2=0x7f37940b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37940d1d20 0x7f37940b1700 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f378400ba80 tx=0x7f378400bf40 comp rx=0 tx=0).stop 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 shutdown_connections 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f37740678c0 0x7f3774069d80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f379415e980 0x7f37940af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f379415e5a0 0x7f37940b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 --2- 172.21.15.3:0/786428903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37940d1d20 0x7f37940b1700 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 >> 172.21.15.3:0/786428903 conn(0x7f37940626e0 msgr2=0x7f37940b65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 shutdown_connections 2024-09-12T12:25:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:49.895+0000 7f379ab6a640 1 -- 172.21.15.3:0/786428903 wait complete. 2024-09-12T12:25:49.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:25:50.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f847781b640 1 Processor -- start 2024-09-12T12:25:50.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f847781b640 1 -- start start 2024-09-12T12:25:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f847781b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f84700b38b0 0x7f84700b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f847781b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84700d00b0 con 0x7f84700d04e0 2024-09-12T12:25:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f847781b640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84700cfdd0 con 0x7f84700b4c70 2024-09-12T12:25:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f847781b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f847006db80 con 0x7f84700b38b0 2024-09-12T12:25:50.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f8475db8640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f84700d04e0 0x7f84700cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45030/0 (socket says 172.21.15.3:45030) 2024-09-12T12:25:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f8475db8640 1 -- 172.21.15.3:0/3803438894 learned_addr learned my addr 172.21.15.3:0/3803438894 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f8474db6640 1 --2- 172.21.15.3:0/3803438894 >> v2:172.21.15.3:3300/0 conn(0x7f84700b38b0 0x7f84700b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f8474db6640 1 -- 172.21.15.3:0/3803438894 >> v1:172.21.15.142:6789/0 conn(0x7f84700b4c70 legacy=0x7f847006e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f8474db6640 1 -- 172.21.15.3:0/3803438894 >> v1:172.21.15.3:6789/0 conn(0x7f84700d04e0 legacy=0x7f84700cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.035+0000 7f8474db6640 1 -- 172.21.15.3:0/3803438894 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847006de60 con 0x7f84700b38b0 2024-09-12T12:25:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8474db6640 1 --2- 172.21.15.3:0/3803438894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700b3d10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f84600097c0 tx=0x7f846001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11528e9dc0d86718 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8467fff640 1 -- 172.21.15.3:0/3803438894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84600b0070 con 0x7f84700b38b0 2024-09-12T12:25:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8467fff640 1 -- 172.21.15.3:0/3803438894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84600ab600 con 0x7f84700b38b0 2024-09-12T12:25:50.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8467fff640 1 -- 172.21.15.3:0/3803438894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84600ab9d0 con 0x7f84700b38b0 2024-09-12T12:25:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- 172.21.15.3:0/3803438894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 msgr2=0x7f84700b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 --2- 172.21.15.3:0/3803438894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700b3d10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f84600097c0 tx=0x7f846001c340 comp rx=0 tx=0).stop 2024-09-12T12:25:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- 172.21.15.3:0/3803438894 shutdown_connections 2024-09-12T12:25:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 --2- 172.21.15.3:0/3803438894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700b3d10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- 172.21.15.3:0/3803438894 >> 172.21.15.3:0/3803438894 conn(0x7f84700628b0 msgr2=0x7f8470062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- 172.21.15.3:0/3803438894 shutdown_connections 2024-09-12T12:25:50.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- 172.21.15.3:0/3803438894 wait complete. 2024-09-12T12:25:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 Processor -- start 2024-09-12T12:25:50.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- start start 2024-09-12T12:25:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84700b4c70 0x7f8470166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84700d04e0 0x7f847014e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f847016ae50 con 0x7f84700b38b0 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f847016acd0 con 0x7f84700d04e0 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f847781b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f847016afd0 con 0x7f84700b4c70 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41028/0 (socket says 172.21.15.3:41028) 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 -- 172.21.15.3:0/3614590509 learned_addr learned my addr 172.21.15.3:0/3614590509 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8475db8640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84700d04e0 0x7f847014e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8474db6640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84700b4c70 0x7f8470166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 -- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84700b4c70 msgr2=0x7f8470166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84700b4c70 0x7f8470166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 -- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84700d04e0 msgr2=0x7f847014e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84700d04e0 0x7f847014e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 -- 172.21.15.3:0/3614590509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847014fd40 con 0x7f84700b38b0 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8474db6640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84700b4c70 0x7f8470166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f84755b7640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700c66f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f846c00b410 tx=0x7f846c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.039+0000 7f8465ffb640 1 -- 172.21.15.3:0/3614590509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c015030 con 0x7f84700b38b0 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f8465ffb640 1 -- 172.21.15.3:0/3614590509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f846c004020 con 0x7f84700b38b0 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84700c5650 con 0x7f84700b38b0 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f8465ffb640 1 -- 172.21.15.3:0/3614590509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c004450 con 0x7f84700b38b0 2024-09-12T12:25:50.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84700c5c10 con 0x7f84700b38b0 2024-09-12T12:25:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84700b3d10 con 0x7f84700b38b0 2024-09-12T12:25:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f8465ffb640 1 -- 172.21.15.3:0/3614590509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f846c018070 con 0x7f84700b38b0 2024-09-12T12:25:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f8465ffb640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84540678c0 0x7f8454069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f8465ffb640 1 -- 172.21.15.3:0/3614590509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f846c0858f0 con 0x7f84700b38b0 2024-09-12T12:25:50.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f8474db6640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84540678c0 0x7f8454069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.043+0000 7f8474db6640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84540678c0 0x7f8454069d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f84600ad000 tx=0x7f8460008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.047+0000 7f8465ffb640 1 -- 172.21.15.3:0/3614590509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f846c056380 con 0x7f84700b38b0 2024-09-12T12:25:50.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.215+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f847006d830 con 0x7f84700b38b0 2024-09-12T12:25:50.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.215+0000 7f8465ffb640 1 -- 172.21.15.3:0/3614590509 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16863 (secure 0 0 0) 0x7f846c055c80 con 0x7f84700b38b0 2024-09-12T12:25:50.219 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:50.220 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":97,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:25:15.057775+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:24:40.677600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-12T12:24:41.990474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/95103479":"2024-09-13T12:25:15.057602+0000","172.21.15.142:6834/95103479":"2024-09-13T12:25:15.057602+0000","172.21.15.3:6835/2492687290":"2024-09-13T12:25:14.060842+0000","172.21.15.3:6833/2492687290":"2024-09-13T12:25:14.060842+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:25:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84540678c0 msgr2=0x7f8454069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84540678c0 0x7f8454069d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f84600ad000 tx=0x7f8460008040 comp rx=0 tx=0).stop 2024-09-12T12:25:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 msgr2=0x7f84700c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700c66f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f846c00b410 tx=0x7f846c00b8d0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 shutdown_connections 2024-09-12T12:25:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f84540678c0 0x7f8454069d80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84700d04e0 0x7f847014e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f84700b4c70 0x7f8470166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 --2- 172.21.15.3:0/3614590509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f84700b38b0 0x7f84700c66f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 >> 172.21.15.3:0/3614590509 conn(0x7f84700628b0 msgr2=0x7f84700ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:50.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 shutdown_connections 2024-09-12T12:25:50.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.219+0000 7f847781b640 1 -- 172.21.15.3:0/3614590509 wait complete. 2024-09-12T12:25:50.240 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:25:50.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.355+0000 7f309178f640 1 Processor -- start 2024-09-12T12:25:50.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.355+0000 7f309178f640 1 -- start start 2024-09-12T12:25:50.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f309178f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f308c15b350 0x7f308c06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f309178f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 0x7f308c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f309178f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c15ef40 con 0x7f308c06ec30 2024-09-12T12:25:50.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f309178f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f308c15fb10 con 0x7f308c15b350 2024-09-12T12:25:50.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f309178f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f308c15f1e0 con 0x7f308c0d0bf0 2024-09-12T12:25:50.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308a7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f308c06ec30 0x7f308c0d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45038/0 (socket says 172.21.15.3:45038) 2024-09-12T12:25:50.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308a7fc640 1 -- 172.21.15.3:0/3229181227 learned_addr learned my addr 172.21.15.3:0/3229181227 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308affd640 1 --2- 172.21.15.3:0/3229181227 >> v2:172.21.15.142:3300/0 conn(0x7f308c15b350 0x7f308c06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462992160 0 0) 0x7f308c15ef40 con 0x7f308c06ec30 2024-09-12T12:25:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308b7fe640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 0x7f308c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f306c005740 con 0x7f308c06ec30 2024-09-12T12:25:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 923478293 0 0) 0x7f306c005740 con 0x7f308c06ec30 2024-09-12T12:25:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 msgr2=0x7f308c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:50.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 0x7f308c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 >> v2:172.21.15.142:3300/0 conn(0x7f308c15b350 msgr2=0x7f308c06e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 --2- 172.21.15.3:0/3229181227 >> v2:172.21.15.142:3300/0 conn(0x7f308c15b350 0x7f308c06e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c15f4c0 con 0x7f308c06ec30 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f3074001680 con 0x7f308c06ec30 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 >> v1:172.21.15.3:6789/0 conn(0x7f308c06ec30 legacy=0x7f308c0d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c005bd0 0x7f306c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 0x7f306c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f306c006f60 0x7f306c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f308c15fb10 con 0x7f306c006f60 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f308c15f1e0 con 0x7f306c005bd0 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f306c00dbe0 con 0x7f308c0d0bf0 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308b7fe640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f306c006f60 0x7f306c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308affd640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c005bd0 0x7f306c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308a7fc640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 0x7f306c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308b7fe640 1 -- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 msgr2=0x7f306c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308b7fe640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c0d0bf0 0x7f306c006a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308b7fe640 1 -- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c005bd0 msgr2=0x7f306c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308b7fe640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c005bd0 0x7f306c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308b7fe640 1 -- 172.21.15.3:0/3229181227 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306c005740 con 0x7f306c006f60 2024-09-12T12:25:50.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.359+0000 7f308affd640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c005bd0 0x7f306c005ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f308b7fe640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f306c006f60 0x7f306c0073e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f307c009b10 tx=0x7f307c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6261aa84ea7ec4f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0b2070 con 0x7f306c006f60 2024-09-12T12:25:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c0a5660 con 0x7f306c006f60 2024-09-12T12:25:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f3089ffb640 1 -- 172.21.15.3:0/3229181227 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0096b0 con 0x7f306c006f60 2024-09-12T12:25:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- 172.21.15.3:0/3229181227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f306c006f60 msgr2=0x7f306c0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 --2- 172.21.15.3:0/3229181227 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f306c006f60 0x7f306c0073e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f307c009b10 tx=0x7f307c01ca70 comp rx=0 tx=0).stop 2024-09-12T12:25:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f308b7fe640 1 -- 172.21.15.3:0/3229181227 reap_dead start 2024-09-12T12:25:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- 172.21.15.3:0/3229181227 shutdown_connections 2024-09-12T12:25:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- 172.21.15.3:0/3229181227 >> 172.21.15.3:0/3229181227 conn(0x7f308c0628b0 msgr2=0x7f308c062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- 172.21.15.3:0/3229181227 shutdown_connections 2024-09-12T12:25:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- 172.21.15.3:0/3229181227 wait complete. 2024-09-12T12:25:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 Processor -- start 2024-09-12T12:25:50.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- start start 2024-09-12T12:25:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c006f60 0x7f308c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c157ad0 0x7f308c157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308c072f20 0x7f308c0733a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f308c0ccce0 con 0x7f308c072f20 2024-09-12T12:25:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f308c0cfa00 con 0x7f306c006f60 2024-09-12T12:25:50.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.363+0000 7f309178f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f308c0cce60 con 0x7f308c157ad0 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308c072f20 0x7f308c0733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308c072f20 0x7f308c0733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41050/0 (socket says 172.21.15.3:41050) 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 -- 172.21.15.3:0/2403910155 learned_addr learned my addr 172.21.15.3:0/2403910155 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308a7fc640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c157ad0 0x7f308c157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308affd640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c006f60 0x7f308c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 -- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c157ad0 msgr2=0x7f308c157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c157ad0 0x7f308c157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 -- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c006f60 msgr2=0x7f308c156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c006f60 0x7f308c156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 -- 172.21.15.3:0/2403910155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c06e990 con 0x7f308c072f20 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308a7fc640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c157ad0 0x7f308c157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308b7fe640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308c072f20 0x7f308c0733a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f307c009b10 tx=0x7f306c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f30837fe640 1 -- 172.21.15.3:0/2403910155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0b2040 con 0x7f308c072f20 2024-09-12T12:25:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f30837fe640 1 -- 172.21.15.3:0/2403910155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c00f000 con 0x7f308c072f20 2024-09-12T12:25:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c0738e0 con 0x7f308c072f20 2024-09-12T12:25:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f30837fe640 1 -- 172.21.15.3:0/2403910155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0096b0 con 0x7f308c072f20 2024-09-12T12:25:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c073e30 con 0x7f308c072f20 2024-09-12T12:25:50.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c06e6f0 con 0x7f308c072f20 2024-09-12T12:25:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f30837fe640 1 -- 172.21.15.3:0/2403910155 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f307c0bd030 con 0x7f308c072f20 2024-09-12T12:25:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f30837fe640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30700678c0 0x7f3070069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f30837fe640 1 -- 172.21.15.3:0/2403910155 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f307c11f420 con 0x7f308c072f20 2024-09-12T12:25:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.367+0000 7f308affd640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30700678c0 0x7f3070069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.371+0000 7f308affd640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30700678c0 0x7f3070069d80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f308c0b29f0 tx=0x7f308400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.375+0000 7f30837fe640 1 -- 172.21.15.3:0/2403910155 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f307c0c4060 con 0x7f308c072f20 2024-09-12T12:25:50.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f308c0cea60 con 0x7f308c072f20 2024-09-12T12:25:50.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f30837fe640 1 -- 172.21.15.3:0/2403910155 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5243 (secure 0 0 0) 0x7f307c0167c0 con 0x7f308c072f20 2024-09-12T12:25:50.574 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:50.575 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":74,"btime":"2024-09-12T12:25:34:876876+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:34.212723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6987,"mds_1":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":1,"incarnation":72,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987,6972]},"id":7}]} 2024-09-12T12:25:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 74 2024-09-12T12:25:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30700678c0 msgr2=0x7f3070069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30700678c0 0x7f3070069d80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f308c0b29f0 tx=0x7f308400b040 comp rx=0 tx=0).stop 2024-09-12T12:25:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308c072f20 msgr2=0x7f308c0733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308c072f20 0x7f308c0733a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f307c009b10 tx=0x7f306c007a50 comp rx=0 tx=0).stop 2024-09-12T12:25:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 shutdown_connections 2024-09-12T12:25:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f30700678c0 0x7f3070069d80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f308c072f20 0x7f308c0733a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f308c157ad0 0x7f308c157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 --2- 172.21.15.3:0/2403910155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f306c006f60 0x7f308c156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 >> 172.21.15.3:0/2403910155 conn(0x7f308c0628b0 msgr2=0x7f308c0b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 shutdown_connections 2024-09-12T12:25:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.571+0000 7f309178f640 1 -- 172.21.15.3:0/2403910155 wait complete. 2024-09-12T12:25:50.592 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:25:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c6341640 1 Processor -- start 2024-09-12T12:25:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c6341640 1 -- start start 2024-09-12T12:25:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c6341640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f93c00b4c70 0x7f93c006e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c6341640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c6341640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93c00d00b0 con 0x7f93c006eba0 2024-09-12T12:25:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c6341640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f93c00d0eb0 con 0x7f93c00b4c70 2024-09-12T12:25:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c6341640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f93c00cfdd0 con 0x7f93c00d04e0 2024-09-12T12:25:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c48de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93bffff640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f93c00b4c70 0x7f93c006e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c48de640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:49936/0 (socket says 172.21.15.3:49936) 2024-09-12T12:25:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c48de640 1 -- 172.21.15.3:0/960169243 learned_addr learned my addr 172.21.15.3:0/960169243 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c48de640 1 -- 172.21.15.3:0/960169243 >> v2:172.21.15.142:3300/0 conn(0x7f93c00b4c70 msgr2=0x7f93c006e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c48de640 1 --2- 172.21.15.3:0/960169243 >> v2:172.21.15.142:3300/0 conn(0x7f93c00b4c70 0x7f93c006e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c48de640 1 -- 172.21.15.3:0/960169243 >> v1:172.21.15.142:6789/0 conn(0x7f93c006eba0 legacy=0x7f93c00b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.707+0000 7f93c48de640 1 -- 172.21.15.3:0/960169243 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c006db80 con 0x7f93c00d04e0 2024-09-12T12:25:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c48de640 1 --2- 172.21.15.3:0/960169243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00d0970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f93c00d1030 tx=0x7f93b801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48b650464fffbb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93beffd640 1 -- 172.21.15.3:0/960169243 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80b0070 con 0x7f93c00d04e0 2024-09-12T12:25:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93beffd640 1 -- 172.21.15.3:0/960169243 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b80ab600 con 0x7f93c00d04e0 2024-09-12T12:25:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93beffd640 1 -- 172.21.15.3:0/960169243 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80ab9d0 con 0x7f93c00d04e0 2024-09-12T12:25:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- 172.21.15.3:0/960169243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 msgr2=0x7f93c00d0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 --2- 172.21.15.3:0/960169243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00d0970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f93c00d1030 tx=0x7f93b801c420 comp rx=0 tx=0).stop 2024-09-12T12:25:50.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- 172.21.15.3:0/960169243 shutdown_connections 2024-09-12T12:25:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 --2- 172.21.15.3:0/960169243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00d0970 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 --2- 172.21.15.3:0/960169243 >> v2:172.21.15.142:3300/0 conn(0x7f93c00b4c70 0x7f93c006e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- 172.21.15.3:0/960169243 >> 172.21.15.3:0/960169243 conn(0x7f93c00628b0 msgr2=0x7f93c0062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- 172.21.15.3:0/960169243 shutdown_connections 2024-09-12T12:25:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- 172.21.15.3:0/960169243 wait complete. 2024-09-12T12:25:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 Processor -- start 2024-09-12T12:25:50.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- start start 2024-09-12T12:25:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93c006eba0 0x7f93c014f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93c00b4c70 0x7f93c014ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00b41d0 con 0x7f93c00b4c70 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00cfb00 con 0x7f93c006eba0 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93c6341640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f93c01692a0 con 0x7f93c00d04e0 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.711+0000 7f93bf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93c00b4c70 0x7f93c014ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93c00b4c70 0x7f93c014ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41054/0 (socket says 172.21.15.3:41054) 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 -- 172.21.15.3:0/898353095 learned_addr learned my addr 172.21.15.3:0/898353095 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93c48de640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bffff640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93c006eba0 0x7f93c014f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 -- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 msgr2=0x7f93c00aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 -- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93c006eba0 msgr2=0x7f93c014f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93c006eba0 0x7f93c014f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 -- 172.21.15.3:0/898353095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c00addb0 con 0x7f93c00b4c70 2024-09-12T12:25:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bf7fe640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93c00b4c70 0x7f93c014ced0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c730 tx=0x7f93b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bcff9640 1 -- 172.21.15.3:0/898353095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4016020 con 0x7f93c00b4c70 2024-09-12T12:25:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c00ae090 con 0x7f93c00b4c70 2024-09-12T12:25:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bcff9640 1 -- 172.21.15.3:0/898353095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b4006d60 con 0x7f93c00b4c70 2024-09-12T12:25:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bcff9640 1 -- 172.21.15.3:0/898353095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4003bc0 con 0x7f93c00b4c70 2024-09-12T12:25:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c00ae650 con 0x7f93c00b4c70 2024-09-12T12:25:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c006e660 con 0x7f93c00b4c70 2024-09-12T12:25:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bcff9640 1 -- 172.21.15.3:0/898353095 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f93b400a850 con 0x7f93c00b4c70 2024-09-12T12:25:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bcff9640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93ac0678c0 0x7f93ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bcff9640 1 -- 172.21.15.3:0/898353095 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f93b4085af0 con 0x7f93c00b4c70 2024-09-12T12:25:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bffff640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93ac0678c0 0x7f93ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:50.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.715+0000 7f93bffff640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93ac0678c0 0x7f93ac069d80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009a50 tx=0x7f93b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:50.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.719+0000 7f93bcff9640 1 -- 172.21.15.3:0/898353095 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f93b4020020 con 0x7f93c00b4c70 2024-09-12T12:25:50.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.915+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93c006d830 con 0x7f93c00b4c70 2024-09-12T12:25:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.915+0000 7f93bcff9640 1 -- 172.21.15.3:0/898353095 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5243 (secure 0 0 0) 0x7f93b40564f0 con 0x7f93c00b4c70 2024-09-12T12:25:50.920 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:50.920 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":74,"btime":"2024-09-12T12:25:34:876876+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T12:24:42.989579+0000","modified":"2024-09-12T12:25:34.212723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":6987,"mds_1":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":1,"incarnation":72,"state":"up:active","state_seq":5,"addr":"172.21.15.3:6833/1847705632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1847705632},{"type":"v1","addr":"172.21.15.3:6833","nonce":1847705632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.3:6835/1654386752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":1654386752},{"type":"v1","addr":"172.21.15.3:6835","nonce":1654386752}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987,6972]},"id":7}]} 2024-09-12T12:25:50.920 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 74 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93ac0678c0 msgr2=0x7f93ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93ac0678c0 0x7f93ac069d80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009a50 tx=0x7f93b0008040 comp rx=0 tx=0).stop 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93c00b4c70 msgr2=0x7f93c014ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93c00b4c70 0x7f93c014ced0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c730 tx=0x7f93b400cbf0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 shutdown_connections 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f93ac0678c0 0x7f93ac069d80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93c00d04e0 0x7f93c00aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f93c00b4c70 0x7f93c014ced0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 --2- 172.21.15.3:0/898353095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93c006eba0 0x7f93c014f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 >> 172.21.15.3:0/898353095 conn(0x7f93c00628b0 msgr2=0x7f93c015de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:50.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 shutdown_connections 2024-09-12T12:25:50.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:50.919+0000 7f93c6341640 1 -- 172.21.15.3:0/898353095 wait complete. 2024-09-12T12:25:50.940 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:25:51.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71fc3c640 1 Processor -- start 2024-09-12T12:25:51.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71fc3c640 1 -- start start 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71fc3c640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe7180d04e0 0x7fe7180d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71fc3c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7180ceec0 0x7fe7180cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71fc3c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe718160a90 con 0x7fe71815ff40 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71fc3c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe71815d260 con 0x7fe7180ceec0 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71fc3c640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fe718160d00 con 0x7fe7180d04e0 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71e1d9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7180ceec0 0x7fe7180cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71e1d9640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7180ceec0 0x7fe7180cf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41062/0 (socket says 172.21.15.3:41062) 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71e1d9640 1 -- 172.21.15.3:0/42453128 learned_addr learned my addr 172.21.15.3:0/42453128 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71d9d8640 1 --2- 172.21.15.3:0/42453128 >> v2:172.21.15.142:3300/0 conn(0x7fe7180d04e0 0x7fe7180d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71e1d9640 1 -- 172.21.15.3:0/42453128 >> v2:172.21.15.142:3300/0 conn(0x7fe7180d04e0 msgr2=0x7fe7180d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71e1d9640 1 --2- 172.21.15.3:0/42453128 >> v2:172.21.15.142:3300/0 conn(0x7fe7180d04e0 0x7fe7180d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71e1d9640 1 -- 172.21.15.3:0/42453128 >> v1:172.21.15.3:6789/0 conn(0x7fe71815ff40 legacy=0x7fe718160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.059+0000 7fe71e1d9640 1 -- 172.21.15.3:0/42453128 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71815cc10 con 0x7fe7180ceec0 2024-09-12T12:25:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 --2- 172.21.15.3:0/42453128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7180ceec0 0x7fe7180cf330 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe71806d3d0 tx=0x7fe71001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150cc3131d8373bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71c9d6640 1 -- 172.21.15.3:0/42453128 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100a57a0 con 0x7fe7180ceec0 2024-09-12T12:25:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71c9d6640 1 -- 172.21.15.3:0/42453128 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7100a5940 con 0x7fe7180ceec0 2024-09-12T12:25:51.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71c9d6640 1 -- 172.21.15.3:0/42453128 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100a5d10 con 0x7fe7180ceec0 2024-09-12T12:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- 172.21.15.3:0/42453128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7180ceec0 msgr2=0x7fe7180cf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/42453128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7180ceec0 0x7fe7180cf330 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe71806d3d0 tx=0x7fe71001c9c0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- 172.21.15.3:0/42453128 shutdown_connections 2024-09-12T12:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/42453128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7180ceec0 0x7fe7180cf330 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/42453128 >> v2:172.21.15.142:3300/0 conn(0x7fe7180d04e0 0x7fe7180d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- 172.21.15.3:0/42453128 >> 172.21.15.3:0/42453128 conn(0x7fe7180628b0 msgr2=0x7fe718062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:51.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- 172.21.15.3:0/42453128 shutdown_connections 2024-09-12T12:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- 172.21.15.3:0/42453128 wait complete. 2024-09-12T12:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 Processor -- start 2024-09-12T12:25:51.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- start start 2024-09-12T12:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7180ceec0 0x7fe718072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7180d04e0 0x7fe71814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71815ff40 0x7fe718159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe71815a340 con 0x7fe71815ff40 2024-09-12T12:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe71815a1c0 con 0x7fe7180ceec0 2024-09-12T12:25:51.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71fc3c640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fe71815d6d0 con 0x7fe7180d04e0 2024-09-12T12:25:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71815ff40 0x7fe718159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71815ff40 0x7fe718159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41074/0 (socket says 172.21.15.3:41074) 2024-09-12T12:25:51.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 -- 172.21.15.3:0/1078647633 learned_addr learned my addr 172.21.15.3:0/1078647633 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71d1d7640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7180d04e0 0x7fe71814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71d9d8640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7180ceec0 0x7fe718072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 -- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7180d04e0 msgr2=0x7fe71814e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7180d04e0 0x7fe71814e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 -- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7180ceec0 msgr2=0x7fe718072ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7180ceec0 0x7fe718072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.063+0000 7fe71e1d9640 1 -- 172.21.15.3:0/1078647633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7180b14e0 con 0x7fe71815ff40 2024-09-12T12:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe71e1d9640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71815ff40 0x7fe718159290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe7100af040 tx=0x7fe710019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe70e7fc640 1 -- 172.21.15.3:0/1078647633 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100bc030 con 0x7fe71815ff40 2024-09-12T12:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe70e7fc640 1 -- 172.21.15.3:0/1078647633 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7100b8470 con 0x7fe71815ff40 2024-09-12T12:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71814e640 con 0x7fe71815ff40 2024-09-12T12:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe70e7fc640 1 -- 172.21.15.3:0/1078647633 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100b8840 con 0x7fe71815ff40 2024-09-12T12:25:51.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7180738b0 con 0x7fe71815ff40 2024-09-12T12:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7180cf330 con 0x7fe71815ff40 2024-09-12T12:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe70e7fc640 1 -- 172.21.15.3:0/1078647633 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7100bf070 con 0x7fe71815ff40 2024-09-12T12:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe70e7fc640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6fc0678c0 0x7fe6fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe70e7fc640 1 -- 172.21.15.3:0/1078647633 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7fe710120660 con 0x7fe71815ff40 2024-09-12T12:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe71d9d8640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6fc0678c0 0x7fe6fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.067+0000 7fe71d9d8640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6fc0678c0 0x7fe6fc069d80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe714009790 tx=0x7fe714008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.071+0000 7fe70e7fc640 1 -- 172.21.15.3:0/1078647633 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fe7100c7020 con 0x7fe71815ff40 2024-09-12T12:25:51.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.235+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe71806d890 con 0x7fe71815ff40 2024-09-12T12:25:51.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe70e7fc640 1 -- 172.21.15.3:0/1078647633 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16863 (secure 0 0 0) 0x7fe7100165e0 con 0x7fe71815ff40 2024-09-12T12:25:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:51.241 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":97,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:25:15.057775+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:24:40.677600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-12T12:24:41.990474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.142:6835/95103479":"2024-09-13T12:25:15.057602+0000","172.21.15.142:6834/95103479":"2024-09-13T12:25:15.057602+0000","172.21.15.3:6835/2492687290":"2024-09-13T12:25:14.060842+0000","172.21.15.3:6833/2492687290":"2024-09-13T12:25:14.060842+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:25:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6fc0678c0 msgr2=0x7fe6fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6fc0678c0 0x7fe6fc069d80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe714009790 tx=0x7fe714008040 comp rx=0 tx=0).stop 2024-09-12T12:25:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71815ff40 msgr2=0x7fe718159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71815ff40 0x7fe718159290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe7100af040 tx=0x7fe710019760 comp rx=0 tx=0).stop 2024-09-12T12:25:51.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 shutdown_connections 2024-09-12T12:25:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fe6fc0678c0 0x7fe6fc069d80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71815ff40 0x7fe718159290 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe7180d04e0 0x7fe71814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 --2- 172.21.15.3:0/1078647633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7180ceec0 0x7fe718072ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 >> 172.21.15.3:0/1078647633 conn(0x7fe7180628b0 msgr2=0x7fe7180b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 shutdown_connections 2024-09-12T12:25:51.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.239+0000 7fe71fc3c640 1 -- 172.21.15.3:0/1078647633 wait complete. 2024-09-12T12:25:51.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-12T12:25:51.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 Processor -- start 2024-09-12T12:25:51.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 -- start start 2024-09-12T12:25:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f785415fb20 0x7f78540c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78540d0640 0x7f785406f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785406fbb0 0x7f78540c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f78540704f0 con 0x7f785415fb20 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f78540d17a0 con 0x7f78540d0640 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f785a8ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f78540c93b0 con 0x7f785406fbb0 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f785415fb20 0x7f78540c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f785415fb20 0x7f78540c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41080/0 (socket says 172.21.15.3:41080) 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 -- 172.21.15.3:0/2337344526 learned_addr learned my addr 172.21.15.3:0/2337344526 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f78537fe640 1 --2- 172.21.15.3:0/2337344526 >> v2:172.21.15.142:3300/0 conn(0x7f78540d0640 0x7f785406f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7858e87640 1 --2- 172.21.15.3:0/2337344526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785406fbb0 0x7f78540c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 -- 172.21.15.3:0/2337344526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785406fbb0 msgr2=0x7f78540c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 --2- 172.21.15.3:0/2337344526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785406fbb0 0x7f78540c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 -- 172.21.15.3:0/2337344526 >> v2:172.21.15.142:3300/0 conn(0x7f78540d0640 msgr2=0x7f785406f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 --2- 172.21.15.3:0/2337344526 >> v2:172.21.15.142:3300/0 conn(0x7f78540d0640 0x7f785406f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f7853fff640 1 -- 172.21.15.3:0/2337344526 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78540d1c30 con 0x7f785415fb20 2024-09-12T12:25:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.375+0000 7f78537fe640 1 --2- 172.21.15.3:0/2337344526 >> v2:172.21.15.142:3300/0 conn(0x7f78540d0640 0x7f785406f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:51.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f7853fff640 1 --2- 172.21.15.3:0/2337344526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f785415fb20 0x7f78540c8e70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f78540b5180 tx=0x7f784401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ec431872a352e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f7852ffd640 1 -- 172.21.15.3:0/2337344526 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440b0070 con 0x7f785415fb20 2024-09-12T12:25:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f7852ffd640 1 -- 172.21.15.3:0/2337344526 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78440ab600 con 0x7f785415fb20 2024-09-12T12:25:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f7852ffd640 1 -- 172.21.15.3:0/2337344526 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440ab9d0 con 0x7f785415fb20 2024-09-12T12:25:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- 172.21.15.3:0/2337344526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f785415fb20 msgr2=0x7f78540c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 --2- 172.21.15.3:0/2337344526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f785415fb20 0x7f78540c8e70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f78540b5180 tx=0x7f784401c260 comp rx=0 tx=0).stop 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- 172.21.15.3:0/2337344526 shutdown_connections 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 --2- 172.21.15.3:0/2337344526 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785406fbb0 0x7f78540c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 --2- 172.21.15.3:0/2337344526 >> v2:172.21.15.142:3300/0 conn(0x7f78540d0640 0x7f785406f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 --2- 172.21.15.3:0/2337344526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f785415fb20 0x7f78540c8e70 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- 172.21.15.3:0/2337344526 >> 172.21.15.3:0/2337344526 conn(0x7f78540b40c0 msgr2=0x7f78540b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- 172.21.15.3:0/2337344526 shutdown_connections 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- 172.21.15.3:0/2337344526 wait complete. 2024-09-12T12:25:51.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 Processor -- start 2024-09-12T12:25:51.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- start start 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785406fbb0 0x7f785413c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78540d0640 0x7f78541556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785415fb20 0x7f7854155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f785415ff00 con 0x7f78540d0640 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78540d1430 con 0x7f785406fbb0 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f785a8ea640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7854160080 con 0x7f785415fb20 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f78537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78540d0640 0x7f78541556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f78537fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78540d0640 0x7f78541556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41086/0 (socket says 172.21.15.3:41086) 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.379+0000 7f78537fe640 1 -- 172.21.15.3:0/724493712 learned_addr learned my addr 172.21.15.3:0/724493712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7858e87640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785415fb20 0x7f7854155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7853fff640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785406fbb0 0x7f785413c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f78537fe640 1 -- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785415fb20 msgr2=0x7f7854155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f78537fe640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785415fb20 0x7f7854155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f78537fe640 1 -- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785406fbb0 msgr2=0x7f785413c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f78537fe640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785406fbb0 0x7f785413c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f78537fe640 1 -- 172.21.15.3:0/724493712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78541579e0 con 0x7f78540d0640 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7853fff640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785406fbb0 0x7f785413c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f78537fe640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78540d0640 0x7f78541556f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f784800c7f0 tx=0x7f784800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7850ff9640 1 -- 172.21.15.3:0/724493712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848007a30 con 0x7f78540d0640 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7850ff9640 1 -- 172.21.15.3:0/724493712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7848007bd0 con 0x7f78540d0640 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7850ff9640 1 -- 172.21.15.3:0/724493712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78480113f0 con 0x7f78540d0640 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78541fb180 con 0x7f78540d0640 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7854157f80 con 0x7f78540d0640 2024-09-12T12:25:51.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785406f670 con 0x7f78540d0640 2024-09-12T12:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7850ff9640 1 -- 172.21.15.3:0/724493712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7848011590 con 0x7f78540d0640 2024-09-12T12:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7850ff9640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78400678c0 0x7f7840069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7850ff9640 1 -- 172.21.15.3:0/724493712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5881+0+0 (secure 0 0 0) 0x7f7848085e70 con 0x7f78540d0640 2024-09-12T12:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7853fff640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78400678c0 0x7f7840069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.383+0000 7f7853fff640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78400678c0 0x7f7840069d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f78440ad000 tx=0x7f7844008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.387+0000 7f7850ff9640 1 -- 172.21.15.3:0/724493712 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7848015020 con 0x7f78540d0640 2024-09-12T12:25:51.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.579+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f785406e1a0 con 0x7f78540d0640 2024-09-12T12:25:51.583 INFO:tasks.ceph.mon.a.smithi003.stderr:2024-09-12T12:25:51.579+0000 7f0a5d435640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:25:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.583+0000 7f7850ff9640 1 -- 172.21.15.3:0/724493712 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f7848056990 con 0x7f78540d0640 2024-09-12T12:25:51.586 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:25:51.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78400678c0 msgr2=0x7f7840069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78400678c0 0x7f7840069d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f78440ad000 tx=0x7f7844008040 comp rx=0 tx=0).stop 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78540d0640 msgr2=0x7f78541556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78540d0640 0x7f78541556f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f784800c7f0 tx=0x7f784800ccb0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 shutdown_connections 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f78400678c0 0x7f7840069d80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f785415fb20 0x7f7854155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78540d0640 0x7f78541556f0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 --2- 172.21.15.3:0/724493712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f785406fbb0 0x7f785413c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 >> 172.21.15.3:0/724493712 conn(0x7f78540b40c0 msgr2=0x7f78540cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 shutdown_connections 2024-09-12T12:25:51.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.587+0000 7f785a8ea640 1 -- 172.21.15.3:0/724493712 wait complete. 2024-09-12T12:25:51.599 INFO:tasks.ceph.mds.a.smithi003.stderr: -5762> 2024-09-12T12:25:20.067+0000 7fe4c71fb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:25:51.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:25:51.622 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-09-12T12:25:51.634 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-09-12T12:25:51.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd87aa0f640 1 Processor -- start 2024-09-12T12:25:51.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd87aa0f640 1 -- start start 2024-09-12T12:25:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd87aa0f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874158d30 0x7fd87415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd87aa0f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87406f820 con 0x7fd87415f080 2024-09-12T12:25:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd87aa0f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87415f670 con 0x7fd874158950 2024-09-12T12:25:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd87aa0f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd87415f950 con 0x7fd874158d30 2024-09-12T12:25:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd878fac640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd87415f080 0x7fd8740626e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45074/0 (socket says 172.21.15.3:45074) 2024-09-12T12:25:51.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd878fac640 1 -- 172.21.15.3:0/1506716884 learned_addr learned my addr 172.21.15.3:0/1506716884 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd8737fe640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874158d30 0x7fd87415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215343843 0 0) 0x7fd87406f820 con 0x7fd87415f080 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd860003530 con 0x7fd87415f080 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 502421803 0 0) 0x7fd860003530 con 0x7fd87415f080 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874158d30 msgr2=0x7fd87415eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874158d30 0x7fd87415eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 >> v1:172.21.15.142:6789/0 conn(0x7fd874158950 legacy=0x7fd87406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87406e930 con 0x7fd87415f080 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fd86c001a00 con 0x7fd87415f080 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 >> v1:172.21.15.3:6789/0 conn(0x7fd87415f080 legacy=0x7fd8740626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd860003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860004810 0x7fd860004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874158d30 0x7fd8600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd87415f950 con 0x7fd8600039f0 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd86000bab0 con 0x7fd860004810 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd86000bcb0 con 0x7fd874158d30 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd873fff640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd860003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd8737fe640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860004810 0x7fd860004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd873fff640 1 -- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874158d30 msgr2=0x7fd8600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd873fff640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874158d30 0x7fd8600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd873fff640 1 -- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860004810 msgr2=0x7fd860004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:51.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd873fff640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860004810 0x7fd860004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.727+0000 7fd873fff640 1 -- 172.21.15.3:0/1506716884 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd860003530 con 0x7fd8600039f0 2024-09-12T12:25:51.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd873fff640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd860003e10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd864001750 tx=0x7fd86401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c300903d27fc4125 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640013a0 con 0x7fd8600039f0 2024-09-12T12:25:51.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8640052f0 con 0x7fd8600039f0 2024-09-12T12:25:51.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd872ffd640 1 -- 172.21.15.3:0/1506716884 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864003840 con 0x7fd8600039f0 2024-09-12T12:25:51.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- 172.21.15.3:0/1506716884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 msgr2=0x7fd860003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 --2- 172.21.15.3:0/1506716884 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd860003e10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd864001750 tx=0x7fd86401c3c0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd878fac640 1 -- 172.21.15.3:0/1506716884 reap_dead start 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- 172.21.15.3:0/1506716884 shutdown_connections 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- 172.21.15.3:0/1506716884 >> 172.21.15.3:0/1506716884 conn(0x7fd87406bc70 msgr2=0x7fd87406c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- 172.21.15.3:0/1506716884 shutdown_connections 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- 172.21.15.3:0/1506716884 wait complete. 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 Processor -- start 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- start start 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd87414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd874164360 0x7fd874152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874153450 0x7fd8740b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd8740d0380 con 0x7fd8600039f0 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd874063000 con 0x7fd874164360 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd87aa0f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fd8740d0500 con 0x7fd874153450 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd873fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd87414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd873fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd87414ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41126/0 (socket says 172.21.15.3:41126) 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.731+0000 7fd873fff640 1 -- 172.21.15.3:0/28007477 learned_addr learned my addr 172.21.15.3:0/28007477 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd873fff640 1 -- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874153450 msgr2=0x7fd8740b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd873fff640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874153450 0x7fd8740b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd873fff640 1 -- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd874164360 msgr2=0x7fd874152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd873fff640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd874164360 0x7fd874152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd873fff640 1 -- 172.21.15.3:0/28007477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741504f0 con 0x7fd8600039f0 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd873fff640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd87414ffb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd864001750 tx=0x7fd860005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd870ff9640 1 -- 172.21.15.3:0/28007477 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864007a40 con 0x7fd8600039f0 2024-09-12T12:25:51.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd870ff9640 1 -- 172.21.15.3:0/28007477 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8640bb070 con 0x7fd8600039f0 2024-09-12T12:25:51.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741635e0 con 0x7fd8600039f0 2024-09-12T12:25:51.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd870ff9640 1 -- 172.21.15.3:0/28007477 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640013a0 con 0x7fd8600039f0 2024-09-12T12:25:51.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd874163880 con 0x7fd8600039f0 2024-09-12T12:25:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87415eb40 con 0x7fd8600039f0 2024-09-12T12:25:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd870ff9640 1 -- 172.21.15.3:0/28007477 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8640b4040 con 0x7fd8600039f0 2024-09-12T12:25:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd870ff9640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd8540678f0 0x7fd854069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd870ff9640 1 -- 172.21.15.3:0/28007477 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6053+0+0 (secure 0 0 0) 0x7fd86411fcc0 con 0x7fd8600039f0 2024-09-12T12:25:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd8737fe640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd8540678f0 0x7fd854069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:51.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.735+0000 7fd8737fe640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd8540678f0 0x7fd854069db0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd8741584d0 tx=0x7fd85c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:51.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.739+0000 7fd870ff9640 1 -- 172.21.15.3:0/28007477 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fd8640160d0 con 0x7fd8600039f0 2024-09-12T12:25:51.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:51.935+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd87406d0f0 con 0x7fd8600039f0 2024-09-12T12:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.583+0000 7fd870ff9640 1 -- 172.21.15.3:0/28007477 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7fd8640f0850 con 0x7fd8600039f0 2024-09-12T12:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd8540678f0 msgr2=0x7fd854069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd8540678f0 0x7fd854069db0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd8741584d0 tx=0x7fd85c008040 comp rx=0 tx=0).stop 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 msgr2=0x7fd87414ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd87414ffb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fd864001750 tx=0x7fd860005840 comp rx=0 tx=0).stop 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 shutdown_connections 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fd8540678f0 0x7fd854069db0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd874153450 0x7fd8740b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd874164360 0x7fd874152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 --2- 172.21.15.3:0/28007477 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8600039f0 0x7fd87414ffb0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 >> 172.21.15.3:0/28007477 conn(0x7fd87406bc70 msgr2=0x7fd874157200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 shutdown_connections 2024-09-12T12:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.587+0000 7fd87aa0f640 1 -- 172.21.15.3:0/28007477 wait complete. 2024-09-12T12:25:52.614 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:25:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c76a22640 1 Processor -- start 2024-09-12T12:25:52.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c76a22640 1 -- start start 2024-09-12T12:25:52.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c76a22640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c70073610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:52.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c76a22640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c700cf9f0 0x7f9c700cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:52.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c76a22640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c70072290 con 0x7f9c700d2e20 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c76a22640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f9c70073180 con 0x7f9c700cf9f0 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c76a22640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9c70072570 con 0x7f9c700b3950 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c700cf9f0 0x7f9c700cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f9c700cf9f0 0x7f9c700cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41134/0 (socket says 172.21.15.3:41134) 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6f7fe640 1 -- 172.21.15.3:0/416771773 learned_addr learned my addr 172.21.15.3:0/416771773 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6ffff640 1 --2- 172.21.15.3:0/416771773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c70073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6f7fe640 1 -- 172.21.15.3:0/416771773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 msgr2=0x7f9c70073610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6f7fe640 1 --2- 172.21.15.3:0/416771773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c70073610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6f7fe640 1 -- 172.21.15.3:0/416771773 >> v1:172.21.15.142:6789/0 conn(0x7f9c700d2e20 legacy=0x7f9c700b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:52.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.779+0000 7f9c6f7fe640 1 -- 172.21.15.3:0/416771773 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c70072850 con 0x7f9c700cf9f0 2024-09-12T12:25:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c6f7fe640 1 --2- 172.21.15.3:0/416771773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700cf9f0 0x7f9c700cfe10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009870 tx=0x7f9c6401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900f691d074098b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c6effd640 1 -- 172.21.15.3:0/416771773 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640aa610 con 0x7f9c700cf9f0 2024-09-12T12:25:52.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c6effd640 1 -- 172.21.15.3:0/416771773 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c640af070 con 0x7f9c700cf9f0 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c6effd640 1 -- 172.21.15.3:0/416771773 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c640aa9d0 con 0x7f9c700cf9f0 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 -- 172.21.15.3:0/416771773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700cf9f0 msgr2=0x7f9c700cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 --2- 172.21.15.3:0/416771773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700cf9f0 0x7f9c700cfe10 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009870 tx=0x7f9c6401c4f0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 -- 172.21.15.3:0/416771773 shutdown_connections 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 --2- 172.21.15.3:0/416771773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700cf9f0 0x7f9c700cfe10 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 --2- 172.21.15.3:0/416771773 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c70073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 -- 172.21.15.3:0/416771773 >> 172.21.15.3:0/416771773 conn(0x7f9c700d4660 msgr2=0x7f9c700d4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 -- 172.21.15.3:0/416771773 shutdown_connections 2024-09-12T12:25:52.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.783+0000 7f9c76a22640 1 -- 172.21.15.3:0/416771773 wait complete. 2024-09-12T12:25:52.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 Processor -- start 2024-09-12T12:25:52.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 -- start start 2024-09-12T12:25:52.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c700caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:52.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c700cf9f0 0x7f9c7015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700d2e20 0x7f9c700c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9c700b8870 con 0x7f9c700d2e20 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9c700d36a0 con 0x7f9c700cf9f0 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f9c700b3d30 con 0x7f9c700b3950 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700d2e20 0x7f9c700c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700d2e20 0x7f9c700c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41142/0 (socket says 172.21.15.3:41142) 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 -- 172.21.15.3:0/3189830158 learned_addr learned my addr 172.21.15.3:0/3189830158 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c6ffff640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c700caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c6f7fe640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c700cf9f0 0x7f9c7015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 -- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 msgr2=0x7f9c700caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c700caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 -- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c700cf9f0 msgr2=0x7f9c7015a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:52.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c700cf9f0 0x7f9c7015a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 -- 172.21.15.3:0/3189830158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c701596f0 con 0x7f9c700d2e20 2024-09-12T12:25:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c74fbf640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700d2e20 0x7f9c700c9920 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b410 tx=0x7f9c6800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c6cff9640 1 -- 172.21.15.3:0/3189830158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68015020 con 0x7f9c700d2e20 2024-09-12T12:25:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c6cff9640 1 -- 172.21.15.3:0/3189830158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c68011870 con 0x7f9c700d2e20 2024-09-12T12:25:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c6cff9640 1 -- 172.21.15.3:0/3189830158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68011c60 con 0x7f9c700d2e20 2024-09-12T12:25:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c701599d0 con 0x7f9c700d2e20 2024-09-12T12:25:52.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c70159f90 con 0x7f9c700d2e20 2024-09-12T12:25:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c70073610 con 0x7f9c700d2e20 2024-09-12T12:25:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.787+0000 7f9c6cff9640 1 -- 172.21.15.3:0/3189830158 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9c68018020 con 0x7f9c700d2e20 2024-09-12T12:25:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.791+0000 7f9c6cff9640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9c5c0678c0 0x7f9c5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.791+0000 7f9c6cff9640 1 -- 172.21.15.3:0/3189830158 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6053+0+0 (secure 0 0 0) 0x7f9c68086160 con 0x7f9c700d2e20 2024-09-12T12:25:52.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.791+0000 7f9c6ffff640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9c5c0678c0 0x7f9c5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:52.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.791+0000 7f9c6ffff640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9c5c0678c0 0x7f9c5c069d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9c600070b0 tx=0x7f9c60007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:52.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.791+0000 7f9c6cff9640 1 -- 172.21.15.3:0/3189830158 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f9c6801f020 con 0x7f9c700d2e20 2024-09-12T12:25:52.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:52.955+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9c700ca4c0 con 0x7f9c700d2e20 2024-09-12T12:25:53.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.939+0000 7f9c6cff9640 1 -- 172.21.15.3:0/3189830158 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7f9c6800f2e0 con 0x7f9c700d2e20 2024-09-12T12:25:53.944 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:25:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9c5c0678c0 msgr2=0x7f9c5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9c5c0678c0 0x7f9c5c069d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9c600070b0 tx=0x7f9c60007040 comp rx=0 tx=0).stop 2024-09-12T12:25:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700d2e20 msgr2=0x7f9c700c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:53.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700d2e20 0x7f9c700c9920 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b410 tx=0x7f9c6800b8d0 comp rx=0 tx=0).stop 2024-09-12T12:25:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 shutdown_connections 2024-09-12T12:25:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f9c5c0678c0 0x7f9c5c069d80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9c700d2e20 0x7f9c700c9920 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c700cf9f0 0x7f9c7015a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 --2- 172.21.15.3:0/3189830158 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c700b3950 0x7f9c700caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:53.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 >> 172.21.15.3:0/3189830158 conn(0x7f9c700d4660 msgr2=0x7f9c700b55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.943+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 shutdown_connections 2024-09-12T12:25:53.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:53.947+0000 7f9c76a22640 1 -- 172.21.15.3:0/3189830158 wait complete. 2024-09-12T12:25:53.973 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05f9cb640 1 Processor -- start 2024-09-12T12:25:54.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05f9cb640 1 -- start start 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05f9cb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa05815d4e0 0x7fa0580c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05f9cb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0580ccbf0 0x7fa0580cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05f9cb640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05815d8c0 con 0x7fa0580d0450 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05f9cb640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580cd6b0 con 0x7fa0580ccbf0 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05f9cb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580cd990 con 0x7fa05815d4e0 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05cf66640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa0580ccbf0 0x7fa0580cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05df68640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa0580d0450 0x7fa05806d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45084/0 (socket says 172.21.15.3:45084) 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.095+0000 7fa05df68640 1 -- 172.21.15.3:0/1798522503 learned_addr learned my addr 172.21.15.3:0/1798522503 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05d767640 1 --2- 172.21.15.3:0/1798522503 >> v2:172.21.15.142:3300/0 conn(0x7fa05815d4e0 0x7fa0580c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05cf66640 1 -- 172.21.15.3:0/1798522503 >> v2:172.21.15.142:3300/0 conn(0x7fa05815d4e0 msgr2=0x7fa0580c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05cf66640 1 --2- 172.21.15.3:0/1798522503 >> v2:172.21.15.142:3300/0 conn(0x7fa05815d4e0 0x7fa0580c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05cf66640 1 -- 172.21.15.3:0/1798522503 >> v1:172.21.15.3:6789/0 conn(0x7fa0580d0450 legacy=0x7fa05806d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa057fff640 1 -- 172.21.15.3:0/1798522503 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265588512 0 0) 0x7fa05815d8c0 con 0x7fa0580d0450 2024-09-12T12:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05cf66640 1 -- 172.21.15.3:0/1798522503 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0580c6250 con 0x7fa0580ccbf0 2024-09-12T12:25:54.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05d767640 1 --2- 172.21.15.3:0/1798522503 >> v2:172.21.15.142:3300/0 conn(0x7fa05815d4e0 0x7fa0580c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05cf66640 1 --2- 172.21.15.3:0/1798522503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa0580cd080 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0097c0 tx=0x7fa04c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bf298b6587d197 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa057fff640 1 -- 172.21.15.3:0/1798522503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0b0070 con 0x7fa0580ccbf0 2024-09-12T12:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa057fff640 1 -- 172.21.15.3:0/1798522503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa04c0ab600 con 0x7fa0580ccbf0 2024-09-12T12:25:54.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa057fff640 1 -- 172.21.15.3:0/1798522503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0ab9d0 con 0x7fa0580ccbf0 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 -- 172.21.15.3:0/1798522503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 msgr2=0x7fa0580cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/1798522503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa0580cd080 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0097c0 tx=0x7fa04c01c430 comp rx=0 tx=0).stop 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 -- 172.21.15.3:0/1798522503 shutdown_connections 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/1798522503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa0580cd080 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/1798522503 >> v2:172.21.15.142:3300/0 conn(0x7fa05815d4e0 0x7fa0580c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 -- 172.21.15.3:0/1798522503 >> 172.21.15.3:0/1798522503 conn(0x7fa0580628b0 msgr2=0x7fa058062cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 -- 172.21.15.3:0/1798522503 shutdown_connections 2024-09-12T12:25:54.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.099+0000 7fa05f9cb640 1 -- 172.21.15.3:0/1798522503 wait complete. 2024-09-12T12:25:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 Processor -- start 2024-09-12T12:25:54.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 -- start start 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa05815fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0580d0450 0x7fa058150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05815d4e0 0x7fa0580afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa0580c6f90 con 0x7fa0580ccbf0 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa0580d0c70 con 0x7fa05815d4e0 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa0580b2b70 con 0x7fa0580d0450 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa05815fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:54.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa05815fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41156/0 (socket says 172.21.15.3:41156) 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 -- 172.21.15.3:0/3958074293 learned_addr learned my addr 172.21.15.3:0/3958074293 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05df68640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05815d4e0 0x7fa0580afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05cf66640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0580d0450 0x7fa058150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 -- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0580d0450 msgr2=0x7fa058150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0580d0450 0x7fa058150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 -- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05815d4e0 msgr2=0x7fa0580afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05815d4e0 0x7fa0580afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 -- 172.21.15.3:0/3958074293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05815e4f0 con 0x7fa0580ccbf0 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05d767640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa05815fdf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa05000cf60 tx=0x7fa05000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa055ffb640 1 -- 172.21.15.3:0/3958074293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05001a070 con 0x7fa0580ccbf0 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa055ffb640 1 -- 172.21.15.3:0/3958074293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05000d840 con 0x7fa0580ccbf0 2024-09-12T12:25:54.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05815e7d0 con 0x7fa0580ccbf0 2024-09-12T12:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa055ffb640 1 -- 172.21.15.3:0/3958074293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05000dc10 con 0x7fa0580ccbf0 2024-09-12T12:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa058160330 con 0x7fa0580ccbf0 2024-09-12T12:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0580cd080 con 0x7fa0580ccbf0 2024-09-12T12:25:54.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.103+0000 7fa055ffb640 1 -- 172.21.15.3:0/3958074293 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa05001d070 con 0x7fa0580ccbf0 2024-09-12T12:25:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.107+0000 7fa055ffb640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0440678c0 0x7fa044069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.107+0000 7fa055ffb640 1 -- 172.21.15.3:0/3958074293 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5615+0+0 (secure 0 0 0) 0x7fa050086410 con 0x7fa0580ccbf0 2024-09-12T12:25:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.107+0000 7fa05cf66640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0440678c0 0x7fa044069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:54.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.107+0000 7fa05cf66640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0440678c0 0x7fa044069d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa0580d1300 tx=0x7fa04c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:54.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.111+0000 7fa055ffb640 1 -- 172.21.15.3:0/3958074293 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa050056960 con 0x7fa0580ccbf0 2024-09-12T12:25:54.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.275+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa05806d630 con 0x7fa0580ccbf0 2024-09-12T12:25:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.947+0000 7fa055ffb640 1 -- 172.21.15.3:0/3958074293 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7fa05005ac60 con 0x7fa0580ccbf0 2024-09-12T12:25:54.952 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T12:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0440678c0 msgr2=0x7fa044069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0440678c0 0x7fa044069d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa0580d1300 tx=0x7fa04c0a6000 comp rx=0 tx=0).stop 2024-09-12T12:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 msgr2=0x7fa05815fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:54.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa05815fdf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa05000cf60 tx=0x7fa05000cf90 comp rx=0 tx=0).stop 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 shutdown_connections 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa0440678c0 0x7fa044069d80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa05815d4e0 0x7fa0580afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa0580d0450 0x7fa058150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 --2- 172.21.15.3:0/3958074293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0580ccbf0 0x7fa05815fdf0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 >> 172.21.15.3:0/3958074293 conn(0x7fa0580628b0 msgr2=0x7fa058155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 shutdown_connections 2024-09-12T12:25:54.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:54.951+0000 7fa05f9cb640 1 -- 172.21.15.3:0/3958074293 wait complete. 2024-09-12T12:25:54.983 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:25:54.986 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-12T12:25:55.013 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T12:25:55.013 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-09-12T12:25:55.013 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:25:55.013 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-09-12T12:25:55.013 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.014 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:25:55.015 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-09-12T12:25:55.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:25:55.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.018 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: bdev async discard = True 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: bdev enable discard = True 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore allocator = bitmap 2024-09-12T12:25:55.019 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 1/20 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 1/20 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 4/10 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.020 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:55.021 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.022 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: mon down mkfs grace = 300 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: mon op complaint time = 120 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on pool no app = False 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-09-12T12:25:55.023 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: client mount timeout = 600 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = "" 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-09-12T12:25:55.024 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d] 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds = 20 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug mds balancer = 20 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug frag = True 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: mds debug scatterstat = True 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: mds op complaint time = 180 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: mds verify scatter = True 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: osd op complaint time = 180 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: rados mon op timeout = 900 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout: rados osd op timeout = 900 2024-09-12T12:25:55.025 INFO:teuthology.orchestra.run.smithi003.stdout:[client.badguy] 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.026 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.027 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:25:55.028 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.3,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 252c40bb-cfe5-4c51-b4ad-5433711beaac 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-09-12T12:25:55.029 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-09-12T12:25:55.030 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-09-12T12:25:55.031 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:25:55.032 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:55.033 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.034 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: mon down mkfs grace = 300 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on pool no app = False 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-09-12T12:25:55.035 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-09-12T12:25:55.036 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-09-12T12:25:55.037 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-09-12T12:25:55.038 INFO:teuthology.orchestra.run.smithi142.stdout:[client.badguy] 2024-09-12T12:25:55.039 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-12T12:25:55.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6ff13eb640 1 Processor -- start 2024-09-12T12:25:55.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6ff13eb640 1 -- start start 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6ff13eb640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fec15c210 0x7f6fec06e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6ff13eb640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6fec0d0cd0 0x7f6fec15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6ff13eb640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fec160600 con 0x7f6fec0adf60 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6ff13eb640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6fec1608e0 con 0x7f6fec0d0cd0 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6ff13eb640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f6fec160bc0 con 0x7f6fec15c210 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feb7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6fec0d0cd0 0x7f6fec15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feb7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6fec0d0cd0 0x7f6fec15feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41160/0 (socket says 172.21.15.3:41160) 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6fea7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f6fec0adf60 0x7f6fec0d05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45096/0 (socket says 172.21.15.3:45096) 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feb7fe640 1 -- 172.21.15.3:0/505774806 learned_addr learned my addr 172.21.15.3:0/505774806 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:55.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feaffd640 1 --2- 172.21.15.3:0/505774806 >> v2:172.21.15.142:3300/0 conn(0x7f6fec15c210 0x7f6fec06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feb7fe640 1 -- 172.21.15.3:0/505774806 >> v2:172.21.15.142:3300/0 conn(0x7f6fec15c210 msgr2=0x7f6fec06e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feb7fe640 1 --2- 172.21.15.3:0/505774806 >> v2:172.21.15.142:3300/0 conn(0x7f6fec15c210 0x7f6fec06e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feb7fe640 1 -- 172.21.15.3:0/505774806 >> v1:172.21.15.3:6789/0 conn(0x7f6fec0adf60 legacy=0x7f6fec0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6feb7fe640 1 -- 172.21.15.3:0/505774806 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec0cec00 con 0x7f6fec0d0cd0 2024-09-12T12:25:55.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.211+0000 7f6fe9ffb640 1 -- 172.21.15.3:0/505774806 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822027615 0 0) 0x7f6fec160600 con 0x7f6fec0adf60 2024-09-12T12:25:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 --2- 172.21.15.3:0/505774806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec15feb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0b5410 tx=0x7f6fe401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d3aa8747750b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6fe9ffb640 1 -- 172.21.15.3:0/505774806 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40a56a0 con 0x7f6fec0d0cd0 2024-09-12T12:25:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6fe9ffb640 1 -- 172.21.15.3:0/505774806 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe40a5840 con 0x7f6fec0d0cd0 2024-09-12T12:25:55.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6fe9ffb640 1 -- 172.21.15.3:0/505774806 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40a5c10 con 0x7f6fec0d0cd0 2024-09-12T12:25:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- 172.21.15.3:0/505774806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec0d0cd0 msgr2=0x7f6fec15feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/505774806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec15feb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0b5410 tx=0x7f6fe401c8d0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- 172.21.15.3:0/505774806 shutdown_connections 2024-09-12T12:25:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/505774806 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec15feb0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/505774806 >> v2:172.21.15.142:3300/0 conn(0x7f6fec15c210 0x7f6fec06e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- 172.21.15.3:0/505774806 >> 172.21.15.3:0/505774806 conn(0x7f6fec067e00 msgr2=0x7f6fec068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- 172.21.15.3:0/505774806 shutdown_connections 2024-09-12T12:25:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- 172.21.15.3:0/505774806 wait complete. 2024-09-12T12:25:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 Processor -- start 2024-09-12T12:25:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- start start 2024-09-12T12:25:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fec0adf60 0x7f6fec154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec070db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec15c210 0x7f6fec14c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec0cfcc0 con 0x7f6fec15c210 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec0cfb40 con 0x7f6fec0d0cd0 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6ff13eb640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6fec0cfe40 con 0x7f6fec0adf60 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec15c210 0x7f6fec14c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec15c210 0x7f6fec14c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41176/0 (socket says 172.21.15.3:41176) 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 -- 172.21.15.3:0/1052600073 learned_addr learned my addr 172.21.15.3:0/1052600073 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feaffd640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fec0adf60 0x7f6fec154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6fea7fc640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 -- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fec0adf60 msgr2=0x7f6fec154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fec0adf60 0x7f6fec154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 -- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fec0d0cd0 msgr2=0x7f6fec070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6feb7fe640 1 -- 172.21.15.3:0/1052600073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec06d480 con 0x7f6fec15c210 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.215+0000 7f6fea7fc640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6feb7fe640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec15c210 0x7f6fec14c010 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6fe401af80 tx=0x7f6fe4019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6fd37fe640 1 -- 172.21.15.3:0/1052600073 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40bc030 con 0x7f6fec15c210 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6fd37fe640 1 -- 172.21.15.3:0/1052600073 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe40b8680 con 0x7f6fec15c210 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6fd37fe640 1 -- 172.21.15.3:0/1052600073 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4009510 con 0x7f6fec15c210 2024-09-12T12:25:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec155e80 con 0x7f6fec15c210 2024-09-12T12:25:55.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec154230 con 0x7f6fec15c210 2024-09-12T12:25:55.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec06e450 con 0x7f6fec15c210 2024-09-12T12:25:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6fd37fe640 1 -- 172.21.15.3:0/1052600073 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6fe40bf070 con 0x7f6fec15c210 2024-09-12T12:25:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6fd37fe640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6fd80678c0 0x7f6fd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6fd37fe640 1 -- 172.21.15.3:0/1052600073 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f6fe4120bb0 con 0x7f6fec15c210 2024-09-12T12:25:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6feaffd640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6fd80678c0 0x7f6fd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.219+0000 7f6feaffd640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6fd80678c0 0x7f6fd8069d80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00b930 tx=0x7f6fdc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:55.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.223+0000 7f6fd37fe640 1 -- 172.21.15.3:0/1052600073 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6fe40165e0 con 0x7f6fec15c210 2024-09-12T12:25:55.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.387+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6fec0ce840 con 0x7f6fec15c210 2024-09-12T12:25:55.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6fd37fe640 1 -- 172.21.15.3:0/1052600073 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f6fe40f1850 con 0x7f6fec15c210 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6fd80678c0 msgr2=0x7f6fd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6fd80678c0 0x7f6fd8069d80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00b930 tx=0x7f6fdc008040 comp rx=0 tx=0).stop 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec15c210 msgr2=0x7f6fec14c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec15c210 0x7f6fec14c010 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6fe401af80 tx=0x7f6fe4019670 comp rx=0 tx=0).stop 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 shutdown_connections 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6fd80678c0 0x7f6fd8069d80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6fec15c210 0x7f6fec14c010 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fec0d0cd0 0x7f6fec070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 --2- 172.21.15.3:0/1052600073 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fec0adf60 0x7f6fec154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 >> 172.21.15.3:0/1052600073 conn(0x7f6fec067e00 msgr2=0x7f6fec0cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:55.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 shutdown_connections 2024-09-12T12:25:55.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.395+0000 7f6ff13eb640 1 -- 172.21.15.3:0/1052600073 wait complete. 2024-09-12T12:25:55.418 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting' 2024-09-12T12:25:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.539+0000 7fb750478640 1 Processor -- start 2024-09-12T12:25:55.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.539+0000 7fb750478640 1 -- start start 2024-09-12T12:25:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb750478640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb748073cb0 0x7fb74815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb750478640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb7480b30e0 0x7fb7480adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb750478640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7480b5000 con 0x7fb7480ae150 2024-09-12T12:25:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb750478640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb748074090 con 0x7fb748073cb0 2024-09-12T12:25:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb750478640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb7480b3530 con 0x7fb7480b30e0 2024-09-12T12:25:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb748073cb0 0x7fb74815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb748073cb0 0x7fb74815c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41182/0 (socket says 172.21.15.3:41182) 2024-09-12T12:25:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 -- 172.21.15.3:0/3580386688 learned_addr learned my addr 172.21.15.3:0/3580386688 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 -- 172.21.15.3:0/3580386688 >> v2:172.21.15.142:3300/0 conn(0x7fb7480b30e0 msgr2=0x7fb7480adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74da13640 1 --2- 172.21.15.3:0/3580386688 >> v2:172.21.15.142:3300/0 conn(0x7fb7480b30e0 0x7fb7480adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 --2- 172.21.15.3:0/3580386688 >> v2:172.21.15.142:3300/0 conn(0x7fb7480b30e0 0x7fb7480adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 -- 172.21.15.3:0/3580386688 >> v1:172.21.15.142:6789/0 conn(0x7fb7480ae150 legacy=0x7fb7480ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:55.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 -- 172.21.15.3:0/3580386688 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7480b37c0 con 0x7fb748073cb0 2024-09-12T12:25:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74e214640 1 --2- 172.21.15.3:0/3580386688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb748073cb0 0x7fb74815c710 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb74806efc0 tx=0x7fb74401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f13ee02ceb917d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74d212640 1 -- 172.21.15.3:0/3580386688 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7440b1070 con 0x7fb748073cb0 2024-09-12T12:25:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74d212640 1 -- 172.21.15.3:0/3580386688 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7440ac630 con 0x7fb748073cb0 2024-09-12T12:25:55.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb74d212640 1 -- 172.21.15.3:0/3580386688 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7440a8050 con 0x7fb748073cb0 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb750478640 1 -- 172.21.15.3:0/3580386688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb748073cb0 msgr2=0x7fb74815c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.543+0000 7fb750478640 1 --2- 172.21.15.3:0/3580386688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb748073cb0 0x7fb74815c710 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fb74806efc0 tx=0x7fb74401c4e0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- 172.21.15.3:0/3580386688 shutdown_connections 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 --2- 172.21.15.3:0/3580386688 >> v2:172.21.15.142:3300/0 conn(0x7fb7480b30e0 0x7fb7480adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 --2- 172.21.15.3:0/3580386688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb748073cb0 0x7fb74815c710 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- 172.21.15.3:0/3580386688 >> 172.21.15.3:0/3580386688 conn(0x7fb748067e00 msgr2=0x7fb748068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- 172.21.15.3:0/3580386688 shutdown_connections 2024-09-12T12:25:55.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- 172.21.15.3:0/3580386688 wait complete. 2024-09-12T12:25:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 Processor -- start 2024-09-12T12:25:55.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- start start 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb748073cb0 0x7fb748157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7480ae150 0x7fb748099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480b30e0 0x7fb748152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb7480af110 con 0x7fb7480ae150 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7480aef90 con 0x7fb748073cb0 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb748168dc0 con 0x7fb7480b30e0 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7480ae150 0x7fb748099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7480ae150 0x7fb748099910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41194/0 (socket says 172.21.15.3:41194) 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 -- 172.21.15.3:0/3559216559 learned_addr learned my addr 172.21.15.3:0/3559216559 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:55.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74ea15640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480b30e0 0x7fb748152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 -- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480b30e0 msgr2=0x7fb748152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74e214640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb748073cb0 0x7fb748157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480b30e0 0x7fb748152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 -- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb748073cb0 msgr2=0x7fb748157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb748073cb0 0x7fb748157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 -- 172.21.15.3:0/3559216559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7481540a0 con 0x7fb7480ae150 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74ea15640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480b30e0 0x7fb748152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb74da13640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7480ae150 0x7fb748099910 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00c7f0 tx=0x7fb73c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb73affd640 1 -- 172.21.15.3:0/3559216559 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c015070 con 0x7fb7480ae150 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb73affd640 1 -- 172.21.15.3:0/3559216559 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73c004240 con 0x7fb7480ae150 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb748156840 con 0x7fb7480ae150 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb73affd640 1 -- 172.21.15.3:0/3559216559 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c003620 con 0x7fb7480ae150 2024-09-12T12:25:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.547+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb748156e00 con 0x7fb7480ae150 2024-09-12T12:25:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.551+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7480adc10 con 0x7fb7480ae150 2024-09-12T12:25:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.551+0000 7fb73affd640 1 -- 172.21.15.3:0/3559216559 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb73c0039d0 con 0x7fb7480ae150 2024-09-12T12:25:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.551+0000 7fb73affd640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb72c0678c0 0x7fb72c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:55.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.551+0000 7fb73affd640 1 -- 172.21.15.3:0/3559216559 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7fb73c085b30 con 0x7fb7480ae150 2024-09-12T12:25:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.551+0000 7fb74e214640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb72c0678c0 0x7fb72c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:55.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.551+0000 7fb74e214640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb72c0678c0 0x7fb72c069d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb7440ae000 tx=0x7fb744008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:55.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.555+0000 7fb73affd640 1 -- 172.21.15.3:0/3559216559 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb73c056860 con 0x7fb7480ae150 2024-09-12T12:25:55.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.783+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]} v 0) -- 0x7fb748159860 con 0x7fb7480ae150 2024-09-12T12:25:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.951+0000 7fb73affd640 1 -- 172.21.15.3:0/3559216559 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_sessionmap.TestSessionMap.test_version_splitting"]}]=0 v312) ==== 145+0+0 (secure 0 0 0) 0x7fb73c056160 con 0x7fb7480ae150 2024-09-12T12:25:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb72c0678c0 msgr2=0x7fb72c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb72c0678c0 0x7fb72c069d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb7440ae000 tx=0x7fb744008040 comp rx=0 tx=0).stop 2024-09-12T12:25:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7480ae150 msgr2=0x7fb748099910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7480ae150 0x7fb748099910 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00c7f0 tx=0x7fb73c00ccb0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 shutdown_connections 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb72c0678c0 0x7fb72c069d80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7480b30e0 0x7fb748152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb7480ae150 0x7fb748099910 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 --2- 172.21.15.3:0/3559216559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb748073cb0 0x7fb748157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 >> 172.21.15.3:0/3559216559 conn(0x7fb748067e00 msgr2=0x7fb74815d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 shutdown_connections 2024-09-12T12:25:55.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:55.955+0000 7fb750478640 1 -- 172.21.15.3:0/3559216559 wait complete. 2024-09-12T12:25:55.978 INFO:tasks.cephfs_test_runner:That when many sessions are updated, they are correctly ... ok 2024-09-12T12:25:55.979 INFO:tasks.cephfs_test_runner: 2024-09-12T12:25:55.979 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-12T12:25:55.979 INFO:tasks.cephfs_test_runner:Ran 6 tests in 339.020s 2024-09-12T12:25:55.979 INFO:tasks.cephfs_test_runner: 2024-09-12T12:25:55.979 INFO:tasks.cephfs_test_runner:OK 2024-09-12T12:25:55.979 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-12T12:25:55.989 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-12T12:25:56.052 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-12T12:25:56.053 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:25:56.053 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:56.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:25:56.060 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T12:25:56.060 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:56.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339568k,nr_inodes=4084892,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16104 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275300k,nr_inodes=818825,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275304k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-12T12:25:56.108 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:56.108 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:56.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:25:56.115 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-09-12T12:25:56.115 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:25:56.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336424k,nr_inodes=4084106,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16123 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-12T12:25:56.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.163 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-09-12T12:25:56.211 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-12T12:25:56.211 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-12T12:25:56.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.211 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-12T12:25:56.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-12T12:25:56.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-09-12T12:25:56.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.512 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T12:25:56.518 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-12T12:25:56.518 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:25:56.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-12T12:25:56.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:25:56.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:25:56.642 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-12T12:25:56.794 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-12T12:25:56.804 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-12T12:25:56.804 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:25:56.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2ac21e1640 1 Processor -- start 2024-09-12T12:25:56.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2ac21e1640 1 -- start start 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2ac21e1640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2abc0d05b0 0x7f2abc0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2ac21e1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2abc0ae250 0x7f2abc0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2ac21e1640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2abc15c630 con 0x7f2abc0b42f0 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2ac21e1640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2abc0d0ed0 con 0x7f2abc0ae250 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2ac21e1640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2abc0ad0c0 con 0x7f2abc0d05b0 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abaffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f2abc0ae250 0x7f2abc0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abbfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2abc0b42f0 0x7f2abc0b4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45100/0 (socket says 172.21.15.3:45100) 2024-09-12T12:25:56.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abbfff640 1 -- 172.21.15.3:0/2019847145 learned_addr learned my addr 172.21.15.3:0/2019847145 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:56.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abb7fe640 1 --2- 172.21.15.3:0/2019847145 >> v2:172.21.15.142:3300/0 conn(0x7f2abc0d05b0 0x7f2abc0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:56.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abaffd640 1 -- 172.21.15.3:0/2019847145 >> v2:172.21.15.142:3300/0 conn(0x7f2abc0d05b0 msgr2=0x7f2abc0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:56.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abaffd640 1 --2- 172.21.15.3:0/2019847145 >> v2:172.21.15.142:3300/0 conn(0x7f2abc0d05b0 0x7f2abc0d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:56.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abaffd640 1 -- 172.21.15.3:0/2019847145 >> v1:172.21.15.3:6789/0 conn(0x7f2abc0b42f0 legacy=0x7f2abc0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:56.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abaffd640 1 -- 172.21.15.3:0/2019847145 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc0ad3a0 con 0x7f2abc0ae250 2024-09-12T12:25:56.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.923+0000 7f2abaffd640 1 --2- 172.21.15.3:0/2019847145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0ae250 0x7f2abc0ae6e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2abc06d8b0 tx=0x7f2ab001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2aed4ba7d6e8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:56.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2aba7fc640 1 -- 172.21.15.3:0/2019847145 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00ab650 con 0x7f2abc0ae250 2024-09-12T12:25:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2aba7fc640 1 -- 172.21.15.3:0/2019847145 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab00b0070 con 0x7f2abc0ae250 2024-09-12T12:25:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2aba7fc640 1 -- 172.21.15.3:0/2019847145 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00aba10 con 0x7f2abc0ae250 2024-09-12T12:25:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- 172.21.15.3:0/2019847145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0ae250 msgr2=0x7f2abc0ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/2019847145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0ae250 0x7f2abc0ae6e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2abc06d8b0 tx=0x7f2ab001c450 comp rx=0 tx=0).stop 2024-09-12T12:25:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- 172.21.15.3:0/2019847145 shutdown_connections 2024-09-12T12:25:56.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/2019847145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0ae250 0x7f2abc0ae6e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:56.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/2019847145 >> v2:172.21.15.142:3300/0 conn(0x7f2abc0d05b0 0x7f2abc0d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:56.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- 172.21.15.3:0/2019847145 >> 172.21.15.3:0/2019847145 conn(0x7f2abc067e00 msgr2=0x7f2abc068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:56.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- 172.21.15.3:0/2019847145 shutdown_connections 2024-09-12T12:25:56.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- 172.21.15.3:0/2019847145 wait complete. 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 Processor -- start 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- start start 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2abc0ae250 0x7f2abc071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0b42f0 0x7f2abc14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2abc0d05b0 0x7f2abc0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc0aeda0 con 0x7f2abc0b42f0 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc0aec20 con 0x7f2abc0ae250 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2abc1559b0 con 0x7f2abc0d05b0 2024-09-12T12:25:56.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0b42f0 0x7f2abc14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0b42f0 0x7f2abc14a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41212/0 (socket says 172.21.15.3:41212) 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 -- 172.21.15.3:0/582467047 learned_addr learned my addr 172.21.15.3:0/582467047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abbfff640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2abc0d05b0 0x7f2abc0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abb7fe640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2abc0ae250 0x7f2abc071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 -- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2abc0d05b0 msgr2=0x7f2abc0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2abc0d05b0 0x7f2abc0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 -- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2abc0ae250 msgr2=0x7f2abc071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2abc0ae250 0x7f2abc071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 -- 172.21.15.3:0/582467047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc0cb9f0 con 0x7f2abc0b42f0 2024-09-12T12:25:56.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abb7fe640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2abc0ae250 0x7f2abc071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2abaffd640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0b42f0 0x7f2abc14a670 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00ad040 tx=0x7f2ab001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2aa3fff640 1 -- 172.21.15.3:0/582467047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00bf610 con 0x7f2abc0b42f0 2024-09-12T12:25:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2aa3fff640 1 -- 172.21.15.3:0/582467047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab00bf7d0 con 0x7f2abc0b42f0 2024-09-12T12:25:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc0cbcd0 con 0x7f2abc0b42f0 2024-09-12T12:25:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2aa3fff640 1 -- 172.21.15.3:0/582467047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00bfbc0 con 0x7f2abc0b42f0 2024-09-12T12:25:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.927+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc0716d0 con 0x7f2abc0b42f0 2024-09-12T12:25:56.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.931+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc15c830 con 0x7f2abc0b42f0 2024-09-12T12:25:56.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.931+0000 7f2aa3fff640 1 -- 172.21.15.3:0/582467047 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ab00bd020 con 0x7f2abc0b42f0 2024-09-12T12:25:56.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.931+0000 7f2aa3fff640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa80678c0 0x7f2aa8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:56.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.931+0000 7f2aa3fff640 1 -- 172.21.15.3:0/582467047 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f2ab0120520 con 0x7f2abc0b42f0 2024-09-12T12:25:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.931+0000 7f2abb7fe640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa80678c0 0x7f2aa8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:56.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.931+0000 7f2abb7fe640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa80678c0 0x7f2aa8069d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006fd0 tx=0x7f2aac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:56.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:56.935+0000 7f2aa3fff640 1 -- 172.21.15.3:0/582467047 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2ab0016070 con 0x7f2abc0b42f0 2024-09-12T12:25:57.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.087+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2abc151240 con 0x7f2aa80678c0 2024-09-12T12:25:57.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2aa3fff640 1 -- 172.21.15.3:0/582467047 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13484 (secure 0 0 0) 0x7f2abc151240 con 0x7f2aa80678c0 2024-09-12T12:25:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa80678c0 msgr2=0x7f2aa8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa80678c0 0x7f2aa8069d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006fd0 tx=0x7f2aac008040 comp rx=0 tx=0).stop 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0b42f0 msgr2=0x7f2abc14a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0b42f0 0x7f2abc14a670 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00ad040 tx=0x7f2ab001ac90 comp rx=0 tx=0).stop 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 shutdown_connections 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2aa80678c0 0x7f2aa8069d80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2abc0d05b0 0x7f2abc0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2abc0b42f0 0x7f2abc14a670 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 --2- 172.21.15.3:0/582467047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2abc0ae250 0x7f2abc071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.091+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 >> 172.21.15.3:0/582467047 conn(0x7f2abc067e00 msgr2=0x7f2abc0ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.095+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 shutdown_connections 2024-09-12T12:25:57.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.095+0000 7f2ac21e1640 1 -- 172.21.15.3:0/582467047 wait complete. 2024-09-12T12:25:57.112 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-12T12:25:55.783523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260264,"kb_used_data":10760,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714456,"statfs":{"total":773094113280,"available":772827602944,"internally_reserved":0,"allocated":11018240,"data_stored":5394744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24934,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-92,"num_read_kb":-87,"num_write":-156,"num_write_kb":-77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002235"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":311,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T12:25:51.718108+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:25:51.718108+0000","last_peered":"2024-09-12T12:25:51.718108+0000","last_clean":"2024-09-12T12:25:51.718108+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:25:51.718108+0000","last_undegraded":"2024-09-12T12:25:51.718108+0000","last_fullsized":"2024-09-12T12:25:51.718108+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509519,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1249280,"data_stored":541462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":1110853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":525126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":520041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542222,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":542094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542221,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":519841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":524564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1810432,"data_stored":1110763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:25:57.113 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:25:57.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.227+0000 7f4e16841640 1 Processor -- start 2024-09-12T12:25:57.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.227+0000 7f4e16841640 1 -- start start 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e16841640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4e100d05b0 0x7f4e100d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e16841640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e100ae250 0x7f4e100ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e16841640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e1015c630 con 0x7f4e100b42f0 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e16841640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4e100d0ed0 con 0x7f4e100ae250 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e16841640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f4e100ad0c0 con 0x7f4e100d05b0 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e100ae250 0x7f4e100ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4e100ae250 0x7f4e100ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41216/0 (socket says 172.21.15.3:41216) 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 -- 172.21.15.3:0/2519687725 learned_addr learned my addr 172.21.15.3:0/2519687725 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0ffff640 1 --2- 172.21.15.3:0/2519687725 >> v2:172.21.15.142:3300/0 conn(0x7f4e100d05b0 0x7f4e100d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 -- 172.21.15.3:0/2519687725 >> v2:172.21.15.142:3300/0 conn(0x7f4e100d05b0 msgr2=0x7f4e100d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 --2- 172.21.15.3:0/2519687725 >> v2:172.21.15.142:3300/0 conn(0x7f4e100d05b0 0x7f4e100d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 -- 172.21.15.3:0/2519687725 >> v1:172.21.15.142:6789/0 conn(0x7f4e100b42f0 legacy=0x7f4e100b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 -- 172.21.15.3:0/2519687725 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e100ad3a0 con 0x7f4e100ae250 2024-09-12T12:25:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0f7fe640 1 --2- 172.21.15.3:0/2519687725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100ae250 0x7f4e100ae6e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4e1006d8b0 tx=0x7f4e0401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68561ce839b0c654 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0effd640 1 -- 172.21.15.3:0/2519687725 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e040aa650 con 0x7f4e100ae250 2024-09-12T12:25:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0effd640 1 -- 172.21.15.3:0/2519687725 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e040af070 con 0x7f4e100ae250 2024-09-12T12:25:57.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e0effd640 1 -- 172.21.15.3:0/2519687725 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e040aaa10 con 0x7f4e100ae250 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e16841640 1 -- 172.21.15.3:0/2519687725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100ae250 msgr2=0x7f4e100ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.231+0000 7f4e16841640 1 --2- 172.21.15.3:0/2519687725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100ae250 0x7f4e100ae6e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f4e1006d8b0 tx=0x7f4e0401c4f0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- 172.21.15.3:0/2519687725 shutdown_connections 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 --2- 172.21.15.3:0/2519687725 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100ae250 0x7f4e100ae6e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 --2- 172.21.15.3:0/2519687725 >> v2:172.21.15.142:3300/0 conn(0x7f4e100d05b0 0x7f4e100d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- 172.21.15.3:0/2519687725 >> 172.21.15.3:0/2519687725 conn(0x7f4e10067e00 msgr2=0x7f4e10068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- 172.21.15.3:0/2519687725 shutdown_connections 2024-09-12T12:25:57.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- 172.21.15.3:0/2519687725 wait complete. 2024-09-12T12:25:57.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 Processor -- start 2024-09-12T12:25:57.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- start start 2024-09-12T12:25:57.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e100ae250 0x7f4e100b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e100b42f0 0x7f4e100cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100d05b0 0x7f4e100b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4e100aeda0 con 0x7f4e100d05b0 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e100aec20 con 0x7f4e100b42f0 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4e101559b0 con 0x7f4e100ae250 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100d05b0 0x7f4e100b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100d05b0 0x7f4e100b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41218/0 (socket says 172.21.15.3:41218) 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 -- 172.21.15.3:0/3701063648 learned_addr learned my addr 172.21.15.3:0/3701063648 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:57.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e0ffff640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e100ae250 0x7f4e100b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e0f7fe640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e100b42f0 0x7f4e100cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 -- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e100ae250 msgr2=0x7f4e100b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e100ae250 0x7f4e100b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 -- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e100b42f0 msgr2=0x7f4e100cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e100b42f0 0x7f4e100cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 -- 172.21.15.3:0/3701063648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e100b1c70 con 0x7f4e100d05b0 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e0ffff640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e100ae250 0x7f4e100b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e0f7fe640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e100b42f0 0x7f4e100cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e14dde640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100d05b0 0x7f4e100b31e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c7a0 tx=0x7f4e0800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e0cff9640 1 -- 172.21.15.3:0/3701063648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08003b80 con 0x7f4e100d05b0 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e0cff9640 1 -- 172.21.15.3:0/3701063648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e08003d20 con 0x7f4e100d05b0 2024-09-12T12:25:57.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e0cff9640 1 -- 172.21.15.3:0/3701063648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0800aa90 con 0x7f4e100d05b0 2024-09-12T12:25:57.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e100b1f50 con 0x7f4e100d05b0 2024-09-12T12:25:57.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.235+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e100cd0f0 con 0x7f4e100d05b0 2024-09-12T12:25:57.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.239+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1015c830 con 0x7f4e100d05b0 2024-09-12T12:25:57.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.239+0000 7f4e0cff9640 1 -- 172.21.15.3:0/3701063648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4e0800ac30 con 0x7f4e100d05b0 2024-09-12T12:25:57.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.239+0000 7f4e0cff9640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4dfc0678c0 0x7f4dfc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.239+0000 7f4e0cff9640 1 -- 172.21.15.3:0/3701063648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f4e08085b10 con 0x7f4e100d05b0 2024-09-12T12:25:57.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.239+0000 7f4e0ffff640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4dfc0678c0 0x7f4dfc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.239+0000 7f4e0ffff640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4dfc0678c0 0x7f4dfc069d80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4e000097a0 tx=0x7f4e00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.243+0000 7f4e0cff9640 1 -- 172.21.15.3:0/3701063648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f4e1015c630 con 0x7f4e100d05b0 2024-09-12T12:25:57.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.403+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e100b4be0 con 0x7f4dfc0678c0 2024-09-12T12:25:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e0cff9640 1 -- 172.21.15.3:0/3701063648 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13484 (secure 0 0 0) 0x7f4e100b4be0 con 0x7f4dfc0678c0 2024-09-12T12:25:57.410 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:57.410 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4dfc0678c0 msgr2=0x7f4dfc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4dfc0678c0 0x7f4dfc069d80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4e000097a0 tx=0x7f4e00008040 comp rx=0 tx=0).stop 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100d05b0 msgr2=0x7f4e100b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100d05b0 0x7f4e100b31e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c7a0 tx=0x7f4e0800cc60 comp rx=0 tx=0).stop 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 shutdown_connections 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4dfc0678c0 0x7f4dfc069d80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4e100d05b0 0x7f4e100b31e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e100b42f0 0x7f4e100cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 --2- 172.21.15.3:0/3701063648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4e100ae250 0x7f4e100b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.407+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 >> 172.21.15.3:0/3701063648 conn(0x7f4e10067e00 msgr2=0x7f4e1014f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:57.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.411+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 shutdown_connections 2024-09-12T12:25:57.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.411+0000 7f4e16841640 1 -- 172.21.15.3:0/3701063648 wait complete. 2024-09-12T12:25:57.431 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-12T12:25:55.783523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260264,"kb_used_data":10760,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714456,"statfs":{"total":773094113280,"available":772827602944,"internally_reserved":0,"allocated":11018240,"data_stored":5394744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24934,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-92,"num_read_kb":-87,"num_write":-156,"num_write_kb":-77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002235"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":311,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T12:25:51.718108+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:25:51.718108+0000","last_peered":"2024-09-12T12:25:51.718108+0000","last_clean":"2024-09-12T12:25:51.718108+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:25:51.718108+0000","last_undegraded":"2024-09-12T12:25:51.718108+0000","last_fullsized":"2024-09-12T12:25:51.718108+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509519,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1249280,"data_stored":541462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":1110853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":525126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":520041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542222,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":542094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542221,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":519841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":524564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1810432,"data_stored":1110763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:25:57.431 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-12T12:25:57.431 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:25:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.555+0000 7f511140e640 1 Processor -- start 2024-09-12T12:25:57.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f511140e640 1 -- start start 2024-09-12T12:25:57.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f511140e640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f510c15c500 0x7f510c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f511140e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f510c0b1e80 0x7f510c0d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f511140e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510c0d00c0 con 0x7f510c15c8e0 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f511140e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f510c0d0cd0 con 0x7f510c0b1e80 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f511140e640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f510c0d03a0 con 0x7f510c15c500 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f510c0b1e80 0x7f510c0d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510a7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f510c15c8e0 0x7f510c0b1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:45116/0 (socket says 172.21.15.3:45116) 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510a7fc640 1 -- 172.21.15.3:0/1384832114 learned_addr learned my addr 172.21.15.3:0/1384832114 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510affd640 1 --2- 172.21.15.3:0/1384832114 >> v2:172.21.15.142:3300/0 conn(0x7f510c15c500 0x7f510c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510b7fe640 1 -- 172.21.15.3:0/1384832114 >> v2:172.21.15.142:3300/0 conn(0x7f510c15c500 msgr2=0x7f510c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510b7fe640 1 --2- 172.21.15.3:0/1384832114 >> v2:172.21.15.142:3300/0 conn(0x7f510c15c500 0x7f510c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510b7fe640 1 -- 172.21.15.3:0/1384832114 >> v1:172.21.15.3:6789/0 conn(0x7f510c15c8e0 legacy=0x7f510c0b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510b7fe640 1 -- 172.21.15.3:0/1384832114 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c0d0680 con 0x7f510c0b1e80 2024-09-12T12:25:57.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f5109ffb640 1 -- 172.21.15.3:0/1384832114 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2227577163 0 0) 0x7f510c0d00c0 con 0x7f510c15c8e0 2024-09-12T12:25:57.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.559+0000 7f510b7fe640 1 --2- 172.21.15.3:0/1384832114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0d1160 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f510c0d2290 tx=0x7f510401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecbbe7bc9603dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f5109ffb640 1 -- 172.21.15.3:0/1384832114 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51040b2070 con 0x7f510c0b1e80 2024-09-12T12:25:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f5109ffb640 1 -- 172.21.15.3:0/1384832114 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51040a58d0 con 0x7f510c0b1e80 2024-09-12T12:25:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f5109ffb640 1 -- 172.21.15.3:0/1384832114 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51040a5ca0 con 0x7f510c0b1e80 2024-09-12T12:25:57.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- 172.21.15.3:0/1384832114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 msgr2=0x7f510c0d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 --2- 172.21.15.3:0/1384832114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0d1160 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f510c0d2290 tx=0x7f510401c900 comp rx=0 tx=0).stop 2024-09-12T12:25:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- 172.21.15.3:0/1384832114 shutdown_connections 2024-09-12T12:25:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 --2- 172.21.15.3:0/1384832114 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0d1160 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 --2- 172.21.15.3:0/1384832114 >> v2:172.21.15.142:3300/0 conn(0x7f510c15c500 0x7f510c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- 172.21.15.3:0/1384832114 >> 172.21.15.3:0/1384832114 conn(0x7f510c0731b0 msgr2=0x7f510c0735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- 172.21.15.3:0/1384832114 shutdown_connections 2024-09-12T12:25:57.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- 172.21.15.3:0/1384832114 wait complete. 2024-09-12T12:25:57.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 Processor -- start 2024-09-12T12:25:57.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- start start 2024-09-12T12:25:57.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f510c15c500 0x7f510c0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f510c15c8e0 0x7f510c0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f510c1693f0 con 0x7f510c0b1e80 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f510c169270 con 0x7f510c15c500 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f510c169570 con 0x7f510c15c8e0 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41226/0 (socket says 172.21.15.3:41226) 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 -- 172.21.15.3:0/2376243222 learned_addr learned my addr 172.21.15.3:0/2376243222 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510a7fc640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f510c15c500 0x7f510c0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510b7fe640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f510c15c8e0 0x7f510c0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 -- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f510c15c8e0 msgr2=0x7f510c0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f510c15c8e0 0x7f510c0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 -- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f510c15c500 msgr2=0x7f510c0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f510c15c500 0x7f510c0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 -- 172.21.15.3:0/2376243222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c09a470 con 0x7f510c0b1e80 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510a7fc640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f510c15c500 0x7f510c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f510affd640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0c4120 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d800 tx=0x7f50fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f50f37fe640 1 -- 172.21.15.3:0/2376243222 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00b6c0 con 0x7f510c0b1e80 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f50f37fe640 1 -- 172.21.15.3:0/2376243222 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc019070 con 0x7f510c0b1e80 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f50f37fe640 1 -- 172.21.15.3:0/2376243222 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00ba60 con 0x7f510c0b1e80 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c06ec60 con 0x7f510c0b1e80 2024-09-12T12:25:57.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.563+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c09a730 con 0x7f510c0b1e80 2024-09-12T12:25:57.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.567+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c0d1160 con 0x7f510c0b1e80 2024-09-12T12:25:57.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.567+0000 7f50f37fe640 1 -- 172.21.15.3:0/2376243222 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f50fc00bc00 con 0x7f510c0b1e80 2024-09-12T12:25:57.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.567+0000 7f50f37fe640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50f80678c0 0x7f50f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.567+0000 7f50f37fe640 1 -- 172.21.15.3:0/2376243222 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f50fc085ed0 con 0x7f510c0b1e80 2024-09-12T12:25:57.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.567+0000 7f510a7fc640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50f80678c0 0x7f50f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.567+0000 7f510a7fc640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50f80678c0 0x7f50f8069d80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f510c0cb4c0 tx=0x7f5100007cb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.571+0000 7f50f37fe640 1 -- 172.21.15.3:0/2376243222 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f50fc056c90 con 0x7f510c0b1e80 2024-09-12T12:25:57.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.731+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f510c0736c0 con 0x7f50f80678c0 2024-09-12T12:25:57.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f50f37fe640 1 -- 172.21.15.3:0/2376243222 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13484 (secure 0 0 0) 0x7f510c0736c0 con 0x7f50f80678c0 2024-09-12T12:25:57.738 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:57.738 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:25:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50f80678c0 msgr2=0x7f50f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50f80678c0 0x7f50f8069d80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f510c0cb4c0 tx=0x7f5100007cb0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 msgr2=0x7f510c0c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0c4120 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d800 tx=0x7f50fc00dcc0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 shutdown_connections 2024-09-12T12:25:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f50f80678c0 0x7f50f8069d80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f510c15c8e0 0x7f510c0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f510c15c500 0x7f510c0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 --2- 172.21.15.3:0/2376243222 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f510c0b1e80 0x7f510c0c4120 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 >> 172.21.15.3:0/2376243222 conn(0x7f510c0731b0 msgr2=0x7f510c159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 shutdown_connections 2024-09-12T12:25:57.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.735+0000 7f511140e640 1 -- 172.21.15.3:0/2376243222 wait complete. 2024-09-12T12:25:57.756 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-12T12:25:55.783523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260264,"kb_used_data":10760,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714456,"statfs":{"total":773094113280,"available":772827602944,"internally_reserved":0,"allocated":11018240,"data_stored":5394744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24934,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-92,"num_read_kb":-87,"num_write":-156,"num_write_kb":-77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002235"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":311,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-12T12:25:51.718108+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:25:51.718108+0000","last_peered":"2024-09-12T12:25:51.718108+0000","last_clean":"2024-09-12T12:25:51.718108+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:25:51.718108+0000","last_undegraded":"2024-09-12T12:25:51.718108+0000","last_fullsized":"2024-09-12T12:25:51.718108+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509519,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1249280,"data_stored":541462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":1110853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1228800,"data_stored":525126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":520041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542222,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":542094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542221,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":519841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":524564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542221,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32956,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338884,"statfs":{"total":96636764160,"available":96603017216,"internally_reserved":0,"allocated":1810432,"data_stored":1110763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:25:57.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:25:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.879+0000 7f6f23944640 1 Processor -- start 2024-09-12T12:25:57.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f23944640 1 -- start start 2024-09-12T12:25:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f23944640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c15c710 0x7f6f1c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f23944640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f1c0ae880 0x7f6f1c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f23944640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1c0b5000 con 0x7f6f1c073cb0 2024-09-12T12:25:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f23944640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1c074090 con 0x7f6f1c0ae880 2024-09-12T12:25:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f23944640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c0b37c0 con 0x7f6f1c15c710 2024-09-12T12:25:57.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f1c0ae880 0x7f6f1c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f6f1c0ae880 0x7f6f1c0aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41238/0 (socket says 172.21.15.3:41238) 2024-09-12T12:25:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 -- 172.21.15.3:0/942311163 learned_addr learned my addr 172.21.15.3:0/942311163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f20edf640 1 --2- 172.21.15.3:0/942311163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c15c710 0x7f6f1c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 -- 172.21.15.3:0/942311163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c15c710 msgr2=0x7f6f1c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 --2- 172.21.15.3:0/942311163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c15c710 0x7f6f1c0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 -- 172.21.15.3:0/942311163 >> v1:172.21.15.142:6789/0 conn(0x7f6f1c073cb0 legacy=0x7f6f1c0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:57.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 -- 172.21.15.3:0/942311163 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c0b3aa0 con 0x7f6f1c0ae880 2024-09-12T12:25:57.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.883+0000 7f6f21ee1640 1 --2- 172.21.15.3:0/942311163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c0ae880 0x7f6f1c0aecd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009e50 tx=0x7f6f1401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0e326c9de9708d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f13fff640 1 -- 172.21.15.3:0/942311163 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f140a5670 con 0x7f6f1c0ae880 2024-09-12T12:25:57.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f13fff640 1 -- 172.21.15.3:0/942311163 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f140a5810 con 0x7f6f1c0ae880 2024-09-12T12:25:57.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f13fff640 1 -- 172.21.15.3:0/942311163 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f140a5be0 con 0x7f6f1c0ae880 2024-09-12T12:25:57.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- 172.21.15.3:0/942311163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c0ae880 msgr2=0x7f6f1c0aecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 --2- 172.21.15.3:0/942311163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c0ae880 0x7f6f1c0aecd0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009e50 tx=0x7f6f1401c9f0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- 172.21.15.3:0/942311163 shutdown_connections 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 --2- 172.21.15.3:0/942311163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c0ae880 0x7f6f1c0aecd0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 --2- 172.21.15.3:0/942311163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c15c710 0x7f6f1c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- 172.21.15.3:0/942311163 >> 172.21.15.3:0/942311163 conn(0x7f6f1c067e00 msgr2=0x7f6f1c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- 172.21.15.3:0/942311163 shutdown_connections 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- 172.21.15.3:0/942311163 wait complete. 2024-09-12T12:25:57.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 Processor -- start 2024-09-12T12:25:57.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- start start 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c073cb0 0x7f6f1c155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0ae880 0x7f6f1c156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c15c710 0x7f6f1c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c168f40 con 0x7f6f1c073cb0 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c168dc0 con 0x7f6f1c15c710 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c1690c0 con 0x7f6f1c0ae880 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c073cb0 0x7f6f1c155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c073cb0 0x7f6f1c155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41248/0 (socket says 172.21.15.3:41248) 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 -- 172.21.15.3:0/270891262 learned_addr learned my addr 172.21.15.3:0/270891262 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f20edf640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0ae880 0x7f6f1c156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f21ee1640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c15c710 0x7f6f1c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 -- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0ae880 msgr2=0x7f6f1c156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0ae880 0x7f6f1c156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 -- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c15c710 msgr2=0x7f6f1c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c15c710 0x7f6f1c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 -- 172.21.15.3:0/270891262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c0aff70 con 0x7f6f1c073cb0 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f20edf640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0ae880 0x7f6f1c156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f21ee1640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c15c710 0x7f6f1c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f216e0640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c073cb0 0x7f6f1c155920 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c450 tx=0x7f6f1800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f11ffb640 1 -- 172.21.15.3:0/270891262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18012830 con 0x7f6f1c073cb0 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f11ffb640 1 -- 172.21.15.3:0/270891262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18017070 con 0x7f6f1c073cb0 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f11ffb640 1 -- 172.21.15.3:0/270891262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18012c30 con 0x7f6f1c073cb0 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.887+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c0b0250 con 0x7f6f1c073cb0 2024-09-12T12:25:57.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.891+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c0b0af0 con 0x7f6f1c073cb0 2024-09-12T12:25:57.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.891+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c0b0420 con 0x7f6f1c073cb0 2024-09-12T12:25:57.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.891+0000 7f6f11ffb640 1 -- 172.21.15.3:0/270891262 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6f1801c460 con 0x7f6f1c073cb0 2024-09-12T12:25:57.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.891+0000 7f6f11ffb640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6f000678c0 0x7f6f00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:57.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.891+0000 7f6f11ffb640 1 -- 172.21.15.3:0/270891262 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f6f18085880 con 0x7f6f1c073cb0 2024-09-12T12:25:57.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.891+0000 7f6f20edf640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6f000678c0 0x7f6f00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:57.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.891+0000 7f6f20edf640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6f000678c0 0x7f6f00069d80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0cc340 tx=0x7f6f0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:57.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:57.895+0000 7f6f11ffb640 1 -- 172.21.15.3:0/270891262 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f6f180565b0 con 0x7f6f1c073cb0 2024-09-12T12:25:58.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.059+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f1c0aecd0 con 0x7f6f1c073cb0 2024-09-12T12:25:58.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f11ffb640 1 -- 172.21.15.3:0/270891262 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+13356 (secure 0 0 0) 0x7f6f18055eb0 con 0x7f6f1c073cb0 2024-09-12T12:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:25:58.065 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":100,"fsid":"252c40bb-cfe5-4c51-b4ad-5433711beaac","created":"2024-09-12T12:19:24.973409+0000","modified":"2024-09-12T12:25:54.944479+0000","last_up_change":"2024-09-12T12:19:38.173260+0000","last_in_change":"2024-09-12T12:19:27.455538+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:19:37.735209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8fd6b02-9ed1-4355-9614-bb1195121930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6817","nonce":3370158904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6819","nonce":3370158904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6823","nonce":3370158904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":3370158904},{"type":"v1","addr":"172.21.15.3:6821","nonce":3370158904}]},"public_addr":"172.21.15.3:6817/3370158904","cluster_addr":"172.21.15.3:6819/3370158904","heartbeat_back_addr":"172.21.15.3:6823/3370158904","heartbeat_front_addr":"172.21.15.3:6821/3370158904","state":["exists","up"]},{"osd":1,"uuid":"3cf517e3-ffba-4602-a705-293666e71f8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6801","nonce":4123050499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6803","nonce":4123050499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6807","nonce":4123050499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":4123050499},{"type":"v1","addr":"172.21.15.3:6805","nonce":4123050499}]},"public_addr":"172.21.15.3:6801/4123050499","cluster_addr":"172.21.15.3:6803/4123050499","heartbeat_back_addr":"172.21.15.3:6807/4123050499","heartbeat_front_addr":"172.21.15.3:6805/4123050499","state":["exists","up"]},{"osd":2,"uuid":"1619cc70-8f41-4c98-bd54-f14a875ce4ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6825","nonce":3021596546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6827","nonce":3021596546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6831","nonce":3021596546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3021596546},{"type":"v1","addr":"172.21.15.3:6829","nonce":3021596546}]},"public_addr":"172.21.15.3:6825/3021596546","cluster_addr":"172.21.15.3:6827/3021596546","heartbeat_back_addr":"172.21.15.3:6831/3021596546","heartbeat_front_addr":"172.21.15.3:6829/3021596546","state":["exists","up"]},{"osd":3,"uuid":"837afb49-bced-4356-8bd9-d2641faf463c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6809","nonce":63313814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6811","nonce":63313814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6815","nonce":63313814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":63313814},{"type":"v1","addr":"172.21.15.3:6813","nonce":63313814}]},"public_addr":"172.21.15.3:6809/63313814","cluster_addr":"172.21.15.3:6811/63313814","heartbeat_back_addr":"172.21.15.3:6815/63313814","heartbeat_front_addr":"172.21.15.3:6813/63313814","state":["exists","up"]},{"osd":4,"uuid":"0eb6f325-b5a5-4bcf-bffb-2cbbadcea3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6801","nonce":4142884196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6803","nonce":4142884196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6807","nonce":4142884196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":4142884196},{"type":"v1","addr":"172.21.15.142:6805","nonce":4142884196}]},"public_addr":"172.21.15.142:6801/4142884196","cluster_addr":"172.21.15.142:6803/4142884196","heartbeat_back_addr":"172.21.15.142:6807/4142884196","heartbeat_front_addr":"172.21.15.142:6805/4142884196","state":["exists","up"]},{"osd":5,"uuid":"b8016c23-0f0f-4cdc-a4db-9f33f8f3e1f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6817","nonce":4110985109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6819","nonce":4110985109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6823","nonce":4110985109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":4110985109},{"type":"v1","addr":"172.21.15.142:6821","nonce":4110985109}]},"public_addr":"172.21.15.142:6817/4110985109","cluster_addr":"172.21.15.142:6819/4110985109","heartbeat_back_addr":"172.21.15.142:6823/4110985109","heartbeat_front_addr":"172.21.15.142:6821/4110985109","state":["exists","up"]},{"osd":6,"uuid":"db6c35a4-95c7-4815-98ec-b9c3c6e99cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6809","nonce":3052505942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6811","nonce":3052505942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6815","nonce":3052505942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":3052505942},{"type":"v1","addr":"172.21.15.142:6813","nonce":3052505942}]},"public_addr":"172.21.15.142:6809/3052505942","cluster_addr":"172.21.15.142:6811/3052505942","heartbeat_back_addr":"172.21.15.142:6815/3052505942","heartbeat_front_addr":"172.21.15.142:6813/3052505942","state":["exists","up"]},{"osd":7,"uuid":"7f348809-04cd-4d2b-abeb-06c3c1561d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6825","nonce":177123471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6827","nonce":177123471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6831","nonce":177123471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":177123471},{"type":"v1","addr":"172.21.15.142:6829","nonce":177123471}]},"public_addr":"172.21.15.142:6825/177123471","cluster_addr":"172.21.15.142:6827/177123471","heartbeat_back_addr":"172.21.15.142:6831/177123471","heartbeat_front_addr":"172.21.15.142:6829/177123471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.827961+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.596207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.403752+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.769200+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.401089+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.344231+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.331163+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:19:34.584508+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6835/1654386752":"2024-09-13T12:25:51.583748+0000","172.21.15.3:6834/1654386752":"2024-09-13T12:25:51.583748+0000","172.21.15.3:6832/1847705632":"2024-09-13T12:25:51.583719+0000","172.21.15.142:6835/95103479":"2024-09-13T12:25:15.057602+0000","172.21.15.3:6833/1847705632":"2024-09-13T12:25:51.583719+0000","172.21.15.142:6834/95103479":"2024-09-13T12:25:15.057602+0000","172.21.15.3:6835/2492687290":"2024-09-13T12:25:14.060842+0000","172.21.15.3:6833/2492687290":"2024-09-13T12:25:14.060842+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:25:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6f000678c0 msgr2=0x7f6f00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:58.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6f000678c0 0x7f6f00069d80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0cc340 tx=0x7f6f0c008040 comp rx=0 tx=0).stop 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c073cb0 msgr2=0x7f6f1c155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c073cb0 0x7f6f1c155920 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800c450 tx=0x7f6f1800c910 comp rx=0 tx=0).stop 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 shutdown_connections 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f6f000678c0 0x7f6f00069d80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c15c710 0x7f6f1c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0ae880 0x7f6f1c156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 --2- 172.21.15.3:0/270891262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f1c073cb0 0x7f6f1c155920 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 >> 172.21.15.3:0/270891262 conn(0x7f6f1c067e00 msgr2=0x7f6f1c152720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 shutdown_connections 2024-09-12T12:25:58.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:58.063+0000 7f6f23944640 1 -- 172.21.15.3:0/270891262 wait complete. 2024-09-12T12:25:59.086 INFO:tasks.ceph:Scrubbing osd.0 2024-09-12T12:25:59.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:25:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c6ab6640 1 Processor -- start 2024-09-12T12:25:59.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c6ab6640 1 -- start start 2024-09-12T12:25:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c6ab6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71c01523f0 0x7f71c00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c6ab6640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f71c00d08e0 0x7f71c00ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c6ab6640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c00ad160 con 0x7f71c015c910 2024-09-12T12:25:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c6ab6640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f71c00aef10 con 0x7f71c01523f0 2024-09-12T12:25:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c6ab6640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f71c00d0d30 con 0x7f71c00d08e0 2024-09-12T12:25:59.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c4852640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71c01523f0 0x7f71c00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c4852640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f71c01523f0 0x7f71c00d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60936/0 (socket says 172.21.15.3:60936) 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.223+0000 7f71c4852640 1 -- 172.21.15.3:0/1469031470 learned_addr learned my addr 172.21.15.3:0/1469031470 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c5053640 1 --2- 172.21.15.3:0/1469031470 >> v2:172.21.15.142:3300/0 conn(0x7f71c00d08e0 0x7f71c00ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c4852640 1 -- 172.21.15.3:0/1469031470 >> v2:172.21.15.142:3300/0 conn(0x7f71c00d08e0 msgr2=0x7f71c00ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c4852640 1 --2- 172.21.15.3:0/1469031470 >> v2:172.21.15.142:3300/0 conn(0x7f71c00d08e0 0x7f71c00ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c4852640 1 -- 172.21.15.3:0/1469031470 >> v1:172.21.15.142:6789/0 conn(0x7f71c015c910 legacy=0x7f71c00ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c4852640 1 -- 172.21.15.3:0/1469031470 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c00ad440 con 0x7f71c01523f0 2024-09-12T12:25:59.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c4852640 1 --2- 172.21.15.3:0/1469031470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c01523f0 0x7f71c00d03a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f71c0159320 tx=0x7f71b001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7fce516f86ab13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71bf7fe640 1 -- 172.21.15.3:0/1469031470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00aa610 con 0x7f71c01523f0 2024-09-12T12:25:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71bf7fe640 1 -- 172.21.15.3:0/1469031470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b00af070 con 0x7f71c01523f0 2024-09-12T12:25:59.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71bf7fe640 1 -- 172.21.15.3:0/1469031470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00aa9d0 con 0x7f71c01523f0 2024-09-12T12:25:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1469031470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c01523f0 msgr2=0x7f71c00d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 --2- 172.21.15.3:0/1469031470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c01523f0 0x7f71c00d03a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f71c0159320 tx=0x7f71b001c4b0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1469031470 shutdown_connections 2024-09-12T12:25:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 --2- 172.21.15.3:0/1469031470 >> v2:172.21.15.142:3300/0 conn(0x7f71c00d08e0 0x7f71c00ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 --2- 172.21.15.3:0/1469031470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c01523f0 0x7f71c00d03a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1469031470 >> 172.21.15.3:0/1469031470 conn(0x7f71c0067bd0 msgr2=0x7f71c0067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1469031470 shutdown_connections 2024-09-12T12:25:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.227+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1469031470 wait complete. 2024-09-12T12:25:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 Processor -- start 2024-09-12T12:25:59.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 -- start start 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c00d08e0 0x7f71c00713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71c01523f0 0x7f71c014a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c015c910 0x7f71c00b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f71c015e750 con 0x7f71c015c910 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71c015e5d0 con 0x7f71c00d08e0 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f71c015e8d0 con 0x7f71c01523f0 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c015c910 0x7f71c00b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c015c910 0x7f71c00b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60952/0 (socket says 172.21.15.3:60952) 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 -- 172.21.15.3:0/1439590055 learned_addr learned my addr 172.21.15.3:0/1439590055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:59.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 -- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71c01523f0 msgr2=0x7f71c014a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71bffff640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71c01523f0 0x7f71c014a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c4852640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c00d08e0 0x7f71c00713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71c01523f0 0x7f71c014a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 -- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c00d08e0 msgr2=0x7f71c00713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c00d08e0 0x7f71c00713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 -- 172.21.15.3:0/1439590055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c0071930 con 0x7f71c015c910 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71bffff640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f71c01523f0 0x7f71c014a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c4852640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71c00d08e0 0x7f71c00713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c5053640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c015c910 0x7f71c00b03a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d830 tx=0x7f71b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800b6c0 con 0x7f71c015c910 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b8019070 con 0x7f71c015c910 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c0071c00 con 0x7f71c015c910 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800baa0 con 0x7f71c015c910 2024-09-12T12:25:59.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c00721c0 con 0x7f71c015c910 2024-09-12T12:25:59.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.231+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71c00d03a0 con 0x7f71c015c910 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f71b801e460 con 0x7f71c015c910 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bd7fa640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71a00678c0 0x7f71a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f71b80861b0 con 0x7f71c015c910 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bd7fa640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7f71a006d960 0x7f71a006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71a0070430 con 0x7f71a006d960 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f71b808d050 con 0x7f71c015c910 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71c4852640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71a00678c0 0x7f71a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bffff640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7f71a006d960 0x7f71a006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bffff640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7f71a006d960 0x7f71a006fd80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71c4852640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71a00678c0 0x7f71a0069d80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f71b00ae000 tx=0x7f71b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.235+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== osd.0 v2:172.21.15.3:6816/3370158904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71a0070430 con 0x7f71a006d960 2024-09-12T12:25:59.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.247+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 --> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f71c00cc380 con 0x7f71a006d960 2024-09-12T12:25:59.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.247+0000 7f71bd7fa640 1 -- 172.21.15.3:0/1439590055 <== osd.0 v2:172.21.15.3:6816/3370158904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f71c00cc380 con 0x7f71a006d960 2024-09-12T12:25:59.252 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:59.252 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:25:59.252 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.247+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7f71a006d960 msgr2=0x7f71a006fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.247+0000 7f71c6ab6640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:6816/3370158904,v1:172.21.15.3:6817/3370158904] conn(0x7f71a006d960 0x7f71a006fd80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71a00678c0 msgr2=0x7f71a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f71a00678c0 0x7f71a0069d80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f71b00ae000 tx=0x7f71b0008040 comp rx=0 tx=0).stop 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c015c910 msgr2=0x7f71c00b03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 --2- 172.21.15.3:0/1439590055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c015c910 0x7f71c00b03a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d830 tx=0x7f71b800dcf0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c5053640 1 -- 172.21.15.3:0/1439590055 reap_dead start 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 shutdown_connections 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 >> 172.21.15.3:0/1439590055 conn(0x7f71c0067bd0 msgr2=0x7f71c014b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 shutdown_connections 2024-09-12T12:25:59.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.251+0000 7f71c6ab6640 1 -- 172.21.15.3:0/1439590055 wait complete. 2024-09-12T12:25:59.270 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-12T12:25:59.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.383+0000 7f394cba3640 1 Processor -- start 2024-09-12T12:25:59.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.383+0000 7f394cba3640 1 -- start start 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.383+0000 7f394cba3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f394815c710 0x7f39480b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.383+0000 7f394cba3640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39480b5000 con 0x7f39480ae880 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.383+0000 7f394cba3640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39480b37c0 con 0x7f3948073cb0 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.383+0000 7f394cba3640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3948074090 con 0x7f394815c710 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f393ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f394815c710 0x7f39480b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f393ffff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f394815c710 0x7f39480b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60956/0 (socket says 172.21.15.3:60956) 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f393ffff640 1 -- 172.21.15.3:0/1035911145 learned_addr learned my addr 172.21.15.3:0/1035911145 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f3946d9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f39480ae880 0x7f39480d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38024/0 (socket says 172.21.15.3:38024) 2024-09-12T12:25:59.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f393ffff640 1 -- 172.21.15.3:0/1035911145 >> v1:172.21.15.142:6789/0 conn(0x7f3948073cb0 legacy=0x7f39480adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f393ffff640 1 -- 172.21.15.3:0/1035911145 >> v1:172.21.15.3:6789/0 conn(0x7f39480ae880 legacy=0x7f39480d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f393ffff640 1 -- 172.21.15.3:0/1035911145 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39480b3aa0 con 0x7f394815c710 2024-09-12T12:25:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f393ffff640 1 --2- 172.21.15.3:0/1035911145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b30e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3930009870 tx=0x7f393001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca68411b5b77448f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f3945d9b640 1 -- 172.21.15.3:0/1035911145 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300ab610 con 0x7f394815c710 2024-09-12T12:25:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f3945d9b640 1 -- 172.21.15.3:0/1035911145 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39300b0070 con 0x7f394815c710 2024-09-12T12:25:59.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f3945d9b640 1 -- 172.21.15.3:0/1035911145 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300ab9d0 con 0x7f394815c710 2024-09-12T12:25:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f394cba3640 1 -- 172.21.15.3:0/1035911145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 msgr2=0x7f39480b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f394cba3640 1 --2- 172.21.15.3:0/1035911145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b30e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3930009870 tx=0x7f393001c520 comp rx=0 tx=0).stop 2024-09-12T12:25:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f394cba3640 1 -- 172.21.15.3:0/1035911145 shutdown_connections 2024-09-12T12:25:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f394cba3640 1 --2- 172.21.15.3:0/1035911145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b30e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f394cba3640 1 -- 172.21.15.3:0/1035911145 >> 172.21.15.3:0/1035911145 conn(0x7f3948067e00 msgr2=0x7f3948068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f394cba3640 1 -- 172.21.15.3:0/1035911145 shutdown_connections 2024-09-12T12:25:59.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.387+0000 7f394cba3640 1 -- 172.21.15.3:0/1035911145 wait complete. 2024-09-12T12:25:59.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 Processor -- start 2024-09-12T12:25:59.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 -- start start 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3948073cb0 0x7f39480cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39480ae880 0x7f394815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f39480d0e40 con 0x7f394815c710 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f39480d0cc0 con 0x7f39480ae880 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f39480aec90 con 0x7f3948073cb0 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60966/0 (socket says 172.21.15.3:60966) 2024-09-12T12:25:59.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 -- 172.21.15.3:0/4138410504 learned_addr learned my addr 172.21.15.3:0/4138410504 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f393ffff640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39480ae880 0x7f394815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394659c640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3948073cb0 0x7f39480cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 -- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3948073cb0 msgr2=0x7f39480cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3948073cb0 0x7f39480cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 -- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39480ae880 msgr2=0x7f394815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39480ae880 0x7f394815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 -- 172.21.15.3:0/4138410504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394814baf0 con 0x7f394815c710 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f393ffff640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39480ae880 0x7f394815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f3946d9d640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b21a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f394000c7a0 tx=0x7f394000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f393effd640 1 -- 172.21.15.3:0/4138410504 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940017070 con 0x7f394815c710 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f393effd640 1 -- 172.21.15.3:0/4138410504 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394000a640 con 0x7f394815c710 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394814c970 con 0x7f394815c710 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f393effd640 1 -- 172.21.15.3:0/4138410504 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394000a9f0 con 0x7f394815c710 2024-09-12T12:25:59.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.391+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394814cf30 con 0x7f394815c710 2024-09-12T12:25:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.395+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39480af020 con 0x7f394815c710 2024-09-12T12:25:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.395+0000 7f393effd640 1 -- 172.21.15.3:0/4138410504 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f394001a070 con 0x7f394815c710 2024-09-12T12:25:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.395+0000 7f393effd640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39280678c0 0x7f3928069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.395+0000 7f393effd640 1 -- 172.21.15.3:0/4138410504 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f3940084fe0 con 0x7f394815c710 2024-09-12T12:25:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.395+0000 7f394659c640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39280678c0 0x7f3928069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.395+0000 7f394659c640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39280678c0 0x7f3928069d80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f39480ad5b0 tx=0x7f3938009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.399+0000 7f393effd640 1 -- 172.21.15.3:0/4138410504 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f39400556f0 con 0x7f394815c710 2024-09-12T12:25:59.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.567+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f394815c1b0 con 0x7f39280678c0 2024-09-12T12:25:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.567+0000 7f393effd640 1 -- 172.21.15.3:0/4138410504 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f394815c1b0 con 0x7f39280678c0 2024-09-12T12:25:59.570 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 0 to deep-scrub 2024-09-12T12:25:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39280678c0 msgr2=0x7f3928069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39280678c0 0x7f3928069d80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f39480ad5b0 tx=0x7f3938009040 comp rx=0 tx=0).stop 2024-09-12T12:25:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 msgr2=0x7f39480b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b21a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f394000c7a0 tx=0x7f394000cc60 comp rx=0 tx=0).stop 2024-09-12T12:25:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 shutdown_connections 2024-09-12T12:25:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f39280678c0 0x7f3928069d80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f394815c710 0x7f39480b21a0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39480ae880 0x7f394815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 --2- 172.21.15.3:0/4138410504 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3948073cb0 0x7f39480cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 >> 172.21.15.3:0/4138410504 conn(0x7f3948067e00 msgr2=0x7f3948069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 shutdown_connections 2024-09-12T12:25:59.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.571+0000 7f394cba3640 1 -- 172.21.15.3:0/4138410504 wait complete. 2024-09-12T12:25:59.590 INFO:tasks.ceph:Scrubbing osd.1 2024-09-12T12:25:59.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:25:59.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d0dcb8640 1 Processor -- start 2024-09-12T12:25:59.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d0dcb8640 1 -- start start 2024-09-12T12:25:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d0dcb8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 0x7f1d0815b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d0dcb8640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d080b4ad0 con 0x7f1d080b4550 2024-09-12T12:25:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d0dcb8640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d080b4db0 con 0x7f1d0815aeb0 2024-09-12T12:25:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d0dcb8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d080b5090 con 0x7f1d08073290 2024-09-12T12:25:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d07fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1d080b4550 0x7f1d0815b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38038/0 (socket says 172.21.15.3:38038) 2024-09-12T12:25:59.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d07fff640 1 -- 172.21.15.3:0/753062470 learned_addr learned my addr 172.21.15.3:0/753062470 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:25:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d06ffd640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 0x7f1d0815b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908875185 0 0) 0x7f1d080b4ad0 con 0x7f1d080b4550 2024-09-12T12:25:59.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cf0003530 con 0x7f1d080b4550 2024-09-12T12:25:59.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4287551235 0 0) 0x7f1cf0003530 con 0x7f1d080b4550 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 msgr2=0x7f1d0815b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 0x7f1d0815b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 >> v1:172.21.15.142:6789/0 conn(0x7f1d0815aeb0 legacy=0x7f1d0806df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d080b5370 con 0x7f1d080b4550 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f1cf8001680 con 0x7f1d080b4550 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 >> v1:172.21.15.3:6789/0 conn(0x7f1d080b4550 legacy=0x7f1d0815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf00039f0 0x7f1cf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf0004810 0x7f1cf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 0x7f1cf00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d080b5090 con 0x7f1cf00039f0 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf000ba80 con 0x7f1cf0004810 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf000bc50 con 0x7f1d08073290 2024-09-12T12:25:59.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d077fe640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf00039f0 0x7f1cf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d06ffd640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf0004810 0x7f1cf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d07fff640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 0x7f1cf00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d077fe640 1 -- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 msgr2=0x7f1cf00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d077fe640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d08073290 0x7f1cf00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d077fe640 1 -- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf0004810 msgr2=0x7f1cf0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d077fe640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf0004810 0x7f1cf0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.719+0000 7f1d077fe640 1 -- 172.21.15.3:0/753062470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf0003530 con 0x7f1cf00039f0 2024-09-12T12:25:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d077fe640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf00039f0 0x7f1cf0003dd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1d00007fd0 tx=0x7f1d0001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cb1b77e5c7b517 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d000b0070 con 0x7f1cf00039f0 2024-09-12T12:25:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d000ab600 con 0x7f1cf00039f0 2024-09-12T12:25:59.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d067fc640 1 -- 172.21.15.3:0/753062470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00001460 con 0x7f1cf00039f0 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/753062470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf00039f0 msgr2=0x7f1cf0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 --2- 172.21.15.3:0/753062470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1cf00039f0 0x7f1cf0003dd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1d00007fd0 tx=0x7f1d0001c4c0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d07fff640 1 -- 172.21.15.3:0/753062470 reap_dead start 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/753062470 shutdown_connections 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/753062470 >> 172.21.15.3:0/753062470 conn(0x7f1d080b2840 msgr2=0x7f1d080b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/753062470 shutdown_connections 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/753062470 wait complete. 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 Processor -- start 2024-09-12T12:25:59.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.723+0000 7f1d0dcb8640 1 -- start start 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d081536b0 0x7f1d081f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080ce230 0x7f1d080d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d081fe4d0 0x7f1d081fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081658d0 con 0x7f1d080ce230 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1d08165750 con 0x7f1d081536b0 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f1d08165a50 con 0x7f1d081fe4d0 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080ce230 0x7f1d080d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080ce230 0x7f1d080d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60980/0 (socket says 172.21.15.3:60980) 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 -- 172.21.15.3:0/3123447015 learned_addr learned my addr 172.21.15.3:0/3123447015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d07fff640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d081fe4d0 0x7f1d081fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d077fe640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d081536b0 0x7f1d081f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 -- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d081fe4d0 msgr2=0x7f1d081fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d081fe4d0 0x7f1d081fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 -- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d081536b0 msgr2=0x7f1d081f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d081536b0 0x7f1d081f8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 -- 172.21.15.3:0/3123447015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d081625c0 con 0x7f1d080ce230 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d06ffd640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080ce230 0x7f1d080d01b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00aeb0 tx=0x7f1cfc010580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc019020 con 0x7f1d080ce230 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cfc015450 con 0x7f1d080ce230 2024-09-12T12:25:59.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d081628a0 con 0x7f1d080ce230 2024-09-12T12:25:59.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc015820 con 0x7f1d080ce230 2024-09-12T12:25:59.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d08153a90 con 0x7f1d080ce230 2024-09-12T12:25:59.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d0815b7d0 con 0x7f1d080ce230 2024-09-12T12:25:59.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1cfc01c020 con 0x7f1d080ce230 2024-09-12T12:25:59.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cf40678c0 0x7f1cf4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f1cfc086f60 con 0x7f1d080ce230 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f1cf406d960 0x7f1cf406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 --> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cfc006c00 con 0x7f1cf406d960 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f1cfc087310 con 0x7f1d080ce230 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d077fe640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cf40678c0 0x7f1cf4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.727+0000 7f1d07fff640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f1cf406d960 0x7f1cf406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.731+0000 7f1d07fff640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f1cf406d960 0x7f1cf406fd80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.731+0000 7f1d077fe640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cf40678c0 0x7f1cf4069d80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ad000 tx=0x7f1d00005040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.731+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== osd.1 v2:172.21.15.3:6800/4123050499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cfc006c00 con 0x7f1cf406d960 2024-09-12T12:25:59.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 --> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1d080ce8c0 con 0x7f1cf406d960 2024-09-12T12:25:59.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1ceffff640 1 -- 172.21.15.3:0/3123447015 <== osd.1 v2:172.21.15.3:6800/4123050499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1d080ce8c0 con 0x7f1cf406d960 2024-09-12T12:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:25:59.747 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f1cf406d960 msgr2=0x7f1cf406fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:6800/4123050499,v1:172.21.15.3:6801/4123050499] conn(0x7f1cf406d960 0x7f1cf406fd80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cf40678c0 msgr2=0x7f1cf4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f1cf40678c0 0x7f1cf4069d80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1d000ad000 tx=0x7f1d00005040 comp rx=0 tx=0).stop 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080ce230 msgr2=0x7f1d080d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 --2- 172.21.15.3:0/3123447015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d080ce230 0x7f1d080d01b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00aeb0 tx=0x7f1cfc010580 comp rx=0 tx=0).stop 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d07fff640 1 -- 172.21.15.3:0/3123447015 reap_dead start 2024-09-12T12:25:59.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 shutdown_connections 2024-09-12T12:25:59.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 >> 172.21.15.3:0/3123447015 conn(0x7f1d080b2840 msgr2=0x7f1d081544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:59.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 shutdown_connections 2024-09-12T12:25:59.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.743+0000 7f1d0dcb8640 1 -- 172.21.15.3:0/3123447015 wait complete. 2024-09-12T12:25:59.763 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-12T12:25:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafeb00640 1 Processor -- start 2024-09-12T12:25:59.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafeb00640 1 -- start start 2024-09-12T12:25:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafeb00640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdaf80ae250 0x7fdaf80ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafeb00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fdaf815b620 0x7fdaf815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafeb00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf80ad0c0 con 0x7fdaf80d05b0 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafeb00640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf80ad3a0 con 0x7fdaf80ae250 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafeb00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf80ad680 con 0x7fdaf815b620 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdaf7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdaf80ae250 0x7fdaf80ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdaf7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdaf80ae250 0x7fdaf80ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60986/0 (socket says 172.21.15.3:60986) 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdaf7fff640 1 -- 172.21.15.3:0/2848439320 learned_addr learned my addr 172.21.15.3:0/2848439320 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdafd09d640 1 --2- 172.21.15.3:0/2848439320 >> v2:172.21.15.142:3300/0 conn(0x7fdaf815b620 0x7fdaf815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdaf7fff640 1 -- 172.21.15.3:0/2848439320 >> v2:172.21.15.142:3300/0 conn(0x7fdaf815b620 msgr2=0x7fdaf815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdaf7fff640 1 --2- 172.21.15.3:0/2848439320 >> v2:172.21.15.142:3300/0 conn(0x7fdaf815b620 0x7fdaf815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdaf7fff640 1 -- 172.21.15.3:0/2848439320 >> v1:172.21.15.142:6789/0 conn(0x7fdaf80d05b0 legacy=0x7fdaf80d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:59.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.875+0000 7fdaf7fff640 1 -- 172.21.15.3:0/2848439320 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf80ad960 con 0x7fdaf80ae250 2024-09-12T12:25:59.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdaf7fff640 1 --2- 172.21.15.3:0/2848439320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf80ae250 0x7fdaf80ae6e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009e50 tx=0x7fdae001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f0eef8d9f194c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdaf77fe640 1 -- 172.21.15.3:0/2848439320 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae00a5730 con 0x7fdaf80ae250 2024-09-12T12:25:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdaf77fe640 1 -- 172.21.15.3:0/2848439320 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae00a58d0 con 0x7fdaf80ae250 2024-09-12T12:25:59.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdaf77fe640 1 -- 172.21.15.3:0/2848439320 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae00a5ca0 con 0x7fdaf80ae250 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 -- 172.21.15.3:0/2848439320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf80ae250 msgr2=0x7fdaf80ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 --2- 172.21.15.3:0/2848439320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf80ae250 0x7fdaf80ae6e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009e50 tx=0x7fdae001c980 comp rx=0 tx=0).stop 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 -- 172.21.15.3:0/2848439320 shutdown_connections 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 --2- 172.21.15.3:0/2848439320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf80ae250 0x7fdaf80ae6e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 --2- 172.21.15.3:0/2848439320 >> v2:172.21.15.142:3300/0 conn(0x7fdaf815b620 0x7fdaf815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 -- 172.21.15.3:0/2848439320 >> 172.21.15.3:0/2848439320 conn(0x7fdaf8067e00 msgr2=0x7fdaf8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 -- 172.21.15.3:0/2848439320 shutdown_connections 2024-09-12T12:25:59.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.879+0000 7fdafeb00640 1 -- 172.21.15.3:0/2848439320 wait complete. 2024-09-12T12:25:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 Processor -- start 2024-09-12T12:25:59.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 -- start start 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaf80ae250 0x7fdaf80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdaf80d05b0 0x7fdaf8138280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf815b620 0x7fdaf8153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf815c9b0 con 0x7fdaf815b620 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf815c830 con 0x7fdaf80ae250 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf8167cb0 con 0x7fdaf80d05b0 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf815b620 0x7fdaf8153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf815b620 0x7fdaf8153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60998/0 (socket says 172.21.15.3:60998) 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 -- 172.21.15.3:0/1353753744 learned_addr learned my addr 172.21.15.3:0/1353753744 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafc89c640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaf80ae250 0x7fdaf80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdaf7fff640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdaf80d05b0 0x7fdaf8138280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 -- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdaf80d05b0 msgr2=0x7fdaf8138280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdaf80d05b0 0x7fdaf8138280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 -- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaf80ae250 msgr2=0x7fdaf80cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaf80ae250 0x7fdaf80cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 -- 172.21.15.3:0/1353753744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf80ce2a0 con 0x7fdaf815b620 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafd09d640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf815b620 0x7fdaf8153310 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d560 tx=0x7fdae800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdaf57fa640 1 -- 172.21.15.3:0/1353753744 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80125f0 con 0x7fdaf815b620 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdaf57fa640 1 -- 172.21.15.3:0/1353753744 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae8019070 con 0x7fdaf815b620 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf8147730 con 0x7fdaf815b620 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdaf57fa640 1 -- 172.21.15.3:0/1353753744 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80129f0 con 0x7fdaf815b620 2024-09-12T12:25:59.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf80cdeb0 con 0x7fdaf815b620 2024-09-12T12:25:59.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf8147be0 con 0x7fdaf815b620 2024-09-12T12:25:59.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.883+0000 7fdaf57fa640 1 -- 172.21.15.3:0/1353753744 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdae8012b90 con 0x7fdaf815b620 2024-09-12T12:25:59.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.887+0000 7fdaf57fa640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdad80678c0 0x7fdad8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.887+0000 7fdaf57fa640 1 -- 172.21.15.3:0/1353753744 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7fdae8085ee0 con 0x7fdaf815b620 2024-09-12T12:25:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.887+0000 7fdafc89c640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdad80678c0 0x7fdad8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:59.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.887+0000 7fdafc89c640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdad80678c0 0x7fdad8069d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0004e30 tx=0x7fdaf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:59.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:25:59.891+0000 7fdaf57fa640 1 -- 172.21.15.3:0/1353753744 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fdae8056c10 con 0x7fdaf815b620 2024-09-12T12:26:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.055+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fdaf80b3eb0 con 0x7fdad80678c0 2024-09-12T12:26:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.055+0000 7fdaf57fa640 1 -- 172.21.15.3:0/1353753744 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdaf80b3eb0 con 0x7fdad80678c0 2024-09-12T12:26:00.061 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 1 to deep-scrub 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdad80678c0 msgr2=0x7fdad8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdad80678c0 0x7fdad8069d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0004e30 tx=0x7fdaf0009040 comp rx=0 tx=0).stop 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf815b620 msgr2=0x7fdaf8153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf815b620 0x7fdaf8153310 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d560 tx=0x7fdae800da20 comp rx=0 tx=0).stop 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 shutdown_connections 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fdad80678c0 0x7fdad8069d80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdaf815b620 0x7fdaf8153310 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdaf80d05b0 0x7fdaf8138280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 --2- 172.21.15.3:0/1353753744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdaf80ae250 0x7fdaf80cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 >> 172.21.15.3:0/1353753744 conn(0x7fdaf8067e00 msgr2=0x7fdaf80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 shutdown_connections 2024-09-12T12:26:00.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.059+0000 7fdafeb00640 1 -- 172.21.15.3:0/1353753744 wait complete. 2024-09-12T12:26:00.078 INFO:tasks.ceph:Scrubbing osd.2 2024-09-12T12:26:00.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:26:00.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.211+0000 7f215c12a640 1 Processor -- start 2024-09-12T12:26:00.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.211+0000 7f215c12a640 1 -- start start 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f215c12a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f21540b38a0 0x7f21540cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f215c12a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215406d5a0 con 0x7f21540cad40 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f215c12a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21540cb120 con 0x7f21540cf530 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f215c12a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f21540cc2b0 con 0x7f21540b38a0 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2159ec6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f21540cad40 0x7f21540cee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38040/0 (socket says 172.21.15.3:38040) 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2159ec6640 1 -- 172.21.15.3:0/4262587082 learned_addr learned my addr 172.21.15.3:0/4262587082 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3816351275 0 0) 0x7f215406d5a0 con 0x7f21540cad40 2024-09-12T12:26:00.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f215a6c7640 1 --2- 172.21.15.3:0/4262587082 >> v2:172.21.15.142:3300/0 conn(0x7f21540b38a0 0x7f21540cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c005740 con 0x7f21540cad40 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3214053078 0 0) 0x7f213c005740 con 0x7f21540cad40 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 >> v2:172.21.15.142:3300/0 conn(0x7f21540b38a0 msgr2=0x7f21540cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 --2- 172.21.15.3:0/4262587082 >> v2:172.21.15.142:3300/0 conn(0x7f21540b38a0 0x7f21540cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 >> v1:172.21.15.142:6789/0 conn(0x7f21540cf530 legacy=0x7f21540cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21540cc500 con 0x7f21540cad40 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f214c001a00 con 0x7f21540cad40 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 >> v1:172.21.15.3:6789/0 conn(0x7f21540cad40 legacy=0x7f21540cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f213c005bd0 0x7f213c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f213c006a20 0x7f213c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c0073e0 0x7f213c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21540cc2b0 con 0x7f213c006a20 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f213c00e280 con 0x7f213c0073e0 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f213c00e480 con 0x7f213c005bd0 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f21596c5640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f213c006a20 0x7f213c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2159ec6640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f213c005bd0 0x7f213c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f215a6c7640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c0073e0 0x7f213c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f21596c5640 1 -- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f213c005bd0 msgr2=0x7f213c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f21596c5640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f213c005bd0 0x7f213c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f21596c5640 1 -- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c0073e0 msgr2=0x7f213c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f21596c5640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c0073e0 0x7f213c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f21596c5640 1 -- 172.21.15.3:0/4262587082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c005740 con 0x7f213c006a20 2024-09-12T12:26:00.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f215a6c7640 1 -- 172.21.15.3:0/4262587082 reap_dead start 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f21596c5640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f213c006a20 0x7f213c006ea0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2148001780 tx=0x7f214801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219769dfb0f29edc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480013a0 con 0x7f213c006a20 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21480052f0 con 0x7f213c006a20 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.215+0000 7f2158ec4640 1 -- 172.21.15.3:0/4262587082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148003840 con 0x7f213c006a20 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- 172.21.15.3:0/4262587082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f213c006a20 msgr2=0x7f213c006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f213c006a20 0x7f213c006ea0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2148001780 tx=0x7f214801c400 comp rx=0 tx=0).stop 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- 172.21.15.3:0/4262587082 shutdown_connections 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 --2- 172.21.15.3:0/4262587082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f213c006a20 0x7f213c006ea0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- 172.21.15.3:0/4262587082 >> 172.21.15.3:0/4262587082 conn(0x7f21540b2d80 msgr2=0x7f21540b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- 172.21.15.3:0/4262587082 shutdown_connections 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- 172.21.15.3:0/4262587082 wait complete. 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 Processor -- start 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- start start 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c006a20 0x7f215409aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f215409db70 0x7f215409dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f215415f170 0x7f215415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21540ccda0 con 0x7f215409db70 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f21540ccac0 con 0x7f213c006a20 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f21540cd080 con 0x7f215415f170 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f215409db70 0x7f215409dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f215409db70 0x7f215409dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32786/0 (socket says 172.21.15.3:32786) 2024-09-12T12:26:00.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 -- 172.21.15.3:0/660647405 learned_addr learned my addr 172.21.15.3:0/660647405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215a6c7640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f215415f170 0x7f215415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f2159ec6640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c006a20 0x7f215409aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 -- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f215415f170 msgr2=0x7f215415f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f215415f170 0x7f215415f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 -- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c006a20 msgr2=0x7f215409aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f213c006a20 0x7f215409aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 -- 172.21.15.3:0/660647405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21540cbde0 con 0x7f215409db70 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215a6c7640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f215415f170 0x7f215415f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21596c5640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f215409db70 0x7f215409dff0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f214801c230 tx=0x7f213c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480b8020 con 0x7f215409db70 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2148007230 con 0x7f215409db70 2024-09-12T12:26:00.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21540c5410 con 0x7f215409db70 2024-09-12T12:26:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480013a0 con 0x7f215409db70 2024-09-12T12:26:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.219+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215409e960 con 0x7f215409db70 2024-09-12T12:26:00.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f215406d5a0 con 0x7f215409db70 2024-09-12T12:26:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f21480bb020 con 0x7f215409db70 2024-09-12T12:26:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f21527fc640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21400678f0 0x7f2140069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f214811fc40 con 0x7f215409db70 2024-09-12T12:26:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f21527fc640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f214006d990 0x7f214006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 --> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2140070460 con 0x7f214006d990 2024-09-12T12:26:00.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f2148122340 con 0x7f215409db70 2024-09-12T12:26:00.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f215a6c7640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f214006d990 0x7f214006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f2159ec6640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21400678f0 0x7f2140069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f215a6c7640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f214006d990 0x7f214006fdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f2159ec6640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21400678f0 0x7f2140069db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f214c000f90 tx=0x7f214c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.223+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== osd.2 v2:172.21.15.3:6824/3021596546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2140070460 con 0x7f214006d990 2024-09-12T12:26:00.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 --> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f21540b71d0 con 0x7f214006d990 2024-09-12T12:26:00.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f21527fc640 1 -- 172.21.15.3:0/660647405 <== osd.2 v2:172.21.15.3:6824/3021596546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f21540b71d0 con 0x7f214006d990 2024-09-12T12:26:00.241 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:26:00.241 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:26:00.241 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:26:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f214006d990 msgr2=0x7f214006fdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:6824/3021596546,v1:172.21.15.3:6825/3021596546] conn(0x7f214006d990 0x7f214006fdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21400678f0 msgr2=0x7f2140069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f21400678f0 0x7f2140069db0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f214c000f90 tx=0x7f214c009040 comp rx=0 tx=0).stop 2024-09-12T12:26:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f215409db70 msgr2=0x7f215409dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 --2- 172.21.15.3:0/660647405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f215409db70 0x7f215409dff0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f214801c230 tx=0x7f213c007a50 comp rx=0 tx=0).stop 2024-09-12T12:26:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215a6c7640 1 -- 172.21.15.3:0/660647405 reap_dead start 2024-09-12T12:26:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 shutdown_connections 2024-09-12T12:26:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 >> 172.21.15.3:0/660647405 conn(0x7f21540b2d80 msgr2=0x7f21540d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 shutdown_connections 2024-09-12T12:26:00.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.239+0000 7f215c12a640 1 -- 172.21.15.3:0/660647405 wait complete. 2024-09-12T12:26:00.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-12T12:26:00.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 Processor -- start 2024-09-12T12:26:00.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 -- start start 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa9b4073cb0 0x7fa9b415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9b40b30e0 0x7fa9b40adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b4074090 con 0x7fa9b40b30e0 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa9b40b5000 con 0x7fa9b4073cb0 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b99a5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b40b3530 con 0x7fa9b40ae150 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b27fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9b40b30e0 0x7fa9b40adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b27fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa9b40b30e0 0x7fa9b40adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32802/0 (socket says 172.21.15.3:32802) 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.371+0000 7fa9b27fc640 1 -- 172.21.15.3:0/2281374232 learned_addr learned my addr 172.21.15.3:0/2281374232 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b2ffd640 1 --2- 172.21.15.3:0/2281374232 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4073cb0 0x7fa9b415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b37fe640 1 --2- 172.21.15.3:0/2281374232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b27fc640 1 -- 172.21.15.3:0/2281374232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 msgr2=0x7fa9b40ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b27fc640 1 --2- 172.21.15.3:0/2281374232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b27fc640 1 -- 172.21.15.3:0/2281374232 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4073cb0 msgr2=0x7fa9b415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b27fc640 1 --2- 172.21.15.3:0/2281374232 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4073cb0 0x7fa9b415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b27fc640 1 -- 172.21.15.3:0/2281374232 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b40b37c0 con 0x7fa9b40b30e0 2024-09-12T12:26:00.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b27fc640 1 --2- 172.21.15.3:0/2281374232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40adc10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40d1170 tx=0x7fa9a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1a122e96dad142 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b1ffb640 1 -- 172.21.15.3:0/2281374232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80b0070 con 0x7fa9b40b30e0 2024-09-12T12:26:00.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b1ffb640 1 -- 172.21.15.3:0/2281374232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a80ab600 con 0x7fa9b40b30e0 2024-09-12T12:26:00.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b1ffb640 1 -- 172.21.15.3:0/2281374232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80ab9d0 con 0x7fa9b40b30e0 2024-09-12T12:26:00.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2281374232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 msgr2=0x7fa9b40adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2281374232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40adc10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40d1170 tx=0x7fa9a801c520 comp rx=0 tx=0).stop 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2281374232 shutdown_connections 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2281374232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2281374232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40adc10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2281374232 >> v2:172.21.15.142:3300/0 conn(0x7fa9b4073cb0 0x7fa9b415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2281374232 >> 172.21.15.3:0/2281374232 conn(0x7fa9b4067e00 msgr2=0x7fa9b4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2281374232 shutdown_connections 2024-09-12T12:26:00.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.375+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2281374232 wait complete. 2024-09-12T12:26:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 Processor -- start 2024-09-12T12:26:00.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 -- start start 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4073cb0 0x7fa9b41fa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b415fe80 con 0x7fa9b40b30e0 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b415fd00 con 0x7fa9b4073cb0 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa9b4160000 con 0x7fa9b40ae150 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32804/0 (socket says 172.21.15.3:32804) 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 -- 172.21.15.3:0/2280097776 learned_addr learned my addr 172.21.15.3:0/2280097776 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b2ffd640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4073cb0 0x7fa9b41fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b27fc640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 -- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 msgr2=0x7fa9b40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 -- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4073cb0 msgr2=0x7fa9b41fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4073cb0 0x7fa9b41fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 -- 172.21.15.3:0/2280097776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b41fa880 con 0x7fa9b40b30e0 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b37fe640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40b2480 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00d800 tx=0x7fa9ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa99bfff640 1 -- 172.21.15.3:0/2280097776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00b6c0 con 0x7fa9b40b30e0 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa99bfff640 1 -- 172.21.15.3:0/2280097776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ac019070 con 0x7fa9b40b30e0 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa99bfff640 1 -- 172.21.15.3:0/2280097776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00baa0 con 0x7fa9b40b30e0 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b40ccc20 con 0x7fa9b40b30e0 2024-09-12T12:26:00.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.379+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b40cd1a0 con 0x7fa9b40b30e0 2024-09-12T12:26:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.383+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b40ae880 con 0x7fa9b40b30e0 2024-09-12T12:26:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.383+0000 7fa99bfff640 1 -- 172.21.15.3:0/2280097776 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa9ac00bc40 con 0x7fa9b40b30e0 2024-09-12T12:26:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.383+0000 7fa99bfff640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9a00678c0 0x7fa9a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.383+0000 7fa99bfff640 1 -- 172.21.15.3:0/2280097776 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7fa9ac085e10 con 0x7fa9b40b30e0 2024-09-12T12:26:00.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.383+0000 7fa9b2ffd640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9a00678c0 0x7fa9a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.383+0000 7fa9b2ffd640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9a00678c0 0x7fa9a0069d80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4003fd0 tx=0x7fa9a40039a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.387+0000 7fa99bfff640 1 -- 172.21.15.3:0/2280097776 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fa9ac056bd0 con 0x7fa9b40b30e0 2024-09-12T12:26:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.547+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fa9b415c460 con 0x7fa9a00678c0 2024-09-12T12:26:00.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa99bfff640 1 -- 172.21.15.3:0/2280097776 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa9b415c460 con 0x7fa9a00678c0 2024-09-12T12:26:00.553 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 2 to deep-scrub 2024-09-12T12:26:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9a00678c0 msgr2=0x7fa9a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9a00678c0 0x7fa9a0069d80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4003fd0 tx=0x7fa9a40039a0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 msgr2=0x7fa9b40b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40b2480 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00d800 tx=0x7fa9ac00dcc0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 shutdown_connections 2024-09-12T12:26:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa9a00678c0 0x7fa9a0069d80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9b40b30e0 0x7fa9b40b2480 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9b40ae150 0x7fa9b40b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 --2- 172.21.15.3:0/2280097776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9b4073cb0 0x7fa9b41fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 >> 172.21.15.3:0/2280097776 conn(0x7fa9b4067e00 msgr2=0x7fa9b415f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 shutdown_connections 2024-09-12T12:26:00.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.551+0000 7fa9b99a5640 1 -- 172.21.15.3:0/2280097776 wait complete. 2024-09-12T12:26:00.570 INFO:tasks.ceph:Scrubbing osd.3 2024-09-12T12:26:00.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:26:00.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.695+0000 7f5dbeab7640 1 Processor -- start 2024-09-12T12:26:00.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.695+0000 7f5dbeab7640 1 -- start start 2024-09-12T12:26:00.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.695+0000 7f5dbeab7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5db80b31c0 0x7f5db815c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.695+0000 7f5dbeab7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 0x7f5db80724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db815d160 con 0x7f5db80d03a0 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f5db815d720 con 0x7f5db80b31c0 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5db815d440 con 0x7f5db815ccc0 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5db80b31c0 0x7f5db815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f5db80b31c0 0x7f5db815c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32810/0 (socket says 172.21.15.3:32810) 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 -- 172.21.15.3:0/26357431 learned_addr learned my addr 172.21.15.3:0/26357431 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbd054640 1 --2- 172.21.15.3:0/26357431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 0x7f5db80724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 -- 172.21.15.3:0/26357431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 msgr2=0x7f5db80724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 --2- 172.21.15.3:0/26357431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 0x7f5db80724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 -- 172.21.15.3:0/26357431 >> v1:172.21.15.142:6789/0 conn(0x7f5db80d03a0 legacy=0x7f5db80d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 -- 172.21.15.3:0/26357431 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db80ae9b0 con 0x7f5db80b31c0 2024-09-12T12:26:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db7fff640 1 --2- 172.21.15.3:0/26357431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db815c780 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009870 tx=0x7f5da001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a60b85e5e59f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db77fe640 1 -- 172.21.15.3:0/26357431 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da00aa620 con 0x7f5db80b31c0 2024-09-12T12:26:00.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db77fe640 1 -- 172.21.15.3:0/26357431 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da00af070 con 0x7f5db80b31c0 2024-09-12T12:26:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5db77fe640 1 -- 172.21.15.3:0/26357431 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da00aa9e0 con 0x7f5db80b31c0 2024-09-12T12:26:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- 172.21.15.3:0/26357431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 msgr2=0x7f5db815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 --2- 172.21.15.3:0/26357431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db815c780 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009870 tx=0x7f5da001c4a0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- 172.21.15.3:0/26357431 shutdown_connections 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 --2- 172.21.15.3:0/26357431 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 0x7f5db80724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 --2- 172.21.15.3:0/26357431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db815c780 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- 172.21.15.3:0/26357431 >> 172.21.15.3:0/26357431 conn(0x7f5db8067bd0 msgr2=0x7f5db8067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- 172.21.15.3:0/26357431 shutdown_connections 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.699+0000 7f5dbeab7640 1 -- 172.21.15.3:0/26357431 wait complete. 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 Processor -- start 2024-09-12T12:26:00.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 -- start start 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db81400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db80d03a0 0x7f5db80cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 0x7f5db80cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8169150 con 0x7f5db80b31c0 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8168fd0 con 0x7f5db80d03a0 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f5db81692d0 con 0x7f5db815ccc0 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db81400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db81400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32816/0 (socket says 172.21.15.3:32816) 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 -- 172.21.15.3:0/1356732313 learned_addr learned my addr 172.21.15.3:0/1356732313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5db7fff640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db80d03a0 0x7f5db80cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbd054640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 0x7f5db80cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 -- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 msgr2=0x7f5db80cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5db815ccc0 0x7f5db80cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 -- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db80d03a0 msgr2=0x7f5db80cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db80d03a0 0x7f5db80cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 -- 172.21.15.3:0/1356732313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db815e3d0 con 0x7f5db80b31c0 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5db7fff640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5db80d03a0 0x7f5db80cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbc853640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db81400e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5da800c370 tx=0x7f5da800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80052d0 con 0x7f5db80b31c0 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da8005470 con 0x7f5db80b31c0 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db815e6b0 con 0x7f5db80b31c0 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8007260 con 0x7f5db80b31c0 2024-09-12T12:26:00.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db815ec70 con 0x7f5db80b31c0 2024-09-12T12:26:00.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.703+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5db815f050 con 0x7f5db80b31c0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5da8007400 con 0x7f5db80b31c0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db57fa640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5d980678c0 0x7f5d98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f5da80857a0 con 0x7f5db80b31c0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db57fa640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f5d9806d960 0x7f5d9806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 --> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d98070430 con 0x7f5d9806d960 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f5da8085b50 con 0x7f5db80b31c0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db7fff640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5d980678c0 0x7f5d98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5dbd054640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f5d9806d960 0x7f5d9806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5dbd054640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f5d9806d960 0x7f5d9806fd80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db7fff640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5d980678c0 0x7f5d98069d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5da00ae000 tx=0x7f5da00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.707+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== osd.3 v2:172.21.15.3:6808/63313814 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d98070430 con 0x7f5d9806d960 2024-09-12T12:26:00.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.719+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 --> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5db815c780 con 0x7f5d9806d960 2024-09-12T12:26:00.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.719+0000 7f5db57fa640 1 -- 172.21.15.3:0/1356732313 <== osd.3 v2:172.21.15.3:6808/63313814 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5db815c780 con 0x7f5d9806d960 2024-09-12T12:26:00.724 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:26:00.725 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:26:00.725 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f5d9806d960 msgr2=0x7f5d9806fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:6808/63313814,v1:172.21.15.3:6809/63313814] conn(0x7f5d9806d960 0x7f5d9806fd80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5d980678c0 msgr2=0x7f5d98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f5d980678c0 0x7f5d98069d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5da00ae000 tx=0x7f5da00a7000 comp rx=0 tx=0).stop 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 msgr2=0x7f5db81400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 --2- 172.21.15.3:0/1356732313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5db80b31c0 0x7f5db81400e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5da800c370 tx=0x7f5da800c830 comp rx=0 tx=0).stop 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbd054640 1 -- 172.21.15.3:0/1356732313 reap_dead start 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 shutdown_connections 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 >> 172.21.15.3:0/1356732313 conn(0x7f5db8067bd0 msgr2=0x7f5db8067fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 shutdown_connections 2024-09-12T12:26:00.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.723+0000 7f5dbeab7640 1 -- 172.21.15.3:0/1356732313 wait complete. 2024-09-12T12:26:00.741 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-12T12:26:00.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7bab352640 1 Processor -- start 2024-09-12T12:26:00.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7bab352640 1 -- start start 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7bab352640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ba415c710 0x7f7ba40b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7bab352640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba40ae880 0x7f7ba40aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7bab352640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba40b5000 con 0x7f7ba4073cb0 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7bab352640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba4074090 con 0x7f7ba415c710 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7bab352640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba40b37c0 con 0x7f7ba40ae880 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba88ed640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ba415c710 0x7f7ba40b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba98ef640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba40ae880 0x7f7ba40aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba88ed640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ba415c710 0x7f7ba40b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.3:58492/0 (socket says 172.21.15.3:58492) 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba90ee640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f7ba4073cb0 0x7f7ba40adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.3:58116/0 (socket says 172.21.15.3:58116) 2024-09-12T12:26:00.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba88ed640 1 -- 172.21.15.3:0/3613772499 learned_addr learned my addr 172.21.15.3:0/3613772499 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:00.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba98ef640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba40ae880 0x7f7ba40aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.3:39348/0 (socket says 172.21.15.3:39348) 2024-09-12T12:26:00.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba88ed640 1 -- 172.21.15.3:0/3613772499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba40ae880 msgr2=0x7f7ba40aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba88ed640 1 --2- 172.21.15.3:0/3613772499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba40ae880 0x7f7ba40aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba88ed640 1 -- 172.21.15.3:0/3613772499 >> v1:172.21.15.142:6789/0 conn(0x7f7ba4073cb0 legacy=0x7f7ba40adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.855+0000 7f7ba88ed640 1 -- 172.21.15.3:0/3613772499 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba40b3aa0 con 0x7f7ba415c710 2024-09-12T12:26:00.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7ba88ed640 1 --2- 172.21.15.3:0/3613772499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba415c710 0x7f7ba40b30e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009870 tx=0x7f7b9401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a8e5d9a470ee41e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7b9bfff640 1 -- 172.21.15.3:0/3613772499 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b940ab650 con 0x7f7ba415c710 2024-09-12T12:26:00.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7b9bfff640 1 -- 172.21.15.3:0/3613772499 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b940b0070 con 0x7f7ba415c710 2024-09-12T12:26:00.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7b9bfff640 1 -- 172.21.15.3:0/3613772499 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b940aba10 con 0x7f7ba415c710 2024-09-12T12:26:00.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 -- 172.21.15.3:0/3613772499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba415c710 msgr2=0x7f7ba40b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 --2- 172.21.15.3:0/3613772499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba415c710 0x7f7ba40b30e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009870 tx=0x7f7b9401c560 comp rx=0 tx=0).stop 2024-09-12T12:26:00.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 -- 172.21.15.3:0/3613772499 shutdown_connections 2024-09-12T12:26:00.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 --2- 172.21.15.3:0/3613772499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba40ae880 0x7f7ba40aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 --2- 172.21.15.3:0/3613772499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba415c710 0x7f7ba40b30e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 -- 172.21.15.3:0/3613772499 >> 172.21.15.3:0/3613772499 conn(0x7f7ba4067e00 msgr2=0x7f7ba4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 -- 172.21.15.3:0/3613772499 shutdown_connections 2024-09-12T12:26:00.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.859+0000 7f7bab352640 1 -- 172.21.15.3:0/3613772499 wait complete. 2024-09-12T12:26:00.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 Processor -- start 2024-09-12T12:26:00.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 -- start start 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ba4073cb0 0x7f7ba41502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40ae880 0x7f7ba4150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba415c710 0x7f7ba406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4168f40 con 0x7f7ba4073cb0 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4168dc0 con 0x7f7ba40ae880 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba41690c0 con 0x7f7ba415c710 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ba4073cb0 0x7f7ba41502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ba4073cb0 0x7f7ba41502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32832/0 (socket says 172.21.15.3:32832) 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 -- 172.21.15.3:0/766136781 learned_addr learned my addr 172.21.15.3:0/766136781 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba98ef640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba415c710 0x7f7ba406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba88ed640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40ae880 0x7f7ba4150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 -- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba415c710 msgr2=0x7f7ba406dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba415c710 0x7f7ba406dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 -- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40ae880 msgr2=0x7f7ba4150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40ae880 0x7f7ba4150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 -- 172.21.15.3:0/766136781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba406ce70 con 0x7f7ba4073cb0 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7ba90ee640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ba4073cb0 0x7f7ba41502f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c450 tx=0x7f7ba000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7b99ffb640 1 -- 172.21.15.3:0/766136781 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0018070 con 0x7f7ba4073cb0 2024-09-12T12:26:00.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7b99ffb640 1 -- 172.21.15.3:0/766136781 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba0007480 con 0x7f7ba4073cb0 2024-09-12T12:26:00.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba406d110 con 0x7f7ba4073cb0 2024-09-12T12:26:00.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7b99ffb640 1 -- 172.21.15.3:0/766136781 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00047f0 con 0x7f7ba4073cb0 2024-09-12T12:26:00.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba415eb40 con 0x7f7ba4073cb0 2024-09-12T12:26:00.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba40aecb0 con 0x7f7ba4073cb0 2024-09-12T12:26:00.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.863+0000 7f7b99ffb640 1 -- 172.21.15.3:0/766136781 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ba001b070 con 0x7f7ba4073cb0 2024-09-12T12:26:00.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.867+0000 7f7b99ffb640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b880678c0 0x7f7b88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.867+0000 7f7b99ffb640 1 -- 172.21.15.3:0/766136781 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f7ba0084da0 con 0x7f7ba4073cb0 2024-09-12T12:26:00.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.867+0000 7f7ba88ed640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b880678c0 0x7f7b88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.867+0000 7f7ba88ed640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b880678c0 0x7f7b88069d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7b940af000 tx=0x7f7b940a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:00.871+0000 7f7b99ffb640 1 -- 172.21.15.3:0/766136781 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f7ba0055a30 con 0x7f7ba4073cb0 2024-09-12T12:26:01.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.027+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f7ba41576e0 con 0x7f7b880678c0 2024-09-12T12:26:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7b99ffb640 1 -- 172.21.15.3:0/766136781 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7ba41576e0 con 0x7f7b880678c0 2024-09-12T12:26:01.033 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 3 to deep-scrub 2024-09-12T12:26:01.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b880678c0 msgr2=0x7f7b88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b880678c0 0x7f7b88069d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7b940af000 tx=0x7f7b940a7000 comp rx=0 tx=0).stop 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ba4073cb0 msgr2=0x7f7ba41502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ba4073cb0 0x7f7ba41502f0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c450 tx=0x7f7ba000c910 comp rx=0 tx=0).stop 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 shutdown_connections 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7b880678c0 0x7f7b88069d80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ba415c710 0x7f7ba406dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ba40ae880 0x7f7ba4150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 --2- 172.21.15.3:0/766136781 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ba4073cb0 0x7f7ba41502f0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 >> 172.21.15.3:0/766136781 conn(0x7f7ba4067e00 msgr2=0x7f7ba415a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 shutdown_connections 2024-09-12T12:26:01.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.031+0000 7f7bab352640 1 -- 172.21.15.3:0/766136781 wait complete. 2024-09-12T12:26:01.051 INFO:tasks.ceph:Scrubbing osd.4 2024-09-12T12:26:01.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:26:01.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe19937640 1 Processor -- start 2024-09-12T12:26:01.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe19937640 1 -- start start 2024-09-12T12:26:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe19937640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efe140d0ab0 0x7efe140ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe19937640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe1415c910 0x7efe140ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe19937640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe140ad160 con 0x7efe141523f0 2024-09-12T12:26:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe19937640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7efe140ad440 con 0x7efe1415c910 2024-09-12T12:26:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe19937640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efe140ad720 con 0x7efe140d0ab0 2024-09-12T12:26:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe137fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe1415c910 0x7efe140ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe137fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7efe1415c910 0x7efe140ae800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32838/0 (socket says 172.21.15.3:32838) 2024-09-12T12:26:01.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.175+0000 7efe137fe640 1 -- 172.21.15.3:0/1688730204 learned_addr learned my addr 172.21.15.3:0/1688730204 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:01.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe127fc640 1 --2- 172.21.15.3:0/1688730204 >> v2:172.21.15.142:3300/0 conn(0x7efe140d0ab0 0x7efe140ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe137fe640 1 -- 172.21.15.3:0/1688730204 >> v2:172.21.15.142:3300/0 conn(0x7efe140d0ab0 msgr2=0x7efe140ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:01.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe137fe640 1 --2- 172.21.15.3:0/1688730204 >> v2:172.21.15.142:3300/0 conn(0x7efe140d0ab0 0x7efe140ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe137fe640 1 -- 172.21.15.3:0/1688730204 >> v1:172.21.15.142:6789/0 conn(0x7efe141523f0 legacy=0x7efe140d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe137fe640 1 -- 172.21.15.3:0/1688730204 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe140ada00 con 0x7efe1415c910 2024-09-12T12:26:01.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe127fc640 1 --2- 172.21.15.3:0/1688730204 >> v2:172.21.15.142:3300/0 conn(0x7efe140d0ab0 0x7efe140ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:01.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe137fe640 1 --2- 172.21.15.3:0/1688730204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe1415c910 0x7efe140ae800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efe1406e540 tx=0x7efe0c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3ddf361f1d80f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe11ffb640 1 -- 172.21.15.3:0/1688730204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c00b2d0 con 0x7efe1415c910 2024-09-12T12:26:01.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe11ffb640 1 -- 172.21.15.3:0/1688730204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c00b470 con 0x7efe1415c910 2024-09-12T12:26:01.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe11ffb640 1 -- 172.21.15.3:0/1688730204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0a7bc0 con 0x7efe1415c910 2024-09-12T12:26:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 -- 172.21.15.3:0/1688730204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe1415c910 msgr2=0x7efe140ae800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 --2- 172.21.15.3:0/1688730204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe1415c910 0x7efe140ae800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7efe1406e540 tx=0x7efe0c01ebc0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 -- 172.21.15.3:0/1688730204 shutdown_connections 2024-09-12T12:26:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 --2- 172.21.15.3:0/1688730204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe1415c910 0x7efe140ae800 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 --2- 172.21.15.3:0/1688730204 >> v2:172.21.15.142:3300/0 conn(0x7efe140d0ab0 0x7efe140ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 -- 172.21.15.3:0/1688730204 >> 172.21.15.3:0/1688730204 conn(0x7efe14067bd0 msgr2=0x7efe14067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 -- 172.21.15.3:0/1688730204 shutdown_connections 2024-09-12T12:26:01.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.179+0000 7efe19937640 1 -- 172.21.15.3:0/1688730204 wait complete. 2024-09-12T12:26:01.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 Processor -- start 2024-09-12T12:26:01.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 -- start start 2024-09-12T12:26:01.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe140d0ab0 0x7efe1414c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe141523f0 0x7efe140cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe1415c910 0x7efe140ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe1415e960 con 0x7efe140d0ab0 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe1415e7e0 con 0x7efe141523f0 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efe1415eae0 con 0x7efe1415c910 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe140d0ab0 0x7efe1414c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe140d0ab0 0x7efe1414c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32844/0 (socket says 172.21.15.3:32844) 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 -- 172.21.15.3:0/1202213652 learned_addr learned my addr 172.21.15.3:0/1202213652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe127fc640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe141523f0 0x7efe140cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe137fe640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe1415c910 0x7efe140ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 -- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe1415c910 msgr2=0x7efe140ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe1415c910 0x7efe140ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 -- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe141523f0 msgr2=0x7efe140cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe141523f0 0x7efe140cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 -- 172.21.15.3:0/1202213652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1414cdd0 con 0x7efe140d0ab0 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe12ffd640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe140d0ab0 0x7efe1414c4c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efe0400c420 tx=0x7efe0400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040052d0 con 0x7efe140d0ab0 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe04005470 con 0x7efe140d0ab0 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe140cf510 con 0x7efe140d0ab0 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04007260 con 0x7efe140d0ab0 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe14152a50 con 0x7efe140d0ab0 2024-09-12T12:26:01.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.183+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe1414ca00 con 0x7efe140d0ab0 2024-09-12T12:26:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efe04007400 con 0x7efe140d0ab0 2024-09-12T12:26:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efdfbfff640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efe000678c0 0x7efe00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7efe040859d0 con 0x7efe140d0ab0 2024-09-12T12:26:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efdfbfff640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7efe0006d960 0x7efe0006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 --> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe00070430 con 0x7efe0006d960 2024-09-12T12:26:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7efe04085e30 con 0x7efe140d0ab0 2024-09-12T12:26:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efe127fc640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efe000678c0 0x7efe00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efe137fe640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7efe0006d960 0x7efe0006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efe137fe640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7efe0006d960 0x7efe0006fd80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efe127fc640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efe000678c0 0x7efe00069d80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efe08006e90 tx=0x7efe08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.187+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== osd.4 v2:172.21.15.142:6800/4142884196 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe00070430 con 0x7efe0006d960 2024-09-12T12:26:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 --> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efe140cfa30 con 0x7efe0006d960 2024-09-12T12:26:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efdfbfff640 1 -- 172.21.15.3:0/1202213652 <== osd.4 v2:172.21.15.142:6800/4142884196 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efe140cfa30 con 0x7efe0006d960 2024-09-12T12:26:01.206 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:26:01.206 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:26:01.206 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:26:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7efe0006d960 msgr2=0x7efe0006fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6800/4142884196,v1:172.21.15.142:6801/4142884196] conn(0x7efe0006d960 0x7efe0006fd80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efe000678c0 msgr2=0x7efe00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efe000678c0 0x7efe00069d80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efe08006e90 tx=0x7efe08008040 comp rx=0 tx=0).stop 2024-09-12T12:26:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe140d0ab0 msgr2=0x7efe1414c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 --2- 172.21.15.3:0/1202213652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe140d0ab0 0x7efe1414c4c0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7efe0400c420 tx=0x7efe0400c8e0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe137fe640 1 -- 172.21.15.3:0/1202213652 reap_dead start 2024-09-12T12:26:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 shutdown_connections 2024-09-12T12:26:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 >> 172.21.15.3:0/1202213652 conn(0x7efe14067bd0 msgr2=0x7efe140afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 shutdown_connections 2024-09-12T12:26:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.203+0000 7efe19937640 1 -- 172.21.15.3:0/1202213652 wait complete. 2024-09-12T12:26:01.223 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-12T12:26:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0c32f640 1 Processor -- start 2024-09-12T12:26:01.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0c32f640 1 -- start start 2024-09-12T12:26:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0c32f640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efc0415c710 0x7efc040b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0c32f640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc040b5000 con 0x7efc04073cb0 2024-09-12T12:26:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0c32f640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc040b37c0 con 0x7efc040ae880 2024-09-12T12:26:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0c32f640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7efc04074090 con 0x7efc0415c710 2024-09-12T12:26:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0a0cb640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7efc04073cb0 0x7efc040adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38046/0 (socket says 172.21.15.3:38046) 2024-09-12T12:26:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc0a0cb640 1 -- 172.21.15.3:0/788498997 learned_addr learned my addr 172.21.15.3:0/788498997 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.339+0000 7efc098ca640 1 --2- 172.21.15.3:0/788498997 >> v2:172.21.15.142:3300/0 conn(0x7efc0415c710 0x7efc040b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2137724057 0 0) 0x7efc040b5000 con 0x7efc04073cb0 2024-09-12T12:26:01.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbf0003530 con 0x7efc04073cb0 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10215706 0 0) 0x7efc040b37c0 con 0x7efc040ae880 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc040b5000 con 0x7efc040ae880 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1248406130 0 0) 0x7efbf0003530 con 0x7efc04073cb0 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 >> v2:172.21.15.142:3300/0 conn(0x7efc0415c710 msgr2=0x7efc040b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 --2- 172.21.15.3:0/788498997 >> v2:172.21.15.142:3300/0 conn(0x7efc0415c710 0x7efc040b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 >> v1:172.21.15.142:6789/0 conn(0x7efc040ae880 legacy=0x7efc040d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc040b3aa0 con 0x7efc04073cb0 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7efc00001680 con 0x7efc04073cb0 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 >> v1:172.21.15.3:6789/0 conn(0x7efc04073cb0 legacy=0x7efc040adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efbf0003b50 0x7efbf0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efbf0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efbf00051d0 0x7efbf000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efbf000c020 con 0x7efbf0004810 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efbf000c1d0 con 0x7efbf00051d0 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efbf000c3d0 con 0x7efbf0003b50 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc098ca640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efbf0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc0a0cb640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efbf0003b50 0x7efbf0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc0a8cc640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efbf00051d0 0x7efbf000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc098ca640 1 -- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efbf0003b50 msgr2=0x7efbf0003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc098ca640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efbf0003b50 0x7efbf0003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc098ca640 1 -- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efbf00051d0 msgr2=0x7efbf000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:01.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc098ca640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efbf00051d0 0x7efbf000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc098ca640 1 -- 172.21.15.3:0/788498997 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf0003530 con 0x7efbf0004810 2024-09-12T12:26:01.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc0a8cc640 1 -- 172.21.15.3:0/788498997 reap_dead start 2024-09-12T12:26:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc098ca640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efbf0004c90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009ae0 tx=0x7efbf401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c006a13e6670ab61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40b2070 con 0x7efbf0004810 2024-09-12T12:26:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf40a5660 con 0x7efbf0004810 2024-09-12T12:26:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc090c9640 1 -- 172.21.15.3:0/788498997 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf40096b0 con 0x7efbf0004810 2024-09-12T12:26:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.343+0000 7efc0c32f640 1 -- 172.21.15.3:0/788498997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 msgr2=0x7efbf0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efbf0004c90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009ae0 tx=0x7efbf401ca70 comp rx=0 tx=0).stop 2024-09-12T12:26:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- 172.21.15.3:0/788498997 shutdown_connections 2024-09-12T12:26:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 --2- 172.21.15.3:0/788498997 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efbf0004c90 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- 172.21.15.3:0/788498997 >> 172.21.15.3:0/788498997 conn(0x7efc04067e00 msgr2=0x7efc04068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- 172.21.15.3:0/788498997 shutdown_connections 2024-09-12T12:26:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- 172.21.15.3:0/788498997 wait complete. 2024-09-12T12:26:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 Processor -- start 2024-09-12T12:26:01.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- start start 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efc040cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041534a0 0x7efc0414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc040b2810 0x7efc040ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc040b30e0 con 0x7efbf0004810 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efc040ad6f0 con 0x7efc041534a0 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7efc040b3260 con 0x7efc040b2810 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efc040cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efc040cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32860/0 (socket says 172.21.15.3:32860) 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 -- 172.21.15.3:0/3985249371 learned_addr learned my addr 172.21.15.3:0/3985249371 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:01.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc098ca640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041534a0 0x7efc0414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a8cc640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc040b2810 0x7efc040ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 -- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc040b2810 msgr2=0x7efc040ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc040b2810 0x7efc040ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 -- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041534a0 msgr2=0x7efc0414e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041534a0 0x7efc0414e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 -- 172.21.15.3:0/3985249371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc040cdd50 con 0x7efbf0004810 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a8cc640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc040b2810 0x7efc040ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0a0cb640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efc040cd4b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7efc0000e7b0 tx=0x7efc0000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efbfaffd640 1 -- 172.21.15.3:0/3985249371 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0000c6c0 con 0x7efbf0004810 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efbfaffd640 1 -- 172.21.15.3:0/3985249371 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc00018070 con 0x7efbf0004810 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efbfaffd640 1 -- 172.21.15.3:0/3985249371 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0000cac0 con 0x7efbf0004810 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc040ce030 con 0x7efbf0004810 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.347+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc040b0010 con 0x7efbf0004810 2024-09-12T12:26:01.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.351+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc040aecb0 con 0x7efbf0004810 2024-09-12T12:26:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.351+0000 7efbfaffd640 1 -- 172.21.15.3:0/3985249371 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc0000cc60 con 0x7efbf0004810 2024-09-12T12:26:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.351+0000 7efbfaffd640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efbe80678f0 0x7efbe8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.351+0000 7efbfaffd640 1 -- 172.21.15.3:0/3985249371 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7efc000857a0 con 0x7efbf0004810 2024-09-12T12:26:01.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.351+0000 7efc098ca640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efbe80678f0 0x7efbe8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.351+0000 7efc098ca640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efbe80678f0 0x7efbe8069db0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efbf40af000 tx=0x7efbf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.355+0000 7efbfaffd640 1 -- 172.21.15.3:0/3985249371 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7efc00056540 con 0x7efbf0004810 2024-09-12T12:26:01.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.519+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7efc0415d2c0 con 0x7efbe80678f0 2024-09-12T12:26:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.519+0000 7efbfaffd640 1 -- 172.21.15.3:0/3985249371 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efc0415d2c0 con 0x7efbe80678f0 2024-09-12T12:26:01.524 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 4 to deep-scrub 2024-09-12T12:26:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efbe80678f0 msgr2=0x7efbe8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efbe80678f0 0x7efbe8069db0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efbf40af000 tx=0x7efbf4008040 comp rx=0 tx=0).stop 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 msgr2=0x7efc040cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efc040cd4b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7efc0000e7b0 tx=0x7efc0000ec70 comp rx=0 tx=0).stop 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 shutdown_connections 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7efbe80678f0 0x7efbe8069db0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efc040b2810 0x7efc040ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041534a0 0x7efc0414e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 --2- 172.21.15.3:0/3985249371 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efbf0004810 0x7efc040cd4b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 >> 172.21.15.3:0/3985249371 conn(0x7efc04067e00 msgr2=0x7efc040d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 shutdown_connections 2024-09-12T12:26:01.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.523+0000 7efc0c32f640 1 -- 172.21.15.3:0/3985249371 wait complete. 2024-09-12T12:26:01.544 INFO:tasks.ceph:Scrubbing osd.5 2024-09-12T12:26:01.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:26:01.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.671+0000 7f4cdb714640 1 Processor -- start 2024-09-12T12:26:01.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.671+0000 7f4cdb714640 1 -- start start 2024-09-12T12:26:01.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd415ed60 0x7f4cd406f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cd40ceb70 0x7f4cd415f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cd415e920 con 0x7f4cd415f680 2024-09-12T12:26:01.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd40cf380 con 0x7f4cd40ceb70 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd40cf660 con 0x7f4cd415ed60 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cd40ceb70 0x7f4cd415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4cd40ceb70 0x7f4cd415f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32868/0 (socket says 172.21.15.3:32868) 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 -- 172.21.15.3:0/4024899694 learned_addr learned my addr 172.21.15.3:0/4024899694 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd94b0640 1 --2- 172.21.15.3:0/4024899694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd415ed60 0x7f4cd406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 -- 172.21.15.3:0/4024899694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd415ed60 msgr2=0x7f4cd406f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 --2- 172.21.15.3:0/4024899694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd415ed60 0x7f4cd406f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 -- 172.21.15.3:0/4024899694 >> v1:172.21.15.142:6789/0 conn(0x7f4cd415f680 legacy=0x7f4cd40cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:01.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 -- 172.21.15.3:0/4024899694 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd40cf940 con 0x7f4cd40ceb70 2024-09-12T12:26:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cd8caf640 1 --2- 172.21.15.3:0/4024899694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd40ceb70 0x7f4cd415f140 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009870 tx=0x7f4cc401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdc5ac37a27a9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4ccbfff640 1 -- 172.21.15.3:0/4024899694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40a9610 con 0x7f4cd40ceb70 2024-09-12T12:26:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4ccbfff640 1 -- 172.21.15.3:0/4024899694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc40a97b0 con 0x7f4cd40ceb70 2024-09-12T12:26:01.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4ccbfff640 1 -- 172.21.15.3:0/4024899694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40a9b80 con 0x7f4cd40ceb70 2024-09-12T12:26:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- 172.21.15.3:0/4024899694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd40ceb70 msgr2=0x7f4cd415f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 --2- 172.21.15.3:0/4024899694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd40ceb70 0x7f4cd415f140 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009870 tx=0x7f4cc401c530 comp rx=0 tx=0).stop 2024-09-12T12:26:01.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- 172.21.15.3:0/4024899694 shutdown_connections 2024-09-12T12:26:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 --2- 172.21.15.3:0/4024899694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd40ceb70 0x7f4cd415f140 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 --2- 172.21.15.3:0/4024899694 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd415ed60 0x7f4cd406f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- 172.21.15.3:0/4024899694 >> 172.21.15.3:0/4024899694 conn(0x7f4cd40b26d0 msgr2=0x7f4cd40b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- 172.21.15.3:0/4024899694 shutdown_connections 2024-09-12T12:26:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.675+0000 7f4cdb714640 1 -- 172.21.15.3:0/4024899694 wait complete. 2024-09-12T12:26:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 Processor -- start 2024-09-12T12:26:01.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 -- start start 2024-09-12T12:26:01.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd40ceb70 0x7f4cd414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd415ed60 0x7f4cd4159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd415f680 0x7f4cd4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40ce540 con 0x7f4cd415ed60 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40ce3c0 con 0x7f4cd415f680 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd416b0d0 con 0x7f4cd40ceb70 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd415ed60 0x7f4cd4159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd415ed60 0x7f4cd4159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32878/0 (socket says 172.21.15.3:32878) 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 -- 172.21.15.3:0/2239220012 learned_addr learned my addr 172.21.15.3:0/2239220012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd94b0640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd40ceb70 0x7f4cd414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 -- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd40ceb70 msgr2=0x7f4cd414f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd40ceb70 0x7f4cd414f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 -- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd415f680 msgr2=0x7f4cd4159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd9cb1640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd415f680 0x7f4cd4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4cd415f680 0x7f4cd4159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 -- 172.21.15.3:0/2239220012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd41568e0 con 0x7f4cd415ed60 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd94b0640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4cd40ceb70 0x7f4cd414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cd8caf640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd415ed60 0x7f4cd4159a30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ab040 tx=0x7f4cc401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40bc650 con 0x7f4cd415ed60 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc40bc7f0 con 0x7f4cd415ed60 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd4156bc0 con 0x7f4cd415ed60 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40bcbe0 con 0x7f4cd415ed60 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd4157180 con 0x7f4cd415ed60 2024-09-12T12:26:01.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4cd415f140 con 0x7f4cd415ed60 2024-09-12T12:26:01.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.679+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4cc40c53f0 con 0x7f4cd415ed60 2024-09-12T12:26:01.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cc9ffb640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cb80678c0 0x7f4cb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f4cc41203c0 con 0x7f4cd415ed60 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cc9ffb640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f4cb806d960 0x7f4cb806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 --> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cc40b9e70 con 0x7f4cb806d960 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f4cc4127050 con 0x7f4cd415ed60 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cd94b0640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cb80678c0 0x7f4cb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cd9cb1640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f4cb806d960 0x7f4cb806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cd94b0640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cb80678c0 0x7f4cb8069d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4cd415a850 tx=0x7f4ccc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cd9cb1640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f4cb806d960 0x7f4cb806fd80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.683+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== osd.5 v2:172.21.15.142:6816/4110985109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4cc40b9e70 con 0x7f4cb806d960 2024-09-12T12:26:01.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 --> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4cd4073270 con 0x7f4cb806d960 2024-09-12T12:26:01.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cc9ffb640 1 -- 172.21.15.3:0/2239220012 <== osd.5 v2:172.21.15.142:6816/4110985109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4cd4073270 con 0x7f4cb806d960 2024-09-12T12:26:01.702 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:26:01.702 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:26:01.702 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f4cb806d960 msgr2=0x7f4cb806fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6816/4110985109,v1:172.21.15.142:6817/4110985109] conn(0x7f4cb806d960 0x7f4cb806fd80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cb80678c0 msgr2=0x7f4cb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f4cb80678c0 0x7f4cb8069d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4cd415a850 tx=0x7f4ccc00b040 comp rx=0 tx=0).stop 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd415ed60 msgr2=0x7f4cd4159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 --2- 172.21.15.3:0/2239220012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cd415ed60 0x7f4cd4159a30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ab040 tx=0x7f4cc401ad70 comp rx=0 tx=0).stop 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cd9cb1640 1 -- 172.21.15.3:0/2239220012 reap_dead start 2024-09-12T12:26:01.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 shutdown_connections 2024-09-12T12:26:01.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 >> 172.21.15.3:0/2239220012 conn(0x7f4cd40b26d0 msgr2=0x7f4cd406e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 shutdown_connections 2024-09-12T12:26:01.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.699+0000 7f4cdb714640 1 -- 172.21.15.3:0/2239220012 wait complete. 2024-09-12T12:26:01.718 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-12T12:26:01.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65cc2a5640 1 Processor -- start 2024-09-12T12:26:01.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65cc2a5640 1 -- start start 2024-09-12T12:26:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65cc2a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65c40cb870 0x7f65c40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65cc2a5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65cc2a5640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c40cc190 con 0x7f65c40b0200 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65cc2a5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f65c406de90 con 0x7f65c40cb870 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65cc2a5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f65c406e450 con 0x7f65c406fa80 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65ca041640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65c40cb870 0x7f65c40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65ca041640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f65c40cb870 0x7f65c40cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32890/0 (socket says 172.21.15.3:32890) 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65ca041640 1 -- 172.21.15.3:0/4287638245 learned_addr learned my addr 172.21.15.3:0/4287638245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65c9840640 1 --2- 172.21.15.3:0/4287638245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65ca041640 1 -- 172.21.15.3:0/4287638245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 msgr2=0x7f65c406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65ca041640 1 --2- 172.21.15.3:0/4287638245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c406e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65ca041640 1 -- 172.21.15.3:0/4287638245 >> v1:172.21.15.142:6789/0 conn(0x7f65c40b0200 legacy=0x7f65c406ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.835+0000 7f65ca041640 1 -- 172.21.15.3:0/4287638245 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c415f970 con 0x7f65c40cb870 2024-09-12T12:26:01.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65ca041640 1 --2- 172.21.15.3:0/4287638245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c40cbc50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0098a0 tx=0x7f65bc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630ca41899a1a66d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65c903f640 1 -- 172.21.15.3:0/4287638245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0ab600 con 0x7f65c40cb870 2024-09-12T12:26:01.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65c903f640 1 -- 172.21.15.3:0/4287638245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65bc0b0070 con 0x7f65c40cb870 2024-09-12T12:26:01.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65c903f640 1 -- 172.21.15.3:0/4287638245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0ab9c0 con 0x7f65c40cb870 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 -- 172.21.15.3:0/4287638245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 msgr2=0x7f65c40cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/4287638245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c40cbc50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0098a0 tx=0x7f65bc01c440 comp rx=0 tx=0).stop 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 -- 172.21.15.3:0/4287638245 shutdown_connections 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/4287638245 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c406e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/4287638245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c40cbc50 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 -- 172.21.15.3:0/4287638245 >> 172.21.15.3:0/4287638245 conn(0x7f65c40d1220 msgr2=0x7f65c40d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 -- 172.21.15.3:0/4287638245 shutdown_connections 2024-09-12T12:26:01.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.839+0000 7f65cc2a5640 1 -- 172.21.15.3:0/4287638245 wait complete. 2024-09-12T12:26:01.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 Processor -- start 2024-09-12T12:26:01.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 -- start start 2024-09-12T12:26:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c40b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c40b0200 0x7f65c4157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c4152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65c40b08c0 con 0x7f65c40cb870 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f65c40b0a40 con 0x7f65c40b0200 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f65c415c5b0 con 0x7f65c406fa80 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c4152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c4152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32894/0 (socket says 172.21.15.3:32894) 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 -- 172.21.15.3:0/3273042614 learned_addr learned my addr 172.21.15.3:0/3273042614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65c9840640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c40b0200 0x7f65c4157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca041640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c40b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 -- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 msgr2=0x7f65c40b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c40b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 -- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c40b0200 msgr2=0x7f65c4157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c40b0200 0x7f65c4157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 -- 172.21.15.3:0/3273042614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c4156b80 con 0x7f65c40cb870 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65c9840640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c40b0200 0x7f65c4157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65ca842640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c4152250 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f65b400c3f0 tx=0x7f65b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65c2ffd640 1 -- 172.21.15.3:0/3273042614 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4011790 con 0x7f65c40cb870 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65c2ffd640 1 -- 172.21.15.3:0/3273042614 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b4017070 con 0x7f65c40cb870 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c4156e60 con 0x7f65c40cb870 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65c2ffd640 1 -- 172.21.15.3:0/3273042614 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4011b70 con 0x7f65c40cb870 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c4157420 con 0x7f65c40cb870 2024-09-12T12:26:01.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.843+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c40cbc50 con 0x7f65c40cb870 2024-09-12T12:26:01.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.847+0000 7f65c2ffd640 1 -- 172.21.15.3:0/3273042614 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f65b4011d10 con 0x7f65c40cb870 2024-09-12T12:26:01.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.847+0000 7f65c2ffd640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65b00678c0 0x7f65b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:01.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.847+0000 7f65c2ffd640 1 -- 172.21.15.3:0/3273042614 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f65b4085ec0 con 0x7f65c40cb870 2024-09-12T12:26:01.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.847+0000 7f65ca041640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65b00678c0 0x7f65b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:01.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.847+0000 7f65ca041640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65b00678c0 0x7f65b0069d80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0af000 tx=0x7f65bc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:01.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:01.851+0000 7f65c2ffd640 1 -- 172.21.15.3:0/3273042614 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f65b4056ce0 con 0x7f65c40cb870 2024-09-12T12:26:02.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.015+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f65c40cf4a0 con 0x7f65b00678c0 2024-09-12T12:26:02.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.015+0000 7f65c2ffd640 1 -- 172.21.15.3:0/3273042614 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f65c40cf4a0 con 0x7f65b00678c0 2024-09-12T12:26:02.020 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 5 to deep-scrub 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65b00678c0 msgr2=0x7f65b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65b00678c0 0x7f65b0069d80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0af000 tx=0x7f65bc0a7000 comp rx=0 tx=0).stop 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 msgr2=0x7f65c4152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c4152250 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f65b400c3f0 tx=0x7f65b400c8b0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 shutdown_connections 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f65b00678c0 0x7f65b0069d80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f65c40cb870 0x7f65c4152250 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65c40b0200 0x7f65c4157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 --2- 172.21.15.3:0/3273042614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65c406fa80 0x7f65c40b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 >> 172.21.15.3:0/3273042614 conn(0x7f65c40d1220 msgr2=0x7f65c40cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 shutdown_connections 2024-09-12T12:26:02.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.019+0000 7f65cc2a5640 1 -- 172.21.15.3:0/3273042614 wait complete. 2024-09-12T12:26:02.040 INFO:tasks.ceph:Scrubbing osd.6 2024-09-12T12:26:02.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:26:02.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.167+0000 7f7c5910b640 1 Processor -- start 2024-09-12T12:26:02.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.167+0000 7f7c5910b640 1 -- start start 2024-09-12T12:26:02.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5910b640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7c5415e670 0x7f7c540cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5910b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 0x7f7c540d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5910b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c54162260 con 0x7f7c540d11d0 2024-09-12T12:26:02.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5910b640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f7c540d5490 con 0x7f7c5415e670 2024-09-12T12:26:02.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5910b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c54162540 con 0x7f7c540d4af0 2024-09-12T12:26:02.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5359e640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7c540d11d0 0x7f7c540d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38058/0 (socket says 172.21.15.3:38058) 2024-09-12T12:26:02.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5359e640 1 -- 172.21.15.3:0/2914930907 learned_addr learned my addr 172.21.15.3:0/2914930907 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:02.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5259c640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 0x7f7c540d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/2914930907 >> v2:172.21.15.142:3300/0 conn(0x7f7c5415e670 0x7f7c540cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 310919105 0 0) 0x7f7c54162260 con 0x7f7c540d11d0 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c3c003530 con 0x7f7c540d11d0 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4055677230 0 0) 0x7f7c3c003530 con 0x7f7c540d11d0 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 msgr2=0x7f7c540d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 0x7f7c540d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 >> v2:172.21.15.142:3300/0 conn(0x7f7c5415e670 msgr2=0x7f7c540cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 --2- 172.21.15.3:0/2914930907 >> v2:172.21.15.142:3300/0 conn(0x7f7c5415e670 0x7f7c540cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c54162820 con 0x7f7c540d11d0 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f7c4c001a00 con 0x7f7c540d11d0 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 >> v1:172.21.15.3:6789/0 conn(0x7f7c540d11d0 legacy=0x7f7c540d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c3c0039c0 0x7f7c3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3c004810 0x7f7c3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 0x7f7c3c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7c54162540 con 0x7f7c3c0039c0 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3c00b9d0 con 0x7f7c3c004810 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3c00bb80 con 0x7f7c540d4af0 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c3c0039c0 0x7f7c3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5259c640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3c004810 0x7f7c3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c5359e640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 0x7f7c3c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c52d9d640 1 -- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 msgr2=0x7f7c3c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c540d4af0 0x7f7c3c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c52d9d640 1 -- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3c004810 msgr2=0x7f7c3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c3c004810 0x7f7c3c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.171+0000 7f7c52d9d640 1 -- 172.21.15.3:0/2914930907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3c003530 con 0x7f7c3c0039c0 2024-09-12T12:26:02.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c3c0039c0 0x7f7c3c003de0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009ae0 tx=0x7f7c4401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2136e3fcb0c16e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c440096b0 con 0x7f7c3c0039c0 2024-09-12T12:26:02.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c440a64c0 con 0x7f7c3c0039c0 2024-09-12T12:26:02.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c51d9b640 1 -- 172.21.15.3:0/2914930907 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c440a68b0 con 0x7f7c3c0039c0 2024-09-12T12:26:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 -- 172.21.15.3:0/2914930907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c3c0039c0 msgr2=0x7f7c3c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 --2- 172.21.15.3:0/2914930907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c3c0039c0 0x7f7c3c003de0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009ae0 tx=0x7f7c4401ca50 comp rx=0 tx=0).stop 2024-09-12T12:26:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5359e640 1 -- 172.21.15.3:0/2914930907 reap_dead start 2024-09-12T12:26:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 -- 172.21.15.3:0/2914930907 shutdown_connections 2024-09-12T12:26:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 -- 172.21.15.3:0/2914930907 >> 172.21.15.3:0/2914930907 conn(0x7f7c54072410 msgr2=0x7f7c54072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 -- 172.21.15.3:0/2914930907 shutdown_connections 2024-09-12T12:26:02.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 -- 172.21.15.3:0/2914930907 wait complete. 2024-09-12T12:26:02.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 Processor -- start 2024-09-12T12:26:02.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.175+0000 7f7c5910b640 1 -- start start 2024-09-12T12:26:02.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c5415a180 0x7f7c540cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c541fac80 0x7f7c5415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c540b5260 0x7f7c540b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540cf280 con 0x7f7c5415a180 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540cf100 con 0x7f7c540b5260 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f7c540d4ed0 con 0x7f7c541fac80 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c5415a180 0x7f7c540cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c5415a180 0x7f7c540cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32910/0 (socket says 172.21.15.3:32910) 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 -- 172.21.15.3:0/1821789616 learned_addr learned my addr 172.21.15.3:0/1821789616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5259c640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c541fac80 0x7f7c5415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5359e640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c540b5260 0x7f7c540b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 -- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c541fac80 msgr2=0x7f7c5415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c541fac80 0x7f7c5415f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 -- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c540b5260 msgr2=0x7f7c540b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c540b5260 0x7f7c540b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 -- 172.21.15.3:0/1821789616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c541fb1f0 con 0x7f7c5415a180 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5259c640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7c541fac80 0x7f7c5415f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c52d9d640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c5415a180 0x7f7c540cd920 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7c440b0040 tx=0x7f7c3c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c440a6670 con 0x7f7c5415a180 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c440bf070 con 0x7f7c5415a180 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c541fb4d0 con 0x7f7c5415a180 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c440a6a30 con 0x7f7c5415a180 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c541fba90 con 0x7f7c5415a180 2024-09-12T12:26:02.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c540d51c0 con 0x7f7c5415a180 2024-09-12T12:26:02.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c440a6bd0 con 0x7f7c5415a180 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c400678c0 0x7f7c40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f7c441208b0 con 0x7f7c5415a180 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f7c4006d960 0x7f7c4006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 --> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c440a7ea0 con 0x7f7c4006d960 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f7c44120c60 con 0x7f7c5415a180 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5259c640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c400678c0 0x7f7c40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.179+0000 7f7c5359e640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f7c4006d960 0x7f7c4006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.183+0000 7f7c5359e640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f7c4006d960 0x7f7c4006fd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.183+0000 7f7c5259c640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c400678c0 0x7f7c40069d80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7c541fabe0 tx=0x7f7c48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.183+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== osd.6 v2:172.21.15.142:6808/3052505942 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c440a7ea0 con 0x7f7c4006d960 2024-09-12T12:26:02.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.195+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 --> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7c540ba740 con 0x7f7c4006d960 2024-09-12T12:26:02.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c3b7fe640 1 -- 172.21.15.3:0/1821789616 <== osd.6 v2:172.21.15.142:6808/3052505942 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7c540ba740 con 0x7f7c4006d960 2024-09-12T12:26:02.201 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:26:02.201 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:26:02.201 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:26:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f7c4006d960 msgr2=0x7f7c4006fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6808/3052505942,v1:172.21.15.142:6809/3052505942] conn(0x7f7c4006d960 0x7f7c4006fd80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c400678c0 msgr2=0x7f7c40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f7c400678c0 0x7f7c40069d80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7c541fabe0 tx=0x7f7c48008040 comp rx=0 tx=0).stop 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c5415a180 msgr2=0x7f7c540cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 --2- 172.21.15.3:0/1821789616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c5415a180 0x7f7c540cd920 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f7c440b0040 tx=0x7f7c3c005840 comp rx=0 tx=0).stop 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5359e640 1 -- 172.21.15.3:0/1821789616 reap_dead start 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 shutdown_connections 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 >> 172.21.15.3:0/1821789616 conn(0x7f7c54072410 msgr2=0x7f7c540b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 shutdown_connections 2024-09-12T12:26:02.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.199+0000 7f7c5910b640 1 -- 172.21.15.3:0/1821789616 wait complete. 2024-09-12T12:26:02.219 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-12T12:26:02.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edc700640 1 Processor -- start 2024-09-12T12:26:02.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edc700640 1 -- start start 2024-09-12T12:26:02.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edc700640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed4073cb0 0x7f2ed415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edc700640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ed40b30e0 0x7f2ed40adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edc700640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed40b5000 con 0x7f2ed40ae150 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edc700640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed4074090 con 0x7f2ed40b30e0 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edc700640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed40b3500 con 0x7f2ed4073cb0 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edac9d640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2ed40ae150 0x7f2ed40ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38074/0 (socket says 172.21.15.3:38074) 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2edac9d640 1 -- 172.21.15.3:0/162450119 learned_addr learned my addr 172.21.15.3:0/162450119 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2ed9c9b640 1 --2- 172.21.15.3:0/162450119 >> v2:172.21.15.142:3300/0 conn(0x7f2ed40b30e0 0x7f2ed40adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2eda49c640 1 --2- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed4073cb0 0x7f2ed415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2ed949a640 1 -- 172.21.15.3:0/162450119 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011130638 0 0) 0x7f2ed40b5000 con 0x7f2ed40ae150 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.331+0000 7f2ed949a640 1 -- 172.21.15.3:0/162450119 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ebc003530 con 0x7f2ed40ae150 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed9c9b640 1 -- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed4073cb0 msgr2=0x7f2ed415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed9c9b640 1 --2- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed4073cb0 0x7f2ed415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed9c9b640 1 -- 172.21.15.3:0/162450119 >> v1:172.21.15.3:6789/0 conn(0x7f2ed40ae150 legacy=0x7f2ed40ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed9c9b640 1 -- 172.21.15.3:0/162450119 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed40b37c0 con 0x7f2ed40b30e0 2024-09-12T12:26:02.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed9c9b640 1 --2- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40b30e0 0x7f2ed40adc10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d90 tx=0x7f2ec401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=effd3ac17447c8bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed949a640 1 -- 172.21.15.3:0/162450119 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40b2070 con 0x7f2ed40b30e0 2024-09-12T12:26:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed949a640 1 -- 172.21.15.3:0/162450119 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec40a5830 con 0x7f2ed40b30e0 2024-09-12T12:26:02.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2ed949a640 1 -- 172.21.15.3:0/162450119 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40a5be0 con 0x7f2ed40b30e0 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2edc700640 1 -- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40b30e0 msgr2=0x7f2ed40adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2edc700640 1 --2- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40b30e0 0x7f2ed40adc10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d90 tx=0x7f2ec401c970 comp rx=0 tx=0).stop 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2edc700640 1 -- 172.21.15.3:0/162450119 shutdown_connections 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2edc700640 1 --2- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40b30e0 0x7f2ed40adc10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2edc700640 1 --2- 172.21.15.3:0/162450119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed4073cb0 0x7f2ed415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.335+0000 7f2edc700640 1 -- 172.21.15.3:0/162450119 >> 172.21.15.3:0/162450119 conn(0x7f2ed4067e00 msgr2=0x7f2ed4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- 172.21.15.3:0/162450119 shutdown_connections 2024-09-12T12:26:02.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- 172.21.15.3:0/162450119 wait complete. 2024-09-12T12:26:02.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 Processor -- start 2024-09-12T12:26:02.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- start start 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed4073cb0 0x7f2ed40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40ae150 0x7f2ed40b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed40b30e0 0x7f2ed415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed40af110 con 0x7f2ed4073cb0 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed40aef90 con 0x7f2ed40ae150 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed4168dc0 con 0x7f2ed40b30e0 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed4073cb0 0x7f2ed40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed4073cb0 0x7f2ed40cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32912/0 (socket says 172.21.15.3:32912) 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 -- 172.21.15.3:0/1518632996 learned_addr learned my addr 172.21.15.3:0/1518632996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edac9d640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed40b30e0 0x7f2ed415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2ed9c9b640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40ae150 0x7f2ed40b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 -- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed40b30e0 msgr2=0x7f2ed415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed40b30e0 0x7f2ed415c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 -- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40ae150 msgr2=0x7f2ed40b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40ae150 0x7f2ed40b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 -- 172.21.15.3:0/1518632996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed40ccab0 con 0x7f2ed4073cb0 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2ed9c9b640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40ae150 0x7f2ed40b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2eda49c640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed4073cb0 0x7f2ed40cf3d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800e960 tx=0x7f2ec800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2ed2ffd640 1 -- 172.21.15.3:0/1518632996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8017070 con 0x7f2ed4073cb0 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2ed2ffd640 1 -- 172.21.15.3:0/1518632996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec800ca00 con 0x7f2ed4073cb0 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed40b23c0 con 0x7f2ed4073cb0 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2ed2ffd640 1 -- 172.21.15.3:0/1518632996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800cdb0 con 0x7f2ed4073cb0 2024-09-12T12:26:02.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.339+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed40b2980 con 0x7f2ed4073cb0 2024-09-12T12:26:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.343+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed40adc10 con 0x7f2ed4073cb0 2024-09-12T12:26:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.343+0000 7f2ed2ffd640 1 -- 172.21.15.3:0/1518632996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ec801a070 con 0x7f2ed4073cb0 2024-09-12T12:26:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.343+0000 7f2ed2ffd640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ec00678c0 0x7f2ec0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.343+0000 7f2ed2ffd640 1 -- 172.21.15.3:0/1518632996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f2ec8085f70 con 0x7f2ed4073cb0 2024-09-12T12:26:02.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.343+0000 7f2ed9c9b640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ec00678c0 0x7f2ec0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.343+0000 7f2ed9c9b640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ec00678c0 0x7f2ec0069d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40af000 tx=0x7f2ec40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.347+0000 7f2ed2ffd640 1 -- 172.21.15.3:0/1518632996 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f2ed40ccab0 con 0x7f2ed4073cb0 2024-09-12T12:26:02.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f2ed4155b70 con 0x7f2ec00678c0 2024-09-12T12:26:02.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2ed2ffd640 1 -- 172.21.15.3:0/1518632996 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2ed4155b70 con 0x7f2ec00678c0 2024-09-12T12:26:02.514 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 6 to deep-scrub 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ec00678c0 msgr2=0x7f2ec0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ec00678c0 0x7f2ec0069d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40af000 tx=0x7f2ec40a7070 comp rx=0 tx=0).stop 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed4073cb0 msgr2=0x7f2ed40cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed4073cb0 0x7f2ed40cf3d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800e960 tx=0x7f2ec800ee20 comp rx=0 tx=0).stop 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 shutdown_connections 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f2ec00678c0 0x7f2ec0069d80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ed40b30e0 0x7f2ed415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ed40ae150 0x7f2ed40b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 --2- 172.21.15.3:0/1518632996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed4073cb0 0x7f2ed40cf3d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 >> 172.21.15.3:0/1518632996 conn(0x7f2ed4067e00 msgr2=0x7f2ed4156090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 shutdown_connections 2024-09-12T12:26:02.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.511+0000 7f2edc700640 1 -- 172.21.15.3:0/1518632996 wait complete. 2024-09-12T12:26:02.532 INFO:tasks.ceph:Scrubbing osd.7 2024-09-12T12:26:02.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-12T12:26:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72f917640 1 Processor -- start 2024-09-12T12:26:02.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72f917640 1 -- start start 2024-09-12T12:26:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72f917640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7281523f0 0x7fa7280d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72f917640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 0x7fa7280d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72f917640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7280aee60 con 0x7fa7280ae2c0 2024-09-12T12:26:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72f917640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fa7280ad160 con 0x7fa7281523f0 2024-09-12T12:26:02.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72f917640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa7280ad440 con 0x7fa72815c910 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72deb4640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa7280ae2c0 0x7fa7280ae750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38084/0 (socket says 172.21.15.3:38084) 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72deb4640 1 -- 172.21.15.3:0/2238137154 learned_addr learned my addr 172.21.15.3:0/2238137154 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72d6b3640 1 --2- 172.21.15.3:0/2238137154 >> v2:172.21.15.142:3300/0 conn(0x7fa7281523f0 0x7fa7280d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72ceb2640 1 --2- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 0x7fa7280d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa71ffff640 1 -- 172.21.15.3:0/2238137154 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1286372900 0 0) 0x7fa7280aee60 con 0x7fa7280ae2c0 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa71ffff640 1 -- 172.21.15.3:0/2238137154 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa714003530 con 0x7fa7280ae2c0 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa71ffff640 1 -- 172.21.15.3:0/2238137154 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 310625680 0 0) 0x7fa714003530 con 0x7fa7280ae2c0 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72d6b3640 1 -- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 msgr2=0x7fa7280d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72d6b3640 1 --2- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 0x7fa7280d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72d6b3640 1 -- 172.21.15.3:0/2238137154 >> v1:172.21.15.3:6789/0 conn(0x7fa7280ae2c0 legacy=0x7fa7280ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.659+0000 7fa72d6b3640 1 -- 172.21.15.3:0/2238137154 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280ad720 con 0x7fa7281523f0 2024-09-12T12:26:02.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72d6b3640 1 --2- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7281523f0 0x7fa7280d03a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7240097c0 tx=0x7fa72401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aefbc7ab6c02c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa71ffff640 1 -- 172.21.15.3:0/2238137154 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7240b0070 con 0x7fa7281523f0 2024-09-12T12:26:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa71ffff640 1 -- 172.21.15.3:0/2238137154 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7240ab400 con 0x7fa7281523f0 2024-09-12T12:26:02.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa71ffff640 1 -- 172.21.15.3:0/2238137154 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720001a00 con 0x7fa7281523f0 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 -- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7281523f0 msgr2=0x7fa7280d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 --2- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7281523f0 0x7fa7280d03a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7240097c0 tx=0x7fa72401c390 comp rx=0 tx=0).stop 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 -- 172.21.15.3:0/2238137154 shutdown_connections 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 --2- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 0x7fa7280d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 --2- 172.21.15.3:0/2238137154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7281523f0 0x7fa7280d03a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 -- 172.21.15.3:0/2238137154 >> 172.21.15.3:0/2238137154 conn(0x7fa728067bd0 msgr2=0x7fa728067fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 -- 172.21.15.3:0/2238137154 shutdown_connections 2024-09-12T12:26:02.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.663+0000 7fa72f917640 1 -- 172.21.15.3:0/2238137154 wait complete. 2024-09-12T12:26:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 Processor -- start 2024-09-12T12:26:02.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 -- start start 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7280ae2c0 0x7fa72809d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7281523f0 0x7fa72816a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 0x7fa72815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa72815e960 con 0x7fa7281523f0 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa72815e7e0 con 0x7fa7280ae2c0 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fa72815eae0 con 0x7fa72815c910 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7281523f0 0x7fa72816a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7281523f0 0x7fa72816a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32918/0 (socket says 172.21.15.3:32918) 2024-09-12T12:26:02.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 -- 172.21.15.3:0/114216402 learned_addr learned my addr 172.21.15.3:0/114216402 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72d6b3640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7280ae2c0 0x7fa72809d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72deb4640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 0x7fa72815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 -- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 msgr2=0x7fa72815a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa72815c910 0x7fa72815a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 -- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7280ae2c0 msgr2=0x7fa72809d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7280ae2c0 0x7fa72809d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 -- 172.21.15.3:0/114216402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72815adf0 con 0x7fa7281523f0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72ceb2640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7281523f0 0x7fa72816a730 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa71800e9b0 tx=0x7fa71800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718017070 con 0x7fa7281523f0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71800c9a0 con 0x7fa7281523f0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa728169c50 con 0x7fa7281523f0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71800cd70 con 0x7fa7281523f0 2024-09-12T12:26:02.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72816a210 con 0x7fa7281523f0 2024-09-12T12:26:02.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7280ae750 con 0x7fa7281523f0 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa71801a070 con 0x7fa7281523f0 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.667+0000 7fa71dffb640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa70c0678c0 0x7fa70c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7fa7180863c0 con 0x7fa7281523f0 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa71dffb640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7fa70c06d960 0x7fa70c06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 --> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa70c070430 con 0x7fa70c06d960 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fa718086850 con 0x7fa7281523f0 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa72d6b3640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa70c0678c0 0x7fa70c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa72deb4640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7fa70c06d960 0x7fa70c06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa72d6b3640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa70c0678c0 0x7fa70c069d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7240ad000 tx=0x7fa7240a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa72deb4640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7fa70c06d960 0x7fa70c06fd80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.671+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== osd.7 v2:172.21.15.142:6824/177123471 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa70c070430 con 0x7fa70c06d960 2024-09-12T12:26:02.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 --> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa7280cc380 con 0x7fa70c06d960 2024-09-12T12:26:02.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa71dffb640 1 -- 172.21.15.3:0/114216402 <== osd.7 v2:172.21.15.142:6824/177123471 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa7280cc380 con 0x7fa70c06d960 2024-09-12T12:26:02.690 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T12:26:02.690 INFO:teuthology.orchestra.run.smithi003.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T12:26:02.690 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7fa70c06d960 msgr2=0x7fa70c06fd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6824/177123471,v1:172.21.15.142:6825/177123471] conn(0x7fa70c06d960 0x7fa70c06fd80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa70c0678c0 msgr2=0x7fa70c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fa70c0678c0 0x7fa70c069d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7240ad000 tx=0x7fa7240a6000 comp rx=0 tx=0).stop 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7281523f0 msgr2=0x7fa72816a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 --2- 172.21.15.3:0/114216402 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa7281523f0 0x7fa72816a730 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa71800e9b0 tx=0x7fa71800ee70 comp rx=0 tx=0).stop 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72deb4640 1 -- 172.21.15.3:0/114216402 reap_dead start 2024-09-12T12:26:02.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 shutdown_connections 2024-09-12T12:26:02.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 >> 172.21.15.3:0/114216402 conn(0x7fa728067bd0 msgr2=0x7fa72814d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 shutdown_connections 2024-09-12T12:26:02.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.687+0000 7fa72f917640 1 -- 172.21.15.3:0/114216402 wait complete. 2024-09-12T12:26:02.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-12T12:26:02.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5eef9a640 1 Processor -- start 2024-09-12T12:26:02.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5eef9a640 1 -- start start 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5eef9a640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb5e8073cb0 0x7fb5e815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5eef9a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e80b5000 con 0x7fb5e80ae880 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5eef9a640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5e80b3530 con 0x7fb5e80b30e0 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5eef9a640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e8074090 con 0x7fb5e8073cb0 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb5e80ae880 0x7fb5e80d05b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38098/0 (socket says 172.21.15.3:38098) 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 -- 172.21.15.3:0/736681468 learned_addr learned my addr 172.21.15.3:0/736681468 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1846462763 0 0) 0x7fb5e80b5000 con 0x7fb5e80ae880 2024-09-12T12:26:02.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d4003530 con 0x7fb5e80ae880 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ecd36640 1 --2- 172.21.15.3:0/736681468 >> v2:172.21.15.142:3300/0 conn(0x7fb5e8073cb0 0x7fb5e815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677313243 0 0) 0x7fb5d4003530 con 0x7fb5e80ae880 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 >> v2:172.21.15.142:3300/0 conn(0x7fb5e8073cb0 msgr2=0x7fb5e815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 --2- 172.21.15.3:0/736681468 >> v2:172.21.15.142:3300/0 conn(0x7fb5e8073cb0 0x7fb5e815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 >> v1:172.21.15.142:6789/0 conn(0x7fb5e80b30e0 legacy=0x7fb5e80adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e80b37c0 con 0x7fb5e80ae880 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fb5e0001a00 con 0x7fb5e80ae880 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ecd36640 1 --2- 172.21.15.3:0/736681468 >> v2:172.21.15.142:3300/0 conn(0x7fb5e8073cb0 0x7fb5e815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 >> v1:172.21.15.3:6789/0 conn(0x7fb5e80ae880 legacy=0x7fb5e80d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40039c0 0x7fb5d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4004810 0x7fb5d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d40051d0 0x7fb5d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d400c0f0 con 0x7fb5d40051d0 2024-09-12T12:26:02.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d400c2a0 con 0x7fb5d4004810 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d400c4a0 con 0x7fb5d40039c0 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d40051d0 0x7fb5d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5e7fff640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4004810 0x7fb5d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ecd36640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40039c0 0x7fb5d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 -- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40039c0 msgr2=0x7fb5d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40039c0 0x7fb5d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 -- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4004810 msgr2=0x7fb5d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5d4004810 0x7fb5d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.819+0000 7fb5ed537640 1 -- 172.21.15.3:0/736681468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d4003530 con 0x7fb5d40051d0 2024-09-12T12:26:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 -- 172.21.15.3:0/736681468 reap_dead start 2024-09-12T12:26:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d40051d0 0x7fb5d400b9d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000a860 tx=0x7fb5e001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473baa301f57118a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000a480 con 0x7fb5d40051d0 2024-09-12T12:26:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e00ad630 con 0x7fb5d40051d0 2024-09-12T12:26:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5e77fe640 1 -- 172.21.15.3:0/736681468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00ada00 con 0x7fb5d40051d0 2024-09-12T12:26:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- 172.21.15.3:0/736681468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d40051d0 msgr2=0x7fb5d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d40051d0 0x7fb5d400b9d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000a860 tx=0x7fb5e001d380 comp rx=0 tx=0).stop 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- 172.21.15.3:0/736681468 shutdown_connections 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/736681468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5d40051d0 0x7fb5d400b9d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- 172.21.15.3:0/736681468 >> 172.21.15.3:0/736681468 conn(0x7fb5e8067e00 msgr2=0x7fb5e8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- 172.21.15.3:0/736681468 shutdown_connections 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- 172.21.15.3:0/736681468 wait complete. 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 Processor -- start 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- start start 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40051d0 0x7fb5e80b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e81f1970 0x7fb5e815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e815f090 0x7fb5e80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80ad6f0 con 0x7fb5e815f090 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80b3d80 con 0x7fb5e81f1970 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5eef9a640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e815c710 con 0x7fb5d40051d0 2024-09-12T12:26:02.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e815f090 0x7fb5e80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e815f090 0x7fb5e80ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32940/0 (socket says 172.21.15.3:32940) 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 -- 172.21.15.3:0/1442231800 learned_addr learned my addr 172.21.15.3:0/1442231800 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ecd36640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40051d0 0x7fb5e80b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5e7fff640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e81f1970 0x7fb5e815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 -- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40051d0 msgr2=0x7fb5e80b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40051d0 0x7fb5e80b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 -- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e81f1970 msgr2=0x7fb5e815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e81f1970 0x7fb5e815eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ed537640 1 -- 172.21.15.3:0/1442231800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e80aff80 con 0x7fb5e815f090 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.823+0000 7fb5ecd36640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40051d0 0x7fb5e80b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5ed537640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e815f090 0x7fb5e80ce010 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb5e001d1b0 tx=0x7fb5d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5e57fa640 1 -- 172.21.15.3:0/1442231800 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000a480 con 0x7fb5e815f090 2024-09-12T12:26:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5e57fa640 1 -- 172.21.15.3:0/1442231800 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e00b4070 con 0x7fb5e815f090 2024-09-12T12:26:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e80b0260 con 0x7fb5e815f090 2024-09-12T12:26:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5e57fa640 1 -- 172.21.15.3:0/1442231800 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e00ad9e0 con 0x7fb5e815f090 2024-09-12T12:26:02.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e80b0820 con 0x7fb5e815f090 2024-09-12T12:26:02.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e80aecb0 con 0x7fb5e815f090 2024-09-12T12:26:02.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5e57fa640 1 -- 172.21.15.3:0/1442231800 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5e00b6020 con 0x7fb5e815f090 2024-09-12T12:26:02.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5e57fa640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c80678f0 0x7fb5c8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:02.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5e57fa640 1 -- 172.21.15.3:0/1442231800 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7fb5e011f740 con 0x7fb5e815f090 2024-09-12T12:26:02.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5ecd36640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c80678f0 0x7fb5c8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:02.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.827+0000 7fb5ecd36640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c80678f0 0x7fb5c8069db0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80b1e00 tx=0x7fb5d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:02.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.831+0000 7fb5e57fa640 1 -- 172.21.15.3:0/1442231800 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fb5e0017090 con 0x7fb5e815f090 2024-09-12T12:26:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb5e80afd30 con 0x7fb5c80678f0 2024-09-12T12:26:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5e57fa640 1 -- 172.21.15.3:0/1442231800 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb5e80afd30 con 0x7fb5c80678f0 2024-09-12T12:26:02.998 INFO:teuthology.orchestra.run.smithi003.stderr:instructed osd(s) 7 to deep-scrub 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c80678f0 msgr2=0x7fb5c8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c80678f0 0x7fb5c8069db0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80b1e00 tx=0x7fb5d8008040 comp rx=0 tx=0).stop 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e815f090 msgr2=0x7fb5e80ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e815f090 0x7fb5e80ce010 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb5e001d1b0 tx=0x7fb5d4005840 comp rx=0 tx=0).stop 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 shutdown_connections 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fb5c80678f0 0x7fb5c8069db0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5e815f090 0x7fb5e80ce010 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb5e81f1970 0x7fb5e815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 --2- 172.21.15.3:0/1442231800 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb5d40051d0 0x7fb5e80b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.995+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 >> 172.21.15.3:0/1442231800 conn(0x7fb5e8067e00 msgr2=0x7fb5e80cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.999+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 shutdown_connections 2024-09-12T12:26:03.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:02.999+0000 7fb5eef9a640 1 -- 172.21.15.3:0/1442231800 wait complete. 2024-09-12T12:26:03.016 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:26:03.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00dd701640 1 Processor -- start 2024-09-12T12:26:03.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00dd701640 1 -- start start 2024-09-12T12:26:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00dd701640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f00d815c6c0 0x7f00d8073200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00dd701640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00d8073740 0x7f00d80afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00dd701640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d8074280 con 0x7f00d8067a00 2024-09-12T12:26:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00dd701640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f00d8073cc0 con 0x7f00d8073740 2024-09-12T12:26:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00dd701640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f00d8073fa0 con 0x7f00d815c6c0 2024-09-12T12:26:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00d8073740 0x7f00d80afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:03.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d6ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f00d8067a00 0x7f00d8067de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38114/0 (socket says 172.21.15.3:38114) 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f00d8073740 0x7f00d80afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32954/0 (socket says 172.21.15.3:32954) 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d6ffd640 1 -- 172.21.15.3:0/3827544328 learned_addr learned my addr 172.21.15.3:0/3827544328 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d67fc640 1 --2- 172.21.15.3:0/3827544328 >> v2:172.21.15.142:3300/0 conn(0x7f00d815c6c0 0x7f00d8073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d77fe640 1 -- 172.21.15.3:0/3827544328 >> v2:172.21.15.142:3300/0 conn(0x7f00d815c6c0 msgr2=0x7f00d8073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d77fe640 1 --2- 172.21.15.3:0/3827544328 >> v2:172.21.15.142:3300/0 conn(0x7f00d815c6c0 0x7f00d8073200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d77fe640 1 -- 172.21.15.3:0/3827544328 >> v1:172.21.15.3:6789/0 conn(0x7f00d8067a00 legacy=0x7f00d8067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d5ffb640 1 -- 172.21.15.3:0/3827544328 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014028182 0 0) 0x7f00d8074280 con 0x7f00d8067a00 2024-09-12T12:26:03.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.123+0000 7f00d77fe640 1 -- 172.21.15.3:0/3827544328 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d80af220 con 0x7f00d8073740 2024-09-12T12:26:03.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00d77fe640 1 --2- 172.21.15.3:0/3827544328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d80afa20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f00d8072c10 tx=0x7f00d001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fc18c27b8b59fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:03.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00d5ffb640 1 -- 172.21.15.3:0/3827544328 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00b2070 con 0x7f00d8073740 2024-09-12T12:26:03.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00d5ffb640 1 -- 172.21.15.3:0/3827544328 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d00a57f0 con 0x7f00d8073740 2024-09-12T12:26:03.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00d5ffb640 1 -- 172.21.15.3:0/3827544328 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00a5bc0 con 0x7f00d8073740 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- 172.21.15.3:0/3827544328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 msgr2=0x7f00d80afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 --2- 172.21.15.3:0/3827544328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d80afa20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f00d8072c10 tx=0x7f00d001c930 comp rx=0 tx=0).stop 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- 172.21.15.3:0/3827544328 shutdown_connections 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 --2- 172.21.15.3:0/3827544328 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d80afa20 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 --2- 172.21.15.3:0/3827544328 >> v2:172.21.15.142:3300/0 conn(0x7f00d815c6c0 0x7f00d8073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- 172.21.15.3:0/3827544328 >> 172.21.15.3:0/3827544328 conn(0x7f00d80b2d80 msgr2=0x7f00d80b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- 172.21.15.3:0/3827544328 shutdown_connections 2024-09-12T12:26:03.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- 172.21.15.3:0/3827544328 wait complete. 2024-09-12T12:26:03.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 Processor -- start 2024-09-12T12:26:03.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- start start 2024-09-12T12:26:03.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00d8067a00 0x7f00d81538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:03.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d814f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d815c6c0 0x7f00d809a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f00d80b0b00 con 0x7f00d8073740 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f00d80b0980 con 0x7f00d815c6c0 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00dd701640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f00d806fec0 con 0x7f00d8067a00 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00d67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d814f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00d67fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d814f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:32956/0 (socket says 172.21.15.3:32956) 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.127+0000 7f00d67fc640 1 -- 172.21.15.3:0/1279336391 learned_addr learned my addr 172.21.15.3:0/1279336391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d67fc640 1 -- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00d8067a00 msgr2=0x7f00d81538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d67fc640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00d8067a00 0x7f00d81538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d67fc640 1 -- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d815c6c0 msgr2=0x7f00d809a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d67fc640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d815c6c0 0x7f00d809a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d67fc640 1 -- 172.21.15.3:0/1279336391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d814ca30 con 0x7f00d8073740 2024-09-12T12:26:03.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d67fc640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d814f600 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00d800 tx=0x7f00cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:03.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00bf7fe640 1 -- 172.21.15.3:0/1279336391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00b6c0 con 0x7f00d8073740 2024-09-12T12:26:03.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00bf7fe640 1 -- 172.21.15.3:0/1279336391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00cc019070 con 0x7f00d8073740 2024-09-12T12:26:03.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d815d810 con 0x7f00d8073740 2024-09-12T12:26:03.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00bf7fe640 1 -- 172.21.15.3:0/1279336391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc00ba60 con 0x7f00d8073740 2024-09-12T12:26:03.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d815ddd0 con 0x7f00d8073740 2024-09-12T12:26:03.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d8074280 con 0x7f00d8073740 2024-09-12T12:26:03.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00bf7fe640 1 -- 172.21.15.3:0/1279336391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00cc00bc00 con 0x7f00d8073740 2024-09-12T12:26:03.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00bf7fe640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00c40678c0 0x7f00c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:03.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00bf7fe640 1 -- 172.21.15.3:0/1279336391 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f00cc085f90 con 0x7f00d8073740 2024-09-12T12:26:03.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d6ffd640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00c40678c0 0x7f00c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:03.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.131+0000 7f00d6ffd640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00c40678c0 0x7f00c4069d80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00c80015a0 tx=0x7f00c8007970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:03.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.135+0000 7f00bf7fe640 1 -- 172.21.15.3:0/1279336391 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f00cc056cc0 con 0x7f00d8073740 2024-09-12T12:26:03.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.291+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00d814ffc0 con 0x7f00c40678c0 2024-09-12T12:26:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.291+0000 7f00bf7fe640 1 -- 172.21.15.3:0/1279336391 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13508 (secure 0 0 0) 0x7f00d814ffc0 con 0x7f00c40678c0 2024-09-12T12:26:03.296 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:26:03.296 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:26:03.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00c40678c0 msgr2=0x7f00c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00c40678c0 0x7f00c4069d80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00c80015a0 tx=0x7f00c8007970 comp rx=0 tx=0).stop 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 msgr2=0x7f00d814f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d814f600 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00d800 tx=0x7f00cc00dcc0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 shutdown_connections 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f00c40678c0 0x7f00c4069d80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00d815c6c0 0x7f00d809a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00d8073740 0x7f00d814f600 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 --2- 172.21.15.3:0/1279336391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f00d8067a00 0x7f00d81538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 >> 172.21.15.3:0/1279336391 conn(0x7f00d80b2d80 msgr2=0x7f00d80b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 shutdown_connections 2024-09-12T12:26:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:03.295+0000 7f00dd701640 1 -- 172.21.15.3:0/1279336391 wait complete. 2024-09-12T12:26:03.313 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-12T12:26:01.784985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260360,"kb_used_data":10856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714360,"statfs":{"total":773094113280,"available":772827504640,"internally_reserved":0,"allocated":11116544,"data_stored":5435736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27840,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-95,"num_read_kb":-89,"num_write":-169,"num_write_kb":-86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002473"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":315,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T12:25:56.748092+0000","last_change":"2024-09-12T12:19:39.803509+0000","last_active":"2024-09-12T12:25:56.748092+0000","last_peered":"2024-09-12T12:25:56.748092+0000","last_clean":"2024-09-12T12:25:56.748092+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:25:56.748092+0000","last_undegraded":"2024-09-12T12:25:56.748092+0000","last_fullsized":"2024-09-12T12:25:56.748092+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:19:38.173260+0000","last_clean_scrub_stamp":"2024-09-12T12:19:38.173260+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:18:03.970284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509520,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542224,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1835008,"data_stored":1122243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542222,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1835008,"data_stored":1122243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:26:03.316 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-12T12:19:38.173260+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=19, tm_sec=38, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=25, tm_sec=58, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T12:26:03.316 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-12T12:26:17.551 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:26:17.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:26:23.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T12:26:23.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc9032d0640 1 Processor -- start 2024-09-12T12:26:23.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc9032d0640 1 -- start start 2024-09-12T12:26:23.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc9032d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc15c710 0x7fc8fc0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc9032d0640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc8fc0ae880 0x7fc8fc0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc9032d0640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8fc0b5000 con 0x7fc8fc073cb0 2024-09-12T12:26:23.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc9032d0640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7fc8fc074090 con 0x7fc8fc0ae880 2024-09-12T12:26:23.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc9032d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8fc0b37c0 con 0x7fc8fc15c710 2024-09-12T12:26:23.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90106c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc8fc073cb0 0x7fc8fc0adc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39918/0 (socket says 172.21.15.3:39918) 2024-09-12T12:26:23.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90106c640 1 -- 172.21.15.3:0/3159937866 learned_addr learned my addr 172.21.15.3:0/3159937866 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90086b640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc15c710 0x7fc8fc0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3512741368 0 0) 0x7fc8fc0b5000 con 0x7fc8fc073cb0 2024-09-12T12:26:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90186d640 1 --2- 172.21.15.3:0/3159937866 >> v2:172.21.15.142:3300/0 conn(0x7fc8fc0ae880 0x7fc8fc0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8e8003530 con 0x7fc8fc073cb0 2024-09-12T12:26:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1979797342 0 0) 0x7fc8e8003530 con 0x7fc8fc073cb0 2024-09-12T12:26:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc15c710 msgr2=0x7fc8fc0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:23.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc15c710 0x7fc8fc0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 >> v2:172.21.15.142:3300/0 conn(0x7fc8fc0ae880 msgr2=0x7fc8fc0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 --2- 172.21.15.3:0/3159937866 >> v2:172.21.15.142:3300/0 conn(0x7fc8fc0ae880 0x7fc8fc0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc0b3aa0 con 0x7fc8fc073cb0 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7fc8f8001680 con 0x7fc8fc073cb0 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 >> v1:172.21.15.3:6789/0 conn(0x7fc8fc073cb0 legacy=0x7fc8fc0adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80039c0 0x7fc8e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8e8004810 0x7fc8e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8e80051d0 0x7fc8e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc0b37c0 con 0x7fc8e80051d0 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e800bf10 con 0x7fc8e80039c0 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e800c0c0 con 0x7fc8e8004810 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90086b640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc15c710 0x7fc8fc0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90186d640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8e80051d0 0x7fc8e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90106c640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80039c0 0x7fc8e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90086b640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8e8004810 0x7fc8e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90186d640 1 -- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8e8004810 msgr2=0x7fc8e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90186d640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8e8004810 0x7fc8e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90186d640 1 -- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80039c0 msgr2=0x7fc8e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90186d640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8e80039c0 0x7fc8e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.450+0000 7fc90186d640 1 -- 172.21.15.3:0/3159937866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e8003530 con 0x7fc8e80051d0 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc90186d640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8e80051d0 0x7fc8e800b9d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009fc0 tx=0x7fc8f401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6bd39bae766c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40b2070 con 0x7fc8e80051d0 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f40a5cd0 con 0x7fc8e80051d0 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc8f3fff640 1 -- 172.21.15.3:0/3159937866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4009c10 con 0x7fc8e80051d0 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 -- 172.21.15.3:0/3159937866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8e80051d0 msgr2=0x7fc8e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 --2- 172.21.15.3:0/3159937866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8e80051d0 0x7fc8e800b9d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009fc0 tx=0x7fc8f401ccc0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc90186d640 1 -- 172.21.15.3:0/3159937866 reap_dead start 2024-09-12T12:26:23.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 -- 172.21.15.3:0/3159937866 shutdown_connections 2024-09-12T12:26:23.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 -- 172.21.15.3:0/3159937866 >> 172.21.15.3:0/3159937866 conn(0x7fc8fc067e00 msgr2=0x7fc8fc068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:23.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 -- 172.21.15.3:0/3159937866 shutdown_connections 2024-09-12T12:26:23.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 -- 172.21.15.3:0/3159937866 wait complete. 2024-09-12T12:26:23.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 Processor -- start 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.454+0000 7fc9032d0640 1 -- start start 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8fc0b2880 0x7fc8fc1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8fc14eba0 0x7fc8fc1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc157e30 0x7fc8fc1f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc0ae320 con 0x7fc8fc0b2880 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc0ad6f0 con 0x7fc8fc14eba0 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fc8fc0ae4a0 con 0x7fc8fc157e30 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8fc0b2880 0x7fc8fc1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8fc0b2880 0x7fc8fc1556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42156/0 (socket says 172.21.15.3:42156) 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 -- 172.21.15.3:0/1812487133 learned_addr learned my addr 172.21.15.3:0/1812487133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90086b640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8fc14eba0 0x7fc8fc1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90186d640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc157e30 0x7fc8fc1f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 -- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc157e30 msgr2=0x7fc8fc1f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc157e30 0x7fc8fc1f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 -- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8fc14eba0 msgr2=0x7fc8fc1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8fc14eba0 0x7fc8fc1578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 -- 172.21.15.3:0/1812487133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc155d50 con 0x7fc8fc0b2880 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90106c640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8fc0b2880 0x7fc8fc1556c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800eaf0 tx=0x7fc8f80016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc8f1ffb640 1 -- 172.21.15.3:0/1812487133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800c8d0 con 0x7fc8fc0b2880 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc8f1ffb640 1 -- 172.21.15.3:0/1812487133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f8018070 con 0x7fc8fc0b2880 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc156030 con 0x7fc8fc0b2880 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc8f1ffb640 1 -- 172.21.15.3:0/1812487133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800ccb0 con 0x7fc8fc0b2880 2024-09-12T12:26:23.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc1582b0 con 0x7fc8fc0b2880 2024-09-12T12:26:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8fc0aecd0 con 0x7fc8fc0b2880 2024-09-12T12:26:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc8f1ffb640 1 -- 172.21.15.3:0/1812487133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8f8013420 con 0x7fc8fc0b2880 2024-09-12T12:26:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc8f1ffb640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8e00678f0 0x7fc8e0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc8f1ffb640 1 -- 172.21.15.3:0/1812487133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7fc8f8085680 con 0x7fc8fc0b2880 2024-09-12T12:26:23.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.458+0000 7fc90086b640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8e00678f0 0x7fc8e0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.462+0000 7fc90086b640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8e00678f0 0x7fc8e0069db0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc06ead0 tx=0x7fc8ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:23.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.462+0000 7fc8f1ffb640 1 -- 172.21.15.3:0/1812487133 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fc8f8056440 con 0x7fc8fc0b2880 2024-09-12T12:26:23.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.626+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 --> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8fc0ad310 con 0x7fc8e00678f0 2024-09-12T12:26:23.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.626+0000 7fc8f1ffb640 1 -- 172.21.15.3:0/1812487133 <== mgr.4105 v2:172.21.15.142:6832/4257875619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fc8fc0ad310 con 0x7fc8e00678f0 2024-09-12T12:26:23.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:26:23.632 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T12:26:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8e00678f0 msgr2=0x7fc8e0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8e00678f0 0x7fc8e0069db0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc06ead0 tx=0x7fc8ec008040 comp rx=0 tx=0).stop 2024-09-12T12:26:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8fc0b2880 msgr2=0x7fc8fc1556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8fc0b2880 0x7fc8fc1556c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800eaf0 tx=0x7fc8f80016f0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 shutdown_connections 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fc8e00678f0 0x7fc8e0069db0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc8fc157e30 0x7fc8fc1f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8fc14eba0 0x7fc8fc1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 --2- 172.21.15.3:0/1812487133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc8fc0b2880 0x7fc8fc1556c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 >> 172.21.15.3:0/1812487133 conn(0x7fc8fc067e00 msgr2=0x7fc8fc15b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 shutdown_connections 2024-09-12T12:26:23.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.630+0000 7fc9032d0640 1 -- 172.21.15.3:0/1812487133 wait complete. 2024-09-12T12:26:23.651 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-12T12:26:21.790291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260352,"kb_used_data":10856,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714368,"statfs":{"total":773094113280,"available":772827512832,"internally_reserved":0,"allocated":11116544,"data_stored":5435736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003363"},"pg_stats":[{"pgid":"1.0","version":"100'119","reported_seq":326,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-12T12:26:03.199275+0000","last_change":"2024-09-12T12:26:03.199357+0000","last_active":"2024-09-12T12:26:03.199275+0000","last_peered":"2024-09-12T12:26:03.199275+0000","last_clean":"2024-09-12T12:26:03.199275+0000","last_became_active":"2024-09-12T12:19:39.801177+0000","last_became_peered":"2024-09-12T12:19:39.801177+0000","last_unstale":"2024-09-12T12:26:03.199275+0000","last_undegraded":"2024-09-12T12:26:03.199275+0000","last_fullsized":"2024-09-12T12:26:03.199275+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"100'119","last_scrub_stamp":"2024-09-12T12:26:03.199054+0000","last_deep_scrub":"100'119","last_deep_scrub_stamp":"2024-09-12T12:26:03.199054+0000","last_clean_scrub_stamp":"2024-09-12T12:26:03.199054+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:01.306844+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542228,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1835008,"data_stored":1122243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542226,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":531875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542226,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1835008,"data_stored":1122243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:26:23.651 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-12T12:26:23.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb085a5640 1 Processor -- start 2024-09-12T12:26:23.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb085a5640 1 -- start start 2024-09-12T12:26:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb085a5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb000ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb085a5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb0006e670 0x7fbb0006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb085a5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb000cd4f0 con 0x7fbb0015ad40 2024-09-12T12:26:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb085a5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbb000b53e0 con 0x7fbb0006e670 2024-09-12T12:26:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb085a5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb000b4b40 con 0x7fbb000ccbd0 2024-09-12T12:26:23.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb06b42640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbb0006e670 0x7fbb0006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb05b40640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fbb0015ad40 0x7fbb0015b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39934/0 (socket says 172.21.15.3:39934) 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb05b40640 1 -- 172.21.15.3:0/229988513 learned_addr learned my addr 172.21.15.3:0/229988513 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb06341640 1 --2- 172.21.15.3:0/229988513 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb000ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb06b42640 1 -- 172.21.15.3:0/229988513 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 msgr2=0x7fbb000ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb06b42640 1 --2- 172.21.15.3:0/229988513 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb000ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb06b42640 1 -- 172.21.15.3:0/229988513 >> v1:172.21.15.3:6789/0 conn(0x7fbb0015ad40 legacy=0x7fbb0015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb06b42640 1 -- 172.21.15.3:0/229988513 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb000b4e20 con 0x7fbb0006e670 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb06341640 1 --2- 172.21.15.3:0/229988513 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb000ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:23.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.762+0000 7fbb0533f640 1 -- 172.21.15.3:0/229988513 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926319221 0 0) 0x7fbb000cd4f0 con 0x7fbb0015ad40 2024-09-12T12:26:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06b42640 1 --2- 172.21.15.3:0/229988513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0006cd50 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009d90 tx=0x7fbaf801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecaa83e22aca8a39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb0533f640 1 -- 172.21.15.3:0/229988513 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80b2070 con 0x7fbb0006e670 2024-09-12T12:26:23.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb0533f640 1 -- 172.21.15.3:0/229988513 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf80a58e0 con 0x7fbb0006e670 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb0533f640 1 -- 172.21.15.3:0/229988513 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80a5cb0 con 0x7fbb0006e670 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- 172.21.15.3:0/229988513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 msgr2=0x7fbb0006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 --2- 172.21.15.3:0/229988513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0006cd50 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009d90 tx=0x7fbaf801c970 comp rx=0 tx=0).stop 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- 172.21.15.3:0/229988513 shutdown_connections 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 --2- 172.21.15.3:0/229988513 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0006cd50 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 --2- 172.21.15.3:0/229988513 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb000ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- 172.21.15.3:0/229988513 >> 172.21.15.3:0/229988513 conn(0x7fbb00067e00 msgr2=0x7fbb00068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- 172.21.15.3:0/229988513 shutdown_connections 2024-09-12T12:26:23.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- 172.21.15.3:0/229988513 wait complete. 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 Processor -- start 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- start start 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0015c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb0014a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb0015ad40 0x7fbb00144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0006ec40 con 0x7fbb0006e670 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0006eac0 con 0x7fbb0015ad40 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb085a5640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7fbb000cfdd0 con 0x7fbb000ccbd0 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0015c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42168/0 (socket says 172.21.15.3:42168) 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 -- 172.21.15.3:0/3674164717 learned_addr learned my addr 172.21.15.3:0/3674164717 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:23.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb05b40640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb0014a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06b42640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb0015ad40 0x7fbb00144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 -- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 msgr2=0x7fbb0014a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb0014a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 -- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb0015ad40 msgr2=0x7fbb00144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb0015ad40 0x7fbb00144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 -- 172.21.15.3:0/3674164717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0015c7e0 con 0x7fbb0006e670 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb05b40640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb0014a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06b42640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb0015ad40 0x7fbb00144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.766+0000 7fbb06341640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0015c150 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c7c0 tx=0x7fbafc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007a30 con 0x7fbb0006e670 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc007bd0 con 0x7fbb0006e670 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc0113f0 con 0x7fbb0006e670 2024-09-12T12:26:23.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0015cac0 con 0x7fbb0006e670 2024-09-12T12:26:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0015d080 con 0x7fbb0006e670 2024-09-12T12:26:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0006cd50 con 0x7fbb0006e670 2024-09-12T12:26:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbafc011590 con 0x7fbb0006e670 2024-09-12T12:26:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbaf6ffd640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbae40678c0 0x7fbae4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:23.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7fbafc085ca0 con 0x7fbb0006e670 2024-09-12T12:26:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbb05b40640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbae40678c0 0x7fbae4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:23.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.770+0000 7fbb05b40640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbae40678c0 0x7fbae4069d80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0007fd0 tx=0x7fbaf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:23.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.774+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7fbafc015020 con 0x7fbb0006e670 2024-09-12T12:26:23.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.930+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fbb00157320 con 0x7fbb0006e670 2024-09-12T12:26:23.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.938+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v26)=0 v26) ==== 123+0+0 (secure 0 0 0) 0x7fbafc0569e0 con 0x7fbb0006e670 2024-09-12T12:26:23.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.938+0000 7fbaf6ffd640 1 -- 172.21.15.3:0/3674164717 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbafc0562e0 con 0x7fbb0006e670 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.938+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbae40678c0 msgr2=0x7fbae4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.938+0000 7fbb085a5640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbae40678c0 0x7fbae4069d80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0007fd0 tx=0x7fbaf0009040 comp rx=0 tx=0).stop 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 msgr2=0x7fbb0015c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0015c150 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c7c0 tx=0x7fbafc00cc80 comp rx=0 tx=0).stop 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 shutdown_connections 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7fbae40678c0 0x7fbae4069d80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb0015ad40 0x7fbb00144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb000ccbd0 0x7fbb0014a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 --2- 172.21.15.3:0/3674164717 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb0006e670 0x7fbb0015c150 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:23.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 >> 172.21.15.3:0/3674164717 conn(0x7fbb00067e00 msgr2=0x7fbb00159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:23.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 shutdown_connections 2024-09-12T12:26:23.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:23.942+0000 7fbb085a5640 1 -- 172.21.15.3:0/3674164717 wait complete. 2024-09-12T12:26:23.960 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:26:23.960 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-12T12:26:24.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd7155640 1 Processor -- start 2024-09-12T12:26:24.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd7155640 1 -- start start 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd7155640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 0x7f0dd00d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd7155640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0dd00ae250 0x7f0dd00ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd7155640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd015c630 con 0x7f0dd00b42f0 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd7155640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd00d0ed0 con 0x7f0dd00ae250 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd7155640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd00ad0c0 con 0x7f0dd00d05b0 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd56f2640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0dd00b42f0 0x7f0dd00b47a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:39944/0 (socket says 172.21.15.3:39944) 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd56f2640 1 -- 172.21.15.3:0/392217958 learned_addr learned my addr 172.21.15.3:0/392217958 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd4ef1640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 0x7f0dd00d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcffff640 1 --2- 172.21.15.3:0/392217958 >> v2:172.21.15.142:3300/0 conn(0x7f0dd00ae250 0x7f0dd00ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030421954 0 0) 0x7f0dc80014a0 con 0x7f0dd00b42f0 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0db8003530 con 0x7f0dd00b42f0 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 728075405 0 0) 0x7f0db8003530 con 0x7f0dd00b42f0 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 msgr2=0x7f0dd00d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 0x7f0dd00d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 >> v2:172.21.15.142:3300/0 conn(0x7f0dd00ae250 msgr2=0x7f0dd00ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 --2- 172.21.15.3:0/392217958 >> v2:172.21.15.142:3300/0 conn(0x7f0dd00ae250 0x7f0dd00ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd00ad3a0 con 0x7f0dd00b42f0 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2740672686 0 0) 0x7f0dc8001bd0 con 0x7f0dd00b42f0 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 >> v1:172.21.15.3:6789/0 conn(0x7f0dd00b42f0 legacy=0x7f0dd00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 0x7f0db80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8003f40 0x7f0db8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8004d50 0x7f0db80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd00d0ed0 con 0x7f0db8003f40 2024-09-12T12:26:24.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd00ad0c0 con 0x7f0db8004d50 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0db800b9d0 con 0x7f0dd00d05b0 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcffff640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8003f40 0x7f0db8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd56f2640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8004d50 0x7f0db80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd4ef1640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 0x7f0db80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcffff640 1 -- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 msgr2=0x7f0db80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcffff640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00d05b0 0x7f0db80039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcffff640 1 -- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8004d50 msgr2=0x7f0db80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcffff640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8004d50 0x7f0db80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dcffff640 1 -- 172.21.15.3:0/392217958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db8003530 con 0x7f0db8003f40 2024-09-12T12:26:24.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.078+0000 7f0dd56f2640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8004d50 0x7f0db80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:24.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dcffff640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8003f40 0x7f0db8004810 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009ab0 tx=0x7f0dc401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff22bb95060b918f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:24.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40b2070 con 0x7f0db8003f40 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0dc40a5570 con 0x7f0db8003f40 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dcf7fe640 1 -- 172.21.15.3:0/392217958 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40096b0 con 0x7f0db8003f40 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- 172.21.15.3:0/392217958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8003f40 msgr2=0x7f0db8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 --2- 172.21.15.3:0/392217958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8003f40 0x7f0db8004810 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009ab0 tx=0x7f0dc401ca30 comp rx=0 tx=0).stop 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd56f2640 1 -- 172.21.15.3:0/392217958 reap_dead start 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- 172.21.15.3:0/392217958 shutdown_connections 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- 172.21.15.3:0/392217958 >> 172.21.15.3:0/392217958 conn(0x7f0dd0067e00 msgr2=0x7f0dd0068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- 172.21.15.3:0/392217958 shutdown_connections 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- 172.21.15.3:0/392217958 wait complete. 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 Processor -- start 2024-09-12T12:26:24.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- start start 2024-09-12T12:26:24.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8003f40 0x7f0dd00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd014aa00 0x7f0dd00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00afc10 0x7f0dd0151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd00adc40 con 0x7f0dd014aa00 2024-09-12T12:26:24.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd00ad960 con 0x7f0db8003f40 2024-09-12T12:26:24.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.082+0000 7f0dd7155640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd00aebf0 con 0x7f0dd00afc10 2024-09-12T12:26:24.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd014aa00 0x7f0dd00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd014aa00 0x7f0dd00ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42180/0 (socket says 172.21.15.3:42180) 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 -- 172.21.15.3:0/1556539339 learned_addr learned my addr 172.21.15.3:0/1556539339 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dd56f2640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00afc10 0x7f0dd0151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dd4ef1640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8003f40 0x7f0dd00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 -- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00afc10 msgr2=0x7f0dd0151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00afc10 0x7f0dd0151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 -- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8003f40 msgr2=0x7f0dd00b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8003f40 0x7f0dd00b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 -- 172.21.15.3:0/1556539339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd00b03f0 con 0x7f0dd014aa00 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dd56f2640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00afc10 0x7f0dd0151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcffff640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd014aa00 0x7f0dd00ce760 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c10 tx=0x7f0dc4019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcd7fa640 1 -- 172.21.15.3:0/1556539339 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40bd070 con 0x7f0dd014aa00 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcd7fa640 1 -- 172.21.15.3:0/1556539339 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f0dc40ad440 con 0x7f0dd014aa00 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd00b06d0 con 0x7f0dd014aa00 2024-09-12T12:26:24.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcd7fa640 1 -- 172.21.15.3:0/1556539339 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40b2040 con 0x7f0dd014aa00 2024-09-12T12:26:24.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd00b0c90 con 0x7f0dd014aa00 2024-09-12T12:26:24.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd01559b0 con 0x7f0dd014aa00 2024-09-12T12:26:24.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcd7fa640 1 -- 172.21.15.3:0/1556539339 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0dc40c0070 con 0x7f0dd014aa00 2024-09-12T12:26:24.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcd7fa640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0dbc067860 0x7f0dbc069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:24.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dcd7fa640 1 -- 172.21.15.3:0/1556539339 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5201+0+0 (secure 0 0 0) 0x7f0dc411f8e0 con 0x7f0dd014aa00 2024-09-12T12:26:24.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.086+0000 7f0dd4ef1640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0dbc067860 0x7f0dbc069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:24.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.090+0000 7f0dd4ef1640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0dbc067860 0x7f0dbc069d20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009950 tx=0x7f0dc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:24.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.090+0000 7f0dcd7fa640 1 -- 172.21.15.3:0/1556539339 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180250 (secure 0 0 0) 0x7f0dc40167a0 con 0x7f0dd014aa00 2024-09-12T12:26:24.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.286+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dd00cecc0 con 0x7f0dd014aa00 2024-09-12T12:26:24.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.286+0000 7f0dcd7fa640 1 -- 172.21.15.3:0/1556539339 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3908 (secure 0 0 0) 0x7f0dc40efe90 con 0x7f0dd014aa00 2024-09-12T12:26:24.292 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T12:26:24.293 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":76,"btime":"2024-09-12T12:25:52:586750+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5573,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/4160172977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":4160172977},{"type":"v1","addr":"172.21.15.142:6837","nonce":4160172977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6169,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/528245724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":528245724},{"type":"v1","addr":"172.21.15.142:6835","nonce":528245724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":7215,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6833/1632302565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1632302565},{"type":"v1","addr":"172.21.15.3:6833","nonce":1632302565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2033577621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2033577621},{"type":"v1","addr":"172.21.15.3:6835","nonce":2033577621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-09-12T12:26:24.293 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 76 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0dbc067860 msgr2=0x7f0dbc069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0dbc067860 0x7f0dbc069d20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009950 tx=0x7f0dc0008040 comp rx=0 tx=0).stop 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd014aa00 msgr2=0x7f0dd00ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd014aa00 0x7f0dd00ce760 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c10 tx=0x7f0dc4019830 comp rx=0 tx=0).stop 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 shutdown_connections 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:6832/4257875619,v1:172.21.15.142:6833/4257875619] conn(0x7f0dbc067860 0x7f0dbc069d20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0dd00afc10 0x7f0dd0151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0dd014aa00 0x7f0dd00ce760 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 --2- 172.21.15.3:0/1556539339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0db8003f40 0x7f0dd00b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 >> 172.21.15.3:0/1556539339 conn(0x7f0dd0067e00 msgr2=0x7f0dd0073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:24.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 shutdown_connections 2024-09-12T12:26:24.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T12:26:24.290+0000 7f0dd7155640 1 -- 172.21.15.3:0/1556539339 wait complete. 2024-09-12T12:26:24.310 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-12T12:26:24.311 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-12T12:26:24.311 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-12T12:26:24.311 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:24.319 INFO:tasks.ceph.mds.a:Stopped 2024-09-12T12:26:24.320 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-12T12:26:24.320 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:24.375 INFO:tasks.ceph.mds.c:Stopped 2024-09-12T12:26:24.375 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-12T12:26:24.375 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:24.393 INFO:tasks.ceph.mds.b:Stopped 2024-09-12T12:26:24.393 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-12T12:26:24.393 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:24.451 INFO:tasks.ceph.mds.d:Stopped 2024-09-12T12:26:24.451 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-12T12:26:24.451 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-12T12:26:24.451 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:24.539 INFO:tasks.ceph.osd.0:Stopped 2024-09-12T12:26:24.539 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-12T12:26:24.540 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:24.636 INFO:tasks.ceph.osd.1:Stopped 2024-09-12T12:26:24.636 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-12T12:26:24.636 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:28.035 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-12T12:26:30.738 INFO:tasks.ceph.osd.2:Stopped 2024-09-12T12:26:30.738 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-12T12:26:30.738 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:30.796 INFO:tasks.ceph.osd.3:Stopped 2024-09-12T12:26:30.796 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-12T12:26:30.796 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:30.895 INFO:tasks.ceph.osd.4:Stopped 2024-09-12T12:26:30.895 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-12T12:26:30.895 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:30.953 INFO:tasks.ceph.osd.5:Stopped 2024-09-12T12:26:30.953 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-12T12:26:30.953 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:31.015 INFO:tasks.ceph.osd.6:Stopped 2024-09-12T12:26:31.015 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-12T12:26:31.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:31.083 INFO:tasks.ceph.osd.7:Stopped 2024-09-12T12:26:31.083 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-12T12:26:31.084 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-12T12:26:31.084 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:31.112 INFO:tasks.ceph.mgr.y:Stopped 2024-09-12T12:26:31.113 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-12T12:26:31.113 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:31.142 INFO:tasks.ceph.mgr.x:Stopped 2024-09-12T12:26:31.142 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-12T12:26:31.142 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-12T12:26:31.142 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:31.178 INFO:tasks.ceph.mon.a:Stopped 2024-09-12T12:26:31.179 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-12T12:26:31.179 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:31.203 INFO:tasks.ceph.mon.b:Stopped 2024-09-12T12:26:31.204 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-12T12:26:31.204 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T12:26:31.266 INFO:tasks.ceph.mon.c:Stopped 2024-09-12T12:26:31.266 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-12T12:26:31.267 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'client session with non-allowable root' | head -n 1 2024-09-12T12:26:31.286 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:26:31.286 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-12T12:26:31.547 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:26:31.547 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-12T12:26:31.754 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:26:31.755 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-12T12:26:31.928 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:26:31.929 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-12T12:26:32.103 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:26:32.103 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-12T12:26:32.370 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:26:32.370 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-12T12:26:32.533 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:26:32.533 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-12T12:26:32.701 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:26:32.701 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-12T12:26:32.858 INFO:tasks.ceph:Archiving mon data... 2024-09-12T12:26:32.860 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/data/mon.a.tgz 2024-09-12T12:26:32.860 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-12T12:26:32.866 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.vEXuPyywfw 2024-09-12T12:26:32.866 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vEXuPyywfw 2024-09-12T12:26:33.031 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.vEXuPyywfw 2024-09-12T12:26:33.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-12T12:26:33.094 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.vEXuPyywfw is 514KB 2024-09-12T12:26:33.155 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.vEXuPyywfw 2024-09-12T12:26:33.160 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/data/mon.b.tgz 2024-09-12T12:26:33.160 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-09-12T12:26:33.166 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.TPsLb3NDdQ 2024-09-12T12:26:33.166 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TPsLb3NDdQ 2024-09-12T12:26:33.369 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.TPsLb3NDdQ 2024-09-12T12:26:33.429 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.TPsLb3NDdQ is 657KB 2024-09-12T12:26:33.492 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.TPsLb3NDdQ 2024-09-12T12:26:33.497 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/data/mon.c.tgz 2024-09-12T12:26:33.497 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-09-12T12:26:33.546 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.OxgQ9SwuVZ 2024-09-12T12:26:33.546 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.OxgQ9SwuVZ 2024-09-12T12:26:33.745 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.OxgQ9SwuVZ 2024-09-12T12:26:33.810 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.OxgQ9SwuVZ is 657KB 2024-09-12T12:26:33.872 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.OxgQ9SwuVZ 2024-09-12T12:26:33.877 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-12T12:26:33.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T12:26:33.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T12:26:33.939 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T12:26:33.982 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T12:26:33.982 INFO:tasks.ceph:Archiving crash dumps... 2024-09-12T12:26:33.984 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/remote/smithi003/crash 2024-09-12T12:26:33.985 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T12:26:33.999 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/remote/smithi142/crash 2024-09-12T12:26:33.999 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T12:26:34.037 INFO:tasks.ceph:Shutting down logrotate 2024-09-12T12:26:34.038 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:26:34.046 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-12T12:26:34.122 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-12T12:26:34.145 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-12T12:26:34.178 INFO:tasks.ceph:Compressing logs... 2024-09-12T12:26:34.178 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T12:26:34.190 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T12:26:34.209 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2024-09-12T12:26:34.210 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43815.log 2024-09-12T12:26:34.210 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2024-09-12T12:26:34.211 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35339.log 2024-09-12T12:26:34.211 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2024-09-12T12:26:34.211 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43815.log.gz 2024-09-12T12:26:34.211 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35619.log 2024-09-12T12:26:34.212 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2024-09-12T12:26:34.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41975.log 2024-09-12T12:26:34.212 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35339.log.gz 2024-09-12T12:26:34.212 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38280.log 2024-09-12T12:26:34.212 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35619.log.gz 2024-09-12T12:26:34.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42231.log 2024-09-12T12:26:34.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40057.log 2024-09-12T12:26:34.214 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41975.log.gz 2024-09-12T12:26:34.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-09-12T12:26:34.214 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42231.log.gz 2024-09-12T12:26:34.214 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40057.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.38280.log.gz 2024-09-12T12:26:34.214 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38097.log 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40057.log.gz 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34070.log 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38970.log 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38532.log 89.0% 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38097.log.gz 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34070.log.gz 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43672.log 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7%/var/log/ceph/ceph-client.admin.38532.log: -- replaced with /var/log/ceph/ceph-client.admin.38970.log.gz 2024-09-12T12:26:34.216 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33965.log 2024-09-12T12:26:34.218 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38532.log.gz 2024-09-12T12:26:34.218 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-09-12T12:26:34.218 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43672.log: /var/log/ceph/ceph-client.admin.33965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41084.log 2024-09-12T12:26:34.218 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43672.log.gz 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33965.log.gz 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36996.log 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41084.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41084.log.gz 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36996.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41426.log 2024-09-12T12:26:34.219 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36996.log.gz 2024-09-12T12:26:34.221 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2024-09-12T12:26:34.221 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-09-12T12:26:34.221 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37941.log 2024-09-12T12:26:34.221 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41426.log.gz 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37213.log 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44240.log 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37213.log: gzip 89.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.37941.log.gz -- 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr: /var/log/ceph/ceph-client.admin.38680.log 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37213.log.gz 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39145.log 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39509.log 2024-09-12T12:26:34.222 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44240.log.gz 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38680.log: /var/log/ceph/ceph-client.admin.39145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37683.log 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38680.log.gz 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39145.log.gz 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42098.log 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39509.log.gz 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37683.log: /var/log/ceph/ceph-client.admin.42098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35749.log 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37683.log.gz 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36081.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42098.log.gz 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr:gzip 94.0% -5 -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41322.log 2024-09-12T12:26:34.227 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36302.log 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41322.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35749.log.gz 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42257.log 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41322.log.gz 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34064.log 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36302.log.gz 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39601.log 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41776.log 88.7% 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42257.log.gz 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37442.log 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34064.log.gz 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34480.log 2024-09-12T12:26:34.228 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39601.log.gz 2024-09-12T12:26:34.229 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41776.log.gz 2024-09-12T12:26:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40256.log 2024-09-12T12:26:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34480.log: gzip -5 --verbose 88.1% -- /var/log/ceph/ceph-client.admin.40845.log -- replaced with /var/log/ceph/ceph-client.admin.37442.log.gz 2024-09-12T12:26:34.234 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.234 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40256.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34480.log.gz 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34486.log 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40256.log.gz 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33569.log 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.40845.log.gz 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34486.log: /var/log/ceph/ceph-client.admin.33569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36328.log 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34486.log.gz 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33569.log.gz 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36354.log 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43882.log 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36328.log.gz 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36462.log 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43882.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36354.log.gz 2024-09-12T12:26:34.235 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39048.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43882.log.gz 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39813.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34904.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36462.log.gz 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40483.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39048.log.gz -- replaced with /var/log/ceph/ceph-client.admin.39813.log.gz 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34904.log.gz 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39171.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38019.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40483.log.gz 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33599.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37398.log 2024-09-12T12:26:34.236 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39171.log.gz 2024-09-12T12:26:34.237 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46402.log 2024-09-12T12:26:34.237 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37398.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.33599.log.gz -5 2024-09-12T12:26:34.237 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-09-12T12:26:34.237 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37398.log.gz 2024-09-12T12:26:34.237 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-09-12T12:26:34.244 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46402.log.gz 2024-09-12T12:26:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2024-09-12T12:26:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44867.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-09-12T12:26:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44645.log 2024-09-12T12:26:34.244 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-09-12T12:26:34.244 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38201.log 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44645.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41270.log 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38201.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44645.log.gz 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38887.log 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38201.log.gz 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41244.log 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41270.log.gz 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41244.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38887.log.gz 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45683.log 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41244.log.gz 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz 2024-09-12T12:26:34.245 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.45683.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38503.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38466.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45683.log.gzgzip 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43367.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39249.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38503.log.gz 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35253.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39249.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43367.log.gz 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44159.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39249.log.gz 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35253.log.gz 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41348.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41218.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34278.log 2024-09-12T12:26:34.246 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41348.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39483.log 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41348.log.gz 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45605.log 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34278.log.gz 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5/var/log/ceph/ceph-client.admin.39483.log: --verbose -- /var/log/ceph/ceph-client.admin.38149.log 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33653.log 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39483.log.gz 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45605.log.gz 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37832.log 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38149.log.gz 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5/var/log/ceph/ceph-client.admin.33653.log: --verbose -- /var/log/ceph/ceph-client.admin.34479.log 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40294.log 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33653.log.gz 2024-09-12T12:26:34.247 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37832.log.gz 2024-09-12T12:26:34.248 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38440.log 2024-09-12T12:26:34.248 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29188.log 2024-09-12T12:26:34.249 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29116.log 2024-09-12T12:26:34.267 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.29188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29092.log 2024-09-12T12:26:34.267 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.29116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35511.log 2024-09-12T12:26:34.267 INFO:teuthology.orchestra.run.smithi142.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29188.log.gz 2024-09-12T12:26:34.267 INFO:teuthology.orchestra.run.smithi142.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29116.log.gz 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.29092.log: /var/log/ceph/ceph-client.1.35511.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29092.log.gz 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35897.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.1.35511.log.gz 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36417.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.35897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36730.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.36417.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.1.35897.log.gz 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.36730.log: gzip 89.6% -- replaced with /var/log/ceph/ceph-client.1.36417.log.gz -5 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29020.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29068.log 2024-09-12T12:26:34.268 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.29020.log: /var/log/ceph/ceph-client.admin.29068.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29020.log.gz 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr: -5 --verbose -- /var/log/ceph/ceph-client.badguy.36103.log 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29068.log.gz 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.badguy.36103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29044.log 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph.log: 86.7% -- replaced with /var/log/ceph/ceph-client.badguy.36103.log.gz 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29164.log 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29140.log 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.29044.log: 91.6% -- replaced with /var/log/ceph/ceph-client.1.36730.log.gz 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.29164.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29044.log.gz 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29164.log.gz 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-12T12:26:34.269 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.35335.log 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.29140.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29140.log.gz 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.3.35335.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr: 92.4%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36579.log 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.3.35335.log.gz 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.36579.log: 89.3% 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr: -- replaced with /var/log/ceph/ceph-client.1.36579.log.gz 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-12T12:26:34.270 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.badguy.36002.log 2024-09-12T12:26:34.273 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mon.b.log: 93.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-12T12:26:34.273 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-12T12:26:34.273 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.badguy.36002.log: 90.6% -- replaced with /var/log/ceph/ceph-client.badguy.36002.log.gz 2024-09-12T12:26:34.273 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35216.log 2024-09-12T12:26:34.289 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35682.log 2024-09-12T12:26:34.289 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.35216.log: 88.5% -- replaced with /var/log/ceph/ceph-client.1.35216.log.gz 2024-09-12T12:26:34.296 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36259.log 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34479.log.gz/var/log/ceph/ceph-client.admin.40294.log: 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37291.log 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40294.log.gz 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43471.log 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38440.log.gz 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35671.log 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35033.log 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37291.log.gz 2024-09-12T12:26:34.302 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43471.log.gz 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42205.log 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35671.log.gz 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.35033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37317.log 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42205.log: /var/log/ceph/ceph-client.admin.43198.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42205.log.gz 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.0.35033.log.gz 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44446.log 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37317.log.gz 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43341.log 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41192.log 2024-09-12T12:26:34.303 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44446.log: /var/log/ceph/ceph-client.admin.43341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42847.log 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43341.log.gz 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44446.log.gz 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45931.log 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36410.log 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42847.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41192.log.gz 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45931.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.42847.log.gz 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36488.log 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45931.log.gz 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34459.log 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36410.log.gz 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36488.log.gz 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-09-12T12:26:34.304 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44133.log: /var/log/ceph/ceph-client.admin.34459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40535.log 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34459.log.gz 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37715.log 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40535.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz --verbose 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.40509.log 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45631.log 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40535.log.gz 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37715.log.gz 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42548.log 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40509.log: /var/log/ceph/ceph-client.admin.45631.log: gzip -5 --verbose 88.2% -- /var/log/ceph/ceph-client.admin.46089.log -- replaced with /var/log/ceph/ceph-client.admin.40509.log.gz 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45631.log.gz 2024-09-12T12:26:34.305 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35227.log 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45451.log 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42548.log.gz 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46089.log: /var/log/ceph/ceph-client.admin.35227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36514.log 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35227.log.gz 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46089.log.gz 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36593.log 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35082.log 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36514.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45451.log.gz 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40754.log 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36593.log.gz 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36514.log.gz 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41923.log 2024-09-12T12:26:34.306 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40754.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35082.log.gz 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35589.log 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40754.log.gz 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42179.log 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33627.log 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41923.log.gz 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35589.log.gz 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42179.log: /var/log/ceph/ceph-mon.a.log 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44396.log 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42179.log.gz 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33627.log.gz 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39787.log 2024-09-12T12:26:34.307 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.44396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37572.log 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39787.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44396.log.gz 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37572.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39787.log.gz 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36619.log 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37572.log.gz 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44671.log 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43276.log: /var/log/ceph/ceph-client.admin.36619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44583.log 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36619.log.gz 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39197.log 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44671.log: /var/log/ceph/ceph-client.admin.44583.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44671.log.gz 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43419.log 2024-09-12T12:26:34.308 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39197.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44583.log.gz 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40813.log 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39197.log.gz 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38711.log 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43419.log: /var/log/ceph/ceph-client.admin.40813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45212.log 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40813.log.gz 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43419.log.gz 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38918.log 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38711.log: /var/log/ceph/ceph-client.admin.45212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38798.log 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38711.log.gz 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45212.log.gz 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45111.log 2024-09-12T12:26:34.309 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39735.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.38798.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38918.log.gz 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36671.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39735.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45111.log.gz 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38742.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.0.38798.log.gz 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39735.log.gz 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46141.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35313.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.38742.log.gz 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46141.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45001.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36671.log.gz 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46141.log.gz 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39301.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43908.log 2024-09-12T12:26:34.310 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35313.log.gz 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45001.log: /var/log/ceph/ceph-client.admin.39301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39627.log 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45001.log.gz 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39301.log.gz 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37468.log 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39627.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43908.log.gz 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38857.log 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39627.log.gz 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37468.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37468.log.gz 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42717.log 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45142.log/var/log/ceph/ceph-client.admin.46297.log: 2024-09-12T12:26:34.311 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38857.log.gz 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40936.log 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42717.log.gz 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35477.log 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40936.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45142.log.gz 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42609.log 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40936.log.gz 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43250.log 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:gzip 88.0% -5/var/log/ceph/ceph-client.admin.42609.log: --verbose -- replaced with /var/log/ceph/ceph-client.admin.35477.log.gz -- 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr: /var/log/ceph/ceph-client.admin.36913.log 2024-09-12T12:26:34.312 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46454.log 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42609.log.gz 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43250.log.gz 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43646.log 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45241.log 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46454.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36913.log.gz 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40989.log 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46454.log.gz 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43646.log.gz 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41806.log 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45241.log: /var/log/ceph/ceph-client.admin.40989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46219.log 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45241.log.gz 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40989.log.gz 2024-09-12T12:26:34.313 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43165.log 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46219.log: /var/log/ceph/ceph-client.admin.43165.log: gzip -5 --verbose 88.2% -- /var/log/ceph/ceph-client.admin.45553.log -- replaced with /var/log/ceph/ceph-client.admin.46219.log.gz 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41806.log.gz 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35723.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43165.log.gz 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36970.log 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36567.log 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40379.log 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36970.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45553.log.gz 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44051.log 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36970.log.gz 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36567.log.gz 2024-09-12T12:26:34.314 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33991.log 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44051.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40379.log.gz 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40702.log 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44051.log.gz 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38228.log 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40026.log 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33991.log.gz 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38228.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.40702.log.gz 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44292.log 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38228.log.gz 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38045.log 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35562.log 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40026.log.gz 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44292.log: /var/log/ceph/ceph-client.admin.38045.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44292.log.gz 2024-09-12T12:26:34.315 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41049.log 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38045.log.gz 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36833.log 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41049.log: 89.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.35562.log.gz -5 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35853.log 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41049.log.gz/var/log/ceph/ceph-client.admin.36833.log: 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34487.log 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35853.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36833.log.gz 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44556.log 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35853.log.gz 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42795.log 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44556.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34487.log.gz 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35450.log 2024-09-12T12:26:34.316 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44556.log.gz 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38254.log 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43568.log 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42795.log.gz 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38254.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35450.log.gz 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35534.log 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43568.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38254.log.gz 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39457.log 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43568.log.gz 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35534.log: /var/log/ceph/ceph-client.admin.39457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41949.log 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35534.log.gz 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39457.log.gz 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45657.log 2024-09-12T12:26:34.317 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37861.log 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41949.log.gz 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35937.log 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40910.log 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45657.log.gz 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37861.log.gz 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35365.log 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36721.log 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40910.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35937.log.gz 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35365.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40910.log.gz 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36721.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35365.log.gz 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46063.log 2024-09-12T12:26:34.318 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36721.log.gz 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34719.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45860.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46063.log: /var/log/ceph/ceph-client.admin.34719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33340.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46063.log.gz 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0%/var/log/ceph/ceph-client.admin.45860.log: -- replaced with /var/log/ceph/ceph-client.admin.34719.log.gzgzip 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44213.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33340.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45860.log.gz 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35645.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33340.log.gz 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36107.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43698.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44213.log.gz 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35645.log: /var/log/ceph/ceph-client.admin.36107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39392.log 2024-09-12T12:26:34.319 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35645.log.gz 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43698.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36107.log.gz 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45579.log 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43698.log.gz 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40606.log 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45579.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39392.log.gz 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42635.log 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45579.log.gz 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44187.log 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40606.log.gz 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40353.log 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44187.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42635.log.gz 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36436.log 2024-09-12T12:26:34.320 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44187.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37371.log 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36436.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36777.log 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40353.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36436.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44344.log 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46324.log 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37371.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.36777.log: /var/log/ceph/ceph-client.admin.44344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41871.log 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33451.log 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44344.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46324.log: /var/log/ceph/ceph-client.admin.41871.log: 89.0% -- replaced with /var/log/ceph/ceph-client.0.36777.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46324.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41871.log.gz 2024-09-12T12:26:34.321 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45393.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35391.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33451.log.gz 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33913.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45393.log.gz 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41665.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39074.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33913.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35391.log.gz 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41665.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33913.log.gz 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz -5 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39074.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36028.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40457.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40204.log 2024-09-12T12:26:34.322 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39074.log.gz 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36028.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23908.log 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36028.log.gz 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40204.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40204.log.gz 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38175.log 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.tmp-client.admin.23908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39022.log 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23908.log.gz 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46482.log 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34055.log 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38175.log.gz 2024-09-12T12:26:34.323 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2%/var/log/ceph/ceph-client.admin.46482.log: -- replaced with /var/log/ceph/ceph-client.admin.39022.log.gz 2024-09-12T12:26:34.325 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.35682.log: 92.6%/var/log/ceph/ceph-client.1.36259.log: 89.4% -- replaced with /var/log/ceph/ceph-client.1.36259.log.gz 2024-09-12T12:26:34.325 INFO:teuthology.orchestra.run.smithi142.stderr: -- replaced with /var/log/ceph/ceph-client.1.35682.log.gz 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46482.log.gz 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40084.log 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45985.log 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1%/var/log/ceph/ceph-client.admin.40084.log: -- replaced with /var/log/ceph/ceph-client.admin.34055.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42665.log 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40084.log.gz 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45985.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45985.log.gz 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35905.log 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41724.log 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8%/var/log/ceph/ceph-client.admin.35905.log: -- replaced with /var/log/ceph/ceph-client.admin.42665.log.gzgzip 2024-09-12T12:26:34.350 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.44502.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35905.log.gz 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41724.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41724.log.gz 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45267.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.44502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42925.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43393.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42925.log: 89.1% -- replaced with /var/log/ceph/ceph-client.0.44502.log.gz 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45267.log.gz 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42925.log.gz 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34268.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36886.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46115.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4%/var/log/ceph/ceph-client.admin.36886.log: -- replaced with /var/log/ceph/ceph-client.admin.34268.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44717.log 2024-09-12T12:26:34.351 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36886.log.gz 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46115.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46115.log.gz 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39761.log 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37265.log 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1%/var/log/ceph/ceph-client.admin.39761.log: -- replaced with /var/log/ceph/ceph-client.admin.44717.log.gz 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39100.log 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39761.log.gz 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37265.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37265.log.gz 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42438.log 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34067.log 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8%/var/log/ceph/ceph-client.admin.42438.log: -- replaced with /var/log/ceph/ceph-client.admin.39100.log.gz 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2024-09-12T12:26:34.352 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42438.log.gz 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34067.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34067.log.gz 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36133.log 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1%/var/log/ceph/ceph-client.admin.36211.log: -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46193.log 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36133.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36133.log.gz 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46506.log 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4%/var/log/ceph/ceph-client.admin.46506.log: -- replaced with /var/log/ceph/ceph-client.admin.46193.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34271.log 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46506.log.gz/var/log/ceph/ceph-client.admin.36159.log: 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38996.log 2024-09-12T12:26:34.353 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46532.log 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42769.log 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34271.log.gz 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38996.log: /var/log/ceph/ceph-client.admin.46532.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38996.log.gz 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46532.log.gz 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42899.log 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41558.log 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42899.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.44025.log 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42769.log.gz 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42899.log.gz 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.41558.log: 88.8% -- replaced with /var/log/ceph/ceph-client.0.41558.log.gz 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33543.log 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34482.log 2024-09-12T12:26:34.354 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8%/var/log/ceph/ceph-client.admin.33543.log: -- replaced with /var/log/ceph/ceph-client.admin.44025.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46167.log 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33543.log.gz 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34482.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34482.log.gz 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39865.log 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42583.log 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39865.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43057.log 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46167.log.gz 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42583.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39865.log.gz 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42583.log.gz 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34823.log 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.43057.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39683.log 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34823.log.gz 2024-09-12T12:26:34.355 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.log: 89.1% -- replaced with /var/log/ceph/ceph-client.0.43057.log.gz 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42283.log 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34272.log 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42283.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39683.log.gz 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7%/var/log/ceph/ceph-client.admin.34272.log: -- replaced with /var/log/ceph/ceph-client.admin.42283.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41502.log 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41374.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34272.log.gz 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34771.log 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37624.log 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41502.log: /var/log/ceph/ceph-client.admin.34771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41502.log.gz 2024-09-12T12:26:34.356 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37624.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34771.log.gz 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37624.log.gz 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34274.log 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37546.log 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34274.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37915.log 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34274.log.gz 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37546.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37546.log.gz 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44318.log 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4%/var/log/ceph/ceph-client.admin.44318.log: -- replaced with /var/log/ceph/ceph-client.admin.37915.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35775.log 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44318.log.gz 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37780.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-09-12T12:26:34.357 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36267.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8%/var/log/ceph/ceph-client.admin.36267.log: -- replaced with /var/log/ceph/ceph-client.admin.35775.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34071.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36267.log.gz 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37520.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38123.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33731.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38123.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37343.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34071.log.gz 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38123.log.gz 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33731.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33731.log.gz 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42743.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39327.log 2024-09-12T12:26:34.358 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2%/var/log/ceph/ceph-client.admin.42743.log: -- replaced with /var/log/ceph/ceph-client.admin.37343.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38558.log 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42743.log.gz 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39327.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39327.log.gz 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42001.log 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38944.log 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.38558.log.gz 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42001.log.gz 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38944.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38944.log.gz 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44718.log 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41110.log 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0%/var/log/ceph/ceph-client.admin.44718.log: -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37494.log 2024-09-12T12:26:34.359 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44718.log.gz 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41110.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41110.log.gz 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33418.log 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9%/var/log/ceph/ceph-client.admin.33418.log: -- replaced with /var/log/ceph/ceph-client.admin.37494.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35879.log 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33418.log.gz 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40230.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38071.log 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37993.log 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38071.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35879.log.gz 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44266.log 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40405.log 2024-09-12T12:26:34.360 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38071.log.gz 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44266.log: /var/log/ceph/ceph-client.admin.37993.log: 88.9% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37993.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44266.log.gz 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34667.log 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43594.log 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34667.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40405.log.gz 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40676.log 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39360.log 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34667.log.gz 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43594.log.gz 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40676.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40676.log.gz 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34066.log 2024-09-12T12:26:34.361 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42464.log 2024-09-12T12:26:34.363 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-12T12:26:34.363 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4%/var/log/ceph/ceph-client.admin.34066.log: -- replaced with /var/log/ceph/ceph-client.admin.39360.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36239.log 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34066.log.gz 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42464.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42464.log.gz 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38387.log 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37141.log 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.38387.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36239.log.gz 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42342.log 2024-09-12T12:26:34.367 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37141.log: /var/log/ceph/ceph-client.admin.42342.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37141.log.gz 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42342.log.gz 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34797.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45057.log 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.45057.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.38387.log.gz 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.0.45057.log.gz 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-12T12:26:34.368 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35417.log 2024-09-12T12:26:34.369 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8%/var/log/ceph/ceph-mds.a.log: -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-12T12:26:34.373 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37967.log 2024-09-12T12:26:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35417.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35417.log.gz 2024-09-12T12:26:34.374 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43445.log 2024-09-12T12:26:34.375 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37967.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37967.log.gz 2024-09-12T12:26:34.375 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42309.log 2024-09-12T12:26:34.376 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43445.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43445.log.gz 2024-09-12T12:26:34.377 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41692.log 2024-09-12T12:26:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42309.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42309.log.gz 2024-09-12T12:26:34.378 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34483.log 2024-09-12T12:26:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41692.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41692.log.gz 2024-09-12T12:26:34.379 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35827.log 2024-09-12T12:26:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34483.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34483.log.gz 2024-09-12T12:26:34.381 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34051.log 2024-09-12T12:26:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35827.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35827.log.gz 2024-09-12T12:26:34.382 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-09-12T12:26:34.383 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34051.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34051.log.gz 2024-09-12T12:26:34.384 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39223.log 2024-09-12T12:26:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-09-12T12:26:34.385 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44925.log 2024-09-12T12:26:34.386 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39223.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39223.log.gz 2024-09-12T12:26:34.387 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38330.log 2024-09-12T12:26:34.388 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44925.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44925.log.gz 2024-09-12T12:26:34.388 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33809.log 2024-09-12T12:26:34.389 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.38330.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38330.log.gz 2024-09-12T12:26:34.389 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-09-12T12:26:34.390 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33809.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33809.log.gz 2024-09-12T12:26:34.391 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39653.log 2024-09-12T12:26:34.392 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33887.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-09-12T12:26:34.392 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45177.log 2024-09-12T12:26:34.393 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39653.log.gz 2024-09-12T12:26:34.393 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45477.log 2024-09-12T12:26:34.394 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45177.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45177.log.gz 2024-09-12T12:26:34.395 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41750.log 2024-09-12T12:26:34.396 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45477.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45477.log.gz 2024-09-12T12:26:34.396 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2024-09-12T12:26:34.397 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41750.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41750.log.gz 2024-09-12T12:26:34.398 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34017.log 2024-09-12T12:26:34.399 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44610.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2024-09-12T12:26:34.399 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42053.log 2024-09-12T12:26:34.400 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34017.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34017.log.gz 2024-09-12T12:26:34.400 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36185.log 2024-09-12T12:26:34.401 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42053.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42053.log.gz 2024-09-12T12:26:34.402 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40780.log 2024-09-12T12:26:34.403 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36185.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36185.log.gz 2024-09-12T12:26:34.403 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36384.log 2024-09-12T12:26:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40780.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40780.log.gz 2024-09-12T12:26:34.404 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28909.log 2024-09-12T12:26:34.405 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36384.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36384.log.gz 2024-09-12T12:26:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2024-09-12T12:26:34.407 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28909.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.28909.log.gz 2024-09-12T12:26:34.407 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34060.log 2024-09-12T12:26:34.408 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45420.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2024-09-12T12:26:34.409 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46428.log 2024-09-12T12:26:34.410 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34060.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34060.log.gz 2024-09-12T12:26:34.410 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41400.log 2024-09-12T12:26:34.411 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46428.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46428.log.gz 2024-09-12T12:26:34.411 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34693.log 2024-09-12T12:26:34.413 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41400.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41400.log.gz 2024-09-12T12:26:34.413 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43962.log 2024-09-12T12:26:34.414 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34693.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34693.log.gz 2024-09-12T12:26:34.414 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42027.log 2024-09-12T12:26:34.415 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43962.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43962.log.gz 2024-09-12T12:26:34.415 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43001.log 2024-09-12T12:26:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42027.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42027.log.gz 2024-09-12T12:26:34.417 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41897.log 2024-09-12T12:26:34.418 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43001.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43001.log.gz 2024-09-12T12:26:34.418 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37026.log 2024-09-12T12:26:34.419 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41897.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41897.log.gz 2024-09-12T12:26:34.420 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45886.log 2024-09-12T12:26:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37026.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37026.log.gz 2024-09-12T12:26:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40431.log 2024-09-12T12:26:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45886.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45886.log.gz 2024-09-12T12:26:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34481.log 2024-09-12T12:26:34.423 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40431.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40431.log.gz 2024-09-12T12:26:34.424 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-09-12T12:26:34.425 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34481.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34481.log.gz 2024-09-12T12:26:34.425 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2024-09-12T12:26:34.426 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40117.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-09-12T12:26:34.427 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37082.log 2024-09-12T12:26:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43497.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2024-09-12T12:26:34.428 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37598.log 2024-09-12T12:26:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.0.37082.log: 88.2% -- replaced with /var/log/ceph/ceph-client.0.37082.log.gz 2024-09-12T12:26:34.430 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46376.log 2024-09-12T12:26:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37598.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37598.log.gz 2024-09-12T12:26:34.431 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2024-09-12T12:26:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46376.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46376.log.gz 2024-09-12T12:26:34.433 INFO:teuthology.orchestra.run.smithi142.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-12T12:26:34.433 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41136.log 2024-09-12T12:26:34.434 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41638.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2024-09-12T12:26:34.434 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33705.log 2024-09-12T12:26:34.435 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41136.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41136.log.gz 2024-09-12T12:26:34.436 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40650.log 2024-09-12T12:26:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33705.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33705.log.gz 2024-09-12T12:26:34.437 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35801.log 2024-09-12T12:26:34.438 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40650.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40650.log.gz 2024-09-12T12:26:34.438 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2024-09-12T12:26:34.439 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35801.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35801.log.gz 2024-09-12T12:26:34.440 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40143.log 2024-09-12T12:26:34.441 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45317.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gz 2024-09-12T12:26:34.441 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-09-12T12:26:34.443 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40143.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40143.log.gz 2024-09-12T12:26:34.443 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42951.log 2024-09-12T12:26:34.444 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46271.log: 88.6% 90.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-12T12:26:34.444 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-09-12T12:26:34.444 INFO:teuthology.orchestra.run.smithi142.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-12T12:26:34.448 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33517.log 2024-09-12T12:26:34.449 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36054.log 2024-09-12T12:26:34.450 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33517.log: 91.1% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33517.log.gz 2024-09-12T12:26:34.450 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42951.log.gz 2024-09-12T12:26:34.450 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35697.log 2024-09-12T12:26:34.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2024-09-12T12:26:34.451 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36054.log: /var/log/ceph/ceph-client.admin.35697.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36054.log.gz 2024-09-12T12:26:34.451 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35697.log.gz 2024-09-12T12:26:34.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43138.log 2024-09-12T12:26:34.452 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33835.log: -- /var/log/ceph/ceph-client.admin.40728.log 2024-09-12T12:26:34.452 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2024-09-12T12:26:34.453 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43138.log.gz 2024-09-12T12:26:34.453 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45959.log 2024-09-12T12:26:34.453 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2024-09-12T12:26:34.454 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6%/var/log/ceph/ceph-client.admin.45959.log: -- replaced with /var/log/ceph/ceph-client.admin.40728.log.gz 2024-09-12T12:26:34.454 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45959.log.gz 2024-09-12T12:26:34.454 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2024-09-12T12:26:34.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43783.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2024-09-12T12:26:34.455 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38608.log 2024-09-12T12:26:34.456 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44370.log 2024-09-12T12:26:34.456 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz/var/log/ceph/ceph-client.admin.38608.log: 2024-09-12T12:26:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45753.log 2024-09-12T12:26:34.457 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44370.log: 90.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-12T12:26:34.457 INFO:teuthology.orchestra.run.smithi003.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.38608.log.gz 2024-09-12T12:26:34.457 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44370.log.gz 2024-09-12T12:26:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44951.log 2024-09-12T12:26:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-09-12T12:26:34.458 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45753.log: /var/log/ceph/ceph-client.admin.44951.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.45753.log.gz /var/log/ceph/ceph-client.admin.42126.log 2024-09-12T12:26:34.458 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43724.log: 91.1% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-09-12T12:26:34.459 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2024-09-12T12:26:34.459 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44077.log 2024-09-12T12:26:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33757.log 2024-09-12T12:26:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44077.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-09-12T12:26:34.460 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36645.log 2024-09-12T12:26:34.461 INFO:teuthology.orchestra.run.smithi003.stderr:gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44077.log.gz -5 2024-09-12T12:26:34.461 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42821.log 2024-09-12T12:26:34.461 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33757.log: /var/log/ceph/ceph-client.admin.36645.log: 88.6% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36645.log.gz 2024-09-12T12:26:34.461 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33757.log.gz 2024-09-12T12:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39971.log 2024-09-12T12:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34279.log 2024-09-12T12:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7%/var/log/ceph/ceph-client.0.39971.log: -- replaced with /var/log/ceph/ceph-client.admin.42821.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43750.log 2024-09-12T12:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.463 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34279.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34279.log.gz 2024-09-12T12:26:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33861.log 2024-09-12T12:26:34.464 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36859.log 2024-09-12T12:26:34.465 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.33861.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.39971.log.gz 2024-09-12T12:26:34.465 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43750.log.gz 2024-09-12T12:26:34.465 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33861.log.gz 2024-09-12T12:26:34.465 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33679.log 2024-09-12T12:26:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35508.log 2024-09-12T12:26:34.467 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5%/var/log/ceph/ceph-client.admin.33679.log: -- replaced with /var/log/ceph/ceph-client.admin.36859.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45709.log 2024-09-12T12:26:34.467 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.467 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33679.log.gz 2024-09-12T12:26:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35508.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35508.log.gz 2024-09-12T12:26:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39275.log 2024-09-12T12:26:34.467 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.45709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36002.log 2024-09-12T12:26:34.469 INFO:teuthology.orchestra.run.smithi003.stderr: 89.2%/var/log/ceph/ceph-client.admin.39275.log: -- replaced with /var/log/ceph/ceph-client.admin.45709.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2024-09-12T12:26:34.469 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.469 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39275.log.gz 2024-09-12T12:26:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36002.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36002.log.gz 2024-09-12T12:26:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37239.log 2024-09-12T12:26:34.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34276.log 2024-09-12T12:26:34.470 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3%/var/log/ceph/ceph-client.admin.37239.log: -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46350.log 2024-09-12T12:26:34.470 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.470 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37239.log.gz 2024-09-12T12:26:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34276.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34276.log.gz 2024-09-12T12:26:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-09-12T12:26:34.471 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-09-12T12:26:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40580.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46350.log.gz 2024-09-12T12:26:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46037.log 2024-09-12T12:26:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44716.log 2024-09-12T12:26:34.472 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz/var/log/ceph/ceph-client.admin.41166.log: 2024-09-12T12:26:34.472 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46037.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz 2024-09-12T12:26:34.472 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46037.log.gz 2024-09-12T12:26:34.473 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39915.log 2024-09-12T12:26:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39538.log 2024-09-12T12:26:34.474 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9%/var/log/ceph/ceph-client.admin.39915.log: -- replaced with /var/log/ceph/ceph-client.admin.44716.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28793.log 2024-09-12T12:26:34.474 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.474 INFO:teuthology.orchestra.run.smithi003.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39915.log.gz 2024-09-12T12:26:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39538.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39538.log.gz 2024-09-12T12:26:34.474 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39566.log 2024-09-12T12:26:34.475 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-09-12T12:26:34.476 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.39566.log: 90.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29012.log 2024-09-12T12:26:34.476 INFO:teuthology.orchestra.run.smithi003.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28793.log.gz 2024-09-12T12:26:34.476 INFO:teuthology.orchestra.run.smithi003.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39566.log.gz 2024-09-12T12:26:34.476 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42153.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-09-12T12:26:34.476 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39709.log 2024-09-12T12:26:34.476 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41612.log 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9%/var/log/ceph/ceph-client.admin.39709.log: -- replaced with /var/log/ceph/ceph-client.admin.29012.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39709.log.gz 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.41612.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41612.log.gz 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36540.log 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.46245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35287.log 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4%/var/log/ceph/ceph-client.admin.36540.log: -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43620.log 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.478 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36540.log.gz 2024-09-12T12:26:34.479 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.35287.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35287.log.gz 2024-09-12T12:26:34.479 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42374.log 2024-09-12T12:26:34.479 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34260.log 2024-09-12T12:26:34.479 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42374.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43620.log.gz 2024-09-12T12:26:34.479 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2024-09-12T12:26:34.480 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34260.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42374.log.gz 2024-09-12T12:26:34.480 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34260.log.gz 2024-09-12T12:26:34.480 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33783.log 2024-09-12T12:26:34.481 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.44107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35177.log 2024-09-12T12:26:34.481 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4%/var/log/ceph/ceph-client.admin.33783.log: -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2024-09-12T12:26:34.481 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37889.log 2024-09-12T12:26:34.482 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33783.log.gz 2024-09-12T12:26:34.482 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.2.35177.log: 89.7% -- replaced with /var/log/ceph/ceph-client.2.35177.log.gz 2024-09-12T12:26:34.482 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42873.log 2024-09-12T12:26:34.483 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.37889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-09-12T12:26:34.483 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.42873.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37889.log.gz 2024-09-12T12:26:34.483 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40295.log 2024-09-12T12:26:34.483 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-09-12T12:26:34.483 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42873.log.gz 2024-09-12T12:26:34.484 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43111.log: /var/log/ceph/ceph-client.admin.40295.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-09-12T12:26:34.484 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40295.log.gz 2024-09-12T12:26:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34849.log 2024-09-12T12:26:34.485 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.40962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43934.log 2024-09-12T12:26:34.486 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7%/var/log/ceph/ceph-client.admin.34849.log: -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36944.log 2024-09-12T12:26:34.486 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.486 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.43934.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43934.log.gz 2024-09-12T12:26:34.486 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.34849.log.gz 2024-09-12T12:26:34.486 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37806.log 2024-09-12T12:26:34.487 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.36944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-09-12T12:26:34.487 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7%/var/log/ceph/ceph-client.admin.37806.log: -- replaced with /var/log/ceph/ceph-client.admin.36944.log.gz 2024-09-12T12:26:34.487 INFO:teuthology.orchestra.run.smithi003.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37806.log.gz 2024-09-12T12:26:34.487 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.34745.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-09-12T12:26:34.609 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-12T12:26:34.610 INFO:teuthology.orchestra.run.smithi003.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-12T12:26:34.652 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-12T12:26:34.667 INFO:teuthology.orchestra.run.smithi142.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-12T12:26:34.673 INFO:teuthology.orchestra.run.smithi142.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-12T12:26:34.675 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-09-12T12:26:34.675 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m0.449s 2024-09-12T12:26:34.675 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m2.092s 2024-09-12T12:26:34.675 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m0.117s 2024-09-12T12:26:34.723 INFO:teuthology.orchestra.run.smithi003.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-12T12:26:34.754 INFO:teuthology.orchestra.run.smithi003.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-12T12:26:34.757 INFO:teuthology.orchestra.run.smithi003.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-12T12:26:34.783 INFO:teuthology.orchestra.run.smithi003.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-12T12:26:34.785 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T12:26:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m0.592s 2024-09-12T12:26:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m2.832s 2024-09-12T12:26:34.785 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m0.227s 2024-09-12T12:26:34.785 INFO:tasks.ceph:Archiving logs... 2024-09-12T12:26:34.785 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/remote/smithi003/log 2024-09-12T12:26:34.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T12:26:35.427 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/remote/smithi142/log 2024-09-12T12:26:35.428 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T12:26:35.674 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T12:26:35.683 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T12:26:35.683 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T12:26:35.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T12:26:35.779 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T12:26:35.787 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T12:26:35.787 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:26:35.790 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:26:35.801 INFO:teuthology.orchestra.run.smithi003.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T12:26:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:============================================================================== 2024-09-12T12:26:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:*hv01.front.sepi 67.205.162.81 3 u 33 64 377 0.067 +1.406 1.173 2024-09-12T12:26:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:+hv02.front.sepi 63.231.80.2 3 u 36 64 377 0.065 -1.035 1.262 2024-09-12T12:26:35.801 INFO:teuthology.orchestra.run.smithi003.stdout:+hv03.front.sepi 74.6.168.72 3 u 33 64 377 0.100 -0.059 1.466 2024-09-12T12:26:35.801 INFO:teuthology.orchestra.run.smithi003.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-09-12T12:26:35.802 INFO:teuthology.orchestra.run.smithi142.stdout: remote refid st t when poll reach delay offset jitter 2024-09-12T12:26:35.802 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================== 2024-09-12T12:26:35.803 INFO:teuthology.orchestra.run.smithi142.stdout:+hv01.front.sepi 67.205.162.81 3 u 28 64 377 0.108 +0.049 0.947 2024-09-12T12:26:35.803 INFO:teuthology.orchestra.run.smithi142.stdout:+hv02.front.sepi 63.231.80.2 3 u 32 64 377 0.076 -1.387 1.155 2024-09-12T12:26:35.803 INFO:teuthology.orchestra.run.smithi142.stdout:*hv03.front.sepi 74.6.168.72 3 u 29 64 377 0.091 -0.764 1.291 2024-09-12T12:26:35.803 INFO:teuthology.orchestra.run.smithi142.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-09-12T12:26:35.803 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T12:26:35.829 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T12:26:35.829 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T12:26:35.864 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T12:26:35.910 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T12:26:35.948 INFO:teuthology.task.internal:Duration was 1095.174993 seconds 2024-09-12T12:26:35.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T12:26:36.027 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T12:26:36.028 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T12:26:36.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T12:26:36.068 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T12:26:36.069 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T12:26:36.069 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T12:26:36.126 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-09-12T12:26:36.126 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T12:26:36.137 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T12:26:36.137 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T12:26:36.169 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T12:26:36.208 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T12:26:36.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T12:26:36.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T12:26:36.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T12:26:36.335 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T12:26:36.335 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T12:26:36.346 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T12:26:36.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T12:26:36.521 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:26:36.524 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:26:36.535 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-09-12T12:26:36.537 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-09-12T12:26:36.552 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:26:36.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:26:36.593 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:26:36.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:26:36.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T12:26:36.607 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T12:26:36.608 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/remote/smithi003 2024-09-12T12:26:36.608 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T12:26:36.658 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901713/remote/smithi142 2024-09-12T12:26:36.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T12:26:36.678 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T12:26:36.678 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T12:26:36.702 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T12:26:36.726 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T12:26:36.737 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T12:26:36.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T12:26:36.766 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T12:26:36.766 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T12:26:36.768 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T12:26:36.773 INFO:teuthology.orchestra.run.smithi003.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 12:26 /home/ubuntu/cephtest 2024-09-12T12:26:36.773 INFO:teuthology.orchestra.run.smithi142.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 12:26 /home/ubuntu/cephtest 2024-09-12T12:26:36.775 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T12:26:36.799 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T12:26:36.888 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/sessionmap} duration: 1095.174993276596 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T12:26:36.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T12:26:36.970 INFO:teuthology.run:pass